Lux.jl

Bayesian Neural Network

We borrow this tutorial from the official Turing Docs. We will show how the explicit parameterization of Lux enables first-class composability with packages which expect flattened out parameter vectors.

We will use Turing.jl with Lux.jl to implement implementing a classification algorithm. Lets start by importing the relevant libraries.

# Import libraries
using Lux, Turing, CairoMakie, Random, Tracker, Functors, MakiePublication, LinearAlgebra

# Sampling progress
Turing.setprogress!(true);

# Use reverse_diff due to the number of parameters in neural networks
Turing.setadbackend(:tracker)
ERROR: LoadError: ArgumentError: Package Zygote [e88e6eb3-aa80-5325-afca-941959d7151f] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
  [1] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1920
  [2] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1806
  [3] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
  [4] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1797
  [6] macro expansion
    @ Base ./loading.jl:1784 [inlined]
  [7] macro expansion
    @ Base ./lock.jl:267 [inlined]
  [8] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1747
  [9] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
 [10] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
 [11] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1740
 [12] top-level scope
    @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Bijectors/gR4QV/ext/BijectorsZygoteExt.jl:4
 [13] include
    @ Base ./Base.jl:495 [inlined]
 [14] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2216
 [15] top-level scope
    @ stdin:3
in expression starting at /root/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Bijectors/gR4QV/ext/BijectorsZygoteExt.jl:1
in expression starting at stdin:3
┌ Error: Error during loading of extension BijectorsZygoteExt of Bijectors, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Failed to precompile BijectorsZygoteExt [91b77daa-8402-5dd2-b8ee-b16f8be7bd86] to "/root/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/compiled/v1.10/BijectorsZygoteExt/jl_7zEc4k".
│    Stacktrace:
│      [1] error(s::String)
│        @ Base ./error.jl:35
│      [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
│        @ Base ./loading.jl:2462
│      [3] compilecache
│        @ Base ./loading.jl:2334 [inlined]
│      [4] (::Base.var"#968#969"{Base.PkgId})()
│        @ Base ./loading.jl:1968
│      [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
│      [6] #mkpidlock#6
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
│      [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
│      [8] #invokelatest#2
│        @ Base ./essentials.jl:889 [inlined]
│      [9] invokelatest
│        @ Base ./essentials.jl:884 [inlined]
│     [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
│        @ Base ./loading.jl:2977
│     [11] maybe_cachefile_lock
│        @ Base ./loading.jl:2974 [inlined]
│     [12] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1964
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1806
│     [14] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [16] _require_prelocked
│        @ Base ./loading.jl:1797 [inlined]
│     [17] _require_prelocked
│        @ Base ./loading.jl:1796 [inlined]
│     [18] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1289
│     [19] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1324
│     [20] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1158
│     [21] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::Nothing, sourcepath::String, depmods::Vector{Any})
│        @ Base ./loading.jl:1481
│     [22] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
│        @ Base ./loading.jl:1568
│     [23] _require(pkg::Base.PkgId, env::String)
│        @ Base ./loading.jl:1932
│     [24] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1806
│     [25] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [26] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [27] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1797
│     [28] macro expansion
│        @ Base ./loading.jl:1784 [inlined]
│     [29] macro expansion
│        @ Base ./lock.jl:267 [inlined]
│     [30] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1747
│     [31] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [32] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [33] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1740
│     [34] eval
│        @ Core ./boot.jl:385 [inlined]
│     [35] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2070
│     [36] include_string(mapexpr::Function, mod::Module, code::AbstractString, filename::AbstractString)
│        @ Base ./loading.jl:2080 [inlined]
│     [37] #44
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:863 [inlined]
│     [38] (::IOCapture.var"#4#7"{Core.TypeofBottom, Literate.var"#44#45"{String, Bool, Module, String}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}})()
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:161
│     [39] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:515
│     [40] with_logger
│        @ ./logging.jl:627 [inlined]
│     [41] capture(f::Literate.var"#44#45"{String, Bool, Module, String}; rethrow::Type, color::Bool, passthrough::Bool, capture_buffer::IOBuffer)
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:158
│     [42] execute_block(sb::Module, block::String; inputfile::String, fake_source::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:859
│     [43] execute_block
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:844 [inlined]
│     [44] execute_markdown!(io::IOBuffer, sb::Module, block::String, outputdir::String; inputfile::String, fake_source::String, flavor::Literate.DocumenterFlavor, image_formats::Vector{Tuple{MIME, String}}, file_prefix::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:607
│     [45] (::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64})()
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:579
│     [46] cd(f::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64}, dir::String)
│        @ Base.Filesystem ./file.jl:112
│     [47] markdown(inputfile::String, outputdir::String; config::Dict{Any, Any}, kwargs::@Kwargs{execute::Bool, name::String, documenter::Bool, preprocess::Base.Fix1{typeof(preprocess), String}})
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:578
│     [48] markdown
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:542 [inlined]
│     [49] (::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64})()
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:33
│     [50] withenv(::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64}, ::Pair{String, String}, ::Vararg{Pair{String, String}})
│        @ Base ./env.jl:256
│     [51] (::Serialization.__deserialized_types__.var"#1#3")(::Tuple{Int64, Tuple{String, String}})
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:28
│     [52] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
│        @ Base ./essentials.jl:887
│     [53] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:884
│     [54] (::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
│     [55] run_work_thunk(thunk::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
└ @ Base loading.jl:1295
┌ Error: Error during loading of extension StaticArrayInterfaceOffsetArraysExt of StaticArrayInterface, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    ArgumentError: Package StaticArrayInterfaceOffsetArraysExt [a8fab69c-de6f-5f74-aba3-bef19c2d033f] is required but does not seem to be installed:
│     - Run `Pkg.instantiate()` to install all recorded dependencies.
│    
│    Stacktrace:
│      [1] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1920
│      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1806
│      [3] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│      [4] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│      [5] _require_prelocked
│        @ Base ./loading.jl:1797 [inlined]
│      [6] _require_prelocked
│        @ Base ./loading.jl:1796 [inlined]
│      [7] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1289
│      [8] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1324
│      [9] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1158
│     [10] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::Nothing, sourcepath::String, depmods::Vector{Any})
│        @ Base ./loading.jl:1481
│     [11] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
│        @ Base ./loading.jl:1568
│     [12] _require(pkg::Base.PkgId, env::String)
│        @ Base ./loading.jl:1932
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1806
│     [14] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1797
│     [17] macro expansion
│        @ Base ./loading.jl:1784 [inlined]
│     [18] macro expansion
│        @ Base ./lock.jl:267 [inlined]
│     [19] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1747
│     [20] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [21] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [22] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1740
│     [23] eval
│        @ Core ./boot.jl:385 [inlined]
│     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2070
│     [25] include_string(mapexpr::Function, mod::Module, code::AbstractString, filename::AbstractString)
│        @ Base ./loading.jl:2080 [inlined]
│     [26] #44
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:863 [inlined]
│     [27] (::IOCapture.var"#4#7"{Core.TypeofBottom, Literate.var"#44#45"{String, Bool, Module, String}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}})()
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:161
│     [28] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:515
│     [29] with_logger
│        @ ./logging.jl:627 [inlined]
│     [30] capture(f::Literate.var"#44#45"{String, Bool, Module, String}; rethrow::Type, color::Bool, passthrough::Bool, capture_buffer::IOBuffer)
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:158
│     [31] execute_block(sb::Module, block::String; inputfile::String, fake_source::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:859
│     [32] execute_block
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:844 [inlined]
│     [33] execute_markdown!(io::IOBuffer, sb::Module, block::String, outputdir::String; inputfile::String, fake_source::String, flavor::Literate.DocumenterFlavor, image_formats::Vector{Tuple{MIME, String}}, file_prefix::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:607
│     [34] (::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64})()
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:579
│     [35] cd(f::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64}, dir::String)
│        @ Base.Filesystem ./file.jl:112
│     [36] markdown(inputfile::String, outputdir::String; config::Dict{Any, Any}, kwargs::@Kwargs{execute::Bool, name::String, documenter::Bool, preprocess::Base.Fix1{typeof(preprocess), String}})
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:578
│     [37] markdown
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:542 [inlined]
│     [38] (::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64})()
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:33
│     [39] withenv(::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64}, ::Pair{String, String}, ::Vararg{Pair{String, String}})
│        @ Base ./env.jl:256
│     [40] (::Serialization.__deserialized_types__.var"#1#3")(::Tuple{Int64, Tuple{String, String}})
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:28
│     [41] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
│        @ Base ./essentials.jl:887
│     [42] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:884
│     [43] (::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
│     [44] run_work_thunk(thunk::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
│     [45] (::Distributed.var"#109#111"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
└ @ Base loading.jl:1295
┌ Error: Error during loading of extension ComponentArraysTrackerExt of ComponentArrays, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    ArgumentError: Package ComponentArraysTrackerExt [a97f7ea9-bdd4-56dc-b9a4-48fcd061f528] is required but does not seem to be installed:
│     - Run `Pkg.instantiate()` to install all recorded dependencies.
│    
│    Stacktrace:
│      [1] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1920
│      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1806
│      [3] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│      [4] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│      [5] _require_prelocked
│        @ Base ./loading.jl:1797 [inlined]
│      [6] _require_prelocked
│        @ Base ./loading.jl:1796 [inlined]
│      [7] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1289
│      [8] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1324
│      [9] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1158
│     [10] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::Nothing, sourcepath::String, depmods::Vector{Any})
│        @ Base ./loading.jl:1481
│     [11] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
│        @ Base ./loading.jl:1568
│     [12] _require(pkg::Base.PkgId, env::String)
│        @ Base ./loading.jl:1932
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1806
│     [14] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1797
│     [17] macro expansion
│        @ Base ./loading.jl:1784 [inlined]
│     [18] macro expansion
│        @ Base ./lock.jl:267 [inlined]
│     [19] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1747
│     [20] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [21] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [22] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1740
│     [23] eval
│        @ Core ./boot.jl:385 [inlined]
│     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2070
│     [25] include_string(mapexpr::Function, mod::Module, code::AbstractString, filename::AbstractString)
│        @ Base ./loading.jl:2080 [inlined]
│     [26] #44
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:863 [inlined]
│     [27] (::IOCapture.var"#4#7"{Core.TypeofBottom, Literate.var"#44#45"{String, Bool, Module, String}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}})()
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:161
│     [28] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:515
│     [29] with_logger
│        @ ./logging.jl:627 [inlined]
│     [30] capture(f::Literate.var"#44#45"{String, Bool, Module, String}; rethrow::Type, color::Bool, passthrough::Bool, capture_buffer::IOBuffer)
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:158
│     [31] execute_block(sb::Module, block::String; inputfile::String, fake_source::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:859
│     [32] execute_block
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:844 [inlined]
│     [33] execute_markdown!(io::IOBuffer, sb::Module, block::String, outputdir::String; inputfile::String, fake_source::String, flavor::Literate.DocumenterFlavor, image_formats::Vector{Tuple{MIME, String}}, file_prefix::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:607
│     [34] (::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64})()
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:579
│     [35] cd(f::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64}, dir::String)
│        @ Base.Filesystem ./file.jl:112
│     [36] markdown(inputfile::String, outputdir::String; config::Dict{Any, Any}, kwargs::@Kwargs{execute::Bool, name::String, documenter::Bool, preprocess::Base.Fix1{typeof(preprocess), String}})
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:578
│     [37] markdown
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:542 [inlined]
│     [38] (::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64})()
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:33
│     [39] withenv(::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64}, ::Pair{String, String}, ::Vararg{Pair{String, String}})
│        @ Base ./env.jl:256
│     [40] (::Serialization.__deserialized_types__.var"#1#3")(::Tuple{Int64, Tuple{String, String}})
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:28
│     [41] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
│        @ Base ./essentials.jl:887
│     [42] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:884
│     [43] (::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
│     [44] run_work_thunk(thunk::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
│     [45] (::Distributed.var"#109#111"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
└ @ Base loading.jl:1295
┌ Error: Error during loading of extension ZygoteTrackerExt of Zygote, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    ArgumentError: Package ZygoteTrackerExt [2203d700-20ca-554d-9228-defc17349bb5] is required but does not seem to be installed:
│     - Run `Pkg.instantiate()` to install all recorded dependencies.
│    
│    Stacktrace:
│      [1] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1920
│      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1806
│      [3] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│      [4] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│      [5] _require_prelocked
│        @ Base ./loading.jl:1797 [inlined]
│      [6] _require_prelocked
│        @ Base ./loading.jl:1796 [inlined]
│      [7] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1289
│      [8] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1324
│      [9] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1158
│     [10] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::Nothing, sourcepath::String, depmods::Vector{Any})
│        @ Base ./loading.jl:1481
│     [11] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
│        @ Base ./loading.jl:1568
│     [12] _require(pkg::Base.PkgId, env::String)
│        @ Base ./loading.jl:1932
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1806
│     [14] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1797
│     [17] macro expansion
│        @ Base ./loading.jl:1784 [inlined]
│     [18] macro expansion
│        @ Base ./lock.jl:267 [inlined]
│     [19] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1747
│     [20] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [21] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [22] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1740
│     [23] eval
│        @ Core ./boot.jl:385 [inlined]
│     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2070
│     [25] include_string(mapexpr::Function, mod::Module, code::AbstractString, filename::AbstractString)
│        @ Base ./loading.jl:2080 [inlined]
│     [26] #44
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:863 [inlined]
│     [27] (::IOCapture.var"#4#7"{Core.TypeofBottom, Literate.var"#44#45"{String, Bool, Module, String}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}})()
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:161
│     [28] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:515
│     [29] with_logger
│        @ ./logging.jl:627 [inlined]
│     [30] capture(f::Literate.var"#44#45"{String, Bool, Module, String}; rethrow::Type, color::Bool, passthrough::Bool, capture_buffer::IOBuffer)
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:158
│     [31] execute_block(sb::Module, block::String; inputfile::String, fake_source::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:859
│     [32] execute_block
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:844 [inlined]
│     [33] execute_markdown!(io::IOBuffer, sb::Module, block::String, outputdir::String; inputfile::String, fake_source::String, flavor::Literate.DocumenterFlavor, image_formats::Vector{Tuple{MIME, String}}, file_prefix::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:607
│     [34] (::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64})()
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:579
│     [35] cd(f::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64}, dir::String)
│        @ Base.Filesystem ./file.jl:112
│     [36] markdown(inputfile::String, outputdir::String; config::Dict{Any, Any}, kwargs::@Kwargs{execute::Bool, name::String, documenter::Bool, preprocess::Base.Fix1{typeof(preprocess), String}})
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:578
│     [37] markdown
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:542 [inlined]
│     [38] (::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64})()
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:33
│     [39] withenv(::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64}, ::Pair{String, String}, ::Vararg{Pair{String, String}})
│        @ Base ./env.jl:256
│     [40] (::Serialization.__deserialized_types__.var"#1#3")(::Tuple{Int64, Tuple{String, String}})
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:28
│     [41] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
│        @ Base ./essentials.jl:887
│     [42] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:884
│     [43] (::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
│     [44] run_work_thunk(thunk::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
│     [45] (::Distributed.var"#109#111"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
└ @ Base loading.jl:1295
ERROR: LoadError: ArgumentError: Package Zygote [e88e6eb3-aa80-5325-afca-941959d7151f] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
  [1] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1920
  [2] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1806
  [3] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
  [4] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1797
  [6] macro expansion
    @ Base ./loading.jl:1784 [inlined]
  [7] macro expansion
    @ Base ./lock.jl:267 [inlined]
  [8] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1747
  [9] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
 [10] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
 [11] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1740
 [12] top-level scope
    @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LogDensityProblemsAD/OQ0BL/ext/LogDensityProblemsADZygoteExt.jl:11
 [13] include
    @ Base ./Base.jl:495 [inlined]
 [14] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2216
 [15] top-level scope
    @ stdin:3
in expression starting at /root/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LogDensityProblemsAD/OQ0BL/ext/LogDensityProblemsADZygoteExt.jl:1
in expression starting at stdin:3
┌ Error: Error during loading of extension LogDensityProblemsADZygoteExt of LogDensityProblemsAD, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Failed to precompile LogDensityProblemsADZygoteExt [a095c3f1-ce7f-5470-bf63-3f8a43b7458e] to "/root/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/compiled/v1.10/LogDensityProblemsADZygoteExt/jl_zf7Lgn".
│    Stacktrace:
│      [1] error(s::String)
│        @ Base ./error.jl:35
│      [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
│        @ Base ./loading.jl:2462
│      [3] compilecache
│        @ Base ./loading.jl:2334 [inlined]
│      [4] (::Base.var"#968#969"{Base.PkgId})()
│        @ Base ./loading.jl:1968
│      [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
│      [6] #mkpidlock#6
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
│      [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
│      [8] #invokelatest#2
│        @ Base ./essentials.jl:889 [inlined]
│      [9] invokelatest
│        @ Base ./essentials.jl:884 [inlined]
│     [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
│        @ Base ./loading.jl:2977
│     [11] maybe_cachefile_lock
│        @ Base ./loading.jl:2974 [inlined]
│     [12] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1964
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1806
│     [14] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [16] _require_prelocked
│        @ Base ./loading.jl:1797 [inlined]
│     [17] _require_prelocked
│        @ Base ./loading.jl:1796 [inlined]
│     [18] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1289
│     [19] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1324
│     [20] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1158
│     [21] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::Nothing, sourcepath::String, depmods::Vector{Any})
│        @ Base ./loading.jl:1481
│     [22] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
│        @ Base ./loading.jl:1568
│     [23] _require(pkg::Base.PkgId, env::String)
│        @ Base ./loading.jl:1932
│     [24] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1806
│     [25] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [26] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [27] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1797
│     [28] macro expansion
│        @ Base ./loading.jl:1784 [inlined]
│     [29] macro expansion
│        @ Base ./lock.jl:267 [inlined]
│     [30] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1747
│     [31] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [32] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [33] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1740
│     [34] eval
│        @ Core ./boot.jl:385 [inlined]
│     [35] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2070
│     [36] include_string(mapexpr::Function, mod::Module, code::AbstractString, filename::AbstractString)
│        @ Base ./loading.jl:2080 [inlined]
│     [37] #44
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:863 [inlined]
│     [38] (::IOCapture.var"#4#7"{Core.TypeofBottom, Literate.var"#44#45"{String, Bool, Module, String}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}})()
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:161
│     [39] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:515
│     [40] with_logger
│        @ ./logging.jl:627 [inlined]
│     [41] capture(f::Literate.var"#44#45"{String, Bool, Module, String}; rethrow::Type, color::Bool, passthrough::Bool, capture_buffer::IOBuffer)
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:158
│     [42] execute_block(sb::Module, block::String; inputfile::String, fake_source::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:859
│     [43] execute_block
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:844 [inlined]
│     [44] execute_markdown!(io::IOBuffer, sb::Module, block::String, outputdir::String; inputfile::String, fake_source::String, flavor::Literate.DocumenterFlavor, image_formats::Vector{Tuple{MIME, String}}, file_prefix::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:607
│     [45] (::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64})()
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:579
│     [46] cd(f::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64}, dir::String)
│        @ Base.Filesystem ./file.jl:112
│     [47] markdown(inputfile::String, outputdir::String; config::Dict{Any, Any}, kwargs::@Kwargs{execute::Bool, name::String, documenter::Bool, preprocess::Base.Fix1{typeof(preprocess), String}})
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:578
│     [48] markdown
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:542 [inlined]
│     [49] (::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64})()
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:33
│     [50] withenv(::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64}, ::Pair{String, String}, ::Vararg{Pair{String, String}})
│        @ Base ./env.jl:256
│     [51] (::Serialization.__deserialized_types__.var"#1#3")(::Tuple{Int64, Tuple{String, String}})
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:28
│     [52] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
│        @ Base ./essentials.jl:887
│     [53] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:884
│     [54] (::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
│     [55] run_work_thunk(thunk::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
└ @ Base loading.jl:1295
ERROR: LoadError: ArgumentError: Package CUDA [052768ef-5323-5732-b1bb-66c8b64840ba] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
  [1] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1920
  [2] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1806
  [3] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
  [4] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1797
  [6] macro expansion
    @ Base ./loading.jl:1784 [inlined]
  [7] macro expansion
    @ Base ./lock.jl:267 [inlined]
  [8] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1747
  [9] #invoke_in_world#3
    @ Base ./essentials.jl:921 [inlined]
 [10] invoke_in_world
    @ Base ./essentials.jl:918 [inlined]
 [11] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1740
 [12] top-level scope
    @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/AdvancedHMC/AlvV4/ext/AdvancedHMCCUDAExt.jl:5
 [13] include
    @ Base ./Base.jl:495 [inlined]
 [14] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2216
 [15] top-level scope
    @ stdin:3
in expression starting at /root/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/AdvancedHMC/AlvV4/ext/AdvancedHMCCUDAExt.jl:1
in expression starting at stdin:3
┌ Error: Error during loading of extension AdvancedHMCCUDAExt of AdvancedHMC, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Failed to precompile AdvancedHMCCUDAExt [325ebabc-4626-5639-82e6-98efd0e385b0] to "/root/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/compiled/v1.10/AdvancedHMCCUDAExt/jl_j4Twem".
│    Stacktrace:
│      [1] error(s::String)
│        @ Base ./error.jl:35
│      [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
│        @ Base ./loading.jl:2462
│      [3] compilecache
│        @ Base ./loading.jl:2334 [inlined]
│      [4] (::Base.var"#968#969"{Base.PkgId})()
│        @ Base ./loading.jl:1968
│      [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
│      [6] #mkpidlock#6
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
│      [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
│        @ FileWatching.Pidfile ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
│      [8] #invokelatest#2
│        @ Base ./essentials.jl:889 [inlined]
│      [9] invokelatest
│        @ Base ./essentials.jl:884 [inlined]
│     [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
│        @ Base ./loading.jl:2977
│     [11] maybe_cachefile_lock
│        @ Base ./loading.jl:2974 [inlined]
│     [12] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1964
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1806
│     [14] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [16] _require_prelocked
│        @ Base ./loading.jl:1797 [inlined]
│     [17] _require_prelocked
│        @ Base ./loading.jl:1796 [inlined]
│     [18] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1289
│     [19] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1324
│     [20] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1158
│     [21] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::Nothing, sourcepath::String, depmods::Vector{Any})
│        @ Base ./loading.jl:1481
│     [22] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
│        @ Base ./loading.jl:1568
│     [23] _require(pkg::Base.PkgId, env::String)
│        @ Base ./loading.jl:1932
│     [24] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1806
│     [25] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [26] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [27] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1797
│     [28] macro expansion
│        @ Base ./loading.jl:1784 [inlined]
│     [29] macro expansion
│        @ Base ./lock.jl:267 [inlined]
│     [30] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1747
│     [31] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [32] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [33] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1740
│     [34] eval
│        @ Core ./boot.jl:385 [inlined]
│     [35] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2070
│     [36] include_string(mapexpr::Function, mod::Module, code::AbstractString, filename::AbstractString)
│        @ Base ./loading.jl:2080 [inlined]
│     [37] #44
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:863 [inlined]
│     [38] (::IOCapture.var"#4#7"{Core.TypeofBottom, Literate.var"#44#45"{String, Bool, Module, String}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}})()
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:161
│     [39] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:515
│     [40] with_logger
│        @ ./logging.jl:627 [inlined]
│     [41] capture(f::Literate.var"#44#45"{String, Bool, Module, String}; rethrow::Type, color::Bool, passthrough::Bool, capture_buffer::IOBuffer)
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:158
│     [42] execute_block(sb::Module, block::String; inputfile::String, fake_source::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:859
│     [43] execute_block
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:844 [inlined]
│     [44] execute_markdown!(io::IOBuffer, sb::Module, block::String, outputdir::String; inputfile::String, fake_source::String, flavor::Literate.DocumenterFlavor, image_formats::Vector{Tuple{MIME, String}}, file_prefix::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:607
│     [45] (::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64})()
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:579
│     [46] cd(f::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64}, dir::String)
│        @ Base.Filesystem ./file.jl:112
│     [47] markdown(inputfile::String, outputdir::String; config::Dict{Any, Any}, kwargs::@Kwargs{execute::Bool, name::String, documenter::Bool, preprocess::Base.Fix1{typeof(preprocess), String}})
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:578
│     [48] markdown
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:542 [inlined]
│     [49] (::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64})()
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:33
│     [50] withenv(::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64}, ::Pair{String, String}, ::Vararg{Pair{String, String}})
│        @ Base ./env.jl:256
│     [51] (::Serialization.__deserialized_types__.var"#1#3")(::Tuple{Int64, Tuple{String, String}})
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:28
│     [52] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
│        @ Base ./essentials.jl:887
│     [53] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:884
│     [54] (::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
│     [55] run_work_thunk(thunk::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
└ @ Base loading.jl:1295
┌ Error: Error during loading of extension ComponentArraysRecursiveArrayToolsExt of ComponentArrays, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    ArgumentError: Package ComponentArraysRecursiveArrayToolsExt [eb027ef3-389c-5561-b3ad-95c7d3310487] is required but does not seem to be installed:
│     - Run `Pkg.instantiate()` to install all recorded dependencies.
│    
│    Stacktrace:
│      [1] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1920
│      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1806
│      [3] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│      [4] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│      [5] _require_prelocked
│        @ Base ./loading.jl:1797 [inlined]
│      [6] _require_prelocked
│        @ Base ./loading.jl:1796 [inlined]
│      [7] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1289
│      [8] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1324
│      [9] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1158
│     [10] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::Nothing, sourcepath::String, depmods::Vector{Any})
│        @ Base ./loading.jl:1481
│     [11] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
│        @ Base ./loading.jl:1568
│     [12] _require(pkg::Base.PkgId, env::String)
│        @ Base ./loading.jl:1932
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1806
│     [14] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1797
│     [17] macro expansion
│        @ Base ./loading.jl:1784 [inlined]
│     [18] macro expansion
│        @ Base ./lock.jl:267 [inlined]
│     [19] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1747
│     [20] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [21] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [22] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1740
│     [23] eval
│        @ Core ./boot.jl:385 [inlined]
│     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2070
│     [25] include_string(mapexpr::Function, mod::Module, code::AbstractString, filename::AbstractString)
│        @ Base ./loading.jl:2080 [inlined]
│     [26] #44
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:863 [inlined]
│     [27] (::IOCapture.var"#4#7"{Core.TypeofBottom, Literate.var"#44#45"{String, Bool, Module, String}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}})()
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:161
│     [28] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:515
│     [29] with_logger
│        @ ./logging.jl:627 [inlined]
│     [30] capture(f::Literate.var"#44#45"{String, Bool, Module, String}; rethrow::Type, color::Bool, passthrough::Bool, capture_buffer::IOBuffer)
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:158
│     [31] execute_block(sb::Module, block::String; inputfile::String, fake_source::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:859
│     [32] execute_block
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:844 [inlined]
│     [33] execute_markdown!(io::IOBuffer, sb::Module, block::String, outputdir::String; inputfile::String, fake_source::String, flavor::Literate.DocumenterFlavor, image_formats::Vector{Tuple{MIME, String}}, file_prefix::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:607
│     [34] (::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64})()
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:579
│     [35] cd(f::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64}, dir::String)
│        @ Base.Filesystem ./file.jl:112
│     [36] markdown(inputfile::String, outputdir::String; config::Dict{Any, Any}, kwargs::@Kwargs{execute::Bool, name::String, documenter::Bool, preprocess::Base.Fix1{typeof(preprocess), String}})
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:578
│     [37] markdown
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:542 [inlined]
│     [38] (::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64})()
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:33
│     [39] withenv(::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64}, ::Pair{String, String}, ::Vararg{Pair{String, String}})
│        @ Base ./env.jl:256
│     [40] (::Serialization.__deserialized_types__.var"#1#3")(::Tuple{Int64, Tuple{String, String}})
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:28
│     [41] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
│        @ Base ./essentials.jl:887
│     [42] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:884
│     [43] (::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
│     [44] run_work_thunk(thunk::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
│     [45] (::Distributed.var"#109#111"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
└ @ Base loading.jl:1295
┌ Error: Error during loading of extension ComponentArraysSciMLBaseExt of ComponentArrays, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    ArgumentError: Package ComponentArraysSciMLBaseExt [4f110ecc-9415-5188-843d-6e96dffda943] is required but does not seem to be installed:
│     - Run `Pkg.instantiate()` to install all recorded dependencies.
│    
│    Stacktrace:
│      [1] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1920
│      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:1806
│      [3] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│      [4] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│      [5] _require_prelocked
│        @ Base ./loading.jl:1797 [inlined]
│      [6] _require_prelocked
│        @ Base ./loading.jl:1796 [inlined]
│      [7] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1289
│      [8] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1324
│      [9] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1158
│     [10] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::Nothing, sourcepath::String, depmods::Vector{Any})
│        @ Base ./loading.jl:1481
│     [11] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
│        @ Base ./loading.jl:1568
│     [12] _require(pkg::Base.PkgId, env::String)
│        @ Base ./loading.jl:1932
│     [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1806
│     [14] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [15] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:1797
│     [17] macro expansion
│        @ Base ./loading.jl:1784 [inlined]
│     [18] macro expansion
│        @ Base ./lock.jl:267 [inlined]
│     [19] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1747
│     [20] #invoke_in_world#3
│        @ Base ./essentials.jl:921 [inlined]
│     [21] invoke_in_world
│        @ Base ./essentials.jl:918 [inlined]
│     [22] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:1740
│     [23] eval
│        @ Core ./boot.jl:385 [inlined]
│     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2070
│     [25] include_string(mapexpr::Function, mod::Module, code::AbstractString, filename::AbstractString)
│        @ Base ./loading.jl:2080 [inlined]
│     [26] #44
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:863 [inlined]
│     [27] (::IOCapture.var"#4#7"{Core.TypeofBottom, Literate.var"#44#45"{String, Bool, Module, String}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}, IOContext{Base.PipeEndpoint}})()
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:161
│     [28] with_logstate(f::Function, logstate::Any)
│        @ Base.CoreLogging ./logging.jl:515
│     [29] with_logger
│        @ ./logging.jl:627 [inlined]
│     [30] capture(f::Literate.var"#44#45"{String, Bool, Module, String}; rethrow::Type, color::Bool, passthrough::Bool, capture_buffer::IOBuffer)
│        @ IOCapture ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/IOCapture/Rzdxd/src/IOCapture.jl:158
│     [31] execute_block(sb::Module, block::String; inputfile::String, fake_source::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:859
│     [32] execute_block
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:844 [inlined]
│     [33] execute_markdown!(io::IOBuffer, sb::Module, block::String, outputdir::String; inputfile::String, fake_source::String, flavor::Literate.DocumenterFlavor, image_formats::Vector{Tuple{MIME, String}}, file_prefix::String, softscope::Bool)
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:607
│     [34] (::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64})()
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:579
│     [35] cd(f::Literate.var"#28#30"{Dict{String, Any}, IOBuffer, Module, Literate.CodeChunk, Int64}, dir::String)
│        @ Base.Filesystem ./file.jl:112
│     [36] markdown(inputfile::String, outputdir::String; config::Dict{Any, Any}, kwargs::@Kwargs{execute::Bool, name::String, documenter::Bool, preprocess::Base.Fix1{typeof(preprocess), String}})
│        @ Literate ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:578
│     [37] markdown
│        @ ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/Literate/kaTtz/src/Literate.jl:542 [inlined]
│     [38] (::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64})()
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:33
│     [39] withenv(::Serialization.__deserialized_types__.var"#2#4"{String, String, Int64}, ::Pair{String, String}, ::Vararg{Pair{String, String}})
│        @ Base ./env.jl:256
│     [40] (::Serialization.__deserialized_types__.var"#1#3")(::Tuple{Int64, Tuple{String, String}})
│        @ Main /var/lib/buildkite-agent/builds/gpuci-2/julialang/lux-dot-jl/docs/tutorials.jl:28
│     [41] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
│        @ Base ./essentials.jl:887
│     [42] invokelatest(::Any, ::Any, ::Vararg{Any})
│        @ Base ./essentials.jl:884
│     [43] (::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
│     [44] run_work_thunk(thunk::Distributed.var"#110#112"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
│     [45] (::Distributed.var"#109#111"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
│        @ Distributed ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.10/julia-1.10-latest-linux-x86_64/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:286
└ @ Base loading.jl:1295
[ Info: [Turing]: progress logging is enabled globally
[ Info: [AdvancedVI]: global PROGRESS is set as true
┌ Warning: `ADBACKEND` and `setbackend` are deprecated. Please specify the chunk size directly in the sampler constructor, e.g., `HMC(0.1, 5; adtype=AutoForwardDiff(; chunksize=0))`.
│  This function has no effects.
│   caller = ip:0x0
└ @ Core :-1

Generating data

Our goal here is to use a Bayesian neural network to classify points in an artificial dataset. The code below generates data points arranged in a box-like pattern and displays a graph of the dataset we’ll be working with.

# Number of points to generate
N = 80
M = round(Int, N / 4)
rng = Random.default_rng()
Random.seed!(rng, 1234)

# Generate artificial data
x1s = rand(rng, Float32, M) * 4.5f0;
x2s = rand(rng, Float32, M) * 4.5f0;
xt1s = Array([[x1s[i] + 0.5f0; x2s[i] + 0.5f0] for i in 1:M])
x1s = rand(rng, Float32, M) * 4.5f0;
x2s = rand(rng, Float32, M) * 4.5f0;
append!(xt1s, Array([[x1s[i] - 5.0f0; x2s[i] - 5.0f0] for i in 1:M]))

x1s = rand(rng, Float32, M) * 4.5f0;
x2s = rand(rng, Float32, M) * 4.5f0;
xt0s = Array([[x1s[i] + 0.5f0; x2s[i] - 5.0f0] for i in 1:M])
x1s = rand(rng, Float32, M) * 4.5f0;
x2s = rand(rng, Float32, M) * 4.5f0;
append!(xt0s, Array([[x1s[i] - 5.0f0; x2s[i] + 0.5f0] for i in 1:M]))

# Store all the data for later
xs = [xt1s; xt0s]
ts = [ones(2 * M); zeros(2 * M)]

# Plot data points

function plot_data()
    x1 = first.(xt1s)
    y1 = last.(xt1s)
    x2 = first.(xt0s)
    y2 = last.(xt0s)

    fig = with_theme(theme_web()) do
        fig = Figure()
        ax = CairoMakie.Axis(fig[1, 1]; xlabel="x", ylabel="y")

        scatter!(ax, x1, y1; markersize=8, color=:red, strokecolor=:black, strokewidth=1)
        scatter!(ax, x2, y2; markersize=8, color=:blue, strokecolor=:black, strokewidth=1)

        return fig
    end

    return fig
end

plot_data()

Building the Neural Network

The next step is to define a feedforward neural network where we express our parameters as distributions, and not single points as with traditional neural networks. For this we will use Dense to define liner layers and compose them via Chain, both are neural network primitives from Lux. The network nn we will create will have two hidden layers with tanh activations and one output layer with sigmoid activation, as shown below.

The nn is an instance that acts as a function and can take data, parameters and current state as inputs and output predictions. We will define distributions on the neural network parameters.

# Construct a neural network using Lux
nn = Chain(Dense(2 => 3, tanh), Dense(3 => 2, tanh), Dense(2 => 1, sigmoid))

# Initialize the model weights and state
ps, st = Lux.setup(rng, nn)

Lux.parameterlength(nn) # number of paraemters in NN
20

The probabilistic model specification below creates a parameters variable, which has IID normal variables. The parameters represents all parameters of our neural net (weights and biases).

# Create a regularization term and a Gaussian prior variance term.
alpha = 0.09
sig = sqrt(1.0 / alpha)
3.3333333333333335

Construct named tuple from a sampled parameter vector. We could also use ComponentArrays here and simply broadcast to avoid doing this. But let’s do it this way to avoid dependencies.

function vector_to_parameters(ps_new::AbstractVector, ps::NamedTuple)
    @assert length(ps_new) == Lux.parameterlength(ps)
    i = 1
    function get_ps(x)
        z = reshape(view(ps_new, i:(i + length(x) - 1)), size(x))
        i += length(x)
        return z
    end
    return fmap(get_ps, ps)
end

# Specify the probabilistic model.
@model function bayes_nn(xs, ts)
    global st

    # Sample the parameters
    nparameters = Lux.parameterlength(nn)
    parameters ~ MvNormal(zeros(nparameters), Diagonal(abs2.(sig .* ones(nparameters))))

    # Forward NN to make predictions
    preds, st = nn(xs, vector_to_parameters(parameters, ps), st)

    # Observe each prediction.
    for i in 1:length(ts)
        ts[i] ~ Bernoulli(preds[i])
    end
end
bayes_nn (generic function with 2 methods)

Inference can now be performed by calling sample. We use the HMC sampler here.

# Perform inference.
N = 5000
ch = sample(bayes_nn(reduce(hcat, xs), ts), HMC(0.05, 4), N)
Chains MCMC chain (5000×30×1 Array{Float64, 3}):

Iterations        = 1:1:5000
Number of chains  = 1
Samples per chain = 5000
Wall duration     = 21.38 seconds
Compute duration  = 21.38 seconds
parameters        = parameters[1], parameters[2], parameters[3], parameters[4], parameters[5], parameters[6], parameters[7], parameters[8], parameters[9], parameters[10], parameters[11], parameters[12], parameters[13], parameters[14], parameters[15], parameters[16], parameters[17], parameters[18], parameters[19], parameters[20]
internals         = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, numerical_error, step_size, nom_step_size

Summary Statistics
      parameters      mean       std      mcse   ess_bulk   ess_tail      rhat   ess_per_sec
          Symbol   Float64   Float64   Float64    Float64    Float64   Float64       Float64

   parameters[1]   -3.2838    2.1353    0.6061    13.4569    20.5157    1.3553        0.6293
   parameters[2]   -2.5453    2.0214    0.5200    14.3062    38.1263    1.2127        0.6690
   parameters[3]    0.2470    0.5170    0.1303    15.5400    69.9446    1.5131        0.7267
   parameters[4]   -1.1252    1.3161    0.3497    12.6263    27.1661    1.6134        0.5904
   parameters[5]   -1.8002    1.6926    0.4788    12.6602    33.8961    1.5684        0.5920
   parameters[6]    3.9672    2.1695    0.5954    14.0390    38.7946    1.3993        0.6565
   parameters[7]   -2.5286    2.1975    0.6476    12.0525    41.6970    1.6312        0.5636
   parameters[8]    1.8982    2.2701    0.6251    13.7761    28.7269    1.0725        0.6442
   parameters[9]   -1.2549    1.3480    0.2755    23.6932    59.3599    1.0377        1.1079
  parameters[10]   -4.2255    1.5446    0.3788    16.9719    57.0242    1.2471        0.7936
  parameters[11]   -4.6723    1.9392    0.5368    14.1247    53.9306    1.1332        0.6605
  parameters[12]    0.4250    2.4120    0.6992    12.3128    21.2133    1.5221        0.5758
  parameters[13]    2.3588    1.2102    0.2731    20.9996    48.8714    1.0347        0.9820
  parameters[14]   -0.4172    3.8391    1.1613    11.6401    28.0804    1.7871        0.5443
  parameters[15]   -2.1761    1.6085    0.4487    13.4087    25.6770    1.1828        0.6270
  parameters[16]    0.9780    2.6501    0.7593    12.8452    25.3699    1.3569        0.6007
  parameters[17]   -2.9359    1.1614    0.2219    29.2929    32.1933    1.0702        1.3698
  parameters[18]    2.7888    2.9729    0.8878    11.7797    20.3006    1.6940        0.5508
  parameters[19]   -5.9399    1.5947    0.4036    16.3451    27.9753    1.0555        0.7643
  parameters[20]   -3.5665    2.1457    0.6262    12.2139    27.8688    1.7557        0.5711

Quantiles
      parameters      2.5%     25.0%     50.0%     75.0%     97.5%
          Symbol   Float64   Float64   Float64   Float64   Float64

   parameters[1]   -8.9072   -4.7464   -2.5269   -1.7119   -0.7331
   parameters[2]   -7.6220   -3.8540   -2.3673   -1.0140    0.5061
   parameters[3]   -1.0205   -0.0489    0.3157    0.6049    1.0431
   parameters[4]   -5.7669   -1.6373   -1.0735   -0.2033    0.6782
   parameters[5]   -5.8369   -2.9621   -1.4632   -0.6225    0.7054
   parameters[6]    0.6462    2.3803    3.5608    5.1908    9.2960
   parameters[7]   -6.3413   -4.3782   -2.5000   -0.6308    1.3728
   parameters[8]   -2.0000    0.1196    1.9379    3.2783    6.4703
   parameters[9]   -3.4560   -2.1832   -1.3622   -0.4546    1.9927
  parameters[10]   -7.1485   -5.3845   -4.2239   -3.0170   -1.5034
  parameters[11]   -7.7894   -6.4005   -4.7019   -2.9667   -1.4084
  parameters[12]   -3.5029   -1.4492    0.2097    2.3226    5.4709
  parameters[13]    0.4124    1.4219    2.2479    3.1261    4.9592
  parameters[14]   -6.7024   -3.5548   -1.2754    3.5667    5.6695
  parameters[15]   -4.8862   -3.1688   -2.3131   -1.2205    1.4772
  parameters[16]   -3.5627   -1.0369    0.8253    2.8720    6.6413
  parameters[17]   -5.6450   -3.6644   -2.8345   -2.1178   -0.9015
  parameters[18]   -4.3030    0.9028    2.5158    5.1998    7.5944
  parameters[19]   -8.4777   -7.1883   -5.9664   -4.9567   -2.5945
  parameters[20]   -7.5701   -5.3241   -3.5499   -1.8065   -0.2083

Now we extract the parameter samples from the sampled chain as θ (this is of size 5000 x 20 where 5000 is the number of iterations and 20 is the number of parameters). We’ll use these primarily to determine how good our model’s classifier is.

# Extract all weight and bias parameters.
θ = MCMCChains.group(ch, :parameters).value;

Prediction Visualization

# A helper to run the nn through data `x` using parameters `θ`
nn_forward(x, θ) = first(nn(x, vector_to_parameters(θ, ps), st))

# Plot the data we have.
fig = plot_data()

# Find the index that provided the highest log posterior in the chain.
_, i = findmax(ch[:lp])

# Extract the max row value from i.
i = i.I[1]

# Plot the posterior distribution with a contour plot
x1_range = collect(range(-6; stop=6, length=25))
x2_range = collect(range(-6; stop=6, length=25))
Z = [nn_forward([x1, x2], θ[i, :])[1] for x1 in x1_range, x2 in x2_range]
contour!(x1_range, x2_range, Z)
fig

The contour plot above shows that the MAP method is not too bad at classifying our data. Now we can visualize our predictions.

\[p(\tilde{x} | X, \alpha) = \int_{\theta} p(\tilde{x} | \theta) p(\theta | X, \alpha) \approx \sum_{\theta \sim p(\theta | X, \alpha)}f_{\theta}(\tilde{x})\]

The nn_predict function takes the average predicted value from a network parameterized by weights drawn from the MCMC chain.

# Return the average predicted value across multiple weights.
nn_predict(x, θ, num) = mean([first(nn_forward(x, view(θ, i, :))) for i in 1:10:num])
nn_predict (generic function with 1 method)

Next, we use the nn_predict function to predict the value at a sample of points where the x1 and x2 coordinates range between -6 and 6. As we can see below, we still have a satisfactory fit to our data, and more importantly, we can also see where the neural network is uncertain about its predictions much easier–-those regions between cluster boundaries.

Plot the average prediction.

fig = plot_data()

n_end = 1500
x1_range = collect(range(-6; stop=6, length=25))
x2_range = collect(range(-6; stop=6, length=25))
Z = [nn_predict([x1, x2], θ, n_end)[1] for x1 in x1_range, x2 in x2_range]
contour!(x1_range, x2_range, Z)
fig

Suppose we are interested in how the predictive power of our Bayesian neural network evolved between samples. In that case, the following graph displays an animation of the contour plot generated from the network weights in samples 1 to 5,000.

fig = plot_data()
Z = [first(nn_forward([x1, x2], θ[1, :])) for x1 in x1_range, x2 in x2_range]
c = contour!(x1_range, x2_range, Z)
record(fig, "results.gif", 1:250:size(θ, 1)) do i
    fig.current_axis[].title = "Iteration: $i"
    Z = [first(nn_forward([x1, x2], θ[i, :])) for x1 in x1_range, x2 in x2_range]
    c[3] = Z
    return fig
end
"results.gif"


This page was generated using Literate.jl.