Skip to content

Local Worker Loaders

Virtufin.WorkManager.Engines.Local runs the cluster's worker code formats in the calling process, so a WorkerSpec that deploys to the cluster through GatewayWorkManagerClient deploys unchanged to a LocalWorkManager (C# client). It is opt-in and JIT-only: it compiles C# with Roslyn and loads assemblies into collectible AssemblyLoadContexts, so it is for notebooks, tests and tools, not for the NativeAOT Runtime.

await using var local = new LocalWorkManager(new LocalWorkManagerOptions
{
    Loaders = EngineWorkerLoaders.Default(new LocalLoaderOptions { Logger = logger }),
});

Loaders

Loader Mime type What it does
DotNetDllWorkerLoader application/x-dotnet-dll Fetches the nupkg (CodeSource.Url or Content), reads the nuspec, resolves the declared dependencies from NuGet (Virtufin.* at their latest stable from the org feed, as the Runtime's bridge does), loads the worker DLL into its own WorkerLoadContext and finds its IWorker. Deleting the worker unloads the context.
CSharpSourceWorkerLoader text/x-csharp Installs the source's #r "nuget: …" packages, compiles the text with Roslyn against the .NET reference assemblies plus the worker contract's closure (Virtufin.Worker.Host, CloudEvents, Virtufin.Api.Client, logging abstractions), and instantiates the one IWorker. The same WorkerCompiler the CSharpSource.Bridge uses, in-process.
PythonWorkerLoader text/x-python, application/x-python The Runtime's PythonEngine in this process: one python3 subprocess per worker over a Unix socket, a wheel's dependencies installed with uv. Needs python3 (and uv for wheels) on the PATH.
NativeDllWorkerLoader application/x-native-dll The Runtime's NativeDllEngine: the nupkg's runtimes/<rid>/native/ library for this machine's runtime identifier, loaded with NativeLibrary.

EngineWorkerLoaders.Default() returns all four, in that order. The Python and native workers run in the Runtime's own engines (EngineWorkerLoader wraps any IEngine as an IWorker), so they get the same wire protocol as on the cluster; they do not see context.Api, though: a Python worker reaches the platform through its own client (ApiWorker, from the environment) and a native worker through the host's gateway_call (WorkerContexts.EnvironmentApi), so point VIRTUFIN_API_HOST/_GRPC_PORT/_KEY/_TLS at a gateway when such a worker needs one. A C# worker needs none of this.

LocalLoaderOptions:

Option Default Meaning
PackagesDirectory ~/.virtufin/nuget-cache Where resolved NuGet dependencies are cached across runs.
PublicNuGetSource DOTNET_DEFAULT_NUGET_SOURCE_URL or nuget.org The public source; Virtufin.* always comes from the org feed.
Logger none Where the loaders report.
FrameworkReferences ReferenceAssemblies What C# source compiles against: the .NET reference assemblies (as the Runtime), falling back to the running framework's assemblies when the host process has loaded an incompatible Basic.Reference.Assemblies (Verso does); RuntimeAssemblies forces the latter.

The feed credential (worker nupkgs, #r "nuget:" installs, SourceCredential.Virtufin) is VIRTUFIN_REGISTRY_USER/VIRTUFIN_REGISTRY_TOKEN when set, as in the Runtime, else the credential of the feed's package source in the NuGet configuration chain (NuGetConfigCredentials: the current directory's NuGet.Config up to the user's, %VAR% placeholders expanded). A notebook that restores from the feed therefore needs no extra setup.

Code sources locally

CodeSource.Url is fetched by the loader itself (CodeFetcher in Virtufin.WorkManager.Engines, the Runtime's fetcher): https:// with no credential, SourceCredential.Virtufin (Basic from the registry environment variables, only for the registry hosts), Basic and Bearer as given, and file:// for a nupkg you just built. CodeSource.Content is the bytes.

Fidelity

What is identical to the Runtime: the per-event rules (WorkerEventRules: prepare input, finish outputs, route, dead-letter), the code formats and their loading pipeline (nuspec dependencies, #r packages, WorkerLoadContext aliasing of the host-shared assemblies), and the worker context (IWorkerContext with the host's IVirtufinApi, logger, worker id and cancellation).

What differs: the assemblies already loaded in your process are the "host" copies the worker aliases to (a notebook that has loaded a newer Virtufin.Worker.DevKit shares it with the worker); there is no process isolation, no secrets store and no Dapr; and the IVirtufinApi the worker sees is the LocalWorkManager's in-memory one (state with etags, pubsub over the local bus, @every triggers), documented in Worker.DevKit's local host guide.

See also

  • C# client: IWorkManagerClient, WorkerSpec, IWorkerLoader
  • Engines: the Runtime's engines these loaders mirror
  • Worker.Host: the contract and the worker context