Skip to content

lib: remove deprecated APIs and implicit resource creation#688

Draft
mtjhrc wants to merge 34 commits into
containers:mainfrom
mtjhrc:remove-deprecated-func
Draft

lib: remove deprecated APIs and implicit resource creation#688
mtjhrc wants to merge 34 commits into
containers:mainfrom
mtjhrc:remove-deprecated-func

Conversation

@mtjhrc
Copy link
Copy Markdown
Collaborator

@mtjhrc mtjhrc commented May 20, 2026

This PR is a preparatory cleanup before the full 2.0 API (#634).

Depends on
Depends on: #673 "Replace hardcoded init.krun with generic virtual file overlay"

New functionality introduced (replacing old way of doing this)

  • krun_set_oci_config_json(ctx, json) — set init config from OCI container-spec JSON (user shall no longer write it to disk)
  • krun_inject_init(ctx, fs_tag) — explicitly injects the init binary and json config for the init into the guest

Deprecated functions removed

  • krun_set_log_levelkrun_init_log
  • krun_set_root_diskkrun_add_disk3
  • krun_set_data_diskkrun_add_disk3
  • krun_set_passt_fdkrun_add_net_unixstream
  • krun_set_gvproxy_pathkrun_add_net_unixgram
  • krun_set_net_mac → mac parameter on krun_add_net_*
  • krun_set_mapped_volumes — was already returning -EINVAL
  • krun_set_console_outputkrun_add_virtio_console_default
  • krun_get_default_initkrun_inject_init

Old numbered API variants removed

  • krun_add_disk, krun_add_disk2krun_add_disk3
  • krun_add_virtiofs, krun_add_virtiofs2krun_add_virtiofs3
  • krun_add_vsock_portkrun_add_vsock_port2
  • krun_set_gpu_optionskrun_set_gpu_options2

Implicit resource creation removed

  • krun_disable_implicit_init + implicit init injection → krun_inject_init(ctx, fs_tag)
  • krun_disable_implicit_console + implicit console → krun_add_virtio_console_default(ctx, in, out, err)
  • krun_disable_implicit_vsock + implicit vsock with TSI heuristics → krun_add_vsock(ctx, tsi_features)

Init config delivery change

krun_set_exec/krun_set_env/krun_set_workdir/krun_set_rlimits no longer format config into kernel cmdline env vars (KRUN_INIT=, KRUN_WORKDIR=, -- args). Instead, they use newly introduced InitConfig /.krun_config.json on virtiofs.

The kernel cmdline no longer carries:

  • KRUN_INIT=...
  • KRUN_WORKDIR=...
  • KRUN_RLIMITS=... (now injected as an env var in the JSON Env array — see FIXME in code)
  • env vars
  • -- args epilog

Only KRUN_BLOCK_ROOT_DEVICE (+ fstype/options) remains on the cmdline when a block root is configured.

Init side follow-up: The C init (init/init.c) still supports both the cmdline and JSON paths. Eventually (maybe as part of #670), the legacy cmdline parsing can be dropped from init since libkrun will no longer use it.

mtjhrc added 30 commits May 20, 2026 17:46
Move the init binary build script and include_bytes!() from the
devices crate into a new init-blob crate. The passthrough modules
reference the binary as init_blob::INIT_BINARY instead of using
include_bytes! directly.

Inspired by containers#593.
Suggested-by: Geoffrey Goodman <[email protected]>
Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Replace the private next_inode AtomicU64 inside PassthroughFs with a
shared InodeAllocator that is passed in at construction. This lets
multiple layers (e.g. a future virtual-inode overlay) allocate from
the same counter without implicit coordination via reserved ranges.

The allocator starts at ROOT_ID + 2, reserving inode 2 for the
existing init_inode in PassthroughFs. This reservation is removed
in the next commit when init handling moves to AugmentFs.

PassthroughFs::new() and PassthroughFsRo::new() now take an
Arc<InodeAllocator> parameter. FsWorker::new() creates the allocator
and passes it through.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Introduce AugmentFs<T>, a generic overlay that wraps any FileSystem
implementation and intercepts FUSE operations for virtual inodes —
synthetic read-only files and directories backed by static data.
One-shot files can only be looked up once.

Remove all init.krun special-case code (init_inode, init_handle,
INIT_CSTR) from both the Linux and macOS passthrough implementations.
The init.krun virtual file is now configured via VirtualDirEntry in
the krun API layer and handled generically by the overlay.

FsDeviceConfig carries a Vec<VirtualDirEntry> and FsWorker wraps
AugmentFs<PassthroughFs> / AugmentFs<PassthroughFsRo>.

The InodeAllocator now starts at ROOT_ID + 1 since the init_inode
reservation is no longer needed.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Add API to prevent the default init binary (/init.krun) from being
injected into the root filesystem. Follows the existing
krun_disable_implicit_{console,vsock} pattern.

Must be called before krun_set_root().

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Add C APIs to inject virtual files and directories into a virtiofs
device. Entries are backed entirely by host memory (no host file).
Files support one-shot semantics (disappear after the first lookup).

Paths may contain '/' to nest entries inside existing virtual
directories (e.g. krun_fs_add_overlay_dir for "etc", then
krun_fs_add_overlay_file for "etc/hostname"). Intermediate
directories must already exist; -ENOENT / -ENOTDIR is returned
otherwise.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Add API to retrieve the built-in default init binary. Callers that
use krun_disable_implicit_init() can use this to obtain the init
binary and inject it themselves via krun_fs_add_overlay_file().

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
NullFs implements the FileSystem trait with just an empty root
directory. It can be wrapped with AugmentFs to serve virtual
files without any host directory involvement.

Fs::new() now accepts Option<String> for shared_dir — None selects
NullFs. FsDeviceConfig and FsServer gain the corresponding variants.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
krun_set_root_disk_remount no longer creates a temporary empty host
directory. Instead it configures a NullFs-backed virtiofs device
(shared_dir: None) with init.krun overlaid via AugmentFs.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
The temporary root directory hack is gone (replaced by NullFs), so
the ioctl that cleaned it up and the config flag that gated it are
no longer needed. Remove allow_root_dir_delete from FsDeviceConfig,
Fs::new(), passthrough Config, and all call sites.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
The exit-code ioctl is a krun mechanism, not a filesystem operation.
Move it to the AugmentFs overlay where it is handled before any
delegation to the inner filesystem.

The Linux passthrough retains only EXPORT_FD (which needs access to
passthrough-internal handle and export tables). The macOS passthrough
no longer implements ioctl at all.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Boot a VM with a pure NullFs root — no host directory at all. Every
file in the root (init.krun, guest-agent, .krun_config.json, test
data) is injected as a virtual overlay, and /dev, /proc, /sys are
virtual empty directories used as mount points.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Boot from an ext4 block device via krun_set_root_disk_remount. The
virtiofs root uses NullFs with init.krun and virtual mount-point
directories overlaid. The guest verifies it pivoted to the block
device root successfully.

Uses dlsym for krun_add_disk/krun_set_root_disk_remount so the test
compiles without BLK and skips gracefully at runtime.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Build and test with the block device feature so the root-disk-remount
test runs in CI. Install e2fsprogs (provides mke2fs) which the test
needs to create the ext4 disk image.

Assisted-by: OpenCode:claude-opus-4.6
Signed-off-by: Matej Hrica <[email protected]>
Introduce InitConfig, InitConfigBuilder, GuestFile, and from_oci_spec_json()
to construct the .krun_config.json file consumed by the in-guest init.

InitConfig is the abstraction boundary between the host API and the
guest init. guest_files() returns all files init needs on the guest
root filesystem (the init binary + config JSON) as GuestFile structs,
letting the caller decide how to materialize them (virtiofs overlay,
block device, etc.).

The JSON schema currently matches the OCI-image-config subset that
init.c's config_parse_file() expects (Entrypoint, Cmd, Env, WorkingDir,
mounts), but callers should not rely on this — the serialization format
is an internal detail.

Assisted-by: OpenCode:claude-opus-4.6
Replace the five separate ContextConfig fields (exec_path, args, env,
workdir, rlimits) with a single init_config: InitConfig.  The legacy
krun_set_exec/env/workdir/rlimits functions now populate InitConfig
directly instead of formatting strings for the kernel cmdline.

At krun_start_enter time, guest_files() produces the init binary and
.krun_config.json, which are injected into the root virtiofs device
via guest_file_to_virtual_entry().  The kernel cmdline no longer
carries KRUN_INIT, KRUN_WORKDIR, KRUN_RLIMITS, env vars, or -- args.
Only KRUN_BLOCK_ROOT_DEVICE (when applicable) remains on the cmdline.

Assisted-by: OpenCode:claude-opus-4.6
krun_set_oci_config_json(ctx, json) sets the init configuration from
an OCI container-spec JSON string, replacing any config previously set
via krun_set_exec/krun_set_workdir/krun_set_env.

krun_inject_init(ctx, fs_tag) injects the init binary and config JSON
into the specified virtiofs device. This is the explicit replacement
for the implicit init injection that krun_disable_implicit_init opts
out of.

Together these allow a fully explicit init flow:
  krun_set_oci_config_json(ctx, json);  // or krun_set_exec(...)
  krun_inject_init(ctx, KRUN_FS_ROOT_TAG);

Assisted-by: OpenCode:claude-opus-4.6
…th krun_add_disk

Assisted-by: OpenCode:claude-opus-4.6
Port all tests to the explicit init injection flow:
- common.rs: add krun_disable_implicit_init + krun_inject_init
- test_virtiofs_root_ro: same
- test_root_disk_remount: same
- test_augmentfs: replace manual init.krun/config overlay with
  krun_set_oci_config_json + krun_inject_init

No test relies on implicit init injection anymore.

Assisted-by: OpenCode:claude-opus-4.6
Init injection is now fully explicit via krun_inject_init(). The
default behavior is to not inject init — callers must opt in.

Remove the disable_implicit_init field from ContextConfig, the
krun_disable_implicit_init() function, the implicit injection block
in krun_start_enter, and all test calls to krun_disable_implicit_init.

Assisted-by: OpenCode:claude-opus-4.6
These were replaced by krun_add_disk. Also remove the internal
root_block_cfg/data_block_cfg fields and their setters, and simplify
get_block_cfg() now that the legacy compat path is gone.

Assisted-by: OpenCode:claude-opus-4.6
…_set_net_mac

These were replaced by krun_add_net_unixstream, krun_add_net_unixgram,
and krun_add_net_tap (which take mac as a parameter directly).

Also remove the internal LegacyNetworkConfig enum, legacy_net_cfg and
legacy_mac fields, the compat path in krun_start_enter that converted
them to the new net backend, and the now-unused NET_COMPAT_FEATURES
constant.

Assisted-by: OpenCode:claude-opus-4.6
krun_set_log_level set KRUN_NITRO_DEBUG when level==4 (debug), but
krun_init_log did not. Fix the omission so removing krun_set_log_level
doesn't regress nitro debug logging.

Also fix the condition to level >= 4 so that trace (level 5) also
enables nitro debug.

Assisted-by: OpenCode:claude-opus-4.6
Superseded by krun_init_log which provides control over target fd,
log style, and env override options.

Assisted-by: OpenCode:claude-opus-4.6
- krun_add_disk -> krun_add_disk3
- krun_add_vsock_port -> krun_add_vsock_port2
- krun_set_gpu_options -> krun_set_gpu_options2

Assisted-by: OpenCode:claude-opus-4.6
Remove the older versions of functions that have numbered successors:
- krun_add_disk, krun_add_disk2 (superseded by krun_add_disk3)
- krun_add_virtiofs, krun_add_virtiofs2 (superseded by krun_add_virtiofs3)
- krun_add_vsock_port (superseded by krun_add_vsock_port2)
- krun_set_gpu_options (superseded by krun_set_gpu_options2)

Assisted-by: OpenCode:claude-opus-4.6
This function has been returning -EINVAL unconditionally. Remove it.

Assisted-by: OpenCode:claude-opus-4.6
Superseded by krun_inject_init which handles both the init binary
and config JSON injection in one call.

Assisted-by: OpenCode:claude-opus-4.6
Replace implicit console creation with explicit
krun_add_virtio_console_default calls. Also replace
krun_set_console_output in nitro.c with
krun_add_virtio_console_default.

No test or example relies on implicit console injection anymore.

Assisted-by: OpenCode:claude-opus-4.6
mtjhrc added 4 commits May 20, 2026 18:42
…nd implicit console

Console creation is now fully explicit via krun_add_virtio_console_default
or krun_add_virtio_console_multiport. No console is created unless the
caller requests one.

Remove the disable_implicit_console field from VmResources, the implicit
console and serial device creation paths in builder.rs, the console_output
field and setter on VmResources, and krun_set_console_output (kept only
behind cfg(aws-nitro) where NitroEnclave still needs it).

Assisted-by: OpenCode:claude-opus-4.6
- test_tsi_tcp_guest_connect: add krun_add_vsock(ctx, KRUN_TSI_HIJACK_INET)
- test_tsi_tcp_guest_listen: same
- test_vsock_guest_connect: add krun_add_vsock(ctx, 0)
- chroot_vm.c: replace krun_disable_implicit_vsock + vhost-user
  with explicit krun_add_vsock when not using vhost-user-vsock

No test or example relies on implicit vsock creation anymore.

Assisted-by: OpenCode:claude-opus-4.6
Vsock creation is now fully explicit via krun_add_vsock(). No vsock
device is created unless the caller requests one.

Remove the Implicit variant from VsockConfig, the implicit vsock
creation heuristics in krun_start_enter, and krun_disable_implicit_vsock.

Assisted-by: OpenCode:claude-opus-4.6
All krun_disable_implicit_* functions are gone. The 2.0 API requires
explicit resource creation.

Assisted-by: OpenCode:claude-opus-4.6
@ggoodman
Copy link
Copy Markdown
Contributor

There's an interesting API design challenge here where krun_set_exec and some other methods are coupled to the specific init implementation. In other words, there's a contract with a pluggable init.

Another way to do this might be to explore passing the init-specific config when choosing the init binary. It's subtle but it makes it clear that the two things are coupled.

@mtjhrc
Copy link
Copy Markdown
Collaborator Author

mtjhrc commented May 21, 2026

There's an interesting API design challenge here where krun_set_exec and some other methods are coupled to the specific init implementation. In other words, there's a contract with a pluggable init.

This is not the final design, this is only the first step in untangling this, I'll decouple this more. The main motivation for this was to get rid of the buggy argv forwarding via the kernel command line and unify everything around our json config.

Currently the krun_set_exec and the other functions just forward to the init config builder methods defined in blob-init (this way, the output format produced by the builder and the format expected by the init binary stay in sync). In the future, you'll call the builder methods directly, but exposing them cleanly right now was complicated (you'd need something like a separate handle for the builder; a Rust/C bindings generator I'm working on should help with this).

If you wanted to use a different init, its configuration will be completely transparent to libkrun. I think the contract will simply be that you provide some virtual files (libkrun won't care if they are executables or configs or whatever) and add the appropriate kernel arguments (e.g. like set the init= kernel arg).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants