Caker

Virtualization platform for macOS

View project on GitHub

IMDS (Instance Metadata Service)

Caker provides an AWS-style instance metadata service for Linux VMs, reachable from inside the guest over HTTP, the same way EC2’s 169.254.169.254 works. It exposes the instance ID, hostname, MAC address, instance type, and network info, with both IMDSv1 and IMDSv2 (token-based) support.

Enabling it

IMDS is enabled by default for Linux VMs. The server runs inside the imds network’s process (caked networks start imds), not in caked service listen (unless built with USE_IMDS_COORDINATOR_IN_SERVICE) — including in the sandboxed App Store build: the server starts on an unprivileged port on the IMDS gateway as soon as a Linux VM starts, and stops once the last Linux VM stops. Guests can always reach it there directly, no root or sudo required.

caked service listen                        # IMDS reachable by guests on its unprivileged port
caked service listen --imds-port 9000        # override the unprivileged port (default 28080)
Option Description
--imds-port <port> Unprivileged port IMDS listens on (default 28080). Ignored when caked runs as root.

Network and addressing

Every Linux VM gets a dedicated, host-only “imds” network interface — all VMs on a host share the same “imds” virtual switch. Apple’s vmnet.framework only accepts ordinary private subnets (192.168.0.0/16), so the actual subnet is 192.168.169.0/24 with gateway 192.168.169.1, not the AWS-style 169.254.169.x addressing.

A static route for 169.254.169.254/32 via that gateway is added to the guest’s network config (netplan), matching AWS’s own convention (169.254.169.254 isn’t directly on-link in EC2 either — it’s routed there too). Since that address isn’t itself on-link, caked also installs a pf address-alias redirect: traffic to 169.254.169.254 port 80 is redirected to 192.168.169.1 on whichever port IMDS actually bound (80 as root, or --imds-port otherwise) — set up automatically whenever IMDS starts (needs sudo, best-effort, not available in sandboxed builds). The gateway address itself, 192.168.169.1, is always reachable on its own port regardless and needs no such redirect.

Privilege model

caked service listen normally runs unprivileged, but binding the standard port 80 requires root. IMDS itself never needs root — it always binds some port on the gateway that guests can reach with no privilege involved. Behavior depends on context:

  • caked runs as root (service listen --system): IMDS binds 192.168.169.1:80 directly, nothing else needed.
  • caked runs unprivileged (the common case, including sandboxed builds): IMDS binds 192.168.169.1:<imds-port> — guests can already reach it there, no sudo involved.

Independently of the above, a short sudo call also installs the 169.254.169.254 address alias mentioned in Network and addressing — this runs automatically whenever IMDS starts, on both root and unprivileged caked, and is likewise skipped (with the gateway address still fully reachable) in sandboxed builds. This alias is the only thing that exposes IMDS on the standard port 80 when caked runs unprivileged — see caked networks imds-redirect, an internal subcommand.

Using IMDS from the guest

# IMDSv1 (no token required) — replace 28080 with your --imds-port if you changed it;
# port 80 on 169.254.169.254 only works if the pf address alias is set up (see
# Privilege model), and port 80 on 192.168.169.1 only if caked runs as root
curl http://192.168.169.1:28080/latest/meta-data/instance-id
curl http://169.254.169.254/latest/meta-data/instance-id

# IMDSv2: get a token, then use it
TOKEN=$(curl -X PUT "http://192.168.169.1:28080/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
curl -H "X-aws-ec2-metadata-token: $TOKEN" http://192.168.169.1:28080/latest/meta-data/hostname

GET / (the bare root, no latest/meta-data prefix) returns the list of supported API versions, matching a real EC2 instance (1.0, a run of dates, then latest).

Available endpoints under latest/meta-data: instance-id, hostname, local-hostname, local-ipv4, mac, public-hostname, public-ipv4, ami-id, ami-launch-index, instance-type, placement/availability-zone, placement/region, network/interfaces/macs/<mac>/{local-ipv4s,subnet-ipv4-cidr-block,vpc-id}.

public-ipv4 / public-hostname are only available for VMs with a bridged network attachment (e.g. cakectl config vm --network name=bridged) — the closest analogue to an EC2 public IP this host has, since bridged mode puts the VM directly on the physical LAN. public-ipv4 is resolved primarily from the host’s ARP cache for that interface’s MAC address (prodded with a broadcast ping on a cache miss) — reliable from the guest’s second boot onward (or as soon as a DHCP lease changes), since cloud-init also enables the arp_notify sysctl so the guest announces its own address with a gratuitous ARP whenever the interface comes up or its address changes; the kernel only fires that announce on an interface event, not retroactively when the sysctl is applied, so the guest’s very first DHCP-acquired address on its first boot isn’t covered by it. If the ARP cache still comes up empty — on that first boot, or permanently on the entitled native bridging path, where the host never sees the guest’s traffic at all — caker falls back to asking the in-guest cakeagent directly for its network interfaces. public-hostname then attempts a reverse DNS (PTR) lookup on the resolved address through the system resolver — covering both real PTR records (if the LAN’s router/DNS publishes one) and mDNS/Bonjour .local names, since macOS transparently routes that lookup through mDNSResponder — falling back to <hostname>.local if neither resolves. With no bridged attachment, both endpoints return 404, matching a real EC2 instance with no public IP.

Limitations

  • VM launched via caked vmrun outside StartHandler (e.g. a standalone process): the VM still gets its network interface and netplan route, but lifecycle events are sent by StartHandler to the imds network; without them the VM is only registered the next time the imds network starts.
  • Reaching IMDS at the AWS-style 169.254.169.254 address needs sudo for its pf address-alias redirect, so it’s not available in the sandboxed App Store build. IMDS itself still works there, reachable at the real gateway address on its unprivileged port.