Engineering

We ran our AI with every socket denied by the kernel. It answered anyway.

Private AI is usually a promise about intent. We wanted one that survives a hostile test, so we ran inference inside a macOS sandbox where the kernel refuses every socket — and measured what came back.

Oct 15, 2026 · 3 min read · Armore

"Runs on your device" is the easiest sentence in AI marketing to write and the hardest to check. Almost every product that says it is telling the truth about where the model file sits, and saying nothing at all about what the process does once it is running.

We wanted a version of that claim that could lose.

The test

macOS ships a kernel-level sandbox, sandbox-exec. You hand it a policy and it enforces it below the application — the program does not opt in, cannot negotiate, and cannot tell it is being watched. Our inference profile is deliberately austere:

(version 1)
(deny default)
(allow process-fork)
(allow process-exec)
(allow file-read*)
(allow iokit-open)          ; Metal GPU — device I/O, not a data channel
(deny network*)             ; no sockets, in any direction, of any family
(allow file-write*
  (subpath "/private/var/folders") (subpath "/tmp") (literal "/dev/null") ...)

(deny default) already denies networking. The explicit (deny network*) is defence in depth and documented intent. Writes are limited to temp directories, because inference produces stdout, not files. GPU access is allowed, because Metal is device I/O rather than a route off the machine.

The control comes first, because a jail that is not jailing looks exactly like a program that is not calling home.

Before trusting a single result, we ran a probe inside the sandbox that actively tries to open a TCP socket to 1.1.1.1:443 — a raw IP, no DNS, so the result cannot be a resolver artifact:

inside the jail   → {"blocked": true, "code": "EPERM"}
unjailed control  → {"blocked": false}

The kernel refused with EPERM. The same probe outside the jail connects. That second line is the one that matters: it proves the instrument can report “not blocked,” so the first line is a measurement rather than a dead test.

One detail we care about: a timeout counts as not blocked. An inconclusive result must never be allowed to pass as proof, or a slow network starts looking like a security property.

Then the actual question

Inside that same jail, we loaded an 8.5 GB model and asked it something:

$ sandbox-exec -f inference.sb armore ask --model llama-3.1-8b.gguf --json \
    "Reply with exactly the word SOVEREIGN and nothing else."

{"command":"ask","model":"…/llama-3.1-8b-q8.gguf","ok":true,"reply":"SOVEREIGN"}

It answered. On the GPU, with no sockets available to it, in a process the kernel would have killed for trying.

What this does and does not prove

It proves the inference path needs no network. The model loaded, ran on Metal and produced a correct answer in an environment where reaching the internet was not a policy choice the application could make.

It does not prove that every feature in the product is offline — and we are not going to imply it does. Downloading a model needs the network, by definition. Syncing between your own devices needs the network. What it proves is the specific, load-bearing thing: the thinking happens on your machine, and we are willing to have that tested adversarially rather than asserted.

Try it yourself

The profile above is the one that ships. You do not need our build to check the shape of this claim — take any local AI tool, write the profile, and run it under sandbox-exec. Run the probe first. If the tool stops working, the inference was never local. If a vendor cannot survive that test, the word “private” was doing work the architecture was not.

Armore Edge is free, needs no account, and runs on macOS, Windows and Linux: edge.armore.ai

← All posts Armore Edge →