callhome what local AI tools send out by default, and the switch for each

the phone-home register

How we measure

This page says how a row on the register earns its grade, in enough detail that you can hold us to it or repeat it. Four grade words run through the whole site, and each one is a ceiling: a row may claim what its grade can show and nothing more. Then the window we watch a program for, what we count as a call out, how deep a look has to be before it can judge a switch, what we mean when we say a switch is proven, what we publish and what we never will, and the instruments the measured rows rest on today.

The four grades

  • Measured. The program ran in a sandbox whose only route out led to our own listener, with every system call recorded and every name lookup logged. Before any run counts, a planted program that calls out on purpose has to show up in the same logs, or the run is void: an instrument that has not seen a call cannot say there was none. A measured row prints its window at both ends, the network condition it ran under, what it exercised and what it did not.
  • Witnessed. The call happened on one of our own machines and left a trace we can show: a journal line, a file the call wrote, a connection in a socket list. It shows that the call happened, on that version, on that day; it does not show its full cadence, and you cannot re-run it. Each witness ships as a redacted extract in the data files, written by a script from the private record, never by hand.
  • Read. The program's source at a named tag or commit, or its maker's own page on a named day, with a permanent link to the exact lines. A read row says "the code at this commit makes a request to X when Y; we did not run it", and where the maker ships binaries, that we did not check that the released build matches the source.
  • Inferred. A dependency's behaviour carried to a program we did not read (a tool that installs ONNX Runtime inherits ONNX Runtime's row), or a package resolution on a named day; the row says why, and names the row it leans on.

A row's words never outrun its grade. "Sends by default" never prints alone: the depth of the look rides in the cell ("yes · measured", "none found · code read at <tag>", "not read · a keyword search found nothing"). A "none found" at read grade prints only when the read enumerated the program's network clients at a named commit; a keyword search prints as a keyword search. A row at "likely" or "inferred" grade about a named app, whose only evidence is a framework's general default, does not print in the table at all: it goes to not yet checked.

The window

The kit this site will ship runs each program through this window: it starts the program and watches it from launch until its own ready line, then ten minutes idle, then one local request of the kind the program exists for (one chat completion, one render, one transcription, one line of speech), then five minutes idle, then a stop through the program's own quit path where it has one, and only then the usual signals; calls made on the way out are counted too, since some libraries flush at exit. Three runs of each arm (one configuration of the program, such as its default or its switch set), in the same window; timings print as the range observed, with the condition they were seen under, never as a bound. A program whose documented or code-read timer is longer than the window gets one extra run with the idle stretched past that timer, where the program can run without a model or a graphics card; a timer longer than any run we made prints as "not observed (period <N>)", never as "none". Long-lived servers get a 24-hour idle soak on top. The extra arms, each named on the row that used it: a relaunch from the same home (next-launch uploads, first-run grace, a queue flushed on start); a seeded queue with the switch set; a volume of requests; one request that fails; a browser loading the program's web interface from inside the sandbox; a state file aged past its period. The measured rows at first release come from the ONNX Runtime probe, whose window was launch to 120 seconds; each row prints its own.

What counts as a call out

At the first depth, a call out is a name lookup, or a connect or send to any address that is not the machine's own loopback. At the second and third depths, the sandbox's own resolver answers each name with an address of its own, one per name, from a block inside the loopback range that nothing else uses; a call out is then every lookup the resolver logs plus every connect or send to that block, on any port and any protocol, whether or not a listener answered. Never counted: genuine loopback outside that block; Unix sockets, which are a program's own parts talking to each other on the same machine (an early version of our parser misread these as network traffic, so the kit now classes them before anything else); and the sandbox's own traffic. A run of retries after a failure prints as "when it cannot connect, it tries N more times in W seconds", never as "it sends N times". Each call is classed by what set it off: background (at start or on a timer, with no action of yours), on request (a side effect of a local request, such as a check made when a model loads), or you asked (an explicit pull, download or build fetch); the register lists all three and treats the first two as the ones this site is about.

The depth decides which verdicts a run can print

Depth 1 sees whether a name is looked up or an address contacted, so it can judge a switch whose effect is "no call at all": ONNX Runtime's, Ollama's, vLLM's, the Hugging Face offline switches, and Gradio's for its version check. Depth 2 adds which host and port each call was for, and the full content of plain-HTTP calls (pip's and uv's user agent, NetworkManager's check). Depth 3, used only where a row says so, completes the encrypted handshake against a throwaway certificate trusted only inside the sandbox, so it can read what a request carries; a program that pins its certificate (accepts only the one it expects) shows up as a failed handshake, and its row says "payload not observable". Only depth 3 can judge a switch that acts on content, or on one path of a host the program also uses for its work (huggingface_hub's telemetry fields, which ride on the same calls that fetch your models; Gradio's analytics, which ride on huggingface.co; telling Ollama's two fetches apart). A verdict below the depth its switch needs prints "not decidable at depth N", which is a true sentence, where "proven" would not be.

What "proved" means

A switch reads "proven on <version>" only when steps 1 and 2 hold in every pre-registered run at the row's own version, build and window, and the cell names the version, the window, the number of runs and every other step not met:

  1. The instrument saw the call. With the switch unset, at least one attempt in the window, in every run. If not, the switch reads "undecided", never "proven": you cannot prove a switch stops what you never saw. Silence with the switch in fewer runs than the plan registered also reads "undecided"; the cell says which.
  2. Silence with the switch. With only the maker's documented switch set, zero attempts and no new files, in every run, at the same version, build and window.
  3. It reached the process. The switch is in the environment of the process that does the work, read after that process's own first log line; or the file form is in the sandbox's home folder and the program's own log says it took.
  4. Side effects recorded, from the maker's own document, and printed beside the switch.
  5. Leftovers tested. Where a default run left a queue on disk, one run seeds that queue and then runs with the switch; the leftover must not be sent.
  6. The window is stated beside the verdict: "silent from launch to N minutes after ready". A timer longer than the window makes the cell read "silent in the window; the N-hour call not observed", unless an extra run covered it.
  7. Re-proven on the next version, and the cell names the version it was proven on. A new release can add a caller; ONNX Runtime 1.29 did.

At first release one switch on the register reads proven: ONNX Runtime's, on 1.30.0, from launch to 120 s, in 3 of 3 pre-registered runs. Two steps were not met: the switch was not read back from the running process (step 3), and a queue left by an earlier run was not seeded and re-run with it (step 5); the cadence past 120 s was not observed. Most switches read "documented; not tested by us" or "found in code; not tested by us", which is the truth of a read row.

What we publish, and what we never will

We publish: the host names a program asked for, as it asked for them (and where we matched a host from an observed address instead, the row says "attributed"); counts per arm and run; seconds from launch and from ready, with the network condition; cadences as observed ranges; switch names, their spellings and their proof states; the files a program leaves behind, by path relative to the home folder; the names and classes of the fields a report carries, never their values; versions and package digests; the operating system and kernel; the window at both ends, in UTC; and what a run did not show.

We never publish: our own network addresses; our machines' names, user names or home paths; a real machine or device identifier, a graphics card's identifier or serial; tokens, including a Hugging Face token used to fetch gated weights; a credential or identifier a maker built into its own program (an error tracker's project key, an analytics measurement ID, an instrumentation key): a host name that embeds one prints with that part replaced by <the maker's project>; raw system-call logs; or the digest of a file before its redaction.

The instruments behind the measured rows

Two instruments stand behind the rows marked measured today, each published with the page that used it, and a measured row says which, with the depth. The first is the sealed-container probe of ONNX Runtime's telemetry on Linux, measured: a container with no network at all, a small name server answering every lookup with a local address, a listener logging the server name each connection asked for and hanging up before the encrypted session began, every network system call traced, six predictions frozen before the first run, and a planted caller to prove the instruments could see; its seven scripts and every run file are in that page's data kit. The second is the no-route sandbox of Ollama and vLLM on one RTX 3090: each server started with only the loopback address, every name lookup sent to a resolver on loopback that refused it, every system call recorded, a window from launch to 120 seconds after ready; it counts attempts and nothing more, and a reading from it reaches this register only after that bench page has printed it, in that page's own words. The one kit this site will ship generalises the two into one harness with the three depths above; until it has passed its own audit and its planted-subject attack (a subject that calls only when a route exists; one that stays quiet when it sees a container; one that calls only on its second launch; one that batches until twenty events; one that flushes only on a graceful exit; one that queues while offline and flushes when a route appears; one that resolves names itself over HTTPS and then calls what it got), no count from it enters a row.

For anyone measuring

Four traps, from our own runs.

  1. An empty queue is not a quiet process. A service whose home folder is read-only to it, or a container since removed, leaves no queue whether or not its reporting ran. Absence of a file is not evidence of silence.
  2. Reading a restarted service's environment too early lies. Straight after a restart, the main process whose environment you read may still be systemd's own launcher, not your program, so a switch reads unset; read it after the program's own first log line (the check on ONNX Runtime's telemetry on Linux, measured says the same).
  3. A test of vLLM's report has to start from a fresh configuration folder. An empty do_not_track file already in .config/vllm, one of the three switches vLLM's own usage page names, stops the report before the instrument can see it, and the run then reads as proof of a switch it never tested. The same applies to any program with a file switch: the sandbox's home folder is made empty, and the preflight records that it was.
  4. Markers can switch a program off before you start. Some libraries go quiet when they see CI, GITHUB_ACTIONS or a container marker: ONNX Runtime treats those two and 11 other such variables as a reason not to report. A run inside a continuous-integration job can look silent either way. The kit strips those names before launch and records which were present.