Improve AI application performance by reducing latency, optimizing embeddings, and lowering cloud inference costs
Performance Fix Foundry Local 1.2 Linux ARM64 Embeddings Offline ASR The Edge Latency Drop: Fixing Latency Spikes by Offloading Embeddings to Foundry Local 1.2 You are paying a full cloud round trip — network, TLS, queue, throttle risk — to turn a twelve-word search query into a vector. That is the most expensive way possible to do one of the cheapest computations in your stack. Foundry Local 1.2 now runs on Linux ARM64, which means embeddings and speech recognition can happen on a Raspberry Pi, a Jetson, or a Graviton instance — offline, unmetered, and in single-digit milliseconds. The failure signature this guide resolves # Application Insights — the embedding call, not the LLM, is your tail latency: name p50 p95 p99 calls/day POST /embeddings (cloud) 89 ms 412 ms 3,847 ms 1,240,000 POST /chat/completions (cloud) 940 ms 1,720 ms 2,910 ms 38,000 ^^^^^^^^ ...