Month: October 2025

This is a write‑up of what finally made ExecuTorch 1.0.0 LLMs run fast on macOS with the XNNPACK backend, after days of chasing Error::OperatorMissing and build issues.

TL;DR

• Exported with export_llm using XNNPACK + TorchAO 8da4w
• Built and included kernels_torchao.xcframework (TorchAO ops) in the app
• Ensured ExecuTorch’s static operator registrations run at runtime by linking the static libs with -Wl,-force_load,<libpath> (per lib)
• Avoided duplicate linking by not also adding those .xcframeworks in the “Link Binary With Libraries” build phase
• Removed -all_load (caused huge duplicate symbols) and fixed malformed linker flags
• Built via the Xcode workspace (Pods) instead of the standalone project

Symptoms

• Loading an XNNPACK PTE failed with: Error::OperatorMissing (20)
• The PTE contained llama::custom_sdpa.out and llama::update_cache.out, but runtime registry didn’t have them
• Portable model also failed prefill with Error::NotSupported (16) before fixes
• Adding -all_load “fixed” registrations but exploded into thousands of duplicate symbols (Skia, etc.)

Root cause

ExecuTorch’s custom operators (e.g. LLM llama::* ops, TorchAO ops) are registered via static initializers created by EXECUTORCH_LIBRARY(...). When linking static libraries inside .xcframeworks on Apple, those initializers may not be pulled in unless you force the linker to load the objects that contain them.

If you simply add the .xcframeworks and no code references the contained symbols, the linker can drop them — the registrations don’t execute — and you get OperatorMissing at runtime.

The fix (linker + layout)

1. Choose a single, consistent way to link

• Do NOT both:
  • add the ExecuTorch .xcframeworks to the target’s “Link Binary With Libraries” phase, and
  • also -force_load their inner static .a libs in Other Linker Flags.
• Pick one. We picked -force_load for the inner .a files and removed the .xcframework items from “Link Binary With Libraries” to avoid duplicates.

2. Force‑load the specific static libraries that contain registrations

For our XNNPACK + LLM + TorchAO build, we needed to force‑load at least these libs (adjust path roots if different):

-Wl,-force_load,$(SRCROOT)/ExecuTorchFrameworks/executorch.xcframework/macos-arm64/libexecutorch_macos.a
-Wl,-force_load,$(SRCROOT)/ExecuTorchFrameworks/executorch_llm.xcframework/macos-arm64/libexecutorch_llm_macos.a
-Wl,-force_load,$(SRCROOT)/ExecuTorchFrameworks/kernels_llm.xcframework/macos-arm64/libkernels_llm_macos.a
-Wl,-force_load,$(SRCROOT)/ExecuTorchFrameworks/kernels_optimized.xcframework/macos-arm64/libkernels_optimized_macos.a
-Wl,-force_load,$(SRCROOT)/ExecuTorchFrameworks/kernels_quantized.xcframework/macos-arm64/libkernels_quantized_macos.a
-Wl,-force_load,$(SRCROOT)/ExecuTorchFrameworks/kernels_torchao.xcframework/macos-arm64/libkernels_torchao_macos.a
-Wl,-force_load,$(SRCROOT)/ExecuTorchFrameworks/backend_xnnpack.xcframework/macos-arm64/libbackend_xnnpack_macos.a
-Wl,-force_load,$(SRCROOT)/ExecuTorchFrameworks/threadpool.xcframework/macos-arm64/libthreadpool_macos.a

Notes:

• Use the single‐token -Wl,-force_load,<path> form in Xcode’s “Other Linker Flags”. It’s much less error‑prone than manually sprinkling -Xlinker tokens.
• Do not use -all_load — it can pull in everything and cause thousands of duplicate symbols with other deps (e.g., Skia).

3. Add header search paths (if you import ObjC headers from ExecuTorch)

$(SRCROOT)/ExecuTorchFrameworks/executorch.xcframework/macos-arm64/Headers
$(SRCROOT)/ExecuTorchFrameworks/executorch_llm.xcframework/macos-arm64/Headers

4. Build with the workspace

• Use AlienTavernMobile.xcworkspace (or your workspace) so CocoaPods products resolve. Building only the .xcodeproj can mislead with unrelated missing libs (e.g., DoubleConversion) and isn’t how the app is normally linked.

TorchAO: include kernels_torchao

When exporting with TorchAO 8da4w, the graph includes torchao::quantize_affine/dequantize_affine/choose_qparams etc. Those kernels are not in kernels_quantized — you need kernels_torchao too.

How we built the Apple frameworks (including TorchAO):

cd <executorch_checkout>
./scripts/build_apple_frameworks.sh --Release --torchao

Then we copied the XCFramework(s) into the app’s tree (e.g., ExecuTorchFrameworks/) and added the -force_load entry for kernels_torchao as shown above.

Verifying the operators

Two good checks that caught the issue:

1. Inspect the PTE to list operators in the plan (Python, optional):

• Convert PTE flatbuffer to JSON and inspect the operator table
• Cross‑check whether those ops should be provided by kernels you linked (e.g., TorchAO, LLM custom ops)

2. Inspect the runtime registry at startup (C++):

• Log whether the registry contains the ops you need (e.g., llama::custom_sdpa.out, llama::update_cache.out)
• If missing, your registrations didn’t run — a linking/initializer issue

Common pitfalls we hit

• Malformed OTHER_LDFLAGS: a stray -Xlinker token on its own makes Xcode treat it as an input path (…/mobile/macos/-Xlinker not found)
• Using -all_load: caused 6k+ duplicate symbols via transitive static libs (Skia, etc.)
• Adding .xcframeworks to the Frameworks phase AND also -force_loading their libs: double‑linking and duplicate symbols
• Forgetting to include kernels_torchao when exporting with TorchAO 8da4w
• Building the .xcodeproj instead of the workspace: spurious CocoaPods missing‑lib warnings

Minimal troubleshooting checklist

1. Confirm the PTE contains only ops you know are provided by the set of ExecuTorch libs you’re linking
2. At runtime, log registry_has_op_function("llama::custom_sdpa.out") and registry_has_op_function("llama::update_cache.out")
3. If missing, fix linking so static initializers run:
   • Remove .xcframeworks from “Link Binary With Libraries” for the ExecuTorch libs you will -force_load
   • Add -Wl,-force_load,<path> for each required .a inside those .xcframeworks
   • Remove -all_load and fix any stray -Xlinker tokens
4. Ensure kernels_torchao is included when using TorchAO quantization
5. Build the workspace, not the project

Why this works

The EXECUTORCH_LIBRARY(namespace, name, fn) macros emit static initializers that register kernels when their object files are linked and loaded. If the linker never pulls in the object containing the initializer, the registration never runs. -force_load guarantees the linker brings in those objects even if nothing directly references them, and the registrations execute.

Suggested docs improvements

• Provide a “linker recipe” for Apple static linking that includes:
  • The list of .a libs to include for common scenarios (portable, XNNPACK, LLM, TorchAO)
  • A warning against mixing the Frameworks phase with -force_load for the same .xcframework
  • Use of -Wl,-force_load,<path> and avoiding -all_load
• Add an explicit note that TorchAO exports require kernels_torchao at runtime
• Include a small runtime snippet to validate op registration (e.g., registry_has_op_function)
• Include a “PTE inspection” tip for debugging OperatorMissing (flatbuffer → JSON → list ops)

References

• ExecuTorch 1.0.0
• XNNPACK backend
• TorchAO 8da4w export path via export_llm
• GitHub issue: https://github.com/pytorch/executorch/issues/14809

Video of it running: https://www.youtube.com/shorts/KHXUIlop-1w

As the government shutdown continues, 42 million Americans face an unthinkable reality: no food assistance in November. The USDA has confirmed it will not issue SNAP benefits if the impasse persists, and has explicitly stated it won’t use its $6 billion contingency fund to bridge the gap. This isn’t just a policy failure—it’s a moral catastrophe unfolding in real time.

The Human Cost of Political Gridlock

SNAP (the Supplemental Nutrition Assistance Program) isn’t an abstract line item in a federal budget. It’s the difference between a child eating breakfast before school or going hungry. It’s a senior citizen on a fixed income being able to afford both medication and groceries. It’s a working parent stretching a paycheck that doesn’t quite cover rent and food.

When 42 million people suddenly lose access to food assistance, we’re not talking about statistics. We’re talking about neighbors, coworkers, and family members facing an impossible choice between paying utilities and feeding their children. We’re talking about food banks that are already stretched thin being overwhelmed by desperate families. We’re talking about a humanitarian crisis on American soil.

The Stark Contrast in Priorities

What makes this situation particularly galling is the context in which it’s happening. While millions of Americans are being told there’s no money to help them eat, we’re simultaneously witnessing government spending on projects that seem to exist in an entirely different moral universe. The construction of luxury ballrooms and vanity projects continues while families wonder how they’ll make it through Thanksgiving.

This isn’t about partisan politics—it’s about fundamental priorities. A nation that can find funding for palatial construction projects but claims it cannot feed its most vulnerable citizens has lost sight of its basic moral obligations.

The Economic Ripple Effect

Beyond the immediate humanitarian crisis, cutting SNAP benefits would trigger devastating economic consequences. SNAP isn’t just a lifeline for recipients—it’s an economic multiplier that benefits entire communities. Every dollar in SNAP benefits generates approximately $1.50 to $1.80 in economic activity, according to USDA research.

Small grocery stores in low-income neighborhoods depend on SNAP dollars to stay afloat. When those benefits disappear, we’re not just hurting families—we’re threatening the viability of local businesses, eliminating jobs, and accelerating food desert conditions in vulnerable communities. The economic damage will extend far beyond November, potentially triggering a cascade of closures and job losses that will take years to recover from.

Why a Strong Safety Net Matters for National Stability

Some argue that government assistance creates dependency, but history tells a different story. SNAP and similar programs aren’t just charitable impulses—they’re investments in social stability and economic resilience. When people can’t feed their families, desperation follows. Crime increases. Health crises escalate. Emergency rooms fill with people suffering from malnutrition-related conditions, shifting costs to a more expensive part of our healthcare system.

Countries with strong social safety nets consistently show lower rates of social unrest, better health outcomes, and more stable economies. The question isn’t whether we can afford to maintain SNAP—it’s whether we can afford not to.

The Slippery Slope We’re On

If we allow 42 million Americans to go without food assistance in November 2025, we set a dangerous precedent. What happens in December? What happens the next time there’s a budget impasse? Once we’ve normalized the idea that the most vulnerable among us can be used as bargaining chips in political negotiations, we’ve crossed a line that will be difficult to uncross.

This moment will define us. Future generations will look back and ask how we allowed this to happen—or how we came together to prevent it.

What You Can Do

This crisis demands immediate action:

Contact your representatives: Call, email, or visit the offices of your senators and congressional representatives. Make it clear that allowing SNAP benefits to lapse is unacceptable and that you’ll hold them accountable at the ballot box.

Support local food banks: Organizations like Feeding America and local food pantries will be on the front lines if SNAP benefits disappear. They need donations and volunteers now more than ever.

Amplify the voices of those affected: Share stories from your community about what SNAP means to real families. Humanize this crisis by putting faces and names to the statistics.

Demand budget transparency: Ask why funding for certain projects continues while nutrition assistance faces cuts. Push for clarity on spending priorities.

Vote: Remember this moment when election day comes. Support candidates who treat food security as the non-negotiable issue it should be.

The Bottom Line

We are one of the wealthiest nations in human history. We have the resources to ensure no one goes hungry. Whether we have the political will and moral courage to do so is the question before us right now.

Allowing 42 million Americans to lose food assistance isn’t an inevitable consequence of budgetary constraints—it’s a choice. And it’s a choice that speaks volumes about who we are as a nation and what we value.

The time to act is now. Before November arrives. Before families face empty cupboards. Before we normalize a level of cruelty that should be unthinkable in a nation as prosperous as ours.

Our silence is complicity. Our action is imperative. The question is simple: What side of history will we be on?