Supercharge Your AI in the Real World

Unity is a tool your AI picks up. RakuAI is a body your AI inhabits — the spatial runtime built for smart glasses, backed by patents that anticipated this moment over a decade ago.

Three vectors converged on me this Saturday morning before I had finished my coffee.

Unity just shipped Native MCP. Smart-glasses hardware is actually on people’s faces now, not in a slide deck. And the LLM labs I have been talking to all want the same thing: a runtime that exposes spatial capabilities through a contract their models can drive, in production, every frame, without a custom integration per vendor.

The engine I started building in 2025, on top of patents I helped file over a decade ago, finally has a name for what it is. It is the runtime your AI inhabits in the real world. Not a tool the model picks up. A body the model lives in.

This post is me writing that down honestly, with the receipts.

Section 1. The patent estate

Five granted United States patents, priority filing July 2012. They are public, they are searchable, they are mine.

• US9210358B2, Entertainment Console. Granted December 2015.
• US9654613B2, Arena Gaming. Granted May 2017.
• US9781244B2, Smart Eyeglasses. Granted October 2017.
• US10027361B2, Point-of-Service. Granted July 2018.
• US10432772B2, Eyeglasses, 21 claims.

Read them together and what they describe is a system. Dual-mode wireless. A smart-glasses-plus-compute-host architecture where the heavy lifting lives off the head. Render offload from the host to the display. Multiplayer world-state synchronization across multiple wearers in the same physical space. World streaming as scene delta rather than encoded video.

The priority date matters. July 2012. That is years before Meta Ray-Bans, before Xreal Air, before Android XR, before the current crop of LLM labs existed in their present form. The architecture in the filings is the architecture the industry is now converging on. We are not chasing the field. We are catching up to ourselves.

I do not lead with patents because patents win arguments. I lead with them because they answer the most common question a serious partner asks in the second meeting, which is “how long have you actually been thinking about this.” The honest answer is two numbers: the patents date to 2012, and the engine has been in active build since 2025. Old IP, new execution.

Section 2. Why glasses need a different runtime

The constraints of smart glasses are not the constraints of consoles, phones, or PCs. If you treat them as a smaller version of any of those, you ship a product that overheats in twenty minutes.

A few of the numbers we design to.

• 18ms motion-to-photon latency target. Predictable latency, not peak framerate. A jittery 90 FPS feels worse than a steady 60.
• Three thermal modes. Performance, Balanced, Low Power. The runtime knows which mode it is in and adjusts pipeline cost per frame. The model knows too, when it asks for resources.
• Phone-as-compute, glasses-as-display. The host is the workhorse. The display is a thin client over a dual-mode link. That is the topology the patents describe.
• Wi-Fi 7 plus optical dual-mode link. Bandwidth for the world delta. Latency budget for the predicted pose.
• 0.5 to 0.8 millimeter spatial precision at 40 centimeters, Kalman-filtered. Sub-millimeter anchoring is what makes a virtual surface feel like a real one. If you have ever calligraphed on a virtual pane, you know the difference between 1mm of jitter and 0.5mm of jitter. It is the difference between writing and scribbling.
• 71-joint full-body tracking. Eye tracking with dwell. Foveated rendering with a 2 to 4x gain. Foveation is not optional in this envelope. It is the difference between a render budget that fits and one that does not.

Now contrast that with the alternatives. Unity is a general-purpose engine designed around 60 FPS PC and console targets. Unreal is VR-first, but the VR it assumes is an RTX 4090 in a tower with a 200-watt thermal budget. Both are great engines. Neither one was designed for a 5-watt thermal envelope on a person’s face.

This is not a slight. It is a category difference.

Section 3. Why AI needs a runtime built for it

I wrote a post in January called AI as Nervous System, Not AI as Factory. The argument was that most engines calling themselves AI-native in 2026 mean they bolted a chat panel onto an existing pipeline. The model produces an artifact. The artifact lands on disk. The runtime loads the artifact. The model never speaks to the runtime again.

That is the factory pattern. It is real, it is useful, it is shipping. It is also not what spatial computing needs.

The nervous-system pattern puts the model in the loop on every frame. Observation in, intent out, world responds, repeat. The model is not a content vendor. It is a participant in the simulation.

The six MCP tools the runtime exposes are the surface where this happens.

• load_world_model(adapter_name, config)
• ingest_frame(adapter_name, frame_data, frame_index, timestamp)
• get_scene_state(include_physics, include_transforms)
• set_render_target(target_type, config)
• start_simulation(tick_rate, max_duration, realtime)
• get_metrics()

Six tools, stdio transport, deny-by-default permissions, full audit log. Any model that speaks Model Context Protocol can drive the runtime through that contract. Claude. ChatGPT. Gemini. Copilot. Whatever ships next.

Multi-vendor by design, because the runtime has no opinion about which model is on the other end. The contract is the boundary. The runtime keeps authority on physics, collision, scoring, multiplayer state. The model contributes intent. Neither side has to know about the other except through six tools.

That is what the nervous-system pattern looks like when you turn it into shipping code.

Section 4. RakuAI versus Unity, said fairly

Unity Native MCP shipped recently and the reception has been positive. I want to be clear about what it is and what it is not, because I think both can be true at once.

Unity MCP is scene manipulation at design time. An agent helps the developer place objects, configure components, build a level. It is a productivity layer for the human-plus-editor loop. That is a real win for a real audience.

RakuAI MCP is world-model orchestration at runtime. The agent is in the loop every frame, contributing to the next state of the world, while the runtime enforces physics and collision and rendering. That is production March 2026, not design time, not editor-assist.

Different categories. Both valid. The pithy version I have started saying out loud:

Unity is a tool your AI picks up. RakuAI is a body your AI inhabits.

If you want an editor that gets smarter, Unity MCP is exciting and you should try it. If you want a runtime that gives your AI a place to live on a pair of glasses, that is a different problem. The patents that make it possible were filed over a decade ago, and we have been building the engine since 2025.

Section 5. Why we are talking about this now

Timing matters in positioning, and I want to be honest about ours.

Smart-glasses hardware partners are picking their runtime in 2026 and 2027. Those decisions get made once and live for a decade. If you are not in those conversations now, you will not be in them at all.

LLM labs are deciding which spatial platforms they integrate with. The early integrations set the defaults. Defaults are sticky.

Unity shipping MCP validated the category. We have been saying “the model should drive the runtime through a typed contract” for years. It is easier to be heard now that a 30-billion-dollar competitor is saying the same thing.

The patent estate gives us IP runway. The patents are granted, the priority date is solid, and the architecture they cover is the architecture the industry is converging on.

And a year of heads-down building since 2025 has produced a runtime that is actually ready. The MCP server is real. The adapters are landing. The thermal envelope is respected. The latency target is hit. The sub-millimeter anchoring works. The eighteen DLLs are green on Linux. The 100 percent test pass rate has held for three months.

This is the moment to plant a flag.

Section 6. What is next

Specific work, named honestly so it gets done.

• Real adapters end-to-end. Runway. Veo. Partner-supplied. The stubs have proved the dispatch path. The next move is one real model behind one real adapter, demoed publicly, end-to-end.
• Production deployment harness. Service template, environment-variable config, health-check endpoint, graceful shutdown, container packaging. Not glamorous. Necessary.
• Multi-provider fallback. When the primary adapter is slow or unavailable, the server routes to a secondary. Shape is straightforward. Tests will be the work.
• Adapter bounty program. Once one adapter ships, publish the contract and invite the ecosystem to write more. Our job stops being “integrate every model” and starts being “review the implementations.”
• Partnership conversations on both sides. Hardware partners on one side, model labs on the other. The runtime is the thing between them. That is the position the patents set up over a decade ago, and the engine has been earning since 2025.

Closing

The runtime your AI inhabits in the real world. The category the patents anticipated over a decade ago, and the engine has been building toward since 2025. Saturday morning, we have a name for it.

If you are at a model lab, a glasses-hardware shop, or a studio that wants to ship spatial experiences in 2027, the contract is ready and the receipts are public. Three links to read next:

• The MCP surface: /mcp
• The smart-glasses runtime: /smart-glasses
• The story for LLM makers: /llm-makers

Saturday. Coffee finished. Back to the engine.