5 Patents | 9,500+ API Endpoints | 18 Native DLLs | The AI-Native Spatial Runtime — Now in Early Access Explore the runtime →
For smart-glasses OEM tech leads

The vision runtime smart-glasses OEMs license rather than build.

RakuAI is an LLM-vendor-neutral spatial runtime OEMs license rather than build. Phase 1 beachhead: Meta Ray-Ban Display PWA — an installable web app from the glasses’ browser, an MCP surface Claude, ChatGPT, Gemini, or Copilot can drive, and a thermal/latency budget tuned for head-worn displays. Phase 2 roadmap: Android XR + native SDK (Q3 2026–2027). We don’t compete with your hardware — we make the AI experiences on it ship sooner.

Written for engineering and partnerships leads at Meta, Samsung, Google, Xreal, Rokid, and others shipping glasses in 2026–2027. Honest about what’s shipped, what’s reference-only, and what’s hardware-gated. No signed OEM partner is claimed.

Member of NVIDIA Inception AWS Activate Microsoft for Startups Read more ↓
Start an OEM partnership → See the OEM phase roadmap ↓
MCP-driven spatial overlay on smart glasses A head wearing smart glasses with a spatial heads-up display overlay. The glasses surface an MCP-driven capability HUD that pins answers to the wearer's real space: an anchor marker labelled "keys", a scene-query result, and an MCP tool-call badge, illustrating an LLM driving the RakuAI runtime from the glasses. Capability HUD MCP-driven · world-locked scene_query("where are my keys") Anchor: kitchen island left of the fruit bowl · 0.6 mm spoken back in the glasses

What ships per platform, today.

RakuAI capabilities mapped to the three platforms that matter in the 2026 window. ✅ shipping. 🟡 partial / flagged. ⏳ on platform roadmap. ❌ no current path. Hardware-gated = depends on the OEM exposing a camera/frame API we don’t control. No signed OEM partnerships are claimed. Footnotes below.

Capability Meta Ray-Ban Display
Shipping		 Android XR (Samsung & partners)
Q3 2026 launch window		 Apple Vision Pro
TBD
PWA-installable web-app entry from the deviceRakuAI distributes as an installable PWA — no app-store gating. ✅Launchable web-app path on Meta AI glasses1 ✅Chrome on Android XR supports PWAs2 ❌visionOS routes to native; we ship a WebXR fallback3
MCP server access (Claude / ChatGPT / Gemini drive the runtime)Hosted MCP over HTTPS + stdio for desktop AI assistants. ✅via the companion phone / cloud MCP ✅Gemini-on-device or cloud MCP both work 🟡cloud MCP works; on-device path is roadmap
Capture-from-glasses flow (scan a space, hand it to the LLM)Raw frames + IMU → cloud reconstruction → explorable splat in minutes. 🟡capture via companion phone today4 ⏳direct from glasses at platform launch ⏳awaits camera-access surface
Real-time scene query (ask “what's in the kitchen?”, get a structured answer)Spatial query layer over the reconstructed scene; ships today. ✅post-capture; live scene-query roadmap ✅same path as Meta today ✅over WebXR + cloud MCP
Multi-LLM driver (Claude, ChatGPT, Gemini, Copilot)Vendor-neutral — the LLM choice is the user's, not the platform's. ✅all four drive RakuAI MCP today ✅all four; Gemini gets first-class on-device ✅cloud-only at launch
  1. Reading from Meta's public Ray-Ban Display launch material: the device supports installable web apps from the glasses' web surface. We have not signed a partnership with Meta and do not claim one — we are compatible with the public web-app entry point.
  2. Android XR follows the standard Chrome PWA install path. We have not signed a partnership with Samsung or Google.
  3. visionOS does not currently expose a glasses-equivalent installable-PWA path. Our visionOS plan is WebXR + a separate native shell as a follow-up.
  4. “Capture from glasses today” means: the user wears Ray-Ban Display, the companion phone runs our PWA, and the phone is the camera-source-of-truth. Direct from-glasses frame capture lands when the OEM exposes the frame API.
For silicon and reference-design partners

The runtime slot in the glasses-class stack is open. We fit it.

The turnkey programs shipping AI eyewear in 2026 hand a brand the chip, the module, and the supply chain, and they market the software stack as AI-agnostic. What no program names today is the spatial runtime that turns the cameras into a world model that the customer's chosen assistant can act on. That is the layer we build. We are silicon-neutral by design, we ride the reference designs rather than compete with them, and the permission and audit model is already in production, so a brand can answer the safety question on day one.

If you are building glasses-class silicon, modules, or reference designs and want a runtime ISV in the stack conversation, talk to us. RakuAI is an NVIDIA Inception member and ships on commodity phone hardware today, which is the honest floor for what your silicon will do better.

Three phases. One licensing model.

The go-to-market is staged by platform readiness — not by RakuAI’s timeline. We meet your device on the public surfaces it already exposes, then deepen the integration once you’re ready.

Phase 1 — Beachhead

Meta Ray-Ban Display PWA · target Q3 2026

  • Entry: Installable PWA from the glasses’ embedded browser — no app-store gating, no SDK install on your device
  • MCP surface: 17 native tools + 6 universal passthrough tools, live for Claude, ChatGPT, Gemini, Copilot
  • Capture: companion-phone-as-camera today; direct from-glasses capture when the OEM’s frame API lands (hardware-gated)
  • Architecture: phone-as-compute (phone runs AI/physics, glasses display + track)
  • Honest label: PWA path is reference-ready; no co-marketing or pre-install with Meta is agreed

Phase 2 — Platform

Android XR + native SDK · 2027

  • Entry: same PWA path generalises across Android XR’s standardised Chrome surface
  • Native SDK: raku-runtime as native DLLs with an OpenXR-compatible session layer — threaded into the OEM’s compositor, hand-tracking pipeline, or display driver
  • SDK docs target: partner-grade docs Q4 2026; 8–12 dev-week integration path
  • Partners: Samsung Galaxy XR + Android XR licensees; Xreal, Rokid, Vision Pro browser on the same staged offer
  • Honest label: Android XR launch window subject to platform schedule; no signed OEM

Phase 3 — Fan-out

Xreal / Rokid / Galaxy XR / Vision Pro · 2027+

  • Long-tail device support via the browser path
  • Offline / intermittent-link resilience for lower-spec glasses
  • Repeatable self-qualify → PWA pilot → SDK pipeline
  • Co-marketed launch app for one Tier 3 strategic partner
  • NVIDIA Inception warm intros into XR OEM ecosystem BD

NVIDIA Inception network: RakuAI was accepted into NVIDIA Inception (May 2026). That network provides warm intros into the XR hardware ecosystem — many glasses OEMs and their silicon/optics suppliers are Inception members or NVIDIA partners.

OEM integration, step by step.

Swipe through the path from first conversation to a co-shipped experience. No SDK lock-in — we meet your device on the public web surfaces it already exposes.

MCP from the glasses, in one diagram.

An LLM on the glasses talks to RakuAI's MCP surface through the PWA — either via a stdio bridge to the companion device or over HTTPS to the hosted MCP. The runtime answers in spatial primitives: anchors, poses, scene queries, splat handles.

MCP-from-glasses architecture diagram An LLM on the glasses sends an MCP request through the installable PWA to the RakuAI runtime; the runtime exposes 17 native MCP tools (5 read-only + 12 mutation) on the production server, of which ~12 are glasses-relevant (6 core capture/scene plus 6 xr_ adapter tools), performs spatial reasoning over the captured scene, and returns a structured answer the assistant speaks back in the glasses. Smart glasses LLM on glasses / on-device assistant "Where did I leave my keys?" … voice reply … "On the kitchen island, left of the fruit bowl." Installable PWA rakuai.com/capture-app/ Camera + IMU + audio WebXR / WebGPU stdio MCP bridge JWT → hosted MCP stdio | https request / response streaming events connect token flow RakuAI MCP 6 core + 6 xr_ adapter tools(of 17 native MCP tools) · capture_create · capture_status · scene_query · scene_describe · capture_list · capture_open · xr_get_session · xr_set_head_pose · xr_simulate_input · xr_create_anchor · xr_list_anchors · xr_remove_anchor raku-runtime C++ engine Spatial reasoning Splat / mesh store Anchors + poses Permission & scope ~9,500 API endpoints 18 native DLLs structured answer { object_id, pose,   confidence } MCP request flow — works the same whether the LLM lives on-device, in the cloud, or on a desktop driving the glasses.

Both halves are real and shipping. The production MCP server exposes 17 native tools (5 read-only + 12 mutation) — see spatial-engine.html for the full inventory. The ~12 glasses-relevant ones are split as: 6 core capture/scene tools (documented on the MCP page), and 6 xr_ adapter tools (session, head-pose, input, and anchor primitives for glasses-driven flows). The hosted MCP runs on Azure Container Apps in East US.

Roadmap, honestly.

Three columns, three timelines. We don't promise what we can't ship, and we flag what depends on platform behaviour we don't control.

Shipping now

Production · May 2026

  • Installable PWA at rakuai.com/capture-app/
  • Capture → cloud reconstruction (GLOMAP + Brush, .spz output)
  • 17 native MCP tools (5 read-only + 12 mutation) live for Claude, ChatGPT, Gemini, Copilot — see spatial-engine.html for the full inventory
  • 6 xr_ adapter tools (session, head-pose, input, anchors)
  • Spatial query layer over reconstructed scenes
  • Multi-tenant accounts, JWT auth, hosted MCP on Azure

Q3 2026 launch window

Aligned to Android XR launch

  • Capture-from-glasses (frame source = Ray-Ban Display / Android XR camera)
  • WebGPU client-side splat training (no-cloud-round-trip path)
  • Live scene-query during capture (not just post-capture)
  • OEM SDK preview behind NDA
  • GPU reconstruction on NVIDIA A10 / G5 / H100 (credits landing)

2027

Subject to platform surfaces

  • visionOS native shell (WebXR fallback until then)
  • On-device SLM for glasses-class compute envelopes
  • Patent-licensing programme for OEMs who want it
  • Co-marketed launch app for one or more OEM partners
  • Real-time sim-to-real for robotics + agentic glasses workflows

What we won't promise: that any specific OEM will pre-install our PWA, that any specific LLM vendor will route through us by default, or that camera APIs we don't control will exist on the timeline we want. We design to the public surfaces and let the platforms decide.

The IP estate behind the runtime.

Five granted US patents, priority July 2012, estate through July 2032. The architecture this page describes is the architecture those patents predicted.

What the claims cover

  • Dual-mode wireless communication (low-latency + high-bandwidth)
  • Smart glasses + phone/compute host architecture
  • Render offload from compute host to glasses
  • Multiplayer world state synchronization
  • World streaming (scene delta delivery, not video)

These patents predate Meta Ray-Bans (2021), Xreal Air (2022), Meta Ray-Ban Display (2024–25), and Android XR (2026). For OEMs who want a clear path forward on architecture-level claims, we offer licensing alongside partnership.

Patent estate timeline: priority July 2012, grants from 2015 to 2018, with the industry shipping AR glasses from 2021 onward, and the estate running through July 2032. 2012 priority 2015–18 5 patents granted 2021–26 industry ships AR glasses 2032 estate runs through

Engineering proof points.

Measured numbers from a runtime engineered against the glasses constraint set since day one. These are not platform claims — they're what RakuAI itself targets and observes.

18 ms
motion-to-photon latency
0.5–0.8 mm
position accuracy @ 40 cm
0.3 mm
pose jitter (Kalman-filtered)
90 Hz
glasses render rate
60 Hz
phone physics rate
~20 Hz / 400–1400 B
scene delta sync (not video)
3 thermal modes
Performance / Balanced / Low Power
71-joint skeleton
full-body tracking
2–4×
foveated rendering performance gain

Phone-as-compute architecture.

The split that the 2012 patents predicted — now the only architecture that ships under the glasses thermal envelope.

Phone-as-compute architecture: the phone runs physics, AI, game state and asset streaming, linked to the glasses over Wi-Fi 7 scene-delta with optical fallback, while the glasses handle render, pose prediction and visual odometry. Phone — compute Physics — 60 Hz fixed AI behavior & game state Asset streaming World model orchestration Glasses — display Render — 90 Hz Pose prediction Foveated rendering IMU + visual odometry scene delta ~20 Hz Wi-Fi 7 · optical fallback sensor uplink no video streaming

Phone runs the world

Physics (60 Hz fixed), AI behavior, game state, asset streaming.

Glasses display the world

Local pose prediction, foveated rendering, IMU + visual odometry.

Link does the rest

Scene delta over Wi-Fi 7 (with optical fallback when available), sensor uplink, no video streaming required.

Device support today.

OpenXR-conformant where possible. Vendor extensions where required. PWA path where available.

Meta Quest 3
Meta Ray-Ban Display
Xreal Air / Air 2 / One
Samsung Android XR
MiRZA AR2
Rokid
Apple Vision Pro (WebXR)
Future OpenXR-conformant devices

Three concrete OEM engagement paths.

Pick the lane that matches your team’s risk appetite. All three start with our contact form. We’ll keep the first reply short and technical.

1. Capability inquiry (Tier 1 — web-app pilot)

You're scoping runtime partners and want a written response on how RakuAI maps to your specific device, OS, and 2026 launch milestones. Send us your platform constraints — we'll reply with what's real today, what's on a hardware-gated path, and what the 1–2 dev-week Tier 1 web-app pilot would look like for your device.

Contact us →

2. Integration spec doc (Tier 2 — native SDK path)

Engineering-level brief covering: PWA entry surface, MCP transport (stdio + HTTPS), capture-flow API, permission model, and the OEM-side hooks we’d recommend for a Tier 2 native-SDK integration. Public doc is in draft — ask and we’ll send the current cut directly along with the Phase 2 SDK integration path (target Q4 2026, 8–12 dev-weeks).

Request the spec →

Public doc is in draft — expect a /docs/oem-integration-spec.html URL in a follow-up.

3. Early-access SDK preview (Tier 3 — strategic partner)

Mutual-NDA access to the xr_ adapter tools, capture-from-glasses preview, WebGPU splat pipeline, and the Phase 2 native-SDK roadmap docs. Tier 3 is reserved for a marquee launch partner committing to a flagship device. Limited seats — NVIDIA Inception connections given priority consideration.

Start the NDA →

We sign mutual NDAs on standard terms; turnaround is days, not weeks.

Go deeper before you write.

The pages OEM partners read most before a first call — plus the phase roadmap above.

Spatial Engine →
The C++ runtime — 18 native DLLs, ~9,500 API endpoints, the platform the MCP surface sits on.
MCP Server →
The 17-tool native MCP surface that lets Claude, ChatGPT, Gemini, and Copilot drive the runtime. Production since March 2026.
Why RakuAI →
The longer argument for an AI-native spatial engine — one API for web, native, and XR.
Engineering blog →
Build log including the MCP adapter post, the NVIDIA Inception announcement, and the Raku Capture write-ups.
NVIDIA Inception Member Powered by AWS Microsoft for Startups

RakuAI is an NVIDIA Inception member. The same NVIDIA-accelerated 3D Gaussian splatting stack we ship in Raku Capture is what powers the glasses-driven scan flow described above. Read more →

NVIDIA Inception is a free program for AI startups; RakuAI's membership does not imply that NVIDIA endorses RakuAI's products. Learn more at nvidia.com/startups.

RakuAI is built on AWS and an AWS Activate participant. Production reconstruction runs on Azure today; AWS GPU capacity (G5 / G6e) is on our roadmap, honestly gated until quota lands.

“AWS” and “AWS Activate” are trademarks of Amazon.com, Inc. or its affiliates. RakuAI’s participation does not imply that AWS endorses RakuAI’s products.