5 Patents | 9,500+ API Endpoints | 18 Native DLLs | The AI-Native Spatial Runtime — Now in Early Access
For robotics CTOs & perception leads

The spatial memory your robots query in plain English.

Every mobile robot — humanoid, AMR, quadruped, drone — needs two things nobody on its core team wants to build: persistent spatial memory that survives reboots, shift changes, and software updates, and an LLM that can reason about it in plain English. RakuAI delivers both as a service: a per-facility Gaussian splat captured once, hosted by us, queryable by any LLM you choose via 17 native MCP tools. We don't build robots, SLAM, or motion planning — we sit alongside them.

This page is written for engineers and partnerships leads at warehouse-automation, humanoid, delivery, and inspection robotics companies. It tries to be honest about where we sit in the stack, what we do and don't build, and where foundation-model robotics firms are customers, not competitors.

Member of NVIDIA Inception AWS Activate Microsoft for Startups Read more ↓

Pilot vs. production — honest status (as of May 2026)

Capability Status What this means for a pilot
MCP server (17 native tools + passthrough) Production Running on Azure Container Apps (East US). Available to integrate now.
5 read-only scene-query tools Production The Phase-1 robotics focus. Query relative position (distance/clearance between named objects), path clearance, occupancy, and a scene summary. Scene-delta ("what changed since T") is Phase-2 roadmap. No real-robot dependency for integration testing.
Splat reconstruction pipeline Pilot GPU training (Azure A10 + AWS G5) pending credit approvals; stub API is live. Pilot captures will use hosted training once credits land.
Real-robot pilots (on-site capture) Planning No real-robot pilots have completed yet. Pilot slots are open. First pilots will co-produce a published case study.
NVIDIA Inception membership Active Accepted May 2026. Inception benefits (Brev, Jetson Orin discount, DLI 50% off) apply to pilots now.
NVIDIA Innovation Lab office hours Applying Application submitted; not yet granted. We reference this benefit as "once approved" only.
AWS Activate ($10K credits) Under review 5–7 business-day review. GPU training backend wired; credits will flip it live when approved.

Where we sit in your stack.

ABOVE perception (we consume images, depth, and poses), BELOW high-level task planning (we serve queries to your planner's LLM). We are the scene-memory + scene-QA layer — nothing more, nothing less. That focus is the pitch.

What RakuAI provides

  • A per-facility Gaussian splat captured once and hosted forever
  • 17 native MCP tools any LLM can call — "what's blocking aisle 7?", "did the pallet move since Tuesday?"
  • Measurable scene queries: relative position between named objects, path clearance for a body of radius r, free-space extents, occupancy status — returned as structured data your planner acts on directly
  • Multi-vendor LLM: Claude, GPT, Gemini, or NVIDIA NIM-hosted open weights — your choice, per workload
  • Hosted reconstruction + query infrastructure, zero infra burden on you

What we explicitly don't build

  • SLAM — Cartographer, RTAB-Map, ORB-SLAM3, NVIDIA cuVSLAM already exist and are good
  • Motion planning — MoveIt, OMPL, Nav2 are solved problems
  • Robot hardware or actuation — we are software-only
  • Sim engines — Isaac Sim, Gazebo, MuJoCo, AWS RoboMaker; we sit alongside, not against
  • Action-policy foundation models — Skild, Physical Intelligence, Covariant own that layer

The wedge in one sentence: a per-facility splat that lives forever + an MCP API any LLM can query — sold per-capture and per-query, hosted, no infra burden on the customer. The moat isn't the splat — it's that the LLM is not vendor-locked, which matters to robot CTOs who already have an LLM relationship and don't want to be told which one to use.

What a robot actually asks — and gets back.

The read-only scene-query API is the Phase-1 robotics focus. It answers object-relationship and path-clearance questions over the stored per-facility scene — relative position between two named objects, free space along a planned segment, who's near a waypoint — and returns structured JSON your planner acts on directly, not prose. Every response below is the real shape these endpoints return today.

The example scene is one captured world, warehouse-aisle-7 — 42 objects: pallets, racks, fixtures (a dock and two charging bays), and two people. Distances are in the engine's world_units, calibrated to the facility's metric scale at capture time. An unknown world returns 404 (existence is never disclosed); an empty world returns honest zeros, never a fabricated obstacle.

"Before I issue per-query calls, what's in this scene and what can I ask?"
{ "object_count": 42, "type_count": 5, "types": [{"type":"pallet","count":21},{"type":"rack","count":12}, {"type":"fixture","count":6},{"type":"person","count":2}, {"type":"untyped","count":1}], "bounds": { "min": [-12.0,0.0,-3.5], "max": [18.4,4.2,9.1] }, "extent": [30.4,4.2,12.6], "unit": "world_units", "queryable": { "distance": true, "clearance_point": true, "clearance_between": true, "occupancy_box": true, "occupancy_radius": true } }
GET /scene-query/{world}/scene/summary — read-only
"What's the relative position of pallet-007 and the charging bay it may be blocking?"
{ "resolved": true, "entity_a": "pallet-007", "entity_b": "charging-bay-2", "center_distance": 2.14, "aabb_gap": 1.03, "overlapping": false, "has_clearance": true, "missing_entities": [], "unit": "world_units" }
GET /scene-query/{world}/distance — read-only
"Can a 0.45-radius AMR drive straight from dock-A3 to charging-bay-2 right now?"
{ "resolved": true, "clear": false, "path_length": 14.5, "body_radius": 0.45, "min_clearance": 0.31, "limiting_entity": { "entity_id": "pallet-007", "name": "pallet-007", "gap": 0.31 }, "unit": "world_units" }
POST /scene-query/{world}/clearance/path — read-only
"Is anyone within 2.0 of the robot's next waypoint?"
{ "resolved": true, "query_kind": "radius", "occupant_count": 1, "occupants": [{ "entity_id": "person-003", "name": "person-003", "gap": 1.12 }], "unit": "world_units" }
POST /scene-query/{world}/occupancy/radius — read-only
"Is there room for a 0.9-wide body to pass between rack-A3 and pallet-014?"
{ "resolved": true, "entity_a": "rack-A3", "entity_b": "pallet-014", "aabb_gap": 0.62, "required_clearance": 0.9, "has_clearance": false, "unit": "world_units" }
POST /scene-query/{world}/clearance/between — read-only
"How much free space is around the drop point [3.0, 0.2, 1.4]?"
{ "resolved": true, "nearest_gap": 0.78, "has_clearance": true, "nearest": { "entity_id": "pallet-014", "name": "pallet-014", "gap": 0.78 }, "considered_entities": 42, "unit": "world_units" }
POST /scene-query/{world}/clearance/point — read-only

These endpoints layer over the 5 read-only scene-memory MCP tools the planner's LLM calls; the runtime returns structured JSON and your LLM and planner interpret it however they choose. Every call is metered — per-query counts persist to durable usage records, with per-pilot rollups and SLA-grade p50/p95/p99 latency + error-rate read back from /scene-query/usage/{summary,by-caller,sla}. That measured volume is what the 90-day pilot converts on: the meter, not the price — pricing stays your decision. Scene-delta ("what changed since shift T") is on the Phase-2 roadmap, not yet shipped — we don't show it as live. The MCP surface also exposes 12 audited mutation tools (anchors, sessions, transforms) — covered in the safety section below.

Safety and authority: how the runtime keeps control.

The runtime is the authority over physics, collision volumes, transforms, and the canonical scene state. An LLM assistant proposes actions through the MCP mutation tools; it does not silently rewrite the scene. Every mutation is audited.

What the runtime owns (non-negotiable)

  • Physics simulation and collision geometry — no external caller can bypass the collision system or set a transform that violates it without an explicit override flag
  • The canonical scene-state record — the ground truth for all queries; only the runtime writes to it
  • Anchor lifecycle and pose authority — anchors are created, updated, and removed only through audited MCP calls, never via direct database writes
  • Session permissions — what a given MCP client is allowed to read vs. mutate is set at session creation, not by the caller at runtime

What an LLM assistant proposes

  • Mutation tool calls (anchor create/update/remove, session open/close) are treated as proposals; the runtime validates them against the current physics and permission state before applying
  • If a proposed transform would collide, the runtime rejects it and returns an error — the LLM does not get a silent success
  • The read-only tools (Phase-1 robotics focus) carry zero mutation risk by construction — they cannot change anything, only observe
  • Multi-step action sequences are broken into individual validated steps, not applied as a batch that could partially succeed

What is audited

  • Every MCP tool call is logged with caller identity, timestamp, input payload, and outcome — read-only and mutation alike
  • Scene-state changes carry a delta record: what changed, when, which MCP session triggered it
  • Pilot customers receive access to the audit log for their facility; it is the basis for the ROI report at week 9–12
  • Nothing in the LLM passthrough path can suppress audit logging — logging happens at the runtime layer, not in the LLM adapter
The practical implication for a pilot: start with the 5 read-only scene-query tools. Your robot's planner asks questions; the runtime answers with structured data; nothing in your facility scene changes. When you're ready to test mutation (anchor management, session control), those calls go through the same validated, audited path. The audit log is yours for the duration of the pilot.

A robot queries its facility, in one diagram.

Your robot's planner (or its LLM) talks to RakuAI's MCP surface over HTTPS — or via a stdio bridge from an on-robot Jetson Orin client. The runtime answers in spatial primitives: anchors, poses, scene queries, splat handles, object deltas.

MCP-from-robot architecture diagram A robot's planner sends an MCP request through an on-robot client or cloud relay to the RakuAI runtime. The runtime exposes 17 native MCP tools: 5 read-only scene-query tools (Phase-1 robotics focus) plus 12 audited mutation tools. The runtime holds authority over physics, collision, and scene state. Every call is audited. The structured answer is returned to the planner. Robot + planner Your perception + task planner LLM "Is aisle 7 clear? Distance to pallet?" { clear: false, distance_m: 1.43 } MCP client on-robot (Jetson Orin) or cloud ROS 2 / fleet-manager hook stdio MCP bridge JWT → hosted MCP offline / edge mode stdio | https request / response streaming events session-scoped perms RakuAI MCP 17 native tools + passthrough READ-ONLY (Phase-1 focus) • scene_query • scene_describe • capture_list • capture_status • capture_open MUTATION (audited) • xr_create_anchor • xr_list_anchors • xr_remove_anchor • xr_get_session • capture_create • + 7 more… raku-runtime C++ engine — authority layer Physics + collision authority Per-facility splat store Anchors + pose history Scene-delta audit log Mutation validation structured answer { object_id, pose, distance_m, changed_at, confidence } Read-only queries carry zero mutation risk. Mutation calls are validated against physics + permissions before applying. All calls audited.

Both halves are real and shipping. The production MCP server exposes 17 native tools (5 read-only + 12 mutation) plus a passthrough surface for LLM-specific extensions — see spatial-engine.html for the full inventory and mcp.html for the documented scene tools. The hosted MCP runs on Azure Container Apps in East US. The LLM is yours to pick — that vendor-neutrality is the moat.

Who this is for.

Four segments, one shared pain: robots collect terabytes of spatial data that get dumped into storage and never queried again. Lead with warehouse for revenue stability; humanoid for visibility.

Warehouse automation

Robots already have SLAM. What they lack is a clean way to answer "what changed in zone B since last shift?" without writing a custom analytics pipeline. Per-site splat unifies QA across the fleet.

Example targetsSymbotic, Locus Robotics, Fetch / Zebra, GreyOrange, Geek+

Humanoid robots

Series B–D startups racing to ship a Fortune 500 pilot in 6–9 months. They buy anything that removes a 6-month engineering line item without locking them into one vendor's LLM stack.

Example targetsFigure, 1X Technologies, Apptronik, Agility Robotics, Unitree

Delivery robots

Sidewalk and last-mile fleets operate across changing urban environments. Persistent scene memory of routes, drop points, and obstacles complements existing navigation without re-instrumenting the robot.

Example targetsStarship Technologies, Serve Robotics

Inspection robots

Spot and quadruped fleets generate massive datasets dumped to storage and never opened. Customers pay per-site for auditable, time-stamped, queryable assets: "show me every valve flagged red in Q3."

Example targetsBoston Dynamics Spot ecosystem, ANYbotics, Cognite

The partner-resource stack behind a pilot.

RakuAI's NVIDIA Inception membership (accepted May 2026) and pending startup-credit programs compound here. Each is honestly labelled by status — we don't claim approvals we don't have.

NVIDIA Isaac platform

Sim · Lab · Perceptor · Manipulator · ROS

  • Complementary, not competitive. Isaac is what your robot sees this second; RakuAI is what your robot remembers about this facility.
  • Co-positioning through Inception channels removes the "isn't this what NVIDIA already does?" objection.

NVIDIA Cosmos

World Foundation Models

  • Capture a real facility with RakuAI → generate domain variations with Cosmos → train perception in Isaac Lab → deploy with persistent RakuAI memory.
  • We own steps 1 and 4; NVIDIA owns 2 and 3 — you don't glue them together.

NVIDIA Omniverse

USD digital-twin simulation

  • RakuAI splats export into Omniverse for digital-twin scenarios.
  • Your splat capture IS your digital twin — no separate CAD modeling sprint. USD export is a near-term lane.

NVIDIA NIM

Inference Microservices

  • Your robot's LLM can be NIM-hosted Llama / Mistral / Mixtral or Claude / GPT / Gemini direct.
  • MCP is the abstraction layer — privacy-sensitive on-prem NIM, high-reasoning cloud Claude, you pick per workload.

NVIDIA Brev & Jetson Orin

Inception benefits

  • Brev GPU cloud as a third concurrent training pool when AWS G5 / Nebius H100 are saturated.
  • Jetson Orin Inception hardware discount makes on-robot MCP-client eval hardware cheap to procure during a pilot.

NVIDIA DLI & Innovation Lab

Training & engineering office hours

  • DLI Inception code unlocks 50% off — co-train your team on Isaac + RakuAI integration.
  • Innovation Lab spatial-AI office hours as a co-development resource. Application status: applying — not yet granted.

AWS RoboMaker & GPU

Cloud sim · G5 / G6e · Greengrass

  • RoboMaker Your sim environment IS your scanned facility — close the sim-to-real gap with the same asset.
  • G5 / G6e for backend recon + perception inference (AWS Activate credit under review).
  • Greengrass our MCP client can ship as a component for fleet deployment.

Honest credit status

Where things actually stand

  • NVIDIA Inception accepted May 2026.
  • AWS Activate $10K under 5–7 day review.
  • Nebius H100 ($150K savings) submitted; not needed for v1.
  • Anthropic Claude Startups submitted, not approved — Claude is our highest-quality menu option today, not a co-marketed claim.

The ROI math.

We're not replacing your perception stack. We're replacing the persistent-memory subsystem nobody wants to own and the scene-QA LLM bridge that gets pushed to "next quarter" forever.

Status quo — build it in-house

  • 1–2 perception engineers spend 6–18 months building persistent scene memory.
  • Loaded cost: $250K–$400K per engineer-year.
  • 6-month single-engineer effort: ~$125K–$200K. 18-month two-engineer effort: ~$750K–$1.2M.
  • Then maintain it forever — ~20–30% of original effort per year.
  • Plus coordination with the LLM team to expose memory as a queryable API.

With RakuAI

  • $0 engineer-months.
  • Pay per-capture: target $0.50–$2.00 per splat reconstruction.
  • Pay per-MCP-query: cents per query (LLM passthrough + small markup).
  • Hosting included.
  • Your perception team ships higher-leverage work instead.

Net savings, year 1: $250K–$1M in engineering cost displaced. Honest caveat: if a prospect already has a happy persistent-memory + LLM-bridge stack, we don't have a sale — and we won't pretend otherwise.

The 90-day free pilot.

Free for the full 90 days. The output is a published, co-marketed case study — the deliverable that makes pilots #2, #3, and #4 easier to land.

Capture & connect

Weeks 1–2

Free splat capture (we scan, or coach your team) plus MCP relay live for 5–10 of your facilities. Start with read-only scene queries — zero invoice during this window.

Integrate

Weeks 3–8

Integration support — your engineers hook the MCP client into your planner. We sit in your Slack. NVIDIA Innovation Lab office hours booked weekly once approved.

ROI report & case study

Weeks 9–12

Audit log review: which queries ran, how many, what changed, engineer-hours displaced. Then a joint published case study — co-marketed via Inception channels.

Conversion offer: per-facility license OR per-capture + per-query metering — we let the first 3 pilots tell us which pricing model wins. The pilot stays free regardless.

Apply for a pilot slot

Pilot slots are open. No real-robot pilots have completed yet — you would be in the first cohort. The commitment is a 90-day integration effort and co-publishing the findings. We supply the MCP surface, capture support, and integration help.

Use our contact form. Include your company name and robot type. We respond within one business day.

Where we sit vs. the rest of the field.

The honest version. Foundation-model robotics firms are integration partners and customers, not competitors.

NVIDIA Isaac

Adjacent, not competitive

  • Real-time perception vs. persistent queryable memory. Inception membership is proof of strategic alignment.

Skild · Physical Intelligence · Covariant

Customers, not competitors

  • They build action-policy foundation models — they need a canonical scene representation. RakuAI provides one. Skild is a lead integration candidate.

Matterport · PolyCam · Luma AI

Consumer / real-estate capture

  • Not in robotics. Some have splat exports, but no MCP, no per-facility long-term hosting, no LLM bridge.

Closed-stack autonomy

Waymo · Tesla FSD · Wayve

  • They do everything themselves — not an addressable market. We don't pretend otherwise.

If you want to go deeper before you write.

The pages robotics teams read most before a first call.

NVIDIA Inception Member Powered by AWS Microsoft for Startups

RakuAI is an NVIDIA Inception member (accepted May 2026). The same NVIDIA-accelerated 3D Gaussian splatting stack we ship in Raku Capture is what powers the per-facility scene memory described above — complementary to NVIDIA Isaac, not competitive with it. Read more →

NVIDIA Inception is a free program for AI startups; RakuAI's membership does not imply that NVIDIA endorses RakuAI's products. Learn more at nvidia.com/startups.

RakuAI is built on AWS and an AWS Activate participant. Production reconstruction runs on Azure today; AWS GPU capacity (G5 / G6e) is on our roadmap, honestly gated until quota lands.

“AWS” and “AWS Activate” are trademarks of Amazon.com, Inc. or its affiliates. RakuAI’s participation does not imply that AWS endorses RakuAI’s products.