Creating Intelligent NPCs with Behavior Trees

In This Tutorial

Understanding Behavior Tree Concepts
Building Your First Patrol AI
Adding Detection and Chase Behavior
Using Blackboard for NPC State
NavMesh Setup for NPC Navigation
Adding Voice Commands for NPC Interaction
Complete Example: Guard NPC with Full AI

FUNDAMENTALS

Understanding Behavior Tree Concepts

A behavior tree is a hierarchical structure that controls NPC decision-making. Unlike finite state machines, behavior trees are modular, reusable, and easy to extend.

Core node types:

Selector (Fallback) — Tries children left-to-right. Succeeds on the first child that succeeds. Think "OR" logic.
Sequence — Runs children left-to-right. Fails on the first child that fails. Think "AND" logic.
Condition — Checks a boolean condition (e.g., "is player visible?").
Action — Performs an action (e.g., "move to target", "play animation").
Decorator — Modifies a child node (e.g., "repeat", "invert", "cooldown").

Here is how a simple guard AI looks as a tree:

Root (Selector) ├── Sequence: "Combat" │ ├── Condition: IsPlayerVisible? │ ├── Action: ChasePlayer │ └── Action: AttackPlayer ├── Sequence: "Investigate" │ ├── Condition: HeardNoise? │ ├── Action: MoveToNoiseSource │ └── Action: LookAround └── Action: "Patrol" └── Action: PatrolWaypoints

The tree is evaluated from top to bottom each frame. The guard first checks if it should fight, then investigate, and falls back to patrolling.

FIRST AI

Building Your First Patrol AI

Let's create an NPC that walks between waypoints in a loop. This is the foundation for most guard and enemy AI.

// Initialize AI subsystem raku_ai_init(); // Create an AI agent tied to a 3D model raku_scene_add_model("guard.glb", 0.0f, 0.0f, 0.0f); raku_ai_create_agent("guard", "behavior_tree"); // Define patrol waypoints float waypoints[][3] = { { 0.0f, 0.0f, 0.0f }, { 10.0f, 0.0f, 0.0f }, { 10.0f, 0.0f, -8.0f }, { 0.0f, 0.0f, -8.0f }, }; raku_ai_agent_set_patrol_points("guard", waypoints, 4); // Configure movement raku_ai_agent_set_speed("guard", 2.0f); // walk speed m/s raku_ai_agent_set_turn_speed("guard", 180.0f); // degrees/s raku_ai_agent_set_stop_distance("guard", 0.3f); // meters // Build a simple patrol behavior tree RakuBTNode* patrol_tree = raku_btree_create_action("patrol_waypoints"); raku_ai_agent_set_btree("guard", patrol_tree);

Add a wait at each waypoint:

// Build a patrol sequence: move to waypoint, wait, then next RakuBTNode* move = raku_btree_create_action("patrol_waypoints"); RakuBTNode* wait = raku_btree_create_action("wait"); raku_btree_action_set_param(wait, "duration", "2.0"); RakuBTNode* patrol_seq = raku_btree_create_sequence(); raku_btree_add_child(patrol_seq, move); raku_btree_add_child(patrol_seq, wait); // Wrap in a repeater so it loops forever RakuBTNode* patrol_loop = raku_btree_create_decorator("repeat", patrol_seq); raku_btree_decorator_set_param(patrol_loop, "count", "-1"); // -1 = infinite raku_ai_agent_set_btree("guard", patrol_loop);

PERCEPTION

Adding Detection and Chase Behavior

Guards need to see and hear the player. Raku provides a perception system with vision cones, hearing ranges, and line-of-sight checks.

// Configure the guard's vision raku_ai_agent_set_vision_range("guard", 15.0f); // meters raku_ai_agent_set_vision_angle("guard", 120.0f); // degrees (field of view) raku_ai_agent_set_hearing_range("guard", 10.0f); // meters // Mark the player as detectable raku_ai_agent_set_detectable("player", true);

Build a behavior tree that reacts to detection:

// Condition nodes: check perception state RakuBTNode* can_see_player = raku_btree_create_condition("can_see_target"); raku_btree_condition_set_param(can_see_player, "target", "player"); RakuBTNode* heard_noise = raku_btree_create_condition("heard_noise"); // Action nodes RakuBTNode* chase = raku_btree_create_action("move_to_target"); raku_btree_action_set_param(chase, "target", "player"); raku_btree_action_set_param(chase, "speed", "4.0"); // run speed RakuBTNode* investigate = raku_btree_create_action("move_to_noise"); RakuBTNode* look_around = raku_btree_create_action("look_around"); raku_btree_action_set_param(look_around, "duration", "3.0"); // Chase sequence: see player -> chase player RakuBTNode* chase_seq = raku_btree_create_sequence(); raku_btree_add_child(chase_seq, can_see_player); raku_btree_add_child(chase_seq, chase); // Investigate sequence: heard noise -> go to noise -> look around RakuBTNode* investigate_seq = raku_btree_create_sequence(); raku_btree_add_child(investigate_seq, heard_noise); raku_btree_add_child(investigate_seq, investigate); raku_btree_add_child(investigate_seq, look_around); // Root selector: try combat, then investigate, then patrol RakuBTNode* root = raku_btree_create_selector(); raku_btree_add_child(root, chase_seq); raku_btree_add_child(root, investigate_seq); raku_btree_add_child(root, patrol_loop); raku_ai_agent_set_btree("guard", root);

STATE

Using Blackboard for NPC State

The blackboard is a shared key-value store attached to each AI agent. Behavior tree nodes read and write blackboard values to share state across the tree.

// Set initial blackboard values raku_ai_blackboard_set_float("guard", "alert_level", 0.0f); raku_ai_blackboard_set_string("guard", "state", "idle"); raku_ai_blackboard_set_bool("guard", "has_weapon", true); raku_ai_blackboard_set_int("guard", "health", 100); // Read blackboard values from anywhere in your code float alert = raku_ai_blackboard_get_float("guard", "alert_level"); const char* state = raku_ai_blackboard_get_string("guard", "state");

Create behavior tree conditions that read the blackboard:

// Condition: check if alert level is high enough to chase RakuBTNode* is_alert = raku_btree_create_condition("blackboard_compare_float"); raku_btree_condition_set_param(is_alert, "key", "alert_level"); raku_btree_condition_set_param(is_alert, "op", "greater"); raku_btree_condition_set_param(is_alert, "value", "0.7"); // Action: write to blackboard when entering combat RakuBTNode* set_combat = raku_btree_create_action("blackboard_set"); raku_btree_action_set_param(set_combat, "key", "state"); raku_btree_action_set_param(set_combat, "value", "combat"); // Gradually increase alert when player is near void on_update(float dt) { float alert = raku_ai_blackboard_get_float("guard", "alert_level"); if (raku_ai_agent_can_see("guard", "player")) { alert += 0.5f * dt; // ramp up while visible } else { alert -= 0.1f * dt; // decay slowly when out of sight } alert = alert < 0.0f ? 0.0f : (alert > 1.0f ? 1.0f : alert); raku_ai_blackboard_set_float("guard", "alert_level", alert); }

NAVIGATION

NavMesh Setup for NPC Navigation

A NavMesh (navigation mesh) defines the walkable areas of your level. NPCs use it to find paths around obstacles, through doorways, and up stairs.

// Generate a NavMesh from the scene geometry raku_ai_navmesh_create("puzzle"); // Configure NavMesh generation parameters raku_ai_navmesh_set_agent_radius(0.4f); // NPC body radius raku_ai_navmesh_set_agent_height(1.8f); // NPC height raku_ai_navmesh_set_step_height(0.3f); // max step (stairs) raku_ai_navmesh_set_slope_angle(45.0f); // max walkable slope // Build the mesh (can take a moment on complex scenes) raku_ai_navmesh_build();

Use the NavMesh for pathfinding:

// Find a path from point A to point B float start[] = { 0.0f, 0.0f, 0.0f }; float end[] = { 10.0f, 0.0f, -5.0f }; float path[64][3]; int path_length = raku_ai_navmesh_find_path(start, end, path, 64); if (path_length > 0) { raku_log(0, "Found path with %d waypoints", path_length); } else { raku_log(1, "No valid path found"); } // Or let an agent navigate automatically raku_ai_agent_move_to("guard", 10.0f, 0.0f, -5.0f);

Add dynamic obstacles that the NavMesh respects:

// Mark an object as a NavMesh obstacle raku_ai_navmesh_add_obstacle("barrel.glb", 0.5f); // radius // When the barrel moves, the NavMesh updates locally // (uses dynamic carving, not full rebuild) // Add off-mesh connections for jumps, ladders, etc. float jump_start[] = { 3.0f, 0.0f, 0.0f }; float jump_end[] = { 3.0f, 2.0f, -3.0f }; raku_ai_navmesh_add_offmesh_link(jump_start, jump_end, true); // bidirectional

VOICE

Adding Voice Commands for NPC Interaction

Use the Raku Voice subsystem to let players talk to NPCs. Combine with the SLM (small language model) for dynamic dialogue responses.

// Initialize voice and SLM subsystems raku_voice_init(); raku_slm_init(); // Register voice commands the NPC understands raku_voice_register_command("follow me", on_follow_command); raku_voice_register_command("stop", on_stop_command); raku_voice_register_command("go to [location]", on_goto_command); raku_voice_register_command("what do you see", on_report_command); // Start listening (uses on-device speech recognition) raku_voice_start_listening();

Handle voice commands with NPC actions:

void on_follow_command(RakuVoiceEvent* event) { raku_ai_blackboard_set_string("guard", "state", "following"); raku_ai_blackboard_set_string("guard", "follow_target", "player"); // Use SLM for a natural-sounding response raku_slm_set_context("You are a medieval guard. The player asked you to follow them."); const char* response = raku_slm_generate("Respond briefly, in character."); raku_audio_play_tts(response); // text-to-speech } void on_stop_command(RakuVoiceEvent* event) { raku_ai_blackboard_set_string("guard", "state", "idle"); raku_ai_agent_stop("guard"); } void on_report_command(RakuVoiceEvent* event) { // Ask the AI what it currently perceives int visible_count = raku_ai_agent_get_visible_count("guard"); char context[256]; sprintf(context, "You are a guard. You can see %d entities. Alert level: %.0f%%.", visible_count, raku_ai_blackboard_get_float("guard", "alert_level") * 100); raku_slm_set_context(context); const char* report = raku_slm_generate("Report what you see, in character."); raku_audio_play_tts(report); }

Privacy: Voice recognition runs entirely on-device using the Raku Voice subsystem. No audio is sent to the cloud. raku_voice_init() is a privileged call that requires user confirmation.

COMPLETE EXAMPLE

Complete Example: Guard NPC with Full AI

A fully functional guard that patrols, detects the player, chases, investigates noises, responds to voice commands, and uses the blackboard for state management.

// === Guard NPC with Full AI === raku_init(); raku_renderer_create_window(1280, 720, "Guard AI Demo", true); raku_renderer_create_camera("main", 60.0f, 0.1f, 200.0f); // Physics for the world float gravity[] = { 0, -9.81f, 0 }; raku_physics_create_world(gravity); // AI and voice raku_ai_init(); raku_voice_init(); raku_slm_init(); // Audio raku_audio_init(32, false); raku_audio_load("sounds/footstep.wav", "footstep"); raku_audio_load("sounds/alert.wav", "alert"); // Scene raku_scene_create("guard_demo"); raku_scene_add_model("level.glb", 0, 0, 0); raku_physics_add_static_collider("level.glb", "mesh"); // NavMesh raku_ai_navmesh_create("guard_demo"); raku_ai_navmesh_set_agent_radius(0.4f); raku_ai_navmesh_set_agent_height(1.8f); raku_ai_navmesh_build(); // Player (simple capsule) raku_scene_add_model("player.glb", -5.0f, 0.0f, 3.0f); raku_ai_agent_set_detectable("player", true); // Guard NPC raku_scene_add_model("guard.glb", 0.0f, 0.0f, 0.0f); raku_ai_create_agent("guard", "behavior_tree"); raku_ai_agent_set_speed("guard", 2.0f); raku_ai_agent_set_turn_speed("guard", 180.0f); raku_ai_agent_set_vision_range("guard", 15.0f); raku_ai_agent_set_vision_angle("guard", 120.0f); raku_ai_agent_set_hearing_range("guard", 10.0f); // Patrol waypoints float waypoints[][3] = { { 0, 0, 0 }, { 8, 0, 0 }, { 8, 0, -6 }, { 0, 0, -6 } }; raku_ai_agent_set_patrol_points("guard", waypoints, 4); // Blackboard raku_ai_blackboard_set_float("guard", "alert_level", 0.0f); raku_ai_blackboard_set_string("guard", "state", "patrol"); raku_ai_blackboard_set_int("guard", "health", 100); // --- Build the full behavior tree --- // Chase branch RakuBTNode* see_player = raku_btree_create_condition("can_see_target"); raku_btree_condition_set_param(see_player, "target", "player"); RakuBTNode* set_alert = raku_btree_create_action("blackboard_set"); raku_btree_action_set_param(set_alert, "key", "state"); raku_btree_action_set_param(set_alert, "value", "combat"); RakuBTNode* play_alert = raku_btree_create_action("play_sound"); raku_btree_action_set_param(play_alert, "id", "alert"); RakuBTNode* chase = raku_btree_create_action("move_to_target"); raku_btree_action_set_param(chase, "target", "player"); raku_btree_action_set_param(chase, "speed", "4.0"); RakuBTNode* chase_seq = raku_btree_create_sequence(); raku_btree_add_child(chase_seq, see_player); raku_btree_add_child(chase_seq, set_alert); raku_btree_add_child(chase_seq, play_alert); raku_btree_add_child(chase_seq, chase); // Investigate branch RakuBTNode* heard = raku_btree_create_condition("heard_noise"); RakuBTNode* go_noise = raku_btree_create_action("move_to_noise"); RakuBTNode* look = raku_btree_create_action("look_around"); raku_btree_action_set_param(look, "duration", "3.0"); RakuBTNode* investigate_seq = raku_btree_create_sequence(); raku_btree_add_child(investigate_seq, heard); raku_btree_add_child(investigate_seq, go_noise); raku_btree_add_child(investigate_seq, look); // Patrol branch RakuBTNode* patrol = raku_btree_create_action("patrol_waypoints"); RakuBTNode* patrol_wait = raku_btree_create_action("wait"); raku_btree_action_set_param(patrol_wait, "duration", "2.0"); RakuBTNode* patrol_seq = raku_btree_create_sequence(); raku_btree_add_child(patrol_seq, patrol); raku_btree_add_child(patrol_seq, patrol_wait); RakuBTNode* patrol_loop = raku_btree_create_decorator("repeat", patrol_seq); raku_btree_decorator_set_param(patrol_loop, "count", "-1"); // Root RakuBTNode* root = raku_btree_create_selector(); raku_btree_add_child(root, chase_seq); raku_btree_add_child(root, investigate_seq); raku_btree_add_child(root, patrol_loop); raku_ai_agent_set_btree("guard", root); // --- Voice commands --- raku_voice_register_command("follow me", on_follow_command); raku_voice_register_command("stop", on_stop_command); raku_voice_start_listening(); // --- Run --- raku_scene_set_update_callback(on_update); raku_scene_start_loop(); raku_shutdown();

Next steps: Combine this NPC with spatial audio for footstep sounds that the player can hear approaching, or with mixed reality to have the guard patrol your real room.