Immersive Audio for XR

In This Tutorial

Audio Engine Setup and Listener
3D Positioned Audio Sources
HRTF for Realistic VR Audio
Reverb Zones for Environment Acoustics
Sound Occlusion Through Geometry
Mixer Groups for Dynamic Audio Control
Complete Example: Immersive Audio Tour

FUNDAMENTALS

Audio Engine Setup and Listener

The Raku audio engine supports up to 256 simultaneous channels with hardware-accelerated spatial processing. The listener represents the player's ears in the 3D world.

// Initialize audio with 64 channels and HRTF enabled raku_audio_init(64, true); // Load audio assets raku_audio_load("sounds/ambient_wind.wav", "wind"); raku_audio_load("sounds/footstep.wav", "footstep"); raku_audio_load("sounds/waterfall.ogg", "waterfall"); raku_audio_load("music/exploration.ogg", "music_explore");

The audio listener automatically tracks the camera (or XR headset) position. You can also set it manually:

// In XR mode, the listener auto-tracks the headset. No setup needed. // For desktop mode, set the listener position manually: raku_audio_set_listener_position(0.0f, 1.7f, 0.0f); raku_audio_set_listener_forward(0.0f, 0.0f, -1.0f); raku_audio_set_listener_up(0.0f, 1.0f, 0.0f); // Set master volume (0.0 = silent, 1.0 = full) raku_audio_set_master_volume(0.8f);

3D AUDIO

3D Positioned Audio Sources

Spatial audio sources exist at a specific point in 3D space. As the listener moves, the volume, panning, and distance attenuation update automatically.

// Play a sound at a 3D position raku_audio_play_spatial("waterfall", 10.0f, 2.0f, -5.0f); // Create a persistent spatial source (for looping ambient sounds) handle src = raku_audio_create_source("waterfall"); raku_audio_source_set_position(src, 10.0f, 2.0f, -5.0f); raku_audio_source_set_loop(src, true); raku_audio_source_set_volume(src, 0.7f); raku_audio_source_play(src);

Configure distance attenuation:

// Set distance attenuation model raku_audio_source_set_min_distance(src, 1.0f); // full volume within 1m raku_audio_source_set_max_distance(src, 50.0f); // inaudible beyond 50m raku_audio_source_set_rolloff(src, "inverse"); // "inverse", "linear", "exponential" // Set the attenuation curve factor (higher = faster falloff) raku_audio_source_set_rolloff_factor(src, 2.0f); // Directional audio (e.g., a speaker pointing in one direction) raku_audio_source_set_cone_angles(src, 60.0f, 120.0f); // inner, outer raku_audio_source_set_cone_outer_volume(src, 0.2f); // volume outside cone raku_audio_source_set_direction(src, 0.0f, 0.0f, -1.0f);

HRTF

HRTF for Realistic VR Audio

Head-Related Transfer Function (HRTF) simulates how sounds are perceived differently based on their direction relative to your head. It enables true "above", "below", and "behind" audio in headphones — essential for VR immersion.

// HRTF is enabled during raku_audio_init(channels, hrtf=true) // You can also toggle it at runtime: raku_audio_set_hrtf_enabled(true); // Choose an HRTF profile (affects how sounds are spatialized) raku_audio_set_hrtf_profile("default"); // generic, works for most people raku_audio_set_hrtf_profile("small_head"); // smaller inter-aural distance raku_audio_set_hrtf_profile("large_head"); // larger inter-aural distance

HRTF works best with headphones (which all VR headsets use). For non-VR contexts:

// Check if the user is in XR (headphones) or desktop (speakers) const char* mode = raku_xr_get_mode(); if (mode != NULL) { // XR mode: HRTF works great with headphones raku_audio_set_hrtf_enabled(true); } else { // Desktop mode: stereo panning is often better for speakers raku_audio_set_hrtf_enabled(false); }

Why HRTF matters in VR: Without HRTF, spatial audio only supports left/right panning. With HRTF, a bird chirping above your head actually sounds like it's above you, and footsteps behind you sound like they're behind you. This dramatically improves presence and immersion.

ENVIRONMENT

Reverb Zones for Environment Acoustics

Reverb zones make different areas of your game sound acoustically distinct. A cathedral should echo, a cave should reverberate, and an outdoor field should be dry.

// Create a reverb zone for a cathedral interior RakuReverbConfig cathedral = { .preset = "cathedral", // built-in preset .position = { 0, 5, 0 }, .radius = 20.0f, // zone radius in meters .mix = 0.8f, // wet/dry mix (0 = dry, 1 = fully wet) }; raku_audio_create_reverb_zone(&cathedral, "cathedral_zone"); // Create a custom reverb for a small cave RakuReverbConfig cave = { .preset = "custom", .position = { 30, -2, 10 }, .radius = 8.0f, .mix = 0.9f, .decay_time = 3.5f, // seconds .early_reflections = 0.6f, // strength of early reflections .late_reverb = 0.8f, // strength of late reverb tail .diffusion = 0.7f, // 0 = distinct echoes, 1 = smooth reverb .density = 0.9f, // reverb density .hf_damping = 0.4f, // high-frequency absorption }; raku_audio_create_reverb_zone(&cave, "cave_zone");

Built-in reverb presets:

// Available presets: // "room" - small room (bedroom, office) // "hall" - medium hall (conference room) // "cathedral" - large reverberant space // "cave" - natural cave with long decay // "bathroom" - small, highly reflective // "forest" - outdoor with subtle early reflections // "underwater" - muffled, low-pass filtered // "arena" - large open space with slap-back echo // "custom" - fully user-defined parameters // Crossfade between zones as the listener moves raku_audio_set_reverb_crossfade(1.5f); // blend over 1.5 seconds

PHYSICS-BASED

Sound Occlusion Through Geometry

Sound occlusion simulates how walls and objects block or muffle audio. When a wall is between the listener and a sound source, the sound should be quieter and lose its high frequencies — just like in real life.

// Enable audio occlusion (uses physics raycasting internally) raku_audio_set_occlusion_enabled(true); // Set how many rays to cast per source (more = smoother, more expensive) raku_audio_set_occlusion_ray_count(4); // 1-8, default 4 // Configure occlusion per material raku_audio_set_material_occlusion("concrete", 0.9f); // heavy blocking raku_audio_set_material_occlusion("wood", 0.5f); // moderate blocking raku_audio_set_material_occlusion("glass", 0.3f); // light blocking raku_audio_set_material_occlusion("curtain", 0.15f); // minimal blocking

Per-source occlusion control:

// Enable occlusion on a specific source handle enemy_voice = raku_audio_create_source("guard_voice"); raku_audio_source_set_position(enemy_voice, 5.0f, 1.5f, -3.0f); raku_audio_source_set_occlusion_enabled(enemy_voice, true); // Configure low-pass filter when occluded raku_audio_source_set_occlusion_lp_frequency(enemy_voice, 800.0f); // When fully occluded, frequencies above 800 Hz are attenuated // Volume reduction when fully occluded raku_audio_source_set_occlusion_volume_factor(enemy_voice, 0.3f); // At full occlusion, volume drops to 30%

Performance: Occlusion raycasting uses the physics system. Ensure your scene has colliders on walls and doors. Keep ray counts low (2-4) on mobile XR hardware. Use raku_audio_set_occlusion_update_rate() to reduce how often raycasts are performed.

MIXING

Mixer Groups for Dynamic Audio Control

Mixer groups let you control categories of sounds together: lower all SFX during dialogue, duck music during combat, or mute ambient sounds in a menu.

// Create mixer groups raku_audio_create_mixer_group("master"); raku_audio_create_mixer_group("music"); raku_audio_create_mixer_group("sfx"); raku_audio_create_mixer_group("voice"); raku_audio_create_mixer_group("ambient"); // Set up hierarchy (all groups route through master) raku_audio_mixer_set_parent("music", "master"); raku_audio_mixer_set_parent("sfx", "master"); raku_audio_mixer_set_parent("voice", "master"); raku_audio_mixer_set_parent("ambient", "master"); // Assign sounds to groups raku_audio_source_set_mixer_group(waterfall_src, "ambient"); raku_audio_source_set_mixer_group(music_src, "music"); raku_audio_source_set_mixer_group(footstep_src, "sfx");

Control groups dynamically:

// Set group volume raku_audio_mixer_set_volume("music", 0.6f); raku_audio_mixer_set_volume("sfx", 1.0f); // Fade a group over time raku_audio_mixer_fade_volume("music", 0.2f, 2.0f); // to 0.2 over 2 seconds // Duck music when voice is playing (sidechain compression) raku_audio_mixer_set_duck("music", "voice", 0.3f, 0.5f); // When "voice" group is active, "music" ducks to 30% over 0.5 seconds // Mute/unmute groups raku_audio_mixer_set_muted("ambient", true); // mute ambient in menus raku_audio_mixer_set_muted("ambient", false); // unmute on return to game // Pause/resume groups (preserves playback position) raku_audio_mixer_pause("sfx"); raku_audio_mixer_resume("sfx");

COMPLETE EXAMPLE

Complete Example: Immersive Audio Tour

A virtual museum tour with spatial audio at each exhibit, reverb that changes as you enter different rooms, voice narration that ducks the music, and occlusion through walls. Designed for VR with HRTF.

// === Immersive Audio Tour === raku_init(); raku_xr_init("vr"); raku_renderer_create_window(1280, 720, "Audio Tour", true); raku_renderer_create_camera("main", 90.0f, 0.01f, 100.0f); // Physics (for audio occlusion raycasts) float gravity[] = { 0, -9.81f, 0 }; raku_physics_create_world(gravity); // Scene raku_scene_create("museum"); raku_scene_add_model("museum.glb", 0, 0, 0); raku_physics_add_static_collider("museum.glb", "mesh"); // --- Audio Engine --- raku_audio_init(64, true); // HRTF on raku_audio_set_occlusion_enabled(true); raku_audio_set_occlusion_ray_count(4); raku_audio_set_material_occlusion("concrete", 0.85f); raku_audio_set_material_occlusion("glass", 0.25f); // --- Mixer Groups --- raku_audio_create_mixer_group("master"); raku_audio_create_mixer_group("music"); raku_audio_create_mixer_group("narration"); raku_audio_create_mixer_group("exhibits"); raku_audio_create_mixer_group("ambient"); raku_audio_mixer_set_parent("music", "master"); raku_audio_mixer_set_parent("narration", "master"); raku_audio_mixer_set_parent("exhibits", "master"); raku_audio_mixer_set_parent("ambient", "master"); // Duck music and exhibits when narration plays raku_audio_mixer_set_duck("music", "narration", 0.2f, 0.8f); raku_audio_mixer_set_duck("exhibits", "narration", 0.4f, 0.5f); // --- Load Audio Assets --- raku_audio_load("music/ambient_museum.ogg", "bg_music"); raku_audio_load("sounds/fountain.wav", "fountain"); raku_audio_load("sounds/clock_tick.wav", "clock"); raku_audio_load("sounds/wind_outside.wav", "wind"); raku_audio_load("narration/welcome.ogg", "narr_welcome"); raku_audio_load("narration/painting_room.ogg", "narr_paintings"); raku_audio_load("narration/sculpture_hall.ogg", "narr_sculpture"); // --- Background Music (non-spatial) --- handle music = raku_audio_create_source("bg_music"); raku_audio_source_set_loop(music, true); raku_audio_source_set_volume(music, 0.4f); raku_audio_source_set_mixer_group(music, "music"); raku_audio_source_set_spatial(music, false); // non-spatial (plays everywhere) raku_audio_source_play(music); // --- Spatial Exhibit Sounds --- // Fountain in the lobby handle fountain = raku_audio_create_source("fountain"); raku_audio_source_set_position(fountain, 0.0f, 1.0f, -3.0f); raku_audio_source_set_loop(fountain, true); raku_audio_source_set_min_distance(fountain, 0.5f); raku_audio_source_set_max_distance(fountain, 15.0f); raku_audio_source_set_occlusion_enabled(fountain, true); raku_audio_source_set_mixer_group(fountain, "exhibits"); raku_audio_source_play(fountain); // Grandfather clock in the painting room handle clock = raku_audio_create_source("clock"); raku_audio_source_set_position(clock, 12.0f, 2.0f, -8.0f); raku_audio_source_set_loop(clock, true); raku_audio_source_set_min_distance(clock, 0.3f); raku_audio_source_set_max_distance(clock, 8.0f); raku_audio_source_set_occlusion_enabled(clock, true); raku_audio_source_set_mixer_group(clock, "exhibits"); raku_audio_source_play(clock); // Wind from outside (through windows) handle wind = raku_audio_create_source("wind"); raku_audio_source_set_position(wind, -5.0f, 3.0f, 0.0f); raku_audio_source_set_loop(wind, true); raku_audio_source_set_volume(wind, 0.3f); raku_audio_source_set_occlusion_enabled(wind, true); raku_audio_source_set_mixer_group(wind, "ambient"); raku_audio_source_play(wind); // --- Reverb Zones --- // Lobby: open with marble floors RakuReverbConfig lobby_reverb = { .preset = "hall", .position = { 0, 3, -3 }, .radius = 10.0f, .mix = 0.5f, }; raku_audio_create_reverb_zone(&lobby_reverb, "lobby"); // Painting room: carpeted, damped RakuReverbConfig painting_reverb = { .preset = "room", .position = { 12, 2, -8 }, .radius = 8.0f, .mix = 0.3f, }; raku_audio_create_reverb_zone(&painting_reverb, "painting_room"); // Sculpture hall: large and echoey RakuReverbConfig sculpture_reverb = { .preset = "cathedral", .position = { -10, 5, -15 }, .radius = 15.0f, .mix = 0.7f, }; raku_audio_create_reverb_zone(&sculpture_reverb, "sculpture_hall"); raku_audio_set_reverb_crossfade(2.0f); // smooth zone transitions // --- Narration Triggers --- // Play narration when entering a room (using physics triggers) void on_enter_lobby(RakuTriggerEvent* event) { handle narr = raku_audio_create_source("narr_welcome"); raku_audio_source_set_spatial(narr, false); raku_audio_source_set_mixer_group(narr, "narration"); raku_audio_source_play(narr); } void on_enter_painting_room(RakuTriggerEvent* event) { handle narr = raku_audio_create_source("narr_paintings"); raku_audio_source_set_spatial(narr, false); raku_audio_source_set_mixer_group(narr, "narration"); raku_audio_source_play(narr); } void on_enter_sculpture_hall(RakuTriggerEvent* event) { handle narr = raku_audio_create_source("narr_sculpture"); raku_audio_source_set_spatial(narr, false); raku_audio_source_set_mixer_group(narr, "narration"); raku_audio_source_play(narr); } // Create trigger zones at room entrances raku_physics_on_trigger_enter("lobby_trigger", on_enter_lobby); raku_physics_on_trigger_enter("painting_trigger", on_enter_painting_room); raku_physics_on_trigger_enter("sculpture_trigger", on_enter_sculpture_hall); // --- Hand tracking for XR navigation --- raku_xr_hand_tracking_enable(true); // --- Run --- raku_scene_start_loop(); raku_xr_shutdown(); raku_shutdown();

Next steps: Combine this audio tour with mixed reality to create an audio tour of your real space, or add AI NPCs as museum guides that respond to voice commands.