Starworld/ENTITY_RENDERING_ENHANCEMENTS.md

Entity Rendering Enhancements for Starworld

Overview

This document describes the enhancements made to starworld to support rendering complex Overte entities with full visual properties including models, textures, colors, and different geometric primitives.

Changes Made

1. Enhanced Entity Data Structure (OverteClient.hpp)

Added EntityType enum:

enum class EntityType {
    Unknown, Box, Sphere, Model, Shape, Light, Text, 
    Zone, Web, ParticleEffect, Line, PolyLine, Grid, Gizmo, Material
};

Extended OverteEntity structure:

struct OverteEntity {
    std::uint64_t id{0};
    std::string name;
    glm::mat4 transform{1.0f};
    
    // NEW: Visual properties
    EntityType type{EntityType::Box};
    std::string modelUrl;      // For Model type entities
    std::string textureUrl;    // Texture/material URL
    glm::vec3 color{1.0f, 1.0f, 1.0f};  // RGB color (0-1 range)
    glm::vec3 dimensions{0.1f, 0.1f, 0.1f};  // Size/scale in meters
    float alpha{1.0f};         // Transparency (0-1)
};

2. Enhanced Entity Packet Parser (OverteClient.cpp)

Updated parseEntityPacket() to extract:

• Entity type classification
• Model URLs (for 3D models)
• Texture URLs
• RGB color values
• Dimensions/scale
• Alpha transparency

Enhanced simulation mode with diverse entity types:

• Red cube (Box type)
• Green sphere (Sphere type)
• Blue model placeholder (Model type)

3. Rust Bridge Visual Properties (bridge/src/lib.rs)

Extended Node structure:

struct Node {
    id: u64,
    name: String,
    transform: Mat4,
    entity_type: u8,       // 0=Unknown, 1=Box, 2=Sphere, 3=Model
    model_url: String,
    texture_url: String,
    color: [f32; 4],       // RGBA
    dimensions: [f32; 3],  // xyz dimensions
}

Added new command types:

• SetModel - Set 3D model URL
• SetTexture - Set texture/material URL
• SetColor - Set RGBA color
• SetDimensions - Set xyz dimensions
• SetEntityType - Set geometric primitive type

New C-ABI export functions:

• sdxr_set_node_model(id, model_url)
• sdxr_set_node_texture(id, texture_url)
• sdxr_set_node_color(id, r, g, b, a)
• sdxr_set_node_dimensions(id, x, y, z)
• sdxr_set_node_entity_type(id, type)

4. Enhanced Rendering (bridge/src/lib.rs - reify())

Implemented type-specific rendering:

• Box entities (type 1): Colored wireframe cubes using entity color
• Sphere entities (type 2): Three orthogonal wireframe circles (XY, XZ, YZ planes) in entity color
• Model entities (type 3): Distinctive octahedron wireframe (placeholder for actual model loading)
• Unknown types: Gray wireframe cube as fallback

Features:

• Respects entity dimensions for accurate sizing
• Uses entity-specific colors
• Applies proper transforms (position, rotation, scale)
• Maintains visual distinction between entity types

5. StardustBridge C++ Interface (StardustBridge.hpp/.cpp)

Added methods:

bool setNodeModel(NodeId id, const std::string& modelUrl);
bool setNodeTexture(NodeId id, const std::string& textureUrl);
bool setNodeColor(NodeId id, const glm::vec3& color, float alpha = 1.0f);
bool setNodeDimensions(NodeId id, const glm::vec3& dimensions);
bool setNodeEntityType(NodeId id, uint8_t entityType);

Updated dynamic library loader to resolve new function pointers from Rust bridge.

6. SceneSync Integration (SceneSync.cpp)

Enhanced entity synchronization:

• Propagates entity type on creation/update
• Sets color and alpha properties
• Configures dimensions
• Handles model and texture URLs
• Updates visual properties when entities change

Testing

Simulation Mode

Run with simulation mode to see example entities:

export STARWORLD_SIMULATE=1
./build/stardust-overte-client

This creates three demo entities:

• CubeA - Red wireframe cube (20cm)
• SphereB - Green wireframe sphere (15cm)
• ModelC - Blue octahedron representing a model entity (25cm)

Live Overte Connection

To connect to a real Overte server:

# Optional: Set credentials
export OVERTE_USERNAME=your_username

# Connect to server
./build/stardust-overte-client ws://domain.example.com:40102

The client will:

1. Perform domain handshake
2. Discover entity server via DomainList packets
3. Send EntityQuery to request all entities
4. Parse and render entities with full visual properties

Architecture

┌─────────────────┐
│ Overte Server   │
│ (Entity Server) │
└────────┬────────┘
         │ UDP Packets (EntityAdd, EntityEdit)
         │ Contains: type, position, rotation, dimensions,
         │          color, modelUrl, textureUrl
         ▼
┌─────────────────────┐
│  OverteClient.cpp   │
│  - parseEntityPacket│
│  - OverteEntity     │
└────────┬────────────┘
         │ consumeUpdatedEntities()
         ▼
┌─────────────────────┐
│   SceneSync.cpp     │
│  - Maps Overte IDs  │
│    to Stardust IDs  │
└────────┬────────────┘
         │ createNode(), setNodeColor(), 
         │ setNodeModel(), etc.
         ▼
┌─────────────────────┐
│ StardustBridge.cpp  │
│  - C++ wrapper      │
│  - dlopen bridge.so │
└────────┬────────────┘
         │ C-ABI calls: sdxr_set_node_*
         ▼
┌─────────────────────┐
│ bridge/lib.rs       │
│  - Command queue    │
│  - Shared state     │
└────────┬────────────┘
         │ BridgeState::reify()
         ▼
┌─────────────────────┐
│  Stardust Server    │
│  - Lines rendering  │
│  - Type-specific    │
│    visualization    │
└─────────────────────┘

Future Enhancements

Short Term

1. Actual model loading - Replace octahedron with loaded .glb/.fbx models using Stardust Model nodes
2. Texture application - Apply textures to rendered entities
3. More entity types - Add support for Light, Text, PolyLine, etc.
4. Performance optimization - Batch updates, reduce command overhead

Medium Term

1. Full mesh rendering - Move beyond wireframes to solid shaded meshes
2. Material support - PBR materials with metallic/roughness
3. Animation - Skeletal animation for rigged models
4. LOD system - Level-of-detail based on distance

Long Term

1. Physics sync - Real-time physics state synchronization
2. Script integration - Execute Overte entity scripts in Stardust context
3. Avatar rendering - Full avatar mesh and animation support
4. Audio spatialization - 3D positional audio from Overte

Build Instructions

# Build Rust bridge
cd bridge
cargo build
cd ..

# Build C++ client
cd build
cmake ..
make
cd ..

# Run
./build/stardust-overte-client

Dependencies

• C++17 - Modern C++ features
• GLM - Math library for vectors/matrices
• OpenSSL - MD5 hashing for protocol signatures
• Rust nightly - For Stardust bridge
• stardust-xr-asteroids - Declarative UI framework
• stardust-xr-fusion - Low-level client API
• tokio - Async runtime

References

• Overte protocol: third_party/overte-src/libraries/networking/
• Entity types: Based on Overte EntityTypes.h
• Packet formats: Overte NLPacket specification
• Stardust API: https://github.com/StardustXR/core