feat: add entity rendering enhancements with visual properties and improved packet parsing

2025-11-08 20:44:42 -05:00
parent a43ba28d20
commit f065e1cdeb
1 changed files with 253 additions and 0 deletions
--- a/ENTITY_RENDERING_ENHANCEMENTS.md
+++ b/ENTITY_RENDERING_ENHANCEMENTS.md
@@ -0,0 +1,253 @@
+# Entity Rendering Enhancements for Starworld
+
+## Overview
+
+This document describes the enhancements made to starworld to support rendering complex Overte entities with full visual properties including models, textures, colors, and different geometric primitives.
+
+## Changes Made
+
+### 1. Enhanced Entity Data Structure (`OverteClient.hpp`)
+
+**Added `EntityType` enum:**
+```cpp
+enum class EntityType {
+    Unknown, Box, Sphere, Model, Shape, Light, Text, 
+    Zone, Web, ParticleEffect, Line, PolyLine, Grid, Gizmo, Material
+};
+```
+
+**Extended `OverteEntity` structure:**
+```cpp
+struct OverteEntity {
+    std::uint64_t id{0};
+    std::string name;
+    glm::mat4 transform{1.0f};
+    
+    // NEW: Visual properties
+    EntityType type{EntityType::Box};
+    std::string modelUrl;      // For Model type entities
+    std::string textureUrl;    // Texture/material URL
+    glm::vec3 color{1.0f, 1.0f, 1.0f};  // RGB color (0-1 range)
+    glm::vec3 dimensions{0.1f, 0.1f, 0.1f};  // Size/scale in meters
+    float alpha{1.0f};         // Transparency (0-1)
+};
+```
+
+### 2. Enhanced Entity Packet Parser (`OverteClient.cpp`)
+
+**Updated `parseEntityPacket()` to extract:**
+- Entity type classification
+- Model URLs (for 3D models)
+- Texture URLs
+- RGB color values
+- Dimensions/scale
+- Alpha transparency
+
+**Enhanced simulation mode** with diverse entity types:
+- Red cube (Box type)
+- Green sphere (Sphere type)  
+- Blue model placeholder (Model type)
+
+### 3. Rust Bridge Visual Properties (`bridge/src/lib.rs`)
+
+**Extended `Node` structure:**
+```rust
+struct Node {
+    id: u64,
+    name: String,
+    transform: Mat4,
+    entity_type: u8,       // 0=Unknown, 1=Box, 2=Sphere, 3=Model
+    model_url: String,
+    texture_url: String,
+    color: [f32; 4],       // RGBA
+    dimensions: [f32; 3],  // xyz dimensions
+}
+```
+
+**Added new command types:**
+- `SetModel` - Set 3D model URL
+- `SetTexture` - Set texture/material URL
+- `SetColor` - Set RGBA color
+- `SetDimensions` - Set xyz dimensions
+- `SetEntityType` - Set geometric primitive type
+
+**New C-ABI export functions:**
+- `sdxr_set_node_model(id, model_url)`
+- `sdxr_set_node_texture(id, texture_url)`
+- `sdxr_set_node_color(id, r, g, b, a)`
+- `sdxr_set_node_dimensions(id, x, y, z)`
+- `sdxr_set_node_entity_type(id, type)`
+
+### 4. Enhanced Rendering (`bridge/src/lib.rs` - `reify()`)
+
+**Implemented type-specific rendering:**
+
+- **Box entities (type 1):** Colored wireframe cubes using entity color
+- **Sphere entities (type 2):** Three orthogonal wireframe circles (XY, XZ, YZ planes) in entity color
+- **Model entities (type 3):** Distinctive octahedron wireframe (placeholder for actual model loading)
+- **Unknown types:** Gray wireframe cube as fallback
+
+**Features:**
+- Respects entity dimensions for accurate sizing
+- Uses entity-specific colors
+- Applies proper transforms (position, rotation, scale)
+- Maintains visual distinction between entity types
+
+### 5. StardustBridge C++ Interface (`StardustBridge.hpp/.cpp`)
+
+**Added methods:**
+```cpp
+bool setNodeModel(NodeId id, const std::string& modelUrl);
+bool setNodeTexture(NodeId id, const std::string& textureUrl);
+bool setNodeColor(NodeId id, const glm::vec3& color, float alpha = 1.0f);
+bool setNodeDimensions(NodeId id, const glm::vec3& dimensions);
+bool setNodeEntityType(NodeId id, uint8_t entityType);
+```
+
+**Updated dynamic library loader** to resolve new function pointers from Rust bridge.
+
+### 6. SceneSync Integration (`SceneSync.cpp`)
+
+**Enhanced entity synchronization:**
+- Propagates entity type on creation/update
+- Sets color and alpha properties
+- Configures dimensions
+- Handles model and texture URLs
+- Updates visual properties when entities change
+
+## Testing
+
+### Simulation Mode
+
+Run with simulation mode to see example entities:
+
+```bash
+export STARWORLD_SIMULATE=1
+./build/stardust-overte-client
+```
+
+This creates three demo entities:
+- **CubeA** - Red wireframe cube (20cm)
+- **SphereB** - Green wireframe sphere (15cm)
+- **ModelC** - Blue octahedron representing a model entity (25cm)
+
+### Live Overte Connection
+
+To connect to a real Overte server:
+
+```bash
+# Optional: Set credentials
+export OVERTE_USERNAME=your_username
+
+# Connect to server
+./build/stardust-overte-client ws://domain.example.com:40102
+```
+
+The client will:
+1. Perform domain handshake
+2. Discover entity server via DomainList packets
+3. Send EntityQuery to request all entities
+4. Parse and render entities with full visual properties
+
+## Architecture
+
+```
+┌─────────────────┐
+│ Overte Server   │
+│ (Entity Server) │
+└────────┬────────┘
+         │ UDP Packets (EntityAdd, EntityEdit)
+         │ Contains: type, position, rotation, dimensions,
+         │          color, modelUrl, textureUrl
+         ▼
+┌─────────────────────┐
+│  OverteClient.cpp   │
+│  - parseEntityPacket│
+│  - OverteEntity     │
+└────────┬────────────┘
+         │ consumeUpdatedEntities()
+         ▼
+┌─────────────────────┐
+│   SceneSync.cpp     │
+│  - Maps Overte IDs  │
+│    to Stardust IDs  │
+└────────┬────────────┘
+         │ createNode(), setNodeColor(), 
+         │ setNodeModel(), etc.
+         ▼
+┌─────────────────────┐
+│ StardustBridge.cpp  │
+│  - C++ wrapper      │
+│  - dlopen bridge.so │
+└────────┬────────────┘
+         │ C-ABI calls: sdxr_set_node_*
+         ▼
+┌─────────────────────┐
+│ bridge/lib.rs       │
+│  - Command queue    │
+│  - Shared state     │
+└────────┬────────────┘
+         │ BridgeState::reify()
+         ▼
+┌─────────────────────┐
+│  Stardust Server    │
+│  - Lines rendering  │
+│  - Type-specific    │
+│    visualization    │
+└─────────────────────┘
+```
+
+## Future Enhancements
+
+### Short Term
+1. **Actual model loading** - Replace octahedron with loaded .glb/.fbx models using Stardust Model nodes
+2. **Texture application** - Apply textures to rendered entities
+3. **More entity types** - Add support for Light, Text, PolyLine, etc.
+4. **Performance optimization** - Batch updates, reduce command overhead
+
+### Medium Term
+1. **Full mesh rendering** - Move beyond wireframes to solid shaded meshes
+2. **Material support** - PBR materials with metallic/roughness
+3. **Animation** - Skeletal animation for rigged models
+4. **LOD system** - Level-of-detail based on distance
+
+### Long Term
+1. **Physics sync** - Real-time physics state synchronization
+2. **Script integration** - Execute Overte entity scripts in Stardust context
+3. **Avatar rendering** - Full avatar mesh and animation support
+4. **Audio spatialization** - 3D positional audio from Overte
+
+## Build Instructions
+
+```bash
+# Build Rust bridge
+cd bridge
+cargo build
+cd ..
+
+# Build C++ client
+cd build
+cmake ..
+make
+cd ..
+
+# Run
+./build/stardust-overte-client
+```
+
+## Dependencies
+
+- **C++17** - Modern C++ features
+- **GLM** - Math library for vectors/matrices
+- **OpenSSL** - MD5 hashing for protocol signatures
+- **Rust nightly** - For Stardust bridge
+- **stardust-xr-asteroids** - Declarative UI framework
+- **stardust-xr-fusion** - Low-level client API
+- **tokio** - Async runtime
+
+## References
+
+- Overte protocol: `third_party/overte-src/libraries/networking/`
+- Entity types: Based on Overte `EntityTypes.h`
+- Packet formats: Overte NLPacket specification
+- Stardust API: https://github.com/StardustXR/core