diff --git a/ENTITY_RENDERING_ENHANCEMENTS.md b/ENTITY_RENDERING_ENHANCEMENTS.md new file mode 100644 index 0000000..cba0974 --- /dev/null +++ b/ENTITY_RENDERING_ENHANCEMENTS.md @@ -0,0 +1,253 @@ +# Entity Rendering Enhancements for Starworld + +## Overview + +This document describes the enhancements made to starworld to support rendering complex Overte entities with full visual properties including models, textures, colors, and different geometric primitives. + +## Changes Made + +### 1. Enhanced Entity Data Structure (`OverteClient.hpp`) + +**Added `EntityType` enum:** +```cpp +enum class EntityType { + Unknown, Box, Sphere, Model, Shape, Light, Text, + Zone, Web, ParticleEffect, Line, PolyLine, Grid, Gizmo, Material +}; +``` + +**Extended `OverteEntity` structure:** +```cpp +struct OverteEntity { + std::uint64_t id{0}; + std::string name; + glm::mat4 transform{1.0f}; + + // NEW: Visual properties + EntityType type{EntityType::Box}; + std::string modelUrl; // For Model type entities + std::string textureUrl; // Texture/material URL + glm::vec3 color{1.0f, 1.0f, 1.0f}; // RGB color (0-1 range) + glm::vec3 dimensions{0.1f, 0.1f, 0.1f}; // Size/scale in meters + float alpha{1.0f}; // Transparency (0-1) +}; +``` + +### 2. Enhanced Entity Packet Parser (`OverteClient.cpp`) + +**Updated `parseEntityPacket()` to extract:** +- Entity type classification +- Model URLs (for 3D models) +- Texture URLs +- RGB color values +- Dimensions/scale +- Alpha transparency + +**Enhanced simulation mode** with diverse entity types: +- Red cube (Box type) +- Green sphere (Sphere type) +- Blue model placeholder (Model type) + +### 3. Rust Bridge Visual Properties (`bridge/src/lib.rs`) + +**Extended `Node` structure:** +```rust +struct Node { + id: u64, + name: String, + transform: Mat4, + entity_type: u8, // 0=Unknown, 1=Box, 2=Sphere, 3=Model + model_url: String, + texture_url: String, + color: [f32; 4], // RGBA + dimensions: [f32; 3], // xyz dimensions +} +``` + +**Added new command types:** +- `SetModel` - Set 3D model URL +- `SetTexture` - Set texture/material URL +- `SetColor` - Set RGBA color +- `SetDimensions` - Set xyz dimensions +- `SetEntityType` - Set geometric primitive type + +**New C-ABI export functions:** +- `sdxr_set_node_model(id, model_url)` +- `sdxr_set_node_texture(id, texture_url)` +- `sdxr_set_node_color(id, r, g, b, a)` +- `sdxr_set_node_dimensions(id, x, y, z)` +- `sdxr_set_node_entity_type(id, type)` + +### 4. Enhanced Rendering (`bridge/src/lib.rs` - `reify()`) + +**Implemented type-specific rendering:** + +- **Box entities (type 1):** Colored wireframe cubes using entity color +- **Sphere entities (type 2):** Three orthogonal wireframe circles (XY, XZ, YZ planes) in entity color +- **Model entities (type 3):** Distinctive octahedron wireframe (placeholder for actual model loading) +- **Unknown types:** Gray wireframe cube as fallback + +**Features:** +- Respects entity dimensions for accurate sizing +- Uses entity-specific colors +- Applies proper transforms (position, rotation, scale) +- Maintains visual distinction between entity types + +### 5. StardustBridge C++ Interface (`StardustBridge.hpp/.cpp`) + +**Added methods:** +```cpp +bool setNodeModel(NodeId id, const std::string& modelUrl); +bool setNodeTexture(NodeId id, const std::string& textureUrl); +bool setNodeColor(NodeId id, const glm::vec3& color, float alpha = 1.0f); +bool setNodeDimensions(NodeId id, const glm::vec3& dimensions); +bool setNodeEntityType(NodeId id, uint8_t entityType); +``` + +**Updated dynamic library loader** to resolve new function pointers from Rust bridge. + +### 6. SceneSync Integration (`SceneSync.cpp`) + +**Enhanced entity synchronization:** +- Propagates entity type on creation/update +- Sets color and alpha properties +- Configures dimensions +- Handles model and texture URLs +- Updates visual properties when entities change + +## Testing + +### Simulation Mode + +Run with simulation mode to see example entities: + +```bash +export STARWORLD_SIMULATE=1 +./build/stardust-overte-client +``` + +This creates three demo entities: +- **CubeA** - Red wireframe cube (20cm) +- **SphereB** - Green wireframe sphere (15cm) +- **ModelC** - Blue octahedron representing a model entity (25cm) + +### Live Overte Connection + +To connect to a real Overte server: + +```bash +# Optional: Set credentials +export OVERTE_USERNAME=your_username + +# Connect to server +./build/stardust-overte-client ws://domain.example.com:40102 +``` + +The client will: +1. Perform domain handshake +2. Discover entity server via DomainList packets +3. Send EntityQuery to request all entities +4. Parse and render entities with full visual properties + +## Architecture + +``` +┌─────────────────┐ +│ Overte Server │ +│ (Entity Server) │ +└────────┬────────┘ + │ UDP Packets (EntityAdd, EntityEdit) + │ Contains: type, position, rotation, dimensions, + │ color, modelUrl, textureUrl + ▼ +┌─────────────────────┐ +│ OverteClient.cpp │ +│ - parseEntityPacket│ +│ - OverteEntity │ +└────────┬────────────┘ + │ consumeUpdatedEntities() + ▼ +┌─────────────────────┐ +│ SceneSync.cpp │ +│ - Maps Overte IDs │ +│ to Stardust IDs │ +└────────┬────────────┘ + │ createNode(), setNodeColor(), + │ setNodeModel(), etc. + ▼ +┌─────────────────────┐ +│ StardustBridge.cpp │ +│ - C++ wrapper │ +│ - dlopen bridge.so │ +└────────┬────────────┘ + │ C-ABI calls: sdxr_set_node_* + ▼ +┌─────────────────────┐ +│ bridge/lib.rs │ +│ - Command queue │ +│ - Shared state │ +└────────┬────────────┘ + │ BridgeState::reify() + ▼ +┌─────────────────────┐ +│ Stardust Server │ +│ - Lines rendering │ +│ - Type-specific │ +│ visualization │ +└─────────────────────┘ +``` + +## Future Enhancements + +### Short Term +1. **Actual model loading** - Replace octahedron with loaded .glb/.fbx models using Stardust Model nodes +2. **Texture application** - Apply textures to rendered entities +3. **More entity types** - Add support for Light, Text, PolyLine, etc. +4. **Performance optimization** - Batch updates, reduce command overhead + +### Medium Term +1. **Full mesh rendering** - Move beyond wireframes to solid shaded meshes +2. **Material support** - PBR materials with metallic/roughness +3. **Animation** - Skeletal animation for rigged models +4. **LOD system** - Level-of-detail based on distance + +### Long Term +1. **Physics sync** - Real-time physics state synchronization +2. **Script integration** - Execute Overte entity scripts in Stardust context +3. **Avatar rendering** - Full avatar mesh and animation support +4. **Audio spatialization** - 3D positional audio from Overte + +## Build Instructions + +```bash +# Build Rust bridge +cd bridge +cargo build +cd .. + +# Build C++ client +cd build +cmake .. +make +cd .. + +# Run +./build/stardust-overte-client +``` + +## Dependencies + +- **C++17** - Modern C++ features +- **GLM** - Math library for vectors/matrices +- **OpenSSL** - MD5 hashing for protocol signatures +- **Rust nightly** - For Stardust bridge +- **stardust-xr-asteroids** - Declarative UI framework +- **stardust-xr-fusion** - Low-level client API +- **tokio** - Async runtime + +## References + +- Overte protocol: `third_party/overte-src/libraries/networking/` +- Entity types: Based on Overte `EntityTypes.h` +- Packet formats: Overte NLPacket specification +- Stardust API: https://github.com/StardustXR/core