MayaTheShy/Starworld
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
main
Starworld/docs/DEVELOPER_GUIDE.md

13 KiB
Raw Permalink Blame History

Future Risks & Priorities (6–12 Months)

As Starworld grows, the following risks and challenges will become increasingly important for developers:

  • Performance Scaling: O(n) transform updates and distance queries will not scale for large scenes. Deep hierarchies and lack of spatial indexing will limit performance.
  • Memory/Data Layout: Inefficient memory layout or fragmentation from dynamic hierarchies can degrade performance.
  • Concurrency & Synchronization: Multithreading will require robust synchronization to avoid race conditions and data corruption.
  • Lifecycle & Ownership: Deletion and reparenting in hierarchies can cause dangling references or orphaned children, especially across FFI boundaries.
  • API Misuse: Cycles or invalid parent/child states can cause subtle bugs or crashes if not checked.
  • Spatial Query Correctness: Queries may become stale or incorrect if transforms are not updated atomically.
  • XR Performance: High frame rates and low latency are critical; entity and query systems must be highly optimized.
  • Security/Authority: In multi-user/networked scenarios, lack of permissions or rate-limiting could allow abuse.
  • Extensibility: As more systems (physics, networking, etc.) are added, poor separation of concerns could hinder future growth.
  • Testing/Debugging: More features and edge cases will make testing and debugging harder without good tools.

Recommended Priorities

  • Spatial Indexing: Integrate a spatial data structure (octree, k-d tree, etc.) for fast queries. Update incrementally as entities move.
  • Hierarchy Traversal: Use dirty flags for transforms and consider breadth-first traversal for deep hierarchies.
  • Lifecycle Invariants: Prevent cycles, define deletion/reparenting behavior, and add runtime checks.
  • FFI Safety: Ensure safe memory and ownership across Rust/C++.
  • Concurrency: Design for thread safety and test with simulated concurrent updates/queries.
  • API Hardening: Clearly document constraints and provide safe helpers/utilities.
  • XR Profiling: Benchmark with XR workloads and optimize for frame time and memory.
  • Security: Add permission models and rate-limiting for entity operations in multi-user mode.
  • Testing & Tooling: Add deep hierarchy/reparenting tests, fuzzing, and scene graph/query inspectors.
  • Architecture: Plan for modularity and extensibility to support future systems cleanly.

Starworld Developer Quick Reference

Build Commands

# Full clean build
./scripts/build_and_test.sh

# Build Rust bridge only
cd bridge && cargo build --release

# Build C++ client only
cmake --build build --target starworld -j$(nproc)

# Rebuild bridge (debug mode)
cd bridge && cargo build

# Clean everything
rm -rf build bridge/target

Run Commands

# Simulation mode (no Overte server needed)
STARWORLD_SIMULATE=1 ./build/starworld

# Connect to local domain
./build/starworld --overte=127.0.0.1:40104

# Connect to remote domain
./build/starworld --overte=domain.example.com:40104

# Domain discovery
./build/starworld --discover

# With verbose logging
RUST_LOG=debug ./build/starworld

Test Commands

# Run all tests
./scripts/ci-test.sh

# Run C++ tests only
./build/starworld-tests

# Run Rust tests only
cd bridge && cargo test

# Check code style
cd bridge && cargo fmt --check
cd bridge && cargo clippy

Debugging

Check Domain Server Status

# Is domain server running?
ps aux | grep domain-server

# What ports is it listening on?
sudo ss -ulnp | grep domain-server

# Check for assignment clients
ps aux | grep assignment-client

Check StardustXR Connection

# Is Stardust server running?
ps aux | grep stardust

# Find Stardust socket
ss -lx | grep stardust

# Check Stardust logs
journalctl --user -u stardust -f

Debug Overte Connection

# Test network connectivity
ping 127.0.0.1
nc -zvu 127.0.0.1 40104  # If nc available

# Capture Overte packets (requires root)
sudo tcpdump -i lo -n udp port 40104 -X

# Run with connection logging
./build/starworld --overte=127.0.0.1:40104 2>&1 | tee connection.log

Check Model Cache

# List cached primitive models
ls -lh ~/.cache/starworld/primitives/

# List downloaded models
ls -lh ~/.cache/starworld/models/

# Check export log
cat ~/.cache/starworld/primitives/export_log.txt

# Regenerate primitives
python3 tools/blender_export_simple.py

Common Issues

"Failed to connect to StardustXR compositor"

# Start StardustXR server first
stardust-xr-server &

# Or check if it's running
ps aux | grep stardust

"Rust bridge present but start() failed"

# Rebuild bridge
cd bridge && cargo build --release

# Check library exists
ls -lh bridge/target/release/libstardust_bridge.so

# Check for missing dependencies
ldd build/starworld

"Retrying domain handshake..."

# Domain server not responding - check if running
ps aux | grep domain-server

# Try local server first
./build/starworld --overte=127.0.0.1:40104

# Check firewall for remote domains
sudo ufw status

"Removing silent node" / Connection dropped after 11-18 seconds

# This is a known HMAC verification issue - see NETWORK_PROTOCOL_INVESTIGATION.md
# The client implementation is correct; the issue is server-side

# Check server logs for hash mismatch
sudo journalctl -u overte-domain-server --since "1 minute ago" | grep -i hash

# Workaround options:
# 1. Contact server admin to disable HMAC verification
# 2. Try a different Overte server
# 3. Wait for server-side fix

# Debug packet sending
./build/starworld --overte=127.0.0.1:40104 2>&1 | grep -E "(Ping|Local ID|periodic)"

Nothing renders in XR

# Check if entities exist
RUST_LOG=debug ./build/starworld 2>&1 | grep -i entity

# Use simulation mode
STARWORLD_SIMULATE=1 ./build/starworld

# Check bridge logs
./build/starworld 2>&1 | grep '\[bridge'

File Locations

Source Code

  • src/main.cpp - Entry point, argument parsing
  • src/OverteClient.cpp - Overte protocol implementation
  • src/StardustBridge.cpp - C++/Rust bridge interface
  • src/SceneSync.cpp - Entity synchronization
  • src/NLPacketCodec.cpp - Packet encoding/decoding
  • src/QDataStream.cpp - Qt serialization
  • bridge/src/lib.rs - Rust StardustXR client

Configuration

  • CMakeLists.txt - C++ build configuration
  • bridge/Cargo.toml - Rust build configuration
  • .gitea/workflows/ - CI/CD pipelines

Documentation

  • README.md - Main documentation
  • docs/NETWORK_PROTOCOL_INVESTIGATION.md - Detailed protocol analysis and HMAC issue
  • docs/OVERTE_AUTH.md - OAuth implementation guide
  • docs/OVERTE_ASSIGNMENT_CLIENT_TASK.md - Protocol details
  • docs/CHANGELOG.md - Version history
  • docs/IMPLEMENTATION_COMPLETE.md - Feature implementation status
  • docs/ENTITY_RENDERING_ENHANCEMENTS.md - Rendering implementation
  • MODELCACHE_IMPLEMENTATION.md - Asset pipeline

Tests

  • tests/TestHarness.cpp - C++ unit tests
  • bridge/src/lib.rs - Rust unit tests (inline)

Key Environment Variables

Variable Purpose Example
STARWORLD_SIMULATE Enable simulation mode 1
STARWORLD_BRIDGE_PATH Override bridge location ./bridge/target/release
OVERTE_UDP_PORT Override UDP port 40104
OVERTE_DISCOVER Enable domain discovery 1
RUST_LOG Rust logging level debug
STARDUSTXR_SOCKET Override Stardust socket /run/user/1000/stardust-socket

Protocol Quick Reference

Port Numbers

  • 40102: HTTP/WebSocket (domain server web interface)
  • 40104: UDP (domain server main communication)
  • Dynamic: Assignment clients (EntityServer, AudioMixer, etc.)

Packet Types

  • 0x1F (31): DomainConnectRequest
  • 0x02 (2): DomainList reply
  • 0x03 (3): DomainListRequest
  • 0x05 (5): Ping
  • 0x06 (6): PingReply
  • 0x10 (16): EntityQuery

Connection Flow

1. Client → Domain: DomainConnectRequest (UDP 40104)
2. Domain → Client: DomainList (session UUID, local ID, assignment clients)
3. Client → Domain: Periodic Ping (keep-alive)
4. Client → EntityServer: EntityQuery (or domain as fallback)
5. EntityServer → Client: EntityData (not yet implemented)

Architecture Diagram

┌─────────────────┐
│  StardustXR     │
│  Compositor     │
└────────┬────────┘
         │ Unix Socket
┌────────▼────────┐
│  Rust Bridge    │
│  (stardust-xr-  │
│   fusion)       │
└────────┬────────┘
         │ C ABI (dlopen)
┌────────▼────────┐
│  StardustBridge │
│  (C++)          │
└────────┬────────┘
         │
┌────────▼────────┐
│  OverteClient   │
│  (C++)          │
└────────┬────────┘
         │ UDP (NLPacket)
┌────────▼────────┐
│  Overte Domain  │
│  Server         │
└─────────────────┘

Useful Commands

# Find all TODO comments
grep -rn "TODO\|FIXME" src/ bridge/src/

# Count lines of code
cloc src/ bridge/src/

# Generate compile_commands.json
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON build

# Format Rust code
cd bridge && cargo fmt

# Update dependencies
cd bridge && cargo update

# Check for security vulnerabilities
cd bridge && cargo audit

# Build documentation
cd bridge && cargo doc --open

Performance Profiling

# Build with debug symbols
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo build
make -j$(nproc)

# Run with perf
perf record -g ./build/starworld
perf report

# Memory profiling with valgrind
valgrind --leak-check=full ./build/starworld

# Rust flamegraph (requires cargo-flamegraph)
cd bridge && cargo flamegraph

Git Workflow

# Start new feature
git checkout -b feature/my-feature

# Make changes, commit
git add src/OverteClient.cpp
git commit -m "Add entity color support"

# Push and create PR
git push origin feature/my-feature
# Then create PR in Gitea web UI

# Update from main
git fetch origin
git rebase origin/main

Resources

StardustXR

Overte

Overte Protocol Documentation

  • Networking Library: https://github.com/overte-org/overte/tree/master/libraries/networking
  • NLPacket: libraries/networking/src/NLPacket.h - Packet format
  • NodeList: libraries/networking/src/NodeList.cpp - Domain connection
  • PacketHeaders: libraries/networking/src/PacketHeaders.h - Packet types
  • EntityServer: assignment-client/src/entities/EntityServer.cpp - Entity serving
  • DomainServer: domain-server/src/DomainServer.cpp - Domain coordination

Development Tools & Libraries

Technical Specifications

Community & Support

Reference in New Issue View Git Blame Copy Permalink