StardustXR/website
1
0
Fork 0
Code Issues 3 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Added XR & Hand tracking guides

Browse Source
This commit is contained in:
Cyberneticmelon
2025-03-31 21:42:16 -04:00
parent fcf9718299
commit 84719643b7
9 changed files with 65 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
docs/03-dive-deeper/01-brief-overview.md Normal file
View File

@@ -0,0 +1,97 @@
---
sidebar_position: 1
---
# Brief Overview
Stardust XR is an XR [display server](https://itsfoss.com/display-server) designed for Linux-based systems<!-- (possibly most unix-based too, but untested)-->.<br/>
Unlike traditional display servers that focus on 2D screens, Stardust XR introduces new conventions to utilize unbounded 3D space in virtual or real environments.
## Priorities
- **3D Environment Focus**: While supporting 2D clients through the Wayland protocol, Stardust XR primarily aims to create and manage 3D environments.
- **Standard Interfaces**: Utilizes Wayland and OpenXR for 2D and XR app integration.
- **Stardust Protocol**: Implements a special protocol for:
- Creating virtual objects
- Developing 3D UIs around 2D apps (Panel Shells)
- Handling data as Items that can represent files, Wayland surfaces, etc.
- **Personal Device Interaction**: Focuses on individual user interactions rather than collaborative 3D spaces.
- **Flexible Object Interaction**: Users can choose objects that best suit their current scenario, with easy setup for default configurations.
## Architecture
```mermaid
graph LR
OpenXR
OpenXR -->|Frame Data| StereoKit
OpenXR -->|Input Data| StereoKit
subgraph Server[Stardust Server]
StereoKit[StereoKit]
InternalSceneGraph[Internal Scenegraph]
WC1[Wayland Client 1]
DBusConnection[D-Bus Connection]
ClientSceneGraph -->|Drawable Objects| StereoKit
StereoKit -->|Controller, Hand & HMD Data| InternalSceneGraph
InternalSceneGraph -->|Panel Items| ClientSceneGraph
InternalSceneGraph -->|Controller, Hand & HMD Export| DBusConnection
WC1 -->|Panel Item| InternalSceneGraph
WC1 -->|Textures| StereoKit
StereoKit -->|EGL Context| WC1
ClientSceneGraph --> C1
subgraph C1[Stardust Client 1]
ClientSceneGraph[Client Scenegraph]
end
end
StardustSocket[Stardust Socket]
SessionBus[D-Bus Session Bus]
WaylandSocket[Wayland Socket]
WC1 --> WaylandSocket
WaylandSocket --> WC1
DBusConnection --> SessionBus
C1 --> StardustSocket
subgraph StardustClient[Stardust Client 1]
Fusion
Molecules
UserCode[User Code]
Fusion --> Molecules
Fusion --> UserCode
Molecules --> UserCode
subgraph Fusion
libstardustxr[stardust-xr library]
end
end
StardustSocket --> libstardustxr
SessionBus -->|HMD Spatial Reference| Fusion
WaylandClient[Wayland Client]
WaylandSocket --> WaylandClient
WaylandClient --> WaylandSocket
```
## Projects & Components
- [**telescope**](https://github.com/StardustXR/telescope): Stable stardust setup for quick and easy usage
- [**server**](https://github.com/StardustXR/server): Reference implementation of the StardustXR server (similar to Xorg)
- [**core lib**](https://github.com/StardustXR/core): Rust crates and schemas for connecting stardust server and client
- [**website**](https://github.com/StardustXR/website): The location where documentation is done and the website is generated from
- [**protostar**](https://github.com/StardustXR/protostar): App launcher library and examples including futuristic hexagon launcher
- [**black hole**](https://github.com/StardustXR/black-hole): Universal minimization of Stardust objects
- [**gravity**](https://github.com/StardustXR/gravity): Launch programs with a certain offset in 3D space
- [**magnetar**](https://github.com/StardustXR/magnetar): Example workspaces client
- [**atmosphere**](https://github.com/StardustXR/atmosphere): 3D environment with natural momentum-based movement
- [**comet**](https://github.com/StardustXR/comet): Annotate in 3D wherever you want amongst your other apps!

39
docs/03-dive-deeper/02-deep-overview.md Normal file
View File

@@ -0,0 +1,39 @@
---
sidebar_position: 2
---
# Technical Overview
## Server
The server is what renders all the meshes of each client, takes in input from input methods, and spits out input to input handlers according to the SUIS (Spatial Universal Interaction System). As well, the server is going to be part Wayland compositor and OpenXR runtime so all 2D and XR applications can be used together at once (if the hardware allows). It is the glue of Stardust, allowing any number of clients and apps to work in harmony while allowing the user full control in an intuitive way.
As new technologies come out, keeping the same server and trying to progressively upgrade it may lead to bloat and issues, so the protocol is designed to make creating new servers possible without too much hassle or to make servers modular. Old interfaces can be deprecated without breaking the overall system, and new interfaces can be added if needed due to the object-oriented design. The design is highly inspired by game engines, however the reference server is made in StereoKit due to it being more optimal.
### Reference Server
Stardust has a reference server, but unlike other display servers this reference server is just as functional as any other Stardust server would be, because all the functionality is implemented in the clients. The server is just the glue.
The reference server uses StereoKit as its backend for many reasons:
- Much more lightweight than any other framework/other engine
- It was originally designed to run AR apps on HoloLens, and now it works on Quest
- It's able to render thousands of objects on a HoloLens without breaking a sweat
- Does not include unnecessary features that slow down performance
- Physics and UI are not needed for Stardust, but those do not actively run and can always have an option to not be compiled at all.
- Uses C-style C++ meaning not bytecode or interpreted (however has a C# API that is not used by Stardust)
- Uses immediate mode rendering meaning the logic is many times easier to implement
- Exposes its rendering insides whenever needed
- Very needed for Wayland when it needs to display EGL Wayland surface textures on meshes
- Overriding the depthbuffer per-pixel in fragment shader is useful for XR depth compositing
- Is constantly getting new updates faster than most engines
- World semantics for OpenXR was implemented ~1 week after the official OpenXR extension release
- Is optimized to load resources JIT
- Many game engines are best at caching their assets, but this is not practical in a display server
- Uses OpenXR as its backend out of the box
## IPC
The Stardust XR IPC uses Unix domain sockets for communication and a protocol based on flatbuffers/flexbuffers.
## Protocol
The Stardust XR protocol is object-oriented, depending on a scenegraph on the server end and scenegraph or functionally equivalent structure on the client end. This allows easier implementation of the protocol while still remaining fast, reliable, and extensible.
## Clients
The clients tell the server what models to draw, where they would like to accept input in, etc. Stardust clients are much more suited to widgets or user interfaces to 2D/XR apps.

29
docs/03-dive-deeper/03-make-clients.md Normal file
View File

@@ -0,0 +1,29 @@
---
sidebar_position: 3
---
# Making Clients
A Stardust client is at its core an executable that connects to a stardust server instance at `$XDG_RUNTIME_DIR/stardust-[0-32]`, sends and receives messages according to the Stardust protocol.
## Rust
Stardust uses Rust due to its speed and reliability in build and runtime, so you will need to install cargo (preferably through [rustup](https://rustup.rs/)). If you're unsure how to use Rust then a great guide is at https://doc.rust-lang.org/book/.
## stardust-xr
`stardust-xr` is a connecting library for servers and clients that deals with the lowest level of connecting to ensure basic compatibility is ensured. As a client developer you most likely won't need it, but it's good to know about it.
## Fusion
`stardust-xr-fusion` is the high level library you'll probably be using to create your clients. It features structs for all Stardust objects and trait-based + async systems for handling events from the server.
To get started, create a new Rust project using
```bash
cargo new
```
and include the `stardust-xr-fusion` dependency in your `cargo.toml`. Documentation will be available at https://docs.rs/crate/stardust-xr-fusion/latest.
## Molecules
`stardust-xr-molecules` is an even higher level widget library built on Fusion similar to [MRTK](https://github.com/microsoft/MixedRealityToolkit-Unity) to get your clients up and running faster. Documentation will be available at https://docs.rs/crate/stardust-xr-molecules/latest.

5
docs/03-dive-deeper/09-terminology.md Normal file
View File

@@ -0,0 +1,5 @@
# Glossary
- **Item**: a virtual object that represents data
- **Panel Shells**: 3D UIs around 2D apps
- ...

4
docs/03-dive-deeper/_category_.json Normal file
View File

@@ -0,0 +1,4 @@
{
"label": "Dive deeper",
"position": 3
}