Initial upload

docs/stardust-protocol/_category_.json

@@ -0,0 +1,4 @@
+{
+	"label": "Protocol",
+	"position": 2
+}

docs/stardust-protocol/client/_category_.json

@@ -0,0 +1,4 @@
+{
+	"label": "Client",
+	"position": 2
+}

docs/stardust-protocol/client/initial-connection.md

@@ -0,0 +1 @@
+# Initial Client Connection

docs/stardust-protocol/protocol.md

@@ -0,0 +1,72 @@
+---
+sidebar_position: 1
+---
+
+# Protocol
+## Data
+Every message uses flatbuffers and most use flexbuffers to add variant data such as method arguments and return values. The flatbuffers Message type is:
+
+```cpp
+namespace StardustXR;
+
+table Message {
+	type: ubyte;
+	id: uint;
+	object: string;
+	method: string;
+	error: string;
+	data:[ubyte] (flexbuffer);
+}
+
+root_type Message;
+```
+
+## Message Types
+All fields not accounted for are assumed to be not included or `0` or `""`
+### Error
+| Field Name | Value |
+|-:|:-|
+| `type` | `0` |
+| `id` | `0` or ID of method call |
+| `object` | Path to the object the error comes from |
+| `method` | What method caused the error (if applicable, otherwise this is an empty string) |
+| `error` | String containing the error message itself |
+| `data` | Empty vector |
+
+### Signal
+Signals are method calls that do not expect a response of any kind.
+| Field Name | Value |
+|-:|:-|
+| `type` | `1` |
+| `object` | Path to object to send signal to |
+| `method` | Name of the method to execute |
+| `data` | Flexbuffer containing arguments to pass to remote method |
+
+### Method Call
+Method calls allow for remote procedure calling and should block the thread they are sent from until a response is heard.
+| Field Name | Value |
+|-:|:-|
+| `type` | `2` |
+| `id` | Dynamically generated ID to identify method return for this method |
+| `object` | Path to object to execute method on |
+| `method` | Name of the method to execute |
+| `data` | Flexbuffer containing arguments to pass to remote method |
+
+### Method Return
+Message to pass the return value back to the program.
+| Field Name | Value |
+|-:|:-|
+| `type` | `3` |
+| `id` | The same as the message call's ID |
+| `data` | Flexbuffer containing return value |
+
+## Data Types
+Stardust does not support the `Map` or `FixedTypedVector` flexbuffers datatypes.
+### Color
+A `Vector` of 4 `float` is treated as a `Color` in the form `RGBA`.
+### Vector2
+A `Vector` of 2 `double` is treated as a `Vector2`.
+### Vector3
+A `Vector` of 3 `double` is treated as a `Vector3`.
+### Quaternion
+A `Vector` of 4 `double` is treated as a `Quaternion`.

docs/stardust-protocol/server/_category_.json

@@ -0,0 +1,4 @@
+{
+	"label": "Server",
+	"position": 3
+}

docs/stardust-protocol/server/objects/_category_.json

@@ -0,0 +1,4 @@
+{
+	"label": "Server Objects",
+	"position": 2
+}

docs/stardust-protocol/server/objects/environment.md

@@ -0,0 +1,11 @@
+# /environment
+
+## Signals
+### `visible(bool enable)`
+Enable or disables the skybox visibility (lighting remains active). This is not meant to be used for an additive AR mode as StereoKit will perform that when the OpenXR runtime indicates the layer's blend mode is additive.
+
+### `setSkytex(string path)`
+Sets the sky texture to be the image (HDR) at the absolute `path`.
+
+### `setLighting(string path)`
+Sets the ambient lighting (using spherical harmonics) to the HDR at the absolute `path`.

docs/stardust-protocol/server/objects/field.md

@@ -0,0 +1,11 @@
+# /field
+
+Object managing fields for interaction.
+
+## Signals
+### `createBoxField(string name, Vector3 origin, Quaternion orientation, Vector3 size)`
+Creates a new [[BoxField]] at `/field/[name]`.
+
+
+### `createSphereField(string name, Vector3 origin, float radius)`
+Creates a new [[SphereField]] at `/field/[name]`.

docs/stardust-protocol/server/objects/hmd.md

@@ -0,0 +1,9 @@
+# /hmd
+
+Reference object to the HMD being used to render content and track head position.
+
+## Signals
+### `setPosition(Vector3 position)`
+### `setRotation(Quaternion rotation)`
+### `setPose(Vector3 position, Quaternion rotation)`
+### `setIPD(double meters)`

docs/stardust-protocol/server/objects/input.md

@@ -0,0 +1,10 @@
+# /input
+
+Object managing input and interaction.
+
+## Signals
+### `registerPointerInput(string name, Vector3 origin, Vector3 direction, float tilt)`
+Creates a new [[Pointerinput]] at `/input/methods/[name]` with the origin, direction, and tilt specified.
+
+### `registerInputHandler(string name, string field, string spacePath, Vector3 position, Quat rotation, string callbackPath, string callbackMethod)`
+Creates a new [[InputHandler]] with the specified `name` and `field`, located relative to the space referenced in `spacePath` and the `postion` + `rotation`.

docs/stardust-protocol/server/objects/lifecycle.md

@@ -0,0 +1,10 @@
+# /lifecycle
+Object representing the life cycle of the Stardust server, from start to each frame to end.
+
+## Signals
+### `subscribeLogicStep(string path, string method)`
+Executes `method` on object at `path` on the client when the main logic loop of the server starts each frame.
+
+`method` must have arguments of `(double delta, double timeToRender)`. 
+
+`delta` gives the time in seconds between the current time and the last time the logic loop executed while `timeToRender` gives the time in ms since epoch when the current frame is expected to reach the user's eyes.

docs/stardust-protocol/server/objects/model.md

@@ -0,0 +1,7 @@
+# /model
+
+Object managing graphical models.
+
+## Signals
+### `createFromFile(string name, string path)`
+Creates a new [[Model]] at `/model/[name]`.

docs/stardust-protocol/server/objects/spatial.md

@@ -0,0 +1,7 @@
+# /spatial
+
+Object managing spatial nodes for parenting.
+
+## Signals
+### `create(string name, Vector3 origin, Quat orientation, Vector3 scale, bool translatable, bool rotatable, bool scalable)`
+Creates a new [[Spatial]] at `/spatial/[name]`.

docs/stardust-protocol/server/types/Model.md

@@ -0,0 +1,37 @@
+---
+sidebar_position: 3
+---
+
+# Model
+### Derived from [[Spatial]]
+A graphical model.
+
+## Properties
+### `setOrigin(Vector3 origin)`
+Sets the origin to `origin`. 
+
+### `setOrientation(Quat orientation)`
+Sets the orientation to `orientation`. 
+
+### `setPose(Vector3 origin, Quat orientation)`
+Sets the pose using `origin` and `orientation` absolutely.
+
+### `destroy()`
+Destroys the model instance. Resources like the mesh and material and textures may be in memory still, but the resource manager will clean it up if needed.
+
+### `setMaterialProperty(uint32 submeshIndex, string propertyName, float value)`
+### `setMaterialProperty(uint32 submeshIndex, string propertyName, Color value)`
+### `setMaterialProperty(uint32 submeshIndex, string propertyName, string value)`
+Sets the material property `propertyName` on this model's submesh at `submeshIndex` to `value`. Valid material names are:
+
+| Name         | Type         | Description                                                                                  |
+|--------------|--------------|----------------------------------------------------------------------------------------------|
+|`color`       | `Color`      | Tint/multiply the shader's output color by this amount. (default is opaque white)            |
+|`roughness`   | `float`      | Scale the metal texture's roughness by this amount                                           |
+|`metallic`    | `float`      | Scale the metal texture's metalness by this amount                                           |
+|`tex_scale`   | `float`      | Scale the UV coordinates of all textures by this amount                                      |
+|`diffuse`     | `string`     | Path to the diffuse texture of the material (if it has one).                                 |
+|`emission`    | `string`     | Path to the emission texture of the material (if it has one).                                |
+|`metal`       | `string`     | Path to the metalness texture of the material (if it has one).                               |
+|`normal`      | `string`     | Path to the normal map texture of the material (if it has one).                              |
+|`occlusion`   | `string`     | Path to the occlusion map texture of the material (if it has one).                           |

docs/stardust-protocol/server/types/Node.md

@@ -0,0 +1,10 @@
+---
+sidebar_position: 1
+---
+
+# Node
+A base type for all othertypes that exist in 3D space.
+
+## Methods
+### `destroy(null)`
+Destroy this node immediately, as well as all its children if it has any.

docs/stardust-protocol/server/types/Spatial.md

@@ -0,0 +1,47 @@
+---
+sidebar_position: 2
+---
+
+# Spatial
+### Derived from [[Node]]
+A base type for types that exist in 3D space, always relative to another space or the engine's world space. Spatials are always relative because in AR or XXR reference spaces change constantly in response to new environments.
+
+## Internal Properties
+Sometimes it makes sense for a Spatial to not be scalable (like in [[Field]]s where scaling dramatically increases the number of steps to raymarch for pointers) or not be rotatable (point lights) or such, so these are disabled for certain Spatial-derived objects, mentioned right after "Derived from [[Spatial]]".
+### `translatable: bool`
+If false, object's origin cannot be modified from a client.
+### `rotatable: bool`
+If false, object's rotation cannot be modified from a client (e.g. point lights, point sound sources).
+### `scalable: bool`
+If false, object's scale cannot be modified from a client (e.g. [[Field]]s, [[PointerInput]]s).
+
+## Methods
+### `move(Vector3 position)`
+Moves the Spatial relative to itself if `translatable` is not `false`.
+
+### `rotate(Quat rotation)`
+Rotates the Spatial relative to itself if `rotatable` is not `false`.
+
+### `rotateAround(Vector3 point, Quat rotation)`
+Rotates the Spatial around `point` (relative to itself) if `rotatable` is not `false`.
+
+### `scale(float scale)`
+Multiplies the Spatial's scale vector by `scale` if `scalable` is not `false`.
+
+### `setOrigin(Vector3 origin)`
+Sets the origin to `origin` if `translatable` is not `false`.
+
+### `setOrientation(Quat orientation)`
+Sets the orientation to `orientation` if `rotatable` is not `false`.
+
+### `setScale(Vector3 scale)`
+Sets the scale to `scale` if `scalable` is not `false`.
+
+### `setPose(Vector3 origin, Quat orientation)`
+Sets the pose relative to the Spatial's space if `translatable` and `rotatable` are true. This method saves on IPC calls compared to `setOrigin` and `setOrientation` in sequence.
+
+### `setTransform(Vector3 origin, Quat orientation)`
+Sets the transform relative to the Spatial's space if `translatable`, `rotatable` and `scalable` are true. This method saves on IPC calls compared to `setOrigin`, `setOrientation`, and `setScale` in sequence.
+
+### `setSpatialParent(string parentPath)`
+Sets the spatial parent of this `Spatial` while keeping the "absolute" position in space intact.

docs/stardust-protocol/server/types/_category_.json

@@ -0,0 +1,4 @@
+{
+	"label": "Node Types",
+	"position": 1
+}

docs/stardust-protocol/server/types/field/BoxField.md

@@ -0,0 +1,6 @@
+# BoxField
+Derived from [[Field]], this Field is in a rectangular prism shape.
+
+## Properties
+### `setSize(Vector3 size)`
+Sets the size of the box.

docs/stardust-protocol/server/types/field/Field.md

@@ -0,0 +1,13 @@
+# Field
+### Derived from [[Spatial]]
+Type that contains methods all Fields share. All fields created are under  `/field`.
+
+## Methods
+### `double distance(string spacePath, Vector3 point)`
+Returns the distance from `point` to the surface of the object, positive if the point is outside and negative if inside. When `point` is far enough away the object will give an approximate distance to the origin instead of the surface.
+
+### `Vector3 normal(string spacePath, Vector3 point)`
+Returns the normal of `point` compared to the surface of the object. Normal always points toward object.
+
+### `Vector3 closestPoint(string spacePath, Vector3 point)`
+Returns the closest point from `point` to the surface of the object. Use this instead of `distance` and `normal` when best to limit IPC calls.

docs/stardust-protocol/server/types/field/SphereField.md

@@ -0,0 +1,6 @@
+# SphereField
+Derived from [[Field]], this Field is in a sphere shape.
+
+## Properties
+### `setRadius(double size)`
+Sets the radius of the sphere.

docs/stardust-protocol/server/types/field/_category_.json

@@ -0,0 +1,4 @@
+{
+	"label": "Fields",
+	"position": 4
+}

docs/stardust-protocol/server/types/input/InputHandler.md

@@ -0,0 +1,23 @@
+---
+sidebar_position: 1
+---
+# InputHandler
+## Derived from [[Spatial]] (`scalable: false`)
+
+A handler to allow clients to recieve input.
+
+## Methods
+### `setOrigin(Vector3 origin)`
+Sets the origin to `origin`. 
+
+### `setOrientation(Quat orientation)`
+Sets the orientation to `orientation`. 
+
+### `setPose(Vector3 origin, Quat orientation)`
+Sets the pose using `origin` and `orientation` relative to the pose's space.
+
+### `setField(string field)`
+Sets the field this input handler will use.
+
+### `setCallback(string callbackPath, string callbackMethod)`
+Sets the callback this input handler will use.

docs/stardust-protocol/server/types/input/InputMethod.md

@@ -0,0 +1,49 @@
+---
+sidebar_position: 2
+---
+# InputMethod
+Represents an input device generically, with specifics being fleshed out in subclasses.
+
+## Internal Properties
+### `type: uint8`
+|   | Name         | Description                                                                                                 |
+|---|--------------|-------------------------------------------------------------------------------------------------------------|
+| 0 | `Global`     | A non-spatial input, used for power buttons and system volume and such                                      |
+| 1 | `Controller` | Contains a pose and datamap for buttons, trackpad, joystick, grip, trigger, etc.                            |
+| 2 | [[PointerInput]]    | Defined by origin, direction, and tilt with datamap for buttons, trackpad, etc.                             |
+| 3 | [[HandInput]]       | Contains a full 27-bone hand skeleton (including forearm) and datamap for abstractions like grip, pinch distance/point, etc. |
+
+### `datamap: Dictionary`
+See #Datamap:
+
+## Internal Methods
+### `distanceToField(string field)`
+Returns the distance from this input to a field in meters. `field` is the name of the field, not the full path.
+
+### `serialize()`
+Returns a serialized version of the InputMethod, each serialized schema is shown under #Serialization in the respective Input subclasses.
+
+## Serialization
+| Key       | Value Type   |
+|-----------|-------------|
+| type    | `int`    |
+| distance    | `float`    |
+
+## Datamap
+The datamap is a dictionary containing abstracted and raw data related to the input that clients can use to analyze input in the most optimal way. It has keys of type `Key`(Flexbuffers-specific, in any other language use a `String`) and values:
+
+| Type      | Example Uses                     | Usage Notes                                                                                                                                        |
+|-----------|----------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|
+| `bool`    | Digital buttons                  | If values are analog, use a `float` instead with `1.0` being considered `true`.                                                                      |
+| `float`   | Analog trigger, grip             | For values such as grip/trigger, make the range `0.0`-`1.0`, for something like a scroll wheel `-0.5`-`0.5`.                                       |
+| `Vector2` | Touchpad, joystick               | For values such as trackpad and joystick, make each component in the range of `-0.5`-`0.5` and calculate+apply deadzone before setting these values. |
+| `Vector3` | Pinch position, grip axis vector | For points and directions, make this a value inside the  InputMethod's local space. Raw hand pose data should not be put in the datamap.           |
+
+Standard supported datamap keys will be put in the individual type's documentation, however you can put more data in and some input handlers may be able to use that data for better interactions.
+
+## Methods
+### `setPosition(Vector3 point)`
+Sets the position of this [[InputMethod]]. This pose also defines the main interaction point.
+
+### `modifyDatamap(Dictionary data)`
+Modifies the datamap of this [[InputMethod]]. Each key here will add to the datamap if not present and set the existing value if present. Any key not present in `data` will be unaffected.

docs/stardust-protocol/server/types/input/_category_.json

@@ -0,0 +1,4 @@
+{
+	"label": "Input",
+	"position": 5
+}

docs/stardust-protocol/server/types/input/inputtypes/HandInput.fbs

@@ -0,0 +1,10 @@
+include "common.fbs";
+namespace StardustXR;
+
+table Hand {
+	finger_joints:[joint] (required);
+
+	palm:joint (required);
+	wrist:joint (required);
+	elbow:joint;
+}

docs/stardust-protocol/server/types/input/inputtypes/HandInput.md

@@ -0,0 +1,57 @@
+# HandInput
+### Derived from [[InputMethod]] and [[Spatial]] (`scalable: false`)
+
+A full 27-bone hand+forearm input with useful abstractions provided through the datamap.
+
+The local space of a hand is where +Z is away from the palm, +Y is from the palm to the fingers, and +X is orthogonal and to the right.
+
+## Serialization
+| Key       | Value Type   |
+|-----------|--------------|
+| origin    | `Vector3`    |
+| direction | `Vector3`    |
+| tilt      | `float`      |
+| datamap   | `Dictionary` |
+
+```js
+[
+	thumb metacarpal,
+	thumb proximal,
+	thumb distal,
+	thumb tip,
+	index metacarpal,
+	index proximal,
+	index intermediate,
+	index distal,
+	index tip,
+	middle metacarpal,
+	middle proximal,
+	middle intermediate,
+	middle distal,
+	middle tip,
+	ring metacarpal,
+	ring proximal,
+	ring intermediate,
+	ring distal,
+	ring tip,
+	little metacarpal,
+	little proximal,
+	little intermediate,
+	little distal,
+	little tip
+]
+```
+(This section is WIP)
+
+## Datamap
+### Required
+| Key             | Value Type                  | Description                                                        |
+|-----------------|-----------------------------|--------------------------------------------------------------------|
+| confidence      | `float` (range `0.0`-`1.0`) | How confident the hand tracker is of the pose of the hand          |
+| isLeft   | `bool` | `true` if this hand is the left hand, `false` if not |
+| pinchStrength   | `float` (range `0.0`-`1.0`) | How much the hand is pinching                                      |
+| pinchDistance   | `float` (>`0.0`)            | The distance in meters between the thumb and index finger |
+| grabStrength    | `float` (range `0.0`-`1.0`) | How much the hand is making a fist                                 |
+
+## Methods
+**TODO**: Properly secured field distance calculation accessible to clients

docs/stardust-protocol/server/types/input/inputtypes/Input.fbs

@@ -0,0 +1,16 @@
+include "PointerInput.fbs";
+include "HandInput.fbs";
+namespace StardustXR;
+
+union InputDataRaw {
+	Pointer,
+	Hand
+}
+
+table InputData {
+	input:InputDataRaw (required);
+	distance:float = 0;
+	datamap:[ubyte] (flexbuffer);
+}
+
+root_type InputData;

docs/stardust-protocol/server/types/input/inputtypes/PointerInput.fbs

@@ -0,0 +1,8 @@
+include "common.fbs";
+namespace StardustXR;
+
+table Pointer {
+	origin:vec3 (required);
+	direction:vec3 (required);
+	tilt:float = 0;
+}

docs/stardust-protocol/server/types/input/inputtypes/PointerInput.md

@@ -0,0 +1,36 @@
+# PointerInput
+### Derived from [[InputMethod]]
+
+An input method for pointers, such as the Huawei VR Glass's remote, a pointer put on an Index controller, Oculus style laser mouse, etc.
+
+The local space for a pointer is where +Z is along the pointer's direction, +X is to the right of the pointer when laying face up, and +Y is upward when laying face up.
+
+## Serialization
+| Key       | Value Type   |
+|-----------|--------------|
+| origin    | `Vector3`    |
+| direction | `Vector3`    |
+| tilt      | `float`      |
+| datamap   | `Dictionary` |
+
+## Datamap
+### Required
+| Key    | Value Type                                | Description                                                                                                                                                                     |
+|--------|-------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| select | `float` (range `0.0`-`1.0`)               | Whether the pointer is attempting to select an object, usually through trigger. 1.0 is selected, 0.0 is not selected, and inbetween is to add smoothness to the interaction.    |
+| grab   | `float` (range `0.0`-`1.0`)               | How much the pointer is attempting to grab an object. 0.0 is not grabbing at all, 1.0 is grabbing the strongest, and inbetween is to add smoothness.                            |
+| scroll | `Vector2` (component range `-0.5`-`0.5`) | The amount to scroll on this input tick (sensitivity determined on the receiving client). Can be used for 2D scrolling as well as adjusting depth/size when grabbing an object. |
+### Built in
+| Key    | Value Type                                | Description                                                                                                                                                                     |
+|--------|-------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| deepestPointDistance | `float` (>=`0.0`) | The distance along the ray where the deepest surface point resides. |
+
+## Methods
+### `setDirection(Vector3 direction)`
+Sets the direction the pointer is pointing in stage space.
+### `setTilt(float angle)`
+Sets the tilt of the pointer (rotation about the pointer direction) clockwise in radians.
+### `setOrientation(Vector3 direction, float angle)`
+Sets the direction and tilt of the pointer to save on IPC calls.
+### `setPose(Vector3 origin, Vector3 direction, float angle)`
+Sets the origin, direction, and tilt of the pointer to save on IPC calls.

docs/stardust-protocol/server/types/input/inputtypes/common.fbs

@@ -0,0 +1,25 @@
+namespace StardustXR;
+
+struct vec3 {
+	x:float;
+	y:float;
+	z:float;
+}
+
+struct quat {
+	w:float;
+	x:float;
+	y:float;
+	z:float;
+}
+
+struct pose {
+	position:vec3;
+	rotation:quat;
+}
+
+struct joint {
+	position:vec3;
+	rotation:quat;
+	radius:float;
+}

docs/stardust-protocol/server/types/input/inputtypes/convert_cpp.sh

@@ -0,0 +1,2 @@
+#!/usr/bin/env bash
+flatc -c *.fbs $@