Recording & playback

namespace · AR51.Unity.SDK.Recorder / .Playback Unity / C#

The recording & playback surface lets you persist incoming mocap frames to a file and replay them back through the live consumers. A recording is a sequence of timed keyframes; each parsed line becomes a KeyframeCache holding one or more KeyframeTasks, and executing a task feeds its payload into the matching consumer (SkeletonConsumer, ObjectTransformConsumer, ObjectDetectionConsumer) exactly as if it had arrived live. SkeletonWebStreamer is the related component that pushes visible bone transforms out to a browser viewer.

Units: Unity world-space positions are Vector3 in meters, rotations are Quaternion, and all keyframe/cache times are seconds (capture times are seconds since the Unix epoch, UTC).

Meters, not centimeters

Every position/length on this page is in meters (Unity convention). AR 51 mocap data is already meters — no conversion is applied on the Unity side.

Vector3Keyframe / QuaternionKeyframe

struct Namespace: AR51.Unity.SDK.Recorder.

Simple timed sample structs for recorded animation curves — one timestamped value each.

public struct Vector3Keyframe    { public float Time; public Vector3 Value;    public Vector3Keyframe(float time, Vector3 value); }
public struct QuaternionKeyframe { public float Time; public Quaternion Value; public QuaternionKeyframe(float time, Quaternion value); }

Field	Type	Description
Time	float	sample time, in seconds
Value	Vector3 / Quaternion	the sampled value — positions in meters, rotations as a Unity quaternion

IKeyframeSource

interface Namespace: AR51.Unity.SDK.Playback.

Implement this to remap recorded keyframes during playback — prepend a skeleton-id prefix, apply a spatial transform to joint positions, and offset capture timestamps. Use it to relocate or retime a recorded performance (e.g. replay a recording into a different anchor space, or merge several recordings under distinct id prefixes). A KeyframeTask applies its assigned Source (when set) before executing.

Member	Returns	Description
Prefix	string	prepended to skeleton ids coming from this source
Transform(Vector3[] positions)	Vector3[]	remap an array of joint positions (meters)
Transform(Vector3 position)	Vector3	remap a single position (meters)
Offset(double time)	double	remap a capture timestamp (seconds)

string Prefix { get; }
Vector3[] Transform(Vector3[] positions);
Vector3 Transform(Vector3 position);
double Offset(double time);

KeyframeTask

abstract class Namespace: AR51.Unity.SDK.Playback.

A single unit of playback work. Execute() replays the task into the live consumers — applying its optional Source remap first. The concrete subclasses each wrap one payload type.

public abstract class KeyframeTask
{
    public IKeyframeSource Source { get; set; }
    public abstract void Execute();
}

Property	Type	Description
Source	IKeyframeSource	optional remap applied before Execute() (null = replay as recorded)

Subclasses

Each subclass holds one payload and routes it to the matching consumer on Execute().

Task	Payload	Execute() replays into
ActionKeyframeTask	Action	runs an arbitrary delegate (markers, side-effects)
SkeletonTask	Skeleton	SkeletonConsumer.Instance.OnSkeleton
ObjectTransformTask	ObjectTransform	ObjectTransformConsumer.Instance.OnObject
ObjectDetectionTask	ObjectDetectionReply (transport)	ObjectDetectionConsumer.Instance.OnObjectDetected

public class ActionKeyframeTask : KeyframeTask         { public ActionKeyframeTask(Action action); public override void Execute(); }
public sealed class SkeletonTask : KeyframeTask        { public Skeleton Skeleton { get; }        public SkeletonTask(Skeleton s); public override void Execute(); }
public sealed class ObjectTransformTask : KeyframeTask { public ObjectTransform ObjectTransform { get; } public ObjectTransformTask(ObjectTransform t); public override void Execute(); }
public sealed class ObjectDetectionTask : KeyframeTask { public ObjectDetectionReply ObjectDetectionReply { get; } public ObjectDetectionTask(ObjectDetectionReply t); public override void Execute(); }

ActionKeyframeTask

Wraps an arbitrary Action; Execute() invokes it. Use to schedule custom side-effects (logging, marker callbacks) on the playback timeline.

SkeletonTask

Holds a Skeleton. Execute() applies the (optionally Source-transformed) skeleton to SkeletonConsumer.Instance.OnSkeleton, driving characters exactly as a live frame would.

ObjectTransformTask

Holds an ObjectTransform. Execute() feeds it to ObjectTransformConsumer.Instance.OnObject.

ObjectDetectionTask

Holds an ObjectDetectionReply (transport/proto type). Execute() feeds it to ObjectDetectionConsumer.Instance.OnObjectDetected.

KeyframeCache

sealed class Namespace: AR51.Unity.SDK.Playback.

One parsed line of a recording: timing, source device, the raw JSON, and the decoded KeyframeTasks. KeyframeParser produces these when loading a recording; you can also build one directly from a Skeleton, an ObjectDetectionReply, or a timestamped Action.

Member	Type	Description
Timing
time	double	absolute capture time, seconds since the Unix epoch
timeSinceStart	double	seconds since the recording started
Source
Source	IKeyframeSource	optional remap applied to this line's tasks
device_name	string	originating device name
device_ip	string	originating device IP
json	string	the raw recorded JSON for this line
Decoded payload
Tasks	KeyframeTask[]	the decoded tasks (populated by ParseTasks)
Skeletons	Skeleton[]	skeletons decoded from this line
ObjectDetectionReplies	ObjectDetectionReply[]	detection replies decoded from this line (transport, read-only)

public KeyframeCache(ReadOnlySpan<char> line);
public KeyframeCache(string line);
public KeyframeCache(Skeleton s, double? overrideTimestamp = null);
public KeyframeCache(ObjectDetectionReply objectDetection, double? overrideTimestamp = null);
public KeyframeCache(double timestamp, Action action);

public void ParseTasks(bool parseBinary = false);   // decode json → Tasks
public override string ToString();

ParseTasks(parseBinary) decodes the line's json into the Tasks array; pass parseBinary: true to also decode packed binary payloads.

KeyframeParser

static class Namespace: AR51.Unity.SDK.Playback.

Loads and parses recording files into KeyframeCache lists — multi-threaded, sorting the result by time.

Method	Returns	Description
EstimateKeyframeCount	long	rough keyframe count from a byte length (pre-size buffers)
Parse	List<KeyframeCache>	parse streams synchronously
ParseAsync	IEnumerator	coroutine wrapper around Parse with progress reporting

EstimateKeyframeCount

public static long EstimateKeyframeCount(long length);

Estimates how many keyframes a recording of length bytes contains — use it to pre-size lists/progress bars before parsing.

Parse

public static List<KeyframeCache> Parse(Stream[] fileStreams, int length,
    CancellationToken? token = null, Action<string, float> onProgress = null, IKeyframeSource[] sources = null);

Parses the given file streams into a time-sorted list of KeyframeCache. Runs multi-threaded.

Parameters

fileStreamsStream[]the recording file stream(s) to read

lengthintexpected keyframe count (e.g. from EstimateKeyframeCount)

tokenCancellationToken?optional cancellation

onProgressAction<string, float>progress callback — message + a 0–1 fraction

sourcesIKeyframeSource[]optional per-stream IKeyframeSource remaps

ParseAsync

public static IEnumerator ParseAsync(Stream[] fileStreams, int length, List<KeyframeCache> keyframes,
    CancellationToken? token = null, Action<string, float> onProgress = null, IKeyframeSource[] sources = null);

Coroutine wrapper around Parse: yield-return it from a Unity coroutine and read the results from the keyframes list you pass in. onProgress reports a 0–1 fraction so you can drive a loading bar.

Exampleinferred

var keyframes = new List<KeyframeCache>();
using var fs = File.OpenRead(recordingPath);
int count = (int)KeyframeParser.EstimateKeyframeCount(fs.Length);
StartCoroutine(KeyframeParser.ParseAsync(
    new Stream[] { fs }, count, keyframes,
    onProgress: (msg, f) => Debug.Log($"{msg} {f:P0}")));   // f is 0..1

note

On WebGL the async signature differs: ParseAsync(StringReader reader, int length, List<KeyframeCache> keyframes, CancellationToken? token, Action<float> onProgress) — a StringReader source and a float-only progress callback.

SkeletonWebStreamer

class · Singleton Unity component singleton

public class SkeletonWebStreamer : Singleton<SkeletonWebStreamer>

Namespace: global (no namespace). A Unity component (singleton) that streams the currently-visible characters' bone transforms over TCP to a web skeleton server, for browser-based viewers. Enable it — and configure the target — via SkeletonConsumer.StreamToWeb / StreamToWebUri, or set the fields directly and enable the component.

Member	Type	Default	Description
HostUri	string	"skeleton-web-server-aci.westeurope.azurecontainer.io"	target web skeleton server host
Port	int	13000	target TCP port

public string HostUri = "skeleton-web-server-aci.westeurope.azurecontainer.io";
public int Port = 13000;
public Task<bool> TryConnectAsync(string hostname = "127.0.0.1", int port = 13000);

Enable the component to start streaming; set HostUri / Port first. The usual path is to flip SkeletonConsumer.StreamToWeb on, which configures and enables this streamer for you.

caution

⚠️ needs confirmation: serialization runs automatically while the component is enabled — TryConnectAsync is normally driven internally, so app code rarely calls it directly.

See also

• SkeletonConsumer — the consumer that SkeletonTask replays into; owns StreamToWeb / StreamToWebUri
• Skeleton — the per-frame payload carried by SkeletonTask and decoded into KeyframeCache
• ObjectTransform — payload for ObjectTransformTask
• ObjectTransformConsumer · ObjectDetectionConsumer — replay targets for the object tasks
• extensions — IsIdentified and the Core↔Unity conversion helpers used around recordings
• Class index — all Unity SDK types