Avatar mapping format

The core never names a Live2D motion or a VRM expression. It speaks a small, renderer-agnostic vocabulary (ADR-0004):

• affect — a point (valence, arousal), each in [-1, 1], plus the feeling currently being voiced (joy, sad, …);
• gestures — nod, shake, tilt_head, wave;
• idle states — idle, listening, thinking, speaking, sleeping;
• gaze — the pointer, the user, or the screen;
• mouth — an open amount, driven by the voice’s loudness.

Each model comes with a mapping, avatar/avatar.json, that translates this vocabulary into that model’s own parts. There are two shapes, chosen by kind.

Where it lives: each character has a directory with an avatar/ folder holding the model and avatar.json (~/Library/Application Support/DeskMate/characters/<id>/avatar/). You rarely write it from nothing — importing a model in the console’s Characters page generates one (see Import) — but you may edit it by hand.

Common fieldsCopy Anchor Link

Expressions: anchors in affect spaceCopy Anchor Link

An anchor is a named face placed at a point in (valence, arousal) space. The renderer picks the anchor nearest the character’s current affect, and sticks with it until another is clearly closer, so the face doesn’t flicker. Switching cross-fades over about 300 ms.

{ "name": "joy", "valence": 0.7, "arousal": 0.3, "…": "…" }

• Give a neutral anchor ("valence": 0, "arousal": 0).
• When the character voices a feeling — [affect joy] in its speech — an anchor named after that word wins over the nearest one. The words are neutral joy excited calm sad angry scared surprised shy bored. Reusing these as anchor names is how you get that behaviour. Their default positions:

VRMCopy Anchor Link

{
  "kind": "vrm",
  "model": "mio.vrm",
  "expressions": [
    { "name": "neutral", "valence": 0, "arousal": 0 },
    { "name": "joy", "valence": 0.7, "arousal": 0.3, "weights": { "happy": 1 } },
    { "name": "excited", "valence": 0.7, "arousal": 0.8,
      "weights": { "happy": 0.8, "surprised": 0.5 } }
  ],
  "gestures": {
    "wave": { "vrma": "wave.vrma" }
  }
}

Expressions. weights mixes the model’s expressions, each between 0 and 1. Names are VRM 1.0 preset names (happy, angry, sad, relaxed, surprised) or the model’s own custom expressions; VRM 0.x presets are renamed on load (joy→happy, sorrow→sad, fun→relaxed, a→aa, …), so write the 1.0 names. Names the model doesn’t have are dropped when the model loads, with one warning. An anchor with no weights is the plain face.

Fixed behaviour you don’t configure.

• Mouth: loudness drives aa.
• Blink: the idle system drives blink; when sleeping the eyes close and the head lowers.
• Gaze: VRM lookAt; the head follows a little, as with Live2D.
• Idle: breathing, small head and spine sway on smooth noise rather than a loop, and random blinks.
• Framing: an orthographic camera, feet at the bottom edge of the window, centered, with room above for the subtitle bubble.

Gestures. By default nod, shake, tilt_head and wave are bone movements written in code (bone-gestures.ts), so any humanoid VRM works without configuration. To use your own animation for one, point at a .vrma file (path relative to avatar/):

"gestures": { "wave": { "vrma": "wave.vrma" } }

Only the humanoid bone rotations in the clip are used, laid over the standing pose, and each bone the clip moves blends in and out over 200 ms. Gaze, expression and position stay the character’s.

Live2DCopy Anchor Link

Live2D is user-supplied: DeskMate doesn’t ship Live2D’s runtime (Cubism Core) or any Live2D model. The user imports Cubism Core themselves, from the Cubism 5 SDK for Web (version 5.x; 6.x isn’t supported). See ADR-0010.

{
  "kind": "live2d",
  "model": "hiyori_pro_t11.model3.json",
  "gestures": {
    "nod": { "motion": { "group": "Tap", "index": 0 } },
    "wave": { "motion": { "group": "Tap@Body", "index": 0 } }
  },
  "idle": {
    "thinking": { "motion": { "group": "FlickDown", "index": 0 } }
  },
  "idleMotion": { "group": "Idle", "everySeconds": [20, 60] },
  "expressions": [
    { "name": "neutral", "valence": 0, "arousal": 0 },
    { "name": "joy", "valence": 0.7, "arousal": 0.3, "expression": "smile" },
    { "name": "sad", "valence": -0.6, "arousal": -0.4,
      "params": { "ParamEyeLSmile": { "value": 0, "blend": "overwrite" } } }
  ]
}

An action (for gestures and idle entries) is one of:

• { "motion": { "group": "Tap", "index": 0 } } — a motion group from model3.json; index defaults to the first.
• { "expression": "smile" } — one of the model’s own .exp3.json expressions.

Gestures are keyed nod, shake, tilt_head, wave; idle states idle, listening, thinking, speaking, sleeping. Anything you omit simply doesn’t animate.

Expression anchors take one of:

• "expression": "<name>" — a named expression from the model;
• "params": { "<ParameterId>": … } — parameter values given inline, for models that ship no .exp3.json. A bare number is added to the parameter; { "value": 0.6, "blend": "add" | "multiply" | "overwrite" } blends as in exp3. Params are layered on top of the motion every frame and faded over about 300 ms. For negative faces use overwrite on ParamEyeLSmile, ParamEyeRSmile and ParamMouthForm, or the smiling eyes in the idle motion will show through.
• neither — the model’s plain face.

idleMotion plays one motion group now and then while purely idle, between the procedural sway: everySeconds is [min, max], default [20, 60]. The engine’s own idle loop, sine sway and auto-blink are turned off in favour of DeskMate’s procedural ones.

Writing a mapping for a new VRM, from zeroCopy Anchor Link

1. Check the model. Open the .vrm in any VRM viewer. Note its license — DeskMate reads allowRedistribution (1.0) or the license name (0.x) from the file and uses it when someone exports a character pack — and list its expressions.
2. Import it. In the console’s Characters page, create or edit a character and set Avatar to A VRM or Live2D model and choose the .vrm. DeskMate copies it into avatar/ and writes an avatar.json. Open it to see what it guessed.
3. Tune the anchors. For each of the ten affect words, make an anchor named after it and give it weights. Prefer the model’s own matching expression where it has one (a Blush expression for shy); otherwise mix standard ones (happy 0.5 + relaxed 0.3). Keep neutral first.
4. See it. Restart DeskMate, or switch to another character and back, so the stage reloads the model. In a development build, the tray’s Debug menu has Affect and Gesture items to try each face and gesture.
5. Optional: your own gesture. Put a .vrma next to the model and reference it from gestures.

What import generatesCopy Anchor Link

When the model comes without an avatar.json:

• Known models — assets/known-avatars/<name>/avatar.json is used when the file name contains the folder name and every motion the mapping uses exists in the model. Today that’s Hiyori.
• Live2D — the Idle motion group becomes the idle motion; the model’s expressions are matched by name (smile, sad, angry, …) to the affect words, plus a neutral anchor. Gestures can’t be guessed, so write them by hand if you want them.
• VRM — only the JSON chunk of the GLB is read. Each affect word uses the model’s own expression of that name if there is one, and otherwise a fixed blend of the standard expressions, minus whatever the model lacks. Gestures use the built-in bone movements.

For a VRM, the author’s terms are also read out of the .vrm into avatar.json’s meta (title, authors, license, allowRedistribution, termsUnclear). That is a cache for the console: every time it matters, DeskMate re-reads the terms from the .vrm itself, so editing meta can’t change what is shared. VRM 0.x models whose license is Other count as not redistributable, because their terms live behind a link a program can’t read.

A built-in avatar’s avatar.json also carries "builtin": true, which makes a character pack store a reference to it rather than the model.