> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vued.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Desktop recall recording design

# Desktop Recall Recording Integration

Design for adding user-controlled desktop recording to Vued Electron with Recall.ai Desktop Recording SDK as the capture layer and the existing tablet-style Vued upload pipeline as the ingestion layer.

## Goals

* Let a signed-in desktop user start and stop recording from the Electron app.
* Capture mixed desktop audio through Recall.ai Desktop Recording SDK.
* Preserve the tablet upload semantics:
  * create a meeting placeholder at start,
  * persist audio and queue metadata durably,
  * create the audio-slice metadata before uploading bytes,
  * retry safely and idempotently,
  * rely on the backend for transcription, speaker ID, summary, title, notes, ACLs, and encrypted desktop sync.
* Keep Recall API keys out of Electron.
* Keep the React renderer on a narrow preload bridge instead of importing native SDK code in UI.

## Non-goals

* Separate mic and system tracks. Recall Desktop SDK currently emits mixed raw audio only for desktop SDK callbacks.
* Replacing the Vued backend transcription/finalization path.
* Broad local plaintext services beyond the existing desktop local runtime.

## References

Android tablet flow:

* `/Users/nirbhaysinghnarang/AndroidStudioProjects/vued/README.md`
* `/Users/nirbhaysinghnarang/AndroidStudioProjects/vued/app/src/main/java/com/nsn8/vued/meeting/MeetingController.kt`
* `/Users/nirbhaysinghnarang/AndroidStudioProjects/vued/app/src/main/java/com/nsn8/vued/net/OutboundQueue.kt`
* `/Users/nirbhaysinghnarang/AndroidStudioProjects/vued/app/src/main/java/com/nsn8/vued/net/VuedApi.kt`

Desktop app bridge:

* `electron/main.js`
* `electron/preload.js`
* `lib/desktop-api.ts`

Recall docs:

* Desktop SDK lifecycle, init, permissions, `prepareDesktopAudioRecording`, `startRecording`, and `realtime-event`: [https://docs.recall.ai/docs/desktop-sdk](https://docs.recall.ai/docs/desktop-sdk)
* Raw mixed audio payload shape: [https://docs.recall.ai/docs/real-time-event-payloads](https://docs.recall.ai/docs/real-time-event-payloads)
* SDK support matrix: [https://docs.recall.ai/docs/dsdk-supported-platforms](https://docs.recall.ai/docs/dsdk-supported-platforms)
* Separate streams limitation: [https://docs.recall.ai/docs/desktop-recording-sdk-faq](https://docs.recall.ai/docs/desktop-recording-sdk-faq)

## Tablet Upload Procedure To Mirror

The Android tablet does this:

1. On start, mint a no-dash UUID meeting ID.
2. Enqueue a durable `MEETING_CREATE` item.
3. Drain opportunistically:
   * `POST /api/v1/meetings`
   * body includes `id`, `title`, `status: "recording"`, `started_at`, optional `roomId`, optional `microphoneId`.
4. On stop, export the recording window to one `.m4a`.
5. Enqueue a durable `MEETING` audio item with deterministic slice ID:
   * `UUID.nameUUIDFromBytes("meeting:<meetingId>")` on Android.
6. Drain in order:
   * wait until the meeting create item is gone,
   * `POST /api/v1/transcript/audio-slices`,
   * persist `metadataDone: true`,
   * `PUT /api/v1/transcript/audio-slices/:sliceId/audio`,
   * headers `x-audio-duration-secs`, `x-audio-size-bytes`,
   * body `audio/mp4`.

Desktop should use the same endpoint order and idempotency model.

## Proposed Electron Architecture

```text theme={null}
React UI
  -> window.vuedDesktop.recording.start()
  -> window.vuedDesktop.recording.stop()

electron/preload.js
  -> IPC bridge only

electron/main.js
  -> registers recording IPC
  -> owns auth/session lookup

electron/recall-recorder.js
  -> initializes RecallAiSdk
  -> requests permissions
  -> prepareDesktopAudioRecording()
  -> startRecording({ windowId, uploadToken })
  -> handles realtime-event audio_mixed_raw.data
  -> writes local PCM/WAV/M4A

electron/desktop-recording-queue.js
  -> durable tablet-style queue
  -> creates Vued meeting
  -> creates Vued audio slice
  -> uploads audio bytes
  -> deletes local audio after success

Vued backend
  -> creates Recall sdk_upload token
  -> receives Vued audio slice upload
  -> existing transcript/finalization pipeline
```

## Recall Upload Token Endpoint

Add a Vued backend endpoint, for example:

```text theme={null}
POST /api/v1/desktop/recordings/recall-upload
```

Electron calls this endpoint with the signed-in user JWT. The backend uses the Recall API key to create an SDK Upload and returns only:

```json theme={null}
{
  "sdkUploadId": "recall-sdk-upload-id",
  "uploadToken": "recall-upload-token",
  "regionApiUrl": "https://us-west-2.recall.ai"
}
```

The SDK upload should request only the realtime callback events we need:

```json theme={null}
{
  "recording_config": {
    "realtime_endpoints": [
      {
        "type": "desktop_sdk_callback",
        "events": [
          "audio_mixed_raw.data",
          "participant_events.speech_on",
          "participant_events.speech_off",
          "participant_events.update"
        ]
      }
    ]
  }
}
```

Open question for backend/Recall configuration: minimize or disable stored Recall media if possible, because Vued is using Recall as capture transport and Vued remains the system of record. If Recall requires a completed SDK upload artifact, keep retention short and store the `sdkUploadId` on Vued meeting metadata for support/debugging.

## Desktop Start Flow

`recording.start({ title?, roomId?, microphoneId? })`

1. Validate signed-in session and `apiBaseUrl`.
2. Ensure no active desktop recording exists.
3. Resolve the stable desktop room for this user:
   * `microphoneId`: `desktop:user:<userId>`.
   * `displayName`: `<User Display Name>'s Desktop`.
   * List org rooms and reuse the matching microphone ID when it exists.
   * Create the room before meeting upload when it does not exist.
   * Server permits members to create only their own `desktop:user:<userId>` room without general `rooms:write`.
4. Initialize Recall SDK if needed:
   * `RecallAiSdk.init({ apiUrl })`
   * macOS: request `accessibility`, `microphone`, `screen-capture`.
5. Mint:
   * `meetingId`: UUID without dashes, to match tablet convention.
   * `sessionId`: one per desktop app process or one per recorder service lifetime.
   * `startedAtSec`: `Date.now() / 1000`.
6. Persist and enqueue `MEETING_CREATE` before starting capture.
7. Drain queue opportunistically so the recording placeholder appears quickly.
8. Ask Vued backend for Recall upload token.
9. Call `RecallAiSdk.prepareDesktopAudioRecording()` to get a manual desktop audio `windowId`.
10. Open local audio writer.
11. Call `RecallAiSdk.startRecording({ windowId, uploadToken })`.
12. Set active state:
    * `status: "recording"`
    * `meetingId`
    * `startedAtSec`
    * `windowId`
    * `sdkUploadId`

If Recall fails after meeting create is queued, stop the local session and leave the meeting create queued only if the UI should show the failed placeholder. Safer default: mark failure before exposing active state and remove a not-yet-drained create item if possible. Once create has reached the backend, use a backend cancel/delete endpoint rather than local deletion.

## Realtime Audio Handling

Recall `audio_mixed_raw.data` provides base64 raw audio:

```text theme={null}
16 kHz mono S16LE PCM
```

Handler:

1. Ignore events unless there is an active recording.
2. Decode `event.data.data.buffer` from base64 into a `Buffer`.
3. Append PCM bytes to the active writer.
4. Track:
   * first/last relative timestamp,
   * byte count,
   * sample count,
   * latest audio timestamp for UI health.

### Encoding Choice

The tablet uploads `.m4a` as `audio/mp4`. Desktop records PCM and uploads the
mixed WAV directly as `audio/wav`.

* The canonical `/api/v1/transcript/audio-slices/:sliceId/audio` route accepts
  M4A, WAV, and MP3.
* The server maps `audio/wav`, `audio/x-wav`, and `audio/wave` to `.wav`.
* This avoids bundling an AAC/M4A encoder in Electron and removes a fragile
  local conversion step.

## Desktop Stop Flow

`recording.stop()`

1. Snapshot active recording state.
2. Call `RecallAiSdk.stopRecording({ windowId })`.
3. Close/finalize the local audio writer.
4. Compute:
   * `endedAtSec`
   * `durationSecs`
   * `sizeBytes`
5. If no audio bytes were captured, surface a user-visible error and optionally cancel/delete the meeting placeholder.
6. Enqueue `MEETING` audio item:
   * deterministic `sliceId` from `meeting:<meetingId>`,
   * `sessionId`,
   * `meetingId`,
   * `startedAtSec`,
   * `endedAtSec`,
   * `durationSecs`,
   * `sizeBytes`,
   * `roomId`.
7. Drain queue.
8. Run desktop sync or targeted record refresh for `meetingId` so the UI shows `finalizing`.
9. Clear active state.

## Durable Queue Design

Create `electron/desktop-recording-queue.js`.

Storage:

```text theme={null}
~/Library/Application Support/Vued[/ Dev]/recording-outbox/
  queue.json
  audio/
    <sliceId>.m4a
    <sliceId>.wav
```

Queue item shapes:

```json theme={null}
{
  "id": "meeting:<meetingId>",
  "kind": "MEETING_CREATE",
  "meetingId": "<meetingId>",
  "title": "Desktop meeting",
  "startedAtSec": 1780000000.123,
  "roomId": null,
  "microphoneId": null
}
```

```json theme={null}
{
  "id": "<sliceId>",
  "kind": "MEETING",
  "sessionId": "<sessionId>",
  "meetingId": "<meetingId>",
  "startedAtSec": 1780000000.123,
  "endedAtSec": 1780000300.456,
  "durationSecs": 300.333,
  "sizeBytes": 1234567,
  "mime": "audio/mp4",
  "extension": "m4a",
  "metadataDone": false,
  "roomId": null
}
```

Drain behavior should match Android:

* Single in-process drain guard.
* Iterate insertion order.
* Never upload `MEETING` while its `MEETING_CREATE` is pending.
* Persist `metadataDone: true` after successful audio-slice metadata creation.
* Delete audio file and queue item after successful bytes upload.
* Keep failed items forever until success or explicit user cleanup.
* Prune orphan audio queue items only if the file is missing.

Desktop can use atomic JSON writes:

1. Write `queue.json.tmp`.
2. `fs.renameSync(tmp, queue.json)`.

## Vued API Calls From Electron

Use the existing desktop session token from `electron/main.js`.

Create meeting:

```http theme={null}
POST /api/v1/meetings
Authorization: Bearer <token>
Content-Type: application/json

{
  "id": "<meetingId>",
  "title": "<title>",
  "status": "recording",
  "started_at": 1780000000.123,
  "roomId": "<optional>",
  "microphoneId": "<optional>"
}
```

Create slice:

```http theme={null}
POST /api/v1/transcript/audio-slices
Authorization: Bearer <token>
Content-Type: application/json

{
  "id": "<sliceId>",
  "sessionId": "<sessionId>",
  "modality": "meeting",
  "linkedRecordId": "<meetingId>",
  "linkedRecordType": "meeting",
  "startedAt": 1780000000.123,
  "endedAt": 1780000300.456,
  "audioDurationSecs": 300.333,
  "audioSizeBytes": 1234567,
  "roomId": "<optional>"
}
```

Upload bytes:

```http theme={null}
PUT /api/v1/transcript/audio-slices/<sliceId>/audio
Authorization: Bearer <token>
x-audio-duration-secs: 300.333
x-audio-size-bytes: 1234567
Content-Type: audio/mp4

<audio bytes>
```

## Preload Bridge

Add to `window.vuedDesktop`:

```ts theme={null}
recording: {
  getStatus(): Promise<DesktopRecordingState>;
  requestPermissions(): Promise<DesktopRecordingState>;
  start(input?: {
    title?: string;
    roomId?: string | null;
    microphoneId?: string | null;
  }): Promise<DesktopRecordingState>;
  stop(): Promise<DesktopRecordingState>;
  retryUploads(): Promise<DesktopRecordingState>;
  onStatus(callback: (state: DesktopRecordingState) => void): () => void;
}
```

State:

```ts theme={null}
type DesktopRecordingState = {
  supported: boolean;
  status:
    | "idle"
    | "initializing"
    | "permission_required"
    | "ready"
    | "starting"
    | "recording"
    | "stopping"
    | "uploading"
    | "error";
  meetingId?: string | null;
  startedAt?: number | null;
  durationSecs?: number | null;
  pendingUploads?: number;
  lastAudioAt?: number | null;
  error?: string | null;
};
```

## UI Placement

MVP:

* Add a compact record control to `/home`.
* Button states:
  * `Start recording`
  * `Recording 04:12`
  * `Stopping`
  * `Upload pending`
* Attach the stable user desktop room to the meeting and audio slice.
* After start, run sync so the recording placeholder appears in the meeting list.
* After stop/upload, targeted refresh/sync should move the meeting from `recording` to `finalizing`.

Avoid mixing this with ambient candidates for the first pass. This is manual desktop recording.

## Packaging And Permissions

Package updates:

* Add `@recallai/desktop-sdk` dependency.
* Add Recall package/native assets to Electron Builder `files`.
* Add SDK/native binaries to `asarUnpack` as needed after inspecting installed package layout.
* Add macOS `Info.plist` usage descriptions if missing:
  * microphone,
  * screen capture,
  * accessibility guidance if required by the SDK.
* Test signing/notarization early. Recall documents SDK-specific signing behavior for macOS.

Supported platforms:

* macOS 13+ Apple Silicon.
* Windows 10+ 64-bit.
* No Linux support through Recall Desktop SDK.

Expose `supported: false` on unsupported platforms and keep the UI hidden/disabled.

## Failure Modes

* Permission denied:
  * status `permission_required`;
  * UI opens OS guidance.
* Recall SDK start fails:
  * close writer;
  * clear active state;
  * leave no audio queue item.
* App quits while recording:
  * best effort stop in `before-quit`;
  * if audio exists, enqueue pending export/audio item.
* Network offline:
  * queue remains;
  * `retryUploads()` and app start drain retry.
* Meeting create succeeds, slice metadata fails:
  * meeting remains `recording`;
  * queue retry eventually advances to upload/finalizing.
* Slice metadata succeeds, byte upload fails:
  * `metadataDone` stays true;
  * retry skips metadata and repeats bytes upload.
* Recall upload completes/fails:
  * useful for diagnostics only in the Vued ingestion path;
  * Vued finalization should depend on Vued audio-slice upload success.

## Implementation Phases

1. Backend token endpoint:
   * create Recall SDK Upload with realtime mixed audio callback.
   * keep Recall API key server-side.
2. Electron recording service skeleton:
   * SDK init,
   * permission status,
   * manual start/stop,
   * state broadcast.
3. Local audio writer:
   * decode `audio_mixed_raw.data`;
   * write WAV or M4A based on backend decision.
4. Desktop recording queue:
   * mirror Android `OutboundQueue`.
   * implement create meeting, create slice, upload bytes.
5. Preload/types/UI:
   * expose `vuedDesktop.recording`.
   * add `/home` record control.
6. Packaging:
   * include Recall native assets.
   * verify packaged app on Apple Silicon macOS and Windows.
7. Hardening:
   * local queued audio encryption,
   * crash recovery,
   * diagnostics,
   * upload retry status in UI.

## Main Open Decisions

* Desktop uploads WAV directly to the existing slice upload endpoint.
* Desktop recordings are room-tagged by default with `desktop:user:<userId>`.
* How short can Recall retention be for SDK uploads when Vued is the source of truth?
* Do we need automatic meeting detection later, or is manual desktop recording the only desired UX for now?