Desktop Recall Recording Integration

Design for adding user-controlled desktop recording to Vued Electron with Recall.ai Desktop Recording SDK as the capture layer and the existing tablet-style Vued upload pipeline as the ingestion layer.

Goals

Let a signed-in desktop user start and stop recording from the Electron app.
Capture mixed desktop audio through Recall.ai Desktop Recording SDK.
Preserve the tablet upload semantics:
- create a meeting placeholder at start,
- persist audio and queue metadata durably,
- create the audio-slice metadata before uploading bytes,
- retry safely and idempotently,
- rely on the backend for transcription, speaker ID, summary, title, notes, ACLs, and encrypted desktop sync.
Keep Recall API keys out of Electron.
Keep the React renderer on a narrow preload bridge instead of importing native SDK code in UI.

Non-goals

Separate mic and system tracks. Recall Desktop SDK currently emits mixed raw audio only for desktop SDK callbacks.
Replacing the Vued backend transcription/finalization path.
Broad local plaintext services beyond the existing desktop local runtime.

References

Android tablet flow:

/Users/nirbhaysinghnarang/AndroidStudioProjects/vued/README.md
/Users/nirbhaysinghnarang/AndroidStudioProjects/vued/app/src/main/java/com/nsn8/vued/meeting/MeetingController.kt
/Users/nirbhaysinghnarang/AndroidStudioProjects/vued/app/src/main/java/com/nsn8/vued/net/OutboundQueue.kt
/Users/nirbhaysinghnarang/AndroidStudioProjects/vued/app/src/main/java/com/nsn8/vued/net/VuedApi.kt

Desktop app bridge:

electron/main.js
electron/preload.js
lib/desktop-api.ts

Recall docs:

Desktop SDK lifecycle, init, permissions, prepareDesktopAudioRecording, startRecording, and realtime-event: https://docs.recall.ai/docs/desktop-sdk
Raw mixed audio payload shape: https://docs.recall.ai/docs/real-time-event-payloads
SDK support matrix: https://docs.recall.ai/docs/dsdk-supported-platforms
Separate streams limitation: https://docs.recall.ai/docs/desktop-recording-sdk-faq

Tablet Upload Procedure To Mirror

The Android tablet does this:

On start, mint a no-dash UUID meeting ID.
Enqueue a durable MEETING_CREATE item.
Drain opportunistically:
- POST /api/v1/meetings
- body includes id, title, status: "recording", started_at, optional roomId, optional microphoneId.
On stop, export the recording window to one .m4a.
Enqueue a durable MEETING audio item with deterministic slice ID:
- UUID.nameUUIDFromBytes("meeting:<meetingId>") on Android.
Drain in order:
- wait until the meeting create item is gone,
- POST /api/v1/transcript/audio-slices,
- persist metadataDone: true,
- PUT /api/v1/transcript/audio-slices/:sliceId/audio,
- headers x-audio-duration-secs, x-audio-size-bytes,
- body audio/mp4.

Desktop should use the same endpoint order and idempotency model.

Proposed Electron Architecture

React UI
  -> window.vuedDesktop.recording.start()
  -> window.vuedDesktop.recording.stop()

electron/preload.js
  -> IPC bridge only

electron/main.js
  -> registers recording IPC
  -> owns auth/session lookup

electron/recall-recorder.js
  -> initializes RecallAiSdk
  -> requests permissions
  -> prepareDesktopAudioRecording()
  -> startRecording({ windowId, uploadToken })
  -> handles realtime-event audio_mixed_raw.data
  -> writes local PCM/WAV/M4A

electron/desktop-recording-queue.js
  -> durable tablet-style queue
  -> creates Vued meeting
  -> creates Vued audio slice
  -> uploads audio bytes
  -> deletes local audio after success

Vued backend
  -> creates Recall sdk_upload token
  -> receives Vued audio slice upload
  -> existing transcript/finalization pipeline

Recall Upload Token Endpoint

Add a Vued backend endpoint, for example:

POST /api/v1/desktop/recordings/recall-upload

Electron calls this endpoint with the signed-in user JWT. The backend uses the Recall API key to create an SDK Upload and returns only:

{
  "sdkUploadId": "recall-sdk-upload-id",
  "uploadToken": "recall-upload-token",
  "regionApiUrl": "https://us-west-2.recall.ai"
}

The SDK upload should request only the realtime callback events we need:

{
  "recording_config": {
    "realtime_endpoints": [
      {
        "type": "desktop_sdk_callback",
        "events": [
          "audio_mixed_raw.data",
          "participant_events.speech_on",
          "participant_events.speech_off",
          "participant_events.update"
        ]
      }
    ]
  }
}

Open question for backend/Recall configuration: minimize or disable stored Recall media if possible, because Vued is using Recall as capture transport and Vued remains the system of record. If Recall requires a completed SDK upload artifact, keep retention short and store the sdkUploadId on Vued meeting metadata for support/debugging.

Desktop Start Flow

recording.start({ title?, roomId?, microphoneId? })

Validate signed-in session and apiBaseUrl.
Ensure no active desktop recording exists.
Resolve the stable desktop room for this user:
- microphoneId: desktop:user:<userId>.
- displayName: <User Display Name>'s Desktop.
- List org rooms and reuse the matching microphone ID when it exists.
- Create the room before meeting upload when it does not exist.
- Server permits members to create only their own desktop:user:<userId> room without general rooms:write.
Initialize Recall SDK if needed:
- RecallAiSdk.init({ apiUrl })
- macOS: request accessibility, microphone, screen-capture.
Mint:
- meetingId: UUID without dashes, to match tablet convention.
- sessionId: one per desktop app process or one per recorder service lifetime.
- startedAtSec: Date.now() / 1000.
Persist and enqueue MEETING_CREATE before starting capture.
Drain queue opportunistically so the recording placeholder appears quickly.
Ask Vued backend for Recall upload token.
Call RecallAiSdk.prepareDesktopAudioRecording() to get a manual desktop audio windowId.
Open local audio writer.
Call RecallAiSdk.startRecording({ windowId, uploadToken }).
Set active state:
- status: "recording"
- meetingId
- startedAtSec
- windowId
- sdkUploadId

If Recall fails after meeting create is queued, stop the local session and leave the meeting create queued only if the UI should show the failed placeholder. Safer default: mark failure before exposing active state and remove a not-yet-drained create item if possible. Once create has reached the backend, use a backend cancel/delete endpoint rather than local deletion.

Realtime Audio Handling

Recall audio_mixed_raw.data provides base64 raw audio:

16 kHz mono S16LE PCM

Handler:

Ignore events unless there is an active recording.
Decode event.data.data.buffer from base64 into a Buffer.
Append PCM bytes to the active writer.
Track:
- first/last relative timestamp,
- byte count,
- sample count,
- latest audio timestamp for UI health.

Encoding Choice

The tablet uploads .m4a as audio/mp4. Desktop records PCM and uploads the mixed WAV directly as audio/wav.

The canonical /api/v1/transcript/audio-slices/:sliceId/audio route accepts M4A, WAV, and MP3.
The server maps audio/wav, audio/x-wav, and audio/wave to .wav.
This avoids bundling an AAC/M4A encoder in Electron and removes a fragile local conversion step.

Desktop Stop Flow

recording.stop()

Snapshot active recording state.
Call RecallAiSdk.stopRecording({ windowId }).
Close/finalize the local audio writer.
Compute:
- endedAtSec
- durationSecs
- sizeBytes
If no audio bytes were captured, surface a user-visible error and optionally cancel/delete the meeting placeholder.
Enqueue MEETING audio item:
- deterministic sliceId from meeting:<meetingId>,
- sessionId,
- meetingId,
- startedAtSec,
- endedAtSec,
- durationSecs,
- sizeBytes,
- roomId.
Drain queue.
Run desktop sync or targeted record refresh for meetingId so the UI shows finalizing.
Clear active state.

Durable Queue Design

Create electron/desktop-recording-queue.js. Storage:

~/Library/Application Support/Vued[/ Dev]/recording-outbox/
  queue.json
  audio/
    <sliceId>.m4a
    <sliceId>.wav

Queue item shapes:

{
  "id": "meeting:<meetingId>",
  "kind": "MEETING_CREATE",
  "meetingId": "<meetingId>",
  "title": "Desktop meeting",
  "startedAtSec": 1780000000.123,
  "roomId": null,
  "microphoneId": null
}

{
  "id": "<sliceId>",
  "kind": "MEETING",
  "sessionId": "<sessionId>",
  "meetingId": "<meetingId>",
  "startedAtSec": 1780000000.123,
  "endedAtSec": 1780000300.456,
  "durationSecs": 300.333,
  "sizeBytes": 1234567,
  "mime": "audio/mp4",
  "extension": "m4a",
  "metadataDone": false,
  "roomId": null
}

Drain behavior should match Android:

Single in-process drain guard.
Iterate insertion order.
Never upload MEETING while its MEETING_CREATE is pending.
Persist metadataDone: true after successful audio-slice metadata creation.
Delete audio file and queue item after successful bytes upload.
Keep failed items forever until success or explicit user cleanup.
Prune orphan audio queue items only if the file is missing.

Desktop can use atomic JSON writes:

Write queue.json.tmp.
fs.renameSync(tmp, queue.json).

Vued API Calls From Electron

Use the existing desktop session token from electron/main.js. Create meeting:

POST /api/v1/meetings
Authorization: Bearer <token>
Content-Type: application/json

{
  "id": "<meetingId>",
  "title": "<title>",
  "status": "recording",
  "started_at": 1780000000.123,
  "roomId": "<optional>",
  "microphoneId": "<optional>"
}

Create slice:

POST /api/v1/transcript/audio-slices
Authorization: Bearer <token>
Content-Type: application/json

{
  "id": "<sliceId>",
  "sessionId": "<sessionId>",
  "modality": "meeting",
  "linkedRecordId": "<meetingId>",
  "linkedRecordType": "meeting",
  "startedAt": 1780000000.123,
  "endedAt": 1780000300.456,
  "audioDurationSecs": 300.333,
  "audioSizeBytes": 1234567,
  "roomId": "<optional>"
}

Upload bytes:

PUT /api/v1/transcript/audio-slices/<sliceId>/audio
Authorization: Bearer <token>
x-audio-duration-secs: 300.333
x-audio-size-bytes: 1234567
Content-Type: audio/mp4

<audio bytes>

Preload Bridge

Add to window.vuedDesktop:

recording: {
  getStatus(): Promise<DesktopRecordingState>;
  requestPermissions(): Promise<DesktopRecordingState>;
  start(input?: {
    title?: string;
    roomId?: string | null;
    microphoneId?: string | null;
  }): Promise<DesktopRecordingState>;
  stop(): Promise<DesktopRecordingState>;
  retryUploads(): Promise<DesktopRecordingState>;
  onStatus(callback: (state: DesktopRecordingState) => void): () => void;
}

State:

type DesktopRecordingState = {
  supported: boolean;
  status:
    | "idle"
    | "initializing"
    | "permission_required"
    | "ready"
    | "starting"
    | "recording"
    | "stopping"
    | "uploading"
    | "error";
  meetingId?: string | null;
  startedAt?: number | null;
  durationSecs?: number | null;
  pendingUploads?: number;
  lastAudioAt?: number | null;
  error?: string | null;
};

UI Placement

MVP:

Add a compact record control to /home.
Button states:
- Start recording
- Recording 04:12
- Stopping
- Upload pending
Attach the stable user desktop room to the meeting and audio slice.
After start, run sync so the recording placeholder appears in the meeting list.
After stop/upload, targeted refresh/sync should move the meeting from recording to finalizing.

Avoid mixing this with ambient candidates for the first pass. This is manual desktop recording.

Packaging And Permissions

Package updates:

Add @recallai/desktop-sdk dependency.
Add Recall package/native assets to Electron Builder files.
Add SDK/native binaries to asarUnpack as needed after inspecting installed package layout.
Add macOS Info.plist usage descriptions if missing:
- microphone,
- screen capture,
- accessibility guidance if required by the SDK.
Test signing/notarization early. Recall documents SDK-specific signing behavior for macOS.

Supported platforms:

macOS 13+ Apple Silicon.
Windows 10+ 64-bit.
No Linux support through Recall Desktop SDK.

Expose supported: false on unsupported platforms and keep the UI hidden/disabled.

Failure Modes

Permission denied:
- status permission_required;
- UI opens OS guidance.
Recall SDK start fails:
- close writer;
- clear active state;
- leave no audio queue item.
App quits while recording:
- best effort stop in before-quit;
- if audio exists, enqueue pending export/audio item.
Network offline:
- queue remains;
- retryUploads() and app start drain retry.
Meeting create succeeds, slice metadata fails:
- meeting remains recording;
- queue retry eventually advances to upload/finalizing.
Slice metadata succeeds, byte upload fails:
- metadataDone stays true;
- retry skips metadata and repeats bytes upload.
Recall upload completes/fails:
- useful for diagnostics only in the Vued ingestion path;
- Vued finalization should depend on Vued audio-slice upload success.

Implementation Phases

Backend token endpoint:
- create Recall SDK Upload with realtime mixed audio callback.
- keep Recall API key server-side.
Electron recording service skeleton:
- SDK init,
- permission status,
- manual start/stop,
- state broadcast.
Local audio writer:
- decode audio_mixed_raw.data;
- write WAV or M4A based on backend decision.
Desktop recording queue:
- mirror Android OutboundQueue.
- implement create meeting, create slice, upload bytes.
Preload/types/UI:
- expose vuedDesktop.recording.
- add /home record control.
Packaging:
- include Recall native assets.
- verify packaged app on Apple Silicon macOS and Windows.
Hardening:
- local queued audio encryption,
- crash recovery,
- diagnostics,
- upload retry status in UI.

Main Open Decisions

Desktop uploads WAV directly to the existing slice upload endpoint.
Desktop recordings are room-tagged by default with desktop:user:<userId>.
How short can Recall retention be for SDK uploads when Vued is the source of truth?
Do we need automatic meeting detection later, or is manual desktop recording the only desired UX for now?

​Desktop Recall Recording Integration

​Goals

​Non-goals

​References

​Tablet Upload Procedure To Mirror

​Proposed Electron Architecture

​Recall Upload Token Endpoint

​Desktop Start Flow

​Realtime Audio Handling

​Encoding Choice

​Desktop Stop Flow

​Durable Queue Design

​Vued API Calls From Electron

​Preload Bridge

​UI Placement

​Packaging And Permissions

​Failure Modes

​Implementation Phases

​Main Open Decisions