Your video, as a time-coded annotation

The shape, with illustrative values (a real read fills every array): speech segments with start, end and an approx flag; frames with their time, a short description and the text visible in them; on-screen text spans; chapters when the source has them; key points each tied to a quote and a time; and a coverage receipt that says what was and was not read.

{
  "version": "1",
  "source": "upload",
  "duration_seconds": 600,
  "speech": [{ "start": 12.0, "end": 18.5, "text": "…", "approx": true }],
  "frames": [{ "t": 4.0, "description": "at most 12 words: what is in view", "ocr": "text on screen" }],
  "on_screen": [{ "start": 8.0, "end": 16.0, "text": "…" }],
  "chapters": [],
  "key_points": [{ "text": "…", "quote": "words from the speech", "t": 40.0, "verified": true }],
  "coverage": { "seconds_transcribed": 600, "largest_frame_gap_seconds": 4 }
}

# the CLI extracts the sound and frames on your machine, then calls the API
pip install linkdigest-mcp            # needs ffmpeg + ffprobe on PATH
LINKDIGEST_API_KEY=ld_live_... linkdigest capture ./clip.mp4 --json

# or over HTTP: one audio part, one frames part per JPEG, one meta part
curl -X POST https://linkdigest.dev/api/v1/capture \
  -H "Authorization: Bearer ld_live_..." \
  -F "audio=@audio.mp3;type=audio/mpeg" \
  -F "frames=@frame_00.jpg;type=image/jpeg" \
  -F 'meta={"page_url":"https://linkdigest.dev/app/upload","duration_seconds":600,"frame_times":[4]}'
# 202 {"id":"…","poll":"/api/v1/capture/…"}

curl "https://linkdigest.dev/api/v1/capture/JOB_ID?format=annotation" \
  -H "Authorization: Bearer ld_live_..."

Or drop a file on /app/upload in the console. Links work too: a public Xiaohongshu, Douyin, Bilibili or YouTube video can be read straight from its link with POST /api/v1/digest and format: "annotation"; a batch of links exports as JSONL, one video per line. Full reference: docs.

An upload is read in pieces of up to 10 minutes; each piece costs 11 credits, so an hour of video is 66 credits. What that is in dollars depends on how you buy credits:

10 free credits on sign-up, no card. A piece that yields nothing readable is not charged for its length. Above a few hundred hours a month, ask in the pilot form. All prices: /pricing.

• Uploads are up to 10 minutes per call; split longer footage (a pilot can take it whole).
• On uploads, one frame every 4 seconds (at most 36 per call) is described; motion between frames is not.
• Speech times are estimated within the pieces the speech model hears, and marked approx.
• No action or task labels, no hand or pose tracking, no depth. Tell us your schema in the pilot request.
• The video file never leaves your machine; the extracted sound and frames are deleted when the job ends, and the annotation stays private to your account.

🇦🇺此时一位留子正在制作她的下午茶 · video © Glen啦 on Xiaohongshu, linked, not copied; the descriptions are LinkDigest's own read · read 2026-11-07 · 2 credits · 40 s · source

16 of 40 frames shown, spread over the clip. Not captured, in the read's own words: no speech transcript: no speech was recognised in the audio; comments: the top 5 of 32 read from the share page; the rest of the thread not read.

The same read as the API returns it, frames cut to the first three:

{
  "version": "1",
  "source": "https://www.xiaohongshu.com/explore/6941175d000000001e02a118",
  "platform": "xiaohongshu",
  "title": "🇦🇺此时一位留子正在制作她的下午茶",
  "duration_seconds": 40,
  "depth": "full",
  "speech": [],
  "frames": [
    {
      "t": 0,
      "description": "A hand drops sliced ginger into a white pot containing fried shallots on a stove.",
      "ocr": "Glen啦\n小红书"
    },
    {
      "t": 1,
      "description": "Raw chicken pieces are poured from a white bowl into the pot with ginger and shallots.",
      "ocr": "Glen啦\n小红书"
    },
    {
      "t": 2,
      "description": "Raw chicken pieces fill the bottom of the white enameled pot on the stovetop.",
      "ocr": "Glen啦\n小红书"
    }
  ],
  "on_screen": [],
  "chapters": [],
  "key_points": [
    {
      "text": "视频展示了在澳大利亚的留学生制作菌菇鸡汤饭的过程。",
      "quote": "此时一位留子正在制作她的下午茶",
      "t": null,
      "verified": true
    },
    {
      "text": "这是一段关于做饭和美食制作的Vlog。",
      "quote": "#菌菇鸡汤饭[话题]#",
      "t": null,
      "verified": true
    },
    {
      "text": "作者认为这道菜味道无敌好吃。",
      "quote": "今天做鸡汤泡饭～ 无敌好吃",
      "t": null,
      "verified": true
    }
  ],
  "coverage": {
    "seconds_transcribed": 0,
    "duration_seconds": 40,
    "largest_frame_gap_seconds": 1,
    "images_read": null,
    "images_total": null,
    "transcript_source": "none",
    "not_captured": [
      "no speech transcript: no speech was recognised in the audio",
      "comments: the top 5 of 32 read from the share page; the rest of the thread not read"
    ],
    "links_found": 0
  }
}

Tell us what you are training, roughly how many hours a month, and the schema you need. A person replies, usually the same day, and annotates a short clip of your own footage as the pilot.

Platforms and live read status: /platforms · 中文：视频标注 API