ZeroDB · RLHF API

Close the loop — production feedback to fine-tuning

Log every AI interaction, attach human thumbs or ratings, and export a DPO- or SFT-ready dataset in JSONL, CSV, or Parquet. Every thumbs-up is a training example you already paid for.

Start logging interactions Get the workbook

Step 1

Log

Record each prompt → response exchange with agent_id, session_id, and context.

Step 2

Collect

Attach human feedback: thumbs_up, thumbs_down, or a 1–5 rating with an optional comment.

Step 3

Measure

Watch average_reward and feedback breakdown per model from the /stats endpoint.

Step 4

Export

Filter by polarity, model, and date. Export JSONL/CSV/Parquet for DPO or SFT.

Zero to first interaction

Mint a project and API key in a single call — no signup form needed.

POST /api/v1/public/instant-db
X-API-Key not required for this call

{
  "agree_terms": true,
  "on_behalf_of": "u_123"
}

# Returns:
# {
#   "api_key": "tmp_...",    <- use as X-API-Key
#   "project_id": "...",     <- use in every RLHF path below
#   "expires_at": "...",     # ~72h lifetime
#   "claim_url": "...",      # claim to make it permanent
#   "limits": { "vectors": 10000, "memory_records": 1000 }
# }

Step 1 — Log the interaction

POST /api/v1/public/zerodb/{project_id}/database/rlhf/interactions
X-API-Key: <your-project-api-key>

{
  "prompt": "How do I export my invoices?",
  "response": "Go to Billing → Invoices and click Export…",
  "agent_id": "support-bot",
  "session_id": "sess_8842",
  "context": { "feature": "billing_help", "user_id": "u_123" }
}

# Returns:
# {
#   "interaction_id": "...",
#   "project_id": "...",
#   "status": "logged",
#   "feedback": "mixed",
#   "reward": 0.0,
#   "timestamp": "2026-08-08T...Z"
# }

Step 2 — Attach human feedback

Feedback is attached via a PUTon the interaction's /feedback sub-resource. There is no standalone feedback POST — log the interaction first.

PUT /api/v1/public/zerodb/{project_id}/database/rlhf/interactions/{interaction_id}/feedback
X-API-Key: <your-project-api-key>

# Thumbs signal (no rating field):
{ "feedback_type": "thumbs_up" }

# Star rating signal (1–5):
{
  "feedback_type": "rating",
  "rating": 5,
  "comment": "Exactly what I needed"
}

Step 3 — Watch quality metrics

GET /api/v1/public/zerodb/{project_id}/database/rlhf/stats
X-API-Key: <your-project-api-key>

# Returns:
# {
#   "total_interactions": 1240,
#   "average_reward": 0.62,
#   "interactions_by_feedback": { "positive": 690, "negative": 120, "mixed": 430 },
#   "interactions_by_model": { "support-bot": 1240 },
#   "total_sessions": 312,
#   "active_sessions": 18,
#   "exported_datasets": 4
# }

Step 4 — Export a training dataset

# Project-scoped export (your project API key):
POST /api/v1/public/zerodb/{project_id}/database/rlhf/export
X-API-Key: <your-project-api-key>

# Admin / global export (admin JWT — all projects, all filters):
GET /api/v1/rlhf/feedback/export
  ?format=jsonl          # jsonl | csv | parquet
  &project_id={project_id}
  &feedback=positive     # positive | negative | mixed
  &date_from=2026-07-01
  &model=support-bot
  &reward_min=0.6
  Authorization: Bearer <admin-jwt>

# JSONL row shape:
# {"interaction_id":"...","prompt":"...","response":"...",
#  "feedback":"positive","reward":0.8,"model":"support-bot",
#  "created_at":"2026-07-15T...Z"}

Building DPO preference pairs

The RLHF API is collection-first — there is no dedicated chosen/rejected pairing endpoint. Build DPO pairs at export time:

Chosen

Exported interactions with feedback "positive" (or rating ≥ 4) to the same prompt.

Rejected

Interactions with feedback "negative" (or rating ≤ 2) to the same or similar prompt.

SFT only

Use the positive set alone for supervised fine-tuning — no pairing needed.

Bulk & review helpers

Endpoint (under /api/v1/public/zerodb/{project_id}/database/rlhf)	Purpose
POST /interactions/batch	Log many interactions in one call
POST /interactions/compare	Compare two candidate responses for the same prompt
POST /interactions/single-review	Submit a single human review
GET /sessions/{session_id}	Read all interactions in a session with their feedback
GET /stats	Project-level quality metrics and feedback breakdown

Admin dashboards

Requires Authorization: Bearer <admin-jwt>

Endpoint	Shows
GET /api/v1/rlhf/dashboard/overview	Total feedback, active models, avg quality, improvement rate, pending reviews
GET /api/v1/rlhf/dashboard/deployments	Per-model interaction counts, avg rating, status
GET /api/v1/rlhf/dashboard/quality/insights	Overall quality score, trend, top issues, recommendations
GET /api/v1/rlhf/dashboard/realtime/status	Active sessions, feedback rate, queue depth, health
GET /api/v1/rlhf/feedback/export	Dataset export (JSONL/CSV/Parquet) across all projects

Patterns & anti-patterns

Patterns

Log everything, judge selectively. Record every interaction; feedback can arrive later.
Tag context. Put feature and user_id in context for per-feature insights.
Consistent scale. Pick thumbs or ratings per UI surface so metrics compare cleanly.
Export narrow. Filter by model, date, and polarity for focused training sets.

Anti-patterns

Feedback with no interaction. Feedback must reference an interaction_id — log first.
Mixing scales randomly. Half thumbs, half ratings on the same feature muddies the signal.
Untagged interactions. No context = no per-feature insight and weaker datasets.
Exporting everything, always. Unfiltered dumps produce noisy, low-quality training data.

Get the RLHF Workbook — From First Interaction to DPO Dataset

A free PDF guide: instrument feedback, log interactions, build preference pairs, and export a fine-tuning dataset — step by step.

Frequently asked questions

What is the ZeroDB RLHF API?

The RLHF API lets you log AI interactions (prompt + response), attach human feedback — thumbs up/down or a 1–5 rating — and export the resulting dataset as JSONL, CSV, or Parquet for DPO or supervised fine-tuning. It is project-scoped under /api/v1/public/zerodb/{project_id}/database/rlhf/.

How do I log an interaction?

POST to /api/v1/public/zerodb/{project_id}/database/rlhf/interactions with prompt, response, agent_id, session_id, and an optional context object. The API returns an interaction_id which you use to attach feedback later.

How do I attach human feedback to an interaction?

Send a PUT to /api/v1/public/zerodb/{project_id}/database/rlhf/interactions/{interaction_id}/feedback. Use feedback_type "thumbs_up" or "thumbs_down" for binary signals, or "rating" with a rating of 1–5 for star signals. An optional comment field captures free-text notes.

How do I build DPO preference pairs from collected feedback?

Export interactions filtered by polarity: treat "positive" (or rating ≥ 4) responses as "chosen" and "negative" (or rating ≤ 2) responses to the same prompt as "rejected". POST to /api/v1/public/zerodb/{project_id}/database/rlhf/export (project-scoped) or GET /api/v1/rlhf/feedback/export with format=jsonl (admin-scoped) to receive the JSONL dataset.

Can I log interactions in bulk?

Yes. POST to /api/v1/public/zerodb/{project_id}/database/rlhf/interactions/batch to log multiple interactions in one call. You can also compare two candidate responses for the same prompt via /rlhf/interactions/compare, or submit a single human review via /rlhf/interactions/single-review.

What export formats are supported?

JSONL, CSV, and Parquet. Filter by format, project_id, date_from/date_to, feedback polarity (positive/negative/mixed), reward_min/reward_max, model, limit, and offset. Use the project stats endpoint first to preview interaction counts and average reward before exporting.