API Reference
Pipeline API
Embedding pipeline status and control endpoints.
Pipeline API
Monitor and control the embedding pipeline. After ingestion, entities are queued for embedding and indexing.
GET /pipeline/status/{ingestion_id}
Get embedding pipeline status for an ingestion batch.
curl https://api.qanatix.com/api/v1/pipeline/status/a1b2c3d4-... \
-H "Authorization: Bearer sk_live_abc123..."Response (200)
{
"ingestion_id": "a1b2c3d4-...",
"total": 150,
"pending": 12,
"processing": 3,
"completed": 130,
"failed": 5
}| Field | Description |
|---|---|
pending | Waiting for embedding |
processing | Currently being embedded |
completed | Successfully embedded and indexed in Qdrant |
failed | Failed after max retries (default: 3) |
POST /pipeline/trigger/{ingestion_id}
Re-trigger the embedding pipeline for pending and failed entities in this ingestion.
curl -X POST https://api.qanatix.com/api/v1/pipeline/trigger/a1b2c3d4-... \
-H "Authorization: Bearer sk_live_abc123..."Response (200)
{
"triggered": 17,
"message": "17 entities queued for embedding"
}Pipeline architecture
Ingest → Queue (SAQ/Redis) → Embed (OpenAI/BGE-M3) → Index (Qdrant) → Done
↓ (on failure)
Retry (up to 3x)
↓ (still fails)
Mark as failed| Setting | Default | Description |
|---|---|---|
PIPELINE_BATCH_SIZE | 20 | Entities per embedding batch |
PIPELINE_MAX_RETRIES | 3 | Max retry attempts |
PIPELINE_WORKER_CONCURRENCY | 10 | Parallel embedding jobs |
PIPELINE_JOB_TIMEOUT | 300s | Timeout per job |
Circuit breaker
If the embedding model fails repeatedly, the circuit breaker opens:
- Threshold: 5 consecutive failures
- Cooldown: 60 seconds
- Effect: New embedding jobs are paused until cooldown expires
This prevents cascading failures when OpenAI/Cohere is down.