Data Ingestion
Database Connectors
Pull data from PostgreSQL, MySQL, MongoDB, and Neo4j.
Database Connectors
Connect QANATIX to your databases and pull data on schedule or on demand. Credentials are encrypted at rest with Fernet (AES-128-CBC + HMAC-SHA256).
Create a connector
curl -X POST https://api.qanatix.com/api/v1/connectors \
-H "Authorization: Bearer sk_live_abc123..." \
-H "Content-Type: application/json" \
-d '{
"name": "Production Postgres",
"connector_type": "postgresql",
"vertical": "manufacturing",
"entity_type": "product",
"connection_config": {
"dsn": "postgresql://user:pass@db.internal:5432/products"
},
"query": "SELECT id, name, sku, price, stock FROM products WHERE updated_at > :last_pull",
"name_column": "name",
"batch_size": 5000
}'Credentials in connection_config are encrypted immediately. They're never returned in API responses — only ***REDACTED***.
Supported databases
PostgreSQL
Uses asyncpg with server-side cursors for memory-efficient streaming of large result sets.
{
"connector_type": "postgresql",
"connection_config": {
"dsn": "postgresql://user:pass@host:5432/dbname"
},
"query": "SELECT * FROM products WHERE updated_at > :last_pull"
}MySQL
Uses asyncmy (Cython-based) with server-side streaming cursors.
{
"connector_type": "mysql",
"connection_config": {
"host": "mysql.internal",
"port": 3306,
"user": "qanatix",
"password": "secret",
"database": "catalog"
},
"query": "SELECT * FROM items LIMIT 10000"
}MongoDB
Uses pymongo AsyncMongoClient for async document retrieval.
{
"connector_type": "mongodb",
"connection_config": {
"uri": "mongodb://user:pass@mongo.internal:27017",
"database": "products",
"collection": "catalog"
},
"query": "{\"status\": \"active\"}"
}The query field is a MongoDB filter document as a JSON string.
Neo4j
Uses neo4j Python driver v6.1 with Rust extension for 3-10x performance.
{
"connector_type": "neo4j",
"connection_config": {
"uri": "bolt://neo4j.internal:7687",
"user": "neo4j",
"password": "secret",
"database": "neo4j"
},
"query": "MATCH (p:Product)-[:MADE_BY]->(s:Supplier) RETURN p.name AS name, p.sku AS sku, s.name AS supplier"
}Pull data
Trigger a pull manually:
curl -X POST https://api.qanatix.com/api/v1/connectors/{connector_id}/pull \
-H "Authorization: Bearer sk_live_abc123..."Response:
{
"status": "completed",
"records_pulled": 2847,
"records_ingested": 2841,
"duplicates_skipped": 6,
"duration_seconds": 12.4
}List connectors
curl https://api.qanatix.com/api/v1/connectors \
-H "Authorization: Bearer sk_live_abc123..."Credentials are always redacted in responses.
Delete a connector
curl -X DELETE https://api.qanatix.com/api/v1/connectors/{connector_id} \
-H "Authorization: Bearer sk_live_abc123..."Security
- Credentials are encrypted with Fernet (AES-128-CBC + HMAC-SHA256) at rest
CONNECTOR_ENCRYPTION_KEYmust be set in production- Key rotation supported via
MultiFernet - Credentials are never returned in API responses
- Connection is established only during pull, then closed immediately