Troubleshooting

Common issues and solutions

This guide covers common issues when running the Conserver and how to resolve them.

Diagnostic Commands

Check System Health

# Redis connectivity
redis-cli ping
# Expected: PONG

# API health
curl -H "x-conserver-api-token: $TOKEN" http://localhost:8000/api/config

# Queue status
redis-cli LLEN incoming_calls
redis-cli LLEN incoming_calls:dlq

View Logs

# Docker Compose logs
docker compose logs -f conserver-api
docker compose logs -f conserver-worker

# Filter for errors
docker compose logs conserver-worker 2>&1 | grep -i error

Connection Issues

Redis Connection Failed

Symptoms:

Workers won't start
"Connection refused" errors
API returns 500 errors

Diagnosis:

# Check Redis is running
docker compose ps redis

# Check Redis connectivity
redis-cli -h localhost -p 6379 ping

# Check Redis logs
docker compose logs redis

Solutions:

Verify Redis URL:

# In .env
REDIS_URL=redis://localhost:6379

# For Docker networking
REDIS_URL=redis://redis:6379

Check Docker networking:

docker network ls
docker network inspect vcon-server_default

Redis memory full:

redis-cli INFO memory
# If used_memory > maxmemory, increase or clear old data

API Authentication Failed

Symptoms:

403 Forbidden responses
"Invalid API Key" errors

Solutions:

Check token configuration:

# Verify environment variable
echo $CONSERVER_API_TOKEN

# Check token file exists
cat /path/to/api_tokens.txt

Check header name:

# Default header
curl -H "x-conserver-api-token: $TOKEN" ...

# Custom header (if CONSERVER_HEADER_NAME is set)
curl -H "x-custom-header: $TOKEN" ...

For external ingress, check ingress_auth config:

ingress_auth:
  my_ingress_list: "correct-api-key"

Processing Issues

vCons Not Being Processed

Symptoms:

Queue depth keeps growing
No worker activity in logs
vCons stuck in ingress list

Diagnosis:

# Check queue depth
redis-cli LLEN incoming_calls

# Check workers are running
docker compose ps

# Check worker logs
docker compose logs conserver-worker --tail 100

Solutions:

Chain not enabled:

chains:
  main:
    enabled: 1  # Must be 1, not 0 or false

Ingress list mismatch:

chains:
  main:
    ingress_lists:
      - incoming_calls  # Must match where vCons are pushed

Workers crashed:
```
docker compose restart conserver-worker
```

Configuration not loaded:

# Check current config
curl -H "x-conserver-api-token: $TOKEN" http://localhost:8000/api/config

High DLQ Count

Symptoms:

vCons accumulating in dead letter queue
Processing errors in logs

Diagnosis:

# Check DLQ depth
redis-cli LLEN incoming_calls:dlq

# Get DLQ contents
curl -H "x-conserver-api-token: $TOKEN" \
  "http://localhost:8000/api/dlq?ingress_list=incoming_calls"

# Check recent errors
docker compose logs conserver-worker 2>&1 | grep -i error | tail 50

Solutions:

Identify the failing link:
- Check logs for which link is failing
- Look for exceptions and stack traces

Common link failures:

API key missing/invalid:

links:
  analyze:
    options:
      OPENAI_API_KEY: ${OPENAI_API_KEY}  # Check env var is set

External service down:

# Test connectivity
curl https://api.openai.com/v1/models -H "Authorization: Bearer $OPENAI_API_KEY"

Timeout:

chains:
  main:
    timeout: 600  # Increase for long transcriptions

Reprocess after fixing:

curl -X POST -H "x-conserver-api-token: $TOKEN" \
  "http://localhost:8000/api/dlq/reprocess?ingress_list=incoming_calls"

Link Timeout

Symptoms:

"Processing timeout" errors
vCons moving to DLQ after timeout period

Solutions:

Increase chain timeout:

chains:
  main:
    timeout: 600  # Seconds

Optimize processing:
- Use faster models
- Skip unnecessary processing steps
- Add sampling to process fewer vCons

Check external API performance:

# Time an API call
time curl https://api.example.com/endpoint

Transcription Issues

No Transcription Generated

Symptoms:

vCon has no transcript analysis
Transcription link completes but adds nothing

Diagnosis:

# Check vCon dialogs
curl -H "x-conserver-api-token: $TOKEN" \
  "http://localhost:8000/api/vcon/$VCON_UUID" | jq '.dialog'

Solutions:

Check dialog type:

// Dialog must be type "recording"
{"type": "recording", "url": "https://..."}

Check duration requirement:

links:
  transcribe:
    options:
      minimum_duration: 30  # Audio must be >= 30 seconds

Check audio URL is accessible:
```
curl -I "https://your-audio-url.mp3"
```

Check API key:

links:
  deepgram:
    options:
      DEEPGRAM_KEY: ${DEEPGRAM_KEY}  # Verify env var

Transcription Already Exists

Symptoms:

"Dialog already transcribed" in logs
Link skipping dialogs

Explanation: Links skip processing if analysis already exists (idempotency).

If you need to re-transcribe:

Delete the vCon and re-submit
Or create a new link with different analysis_type

Storage Issues

Storage Write Failed

Symptoms:

"Failed to save" errors
vCon not appearing in storage backend

Diagnosis:

# Check storage backend connectivity
# For PostgreSQL:
psql -h localhost -U postgres -d vcons -c "SELECT 1"

# For S3:
aws s3 ls s3://your-bucket/

Solutions:

Check credentials:

storages:
  postgres:
    options:
      password: ${POSTGRES_PASSWORD}  # Verify env var

Check network connectivity:

# From container
docker exec conserver-worker ping postgres

Check permissions:
- Database user has write permissions
- S3 IAM role allows PutObject

vCon Not Found

Symptoms:

404 when fetching vCon
"vCon not found" errors

Diagnosis:

# Check Redis
redis-cli EXISTS vcon:$VCON_UUID

# Check sorted set
redis-cli ZSCORE vcons vcon:$VCON_UUID

Solutions:

vCon expired from Redis:
- Check VCON_REDIS_EXPIRY setting
- vCon should be auto-fetched from storage if configured
vCon never stored:
- Check chain has storage backends configured
- Check storage write didn't fail
Wrong UUID format:
- Ensure UUID includes hyphens
- Check for typos

Configuration Issues

Configuration Not Loading

Symptoms:

Changes to config.yml not taking effect
Wrong settings being used

Solutions:

Check config file path:
```
echo $CONSERVER_CONFIG_FILE
```

Validate YAML syntax:

python -c "import yaml; yaml.safe_load(open('config.yml'))"

Restart workers:
```
docker compose restart conserver-worker
```

Use API to update:

curl -X POST -H "x-conserver-api-token: $TOKEN" \
  -H "Content-Type: application/json" \
  -d @config.json \
  http://localhost:8000/api/config

Import/Module Not Found

Symptoms:

"ModuleNotFoundError" in logs
Dynamic imports failing

Solutions:

Check imports section:

imports:
  my_module:
    module: my_module
    pip_name: my-package>=1.0.0  # Correct package name

Check pip_name format:

# PyPI
pip_name: package-name>=1.0.0

# GitHub
pip_name: git+https://github.com/org/repo.git@tag

Check network access:
- Container can reach PyPI/GitHub
- No firewall blocking

Performance Issues

Slow Processing

Symptoms:

High processing times
Queue backing up

Solutions:

Scale workers:

docker compose up -d --scale conserver-worker=5

Use sampling:

links:
  analyze:
    options:
      sampling_rate: 0.1  # Process 10%

Use faster models:

links:
  analyze:
    options:
      model: "gpt-3.5-turbo"  # Faster than gpt-4

Skip unnecessary links:

chains:
  fast_chain:
    links:
      - transcribe
      # Skip heavy analysis for speed

Memory Issues

Symptoms:

OOM errors
Container restarts

Solutions:

Increase container memory:

services:
  conserver-worker:
    deploy:
      resources:
        limits:
          memory: 8G

Configure Redis maxmemory:

redis-cli CONFIG SET maxmemory 2gb
redis-cli CONFIG SET maxmemory-policy allkeys-lru

Use diet link to reduce vCon size:

links:
  slim_down:
    module: links.diet
    options:
      remove_dialog_bodies: true

Getting Help

If you can't resolve an issue:

Check logs carefully - Most errors have clear messages
Search GitHub issues - Someone may have had the same problem
Create a GitHub issue with:
- Conserver version
- Configuration (sanitized)
- Error messages
- Steps to reproduce

PreviousProduction Deployment NextMCP Server

Last updated 1 month ago

Was this helpful?

hashtagDiagnostic Commands

hashtagCheck System Health

hashtagView Logs

hashtagConnection Issues

hashtagRedis Connection Failed

hashtagAPI Authentication Failed

hashtagProcessing Issues

hashtagvCons Not Being Processed

hashtagHigh DLQ Count

hashtagLink Timeout

hashtagTranscription Issues

hashtagNo Transcription Generated

hashtagTranscription Already Exists

hashtagStorage Issues

hashtagStorage Write Failed

hashtagvCon Not Found

hashtagConfiguration Issues

hashtagConfiguration Not Loading

hashtagImport/Module Not Found

hashtagPerformance Issues

hashtagSlow Processing

hashtagMemory Issues

hashtagGetting Help

Diagnostic Commands

Check System Health

View Logs

Connection Issues

Redis Connection Failed

API Authentication Failed

Processing Issues

vCons Not Being Processed

High DLQ Count

Link Timeout

Transcription Issues

No Transcription Generated

Transcription Already Exists

Storage Issues

Storage Write Failed

vCon Not Found

Configuration Issues

Configuration Not Loading

Import/Module Not Found

Performance Issues

Slow Processing

Memory Issues

Getting Help