Skip to Content
Troubleshooting

Troubleshooting

This guide helps resolve common issues when running or developing Eventara.

Installation Issues

Docker Compose Fails to Start

Symptom: docker compose up fails or services don’t start

Possible Causes and Solutions:

  1. Port conflicts:

    # Check if ports are in use lsof -i :5432 # PostgreSQL lsof -i :9092 # Kafka lsof -i :8080 # Spring Boot lsof -i :5173 # Dashboard # Solution: Stop conflicting services or change ports in docker-compose.yaml
  2. Docker resources:

    # Check Docker resources docker system df # Clean up docker system prune -a docker volume prune
  3. Previous containers:

    # Stop and remove all containers docker compose down -v # Restart fresh docker compose up --build -d

Maven Build Fails

Symptom: ./mvnw clean package fails

Solutions:

  1. Check Java version:

    java -version # Should be 21+
  2. Clean and rebuild:

    ./mvnw clean ./mvnw package -DskipTests
  3. Dependency issues:

    # Force update dependencies ./mvnw clean install -U

npm Install Fails

Symptom: npm install in eventara-dashboard fails

Solutions:

  1. Clear cache:

    npm cache clean --force rm -rf node_modules package-lock.json npm install
  2. Check Node version:

    node --version # Should be 18+

Runtime Issues

Events Not Being Ingested

Symptom: POST /api/v1/events returns error or events don’t appear

Debugging Steps:

  1. Check API health:

    curl http://localhost:8080/api/v1/events/health # Expected: {"status":"UP","timestamp":"..."}
  2. Check logs:

    docker compose logs -f springboot | grep ERROR
  3. Test with minimal payload:

    curl -X POST http://localhost:8080/api/v1/events \ -H "Content-Type: application/json" \ -d '{"eventType":"test","source":"test"}'
  4. Verify Kafka connection:

    docker compose logs kafka | grep -i error
  5. Check database connection:

    docker exec postgres14 psql -U postgres -d eventara -c "SELECT COUNT(*) FROM events;"

Kafka Consumer Not Processing Events

Symptom: Events sent to API but not saved to database

Debugging Steps:

  1. Check consumer logs:

    docker compose logs springboot | grep EventConsumer
  2. Verify topic exists:

    docker exec eventara-kafka kafka-topics \ --list \ --bootstrap-server localhost:9092
  3. Check messages in topic:

    docker exec eventara-kafka kafka-console-consumer \ --bootstrap-server localhost:9092 \ --topic eventara.events.raw \ --from-beginning \ --max-messages 10
  4. Check consumer group:

    docker exec eventara-kafka kafka-consumer-groups \ --bootstrap-server localhost:9092 \ --group eventara-consumer-group \ --describe
  5. Consumer lag:

    # If LAG is increasing, consumer is falling behind # Check for errors in logs docker compose logs springboot | grep -A 5 -B 5 "consumer error"

WebSocket Connection Fails

Symptom: Dashboard shows “Connecting…” or connection errors

Debugging Steps:

  1. Check WebSocket endpoint:

    curl -I http://localhost:8080/ws/info # Should return 200 OK
  2. Browser console:

    • Open DevTools (F12)
    • Check Console for errors
    • Check Network tab → WS filter → Look for WebSocket connection
  3. Backend WebSocket logs:

    docker compose logs springboot | grep WebSocket
  4. CORS issues:

    • Check if dashboard URL is allowed
    • Verify @CrossOrigin annotations in controllers
  5. Test with static page:

    • Access http://localhost:8080/static/test-websocket.html
    • Should establish connection and receive messages

Dashboard Not Loading

Symptom: Dashboard page blank or won’t load

Solutions:

  1. Check dashboard service:

    docker compose logs dashboard
  2. Verify API URL:

    • Check eventara-dashboard/.env
    • Should be: VITE_API_URL=http://localhost:8080
  3. Rebuild dashboard:

    cd eventara-dashboard npm install npm run dev
  4. Browser cache:

    • Hard refresh: Ctrl+Shift+R (or Cmd+Shift+R on Mac)
    • Clear browser cache

Database Issues

Flyway Migration Fails

Symptom: Application fails to start with Flyway error

Solutions:

  1. Check migration files:

    ls -la src/main/resources/db/migration/ # Should be: V1__, V2__, etc. (no gaps)
  2. Reset Flyway schema:

    -- Connect to database docker exec -it postgres14 psql -U postgres -d eventara -- Drop Flyway table DROP TABLE IF EXISTS flyway_schema_history CASCADE; -- Restart application
  3. Fresh database:

    docker compose down -v docker compose up -d postgres # Wait for PostgreSQL to be ready docker compose up -d springboot

Cannot Connect to PostgreSQL

Symptom: Connection refused or authentication failed

Solutions:

  1. Check PostgreSQL is running:

    docker compose ps postgres
  2. Verify credentials:

    • Check application.properties
    • Should match docker-compose.yaml environment variables
  3. Test connection:

    docker exec -it postgres14 psql -U postgres -d eventara
  4. Check logs:

    docker compose logs postgres | grep ERROR

Database Query Slow

Symptom: Slow response times or timeouts

Solutions:

  1. Check table size:

    SELECT schemaname, tablename, pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) FROM pg_tables WHERE schemaname = 'public';
  2. Verify indexes:

    SELECT * FROM pg_indexes WHERE tablename = 'events';
  3. Query statistics:

    SELECT * FROM pg_stat_user_tables WHERE relname = 'events';
  4. Vacuum database:

    docker exec postgres14 vacuumdb -U postgres -d eventara -v

Performance Issues

High Memory Usage

Symptom: Application using too much memory or OOM errors

Solutions:

  1. Check JVM heap:

    # Add to docker-compose.yaml environment: JAVA_OPTS: "-Xms512m -Xmx2g"
  2. Heap dump analysis:

    # Enable heap dump on OOM JAVA_OPTS: "-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp" # Analyze with jhat or VisualVM
  3. Metrics service memory:

    • ComprehensiveMetricsService stores last 1000 latency samples
    • Last 24 hours of event timestamps
    • Consider reducing retention if memory-constrained

High CPU Usage

Symptom: CPU at 100% or sluggish performance

Solutions:

  1. Profile application:

    # Use JProfiler, YourKit, or VisualVM # Connect to JVM on port 9010 (if enabled)
  2. Check scheduled tasks:

    docker compose logs springboot | grep "@Scheduled"
  3. Reduce broadcast frequency:

    // In MetricsWebSocketController.java @Scheduled(fixedRate = 5000) // Change from 1000ms to 5000ms

Slow Event Ingestion

Symptom: High latency or timeouts on POST requests

Debugging:

  1. Check processing latency:

    curl http://localhost:8080/api/v1/metrics | jq '.latencyMetrics.overall'
  2. Kafka producer config:

    # Increase batch size for throughput spring.kafka.producer.properties.batch.size=32768 spring.kafka.producer.properties.linger.ms=20
  3. Database connection pool:

    spring.datasource.hikari.maximum-pool-size=20 spring.datasource.hikari.minimum-idle=5
  4. Check consumer lag:

    docker exec eventara-kafka kafka-consumer-groups \ --bootstrap-server localhost:9092 \ --group eventara-consumer-group \ --describe

Alert Rule Issues

Rules Not Triggering

Symptom: Alert rule created but never triggers

Debugging Steps:

  1. Check rule status:

    curl http://localhost:8080/api/v1/rules/{id} # Verify status is "ACTIVE"
  2. Test rule:

    curl -X POST http://localhost:8080/api/v1/rules/test/{id} # Check simulation result
  3. Verify metrics match conditions:

    curl http://localhost:8080/api/v1/metrics | jq '.summary' # Compare with rule threshold
  4. Check DRL compilation:

    docker compose logs springboot | grep "DRL"
  5. Rule execution logs:

    docker compose logs springboot | grep RuleExecutionService

Rule Compilation Errors

Symptom: Error when creating or testing rule

Solutions:

  1. Validate rule configuration:

    • Check all required fields are present
    • Verify operator names are correct
    • Ensure threshold types match metric types
  2. Test before creating:

    curl -X POST http://localhost:8080/api/v1/rules/test \ -H "Content-Type: application/json" \ -d @rule-config.json
  3. Check validation errors:

    # Response includes validationErrors array jq '.validationErrors' response.json

Common Error Messages

”Connection refused”

Cause: Service not running or wrong port

Solution:

# Check service status docker compose ps # Restart service docker compose restart <service-name>

“Table ‘events’ doesn’t exist”

Cause: Flyway migrations not run

Solution:

# Check Flyway status docker compose logs springboot | grep Flyway # Restart to trigger migrations docker compose restart springboot

“Kafka broker not available”

Cause: Kafka not ready or network issue

Solution:

# Wait for Kafka to be ready docker compose logs kafka | grep "started" # Check health docker exec eventara-kafka kafka-broker-api-versions \ --bootstrap-server localhost:9092

“Out of memory error”

Cause: JVM heap too small

Solution:

# In docker-compose.yaml springboot: environment: JAVA_OPTS: "-Xmx4g" # Increase heap size

Debugging Tools

Check All Services Health

#!/bin/bash echo "PostgreSQL:" docker exec postgres14 pg_isready echo -e "\nKafka:" docker exec eventara-kafka kafka-broker-api-versions \ --bootstrap-server localhost:9092 2>&1 | head -n 1 echo -e "\nSpring Boot:" curl -s http://localhost:8080/api/v1/events/health | jq echo -e "\nDashboard:" curl -s -o /dev/null -w "%{http_code}" http://localhost:5173

Monitor Real-Time Logs

# All services docker compose logs -f --tail=100 # Specific service with filtering docker compose logs -f springboot | grep -E "ERROR|WARN"

Kafka Debugging

# List topics docker exec eventara-kafka kafka-topics \ --list \ --bootstrap-server localhost:9092 # Consume messages docker exec eventara-kafka kafka-console-consumer \ --bootstrap-server localhost:9092 \ --topic eventara.events.raw \ --from-beginning # Producer test docker exec -it eventara-kafka kafka-console-producer \ --bootstrap-server localhost:9092 \ --topic eventara.events.raw

Database Debugging

# Connect to PostgreSQL docker exec -it postgres14 psql -U postgres -d eventara # Useful queries: -- Count events SELECT COUNT(*) FROM events; -- Recent events SELECT event_id, event_type, source, timestamp FROM events ORDER BY timestamp DESC LIMIT 10; -- Events by type SELECT event_type, COUNT(*) FROM events GROUP BY event_type ORDER BY COUNT(*) DESC; -- Database size SELECT pg_size_pretty(pg_database_size('eventara'));

Getting Help

If you encounter issues not covered here:

  1. Check logs: Most issues leave traces in logs
  2. GitHub Issues: Search existing issues or create new one
  3. Community: Join discussions on GitHub
  4. Documentation: Review other documentation pages

Reporting Bugs

When reporting issues, include:

  • Eventara version
  • Docker/Java/Node versions
  • Steps to reproduce
  • Error messages and logs
  • Expected vs actual behavior
  • Environment (OS, Docker setup, etc.)
Last updated on