FOCUSCurrently working on ScheduleSomething and LayerdEvents algorithmsLearn More
FOCUSCurrently working on ScheduleSomething and LayerdEvents algorithmsLearn More
FOCUSCurrently working on ScheduleSomething and LayerdEvents algorithmsLearn More
FOCUSCurrently working on ScheduleSomething and LayerdEvents algorithmsLearn More
DevelopmentEngineering Handbook
Observability & Monitoring
DocsdevelopmentEngineering HandbookObservability & Monitoring
GitHub Live Sync

Observability & Monitoring

Live technical documentation fetched from GitHub repository omghante/metapilot/docs/engineering/observability.md

Observability Standards

Guidelines for structured logging, performance metrics, Celery task monitoring, and audit logging.

Structured Logging Rules

MetaPilot uses Python's standard
logging
library configured via Django
LOGGING
settings.

Formatting Standard

Logs use a simple structured format:
text
{levelname} {asctime} {module} {message}
Example Output:
text
INFO 2026-07-24 01:15:22 scheduler_service Claimed 1 jobs for processing
ERROR 2026-07-24 01:16:04 websocket [Inbox WS] Redis connection timeout

Log Level Discipline

  • DEBUG
    : Detailed diagnostic info (SQL queries, internal variable dumps).
  • INFO
    : Normal operational events (Tenant created, Job claimed, WS client connected).
  • WARNING
    : Recoverable non-fatal events (Rate limit hit, invalid login attempt).
  • ERROR
    : Actionable errors requiring investigation (Database failures, API exceptions).
  • CRITICAL
    : Emergency failures causing system shutdown.

Audit Logging

Critical administrative actions are recorded in the
AuditLog
table (
tenants.models.AuditLog
):
  • Super Admin actions (creating/deleting tenants, role modifications).
  • Client configuration changes.
  • High-volume broadcast triggers.
Audit log attributes include:
tenant
,
user
,
action
,
resource
,
ip_address
,
details_json
,
created_at
.

Celery Worker Monitoring

Monitor Celery task queues using standard CLI tools:
bash
# View active tasks and worker stats
celery -A core inspect active
celery -A core events
For production UI monitoring, deploy Flower:
bash
celery -A core flower --port=5555