Runbook: Recovering Failed Broadcast Jobs
Procedure for inspecting, resetting, and re-triggering stalled or failed campaigns.
Procedure Steps
1. Identify Stalled Campaigns
In :
manage.py shellpython
from campaigns.models import Campaign, CampaignStatus
stalled = Campaign.objects.filter(status=CampaignStatus.RUNNING)
print("Stalled campaigns:", stalled.count())
2. Reset Campaign State
Reset status to so Celery Beat picks up remaining recipients:
SCHEDULEDpython
for campaign in stalled:
campaign.status = CampaignStatus.SCHEDULED
campaign.save()
3. Re-trigger Scheduler Worker Job
bash
python manage.py shell -c "from scheduler.tasks import process_scheduler_job; process_scheduler_job.delay()"