New job scheduling delayed

Incident Report for Census

Resolved

This incident has been resolved.
Posted Mar 21, 2023 - 17:37 UTC

Monitoring

We have added capacity to remove a bottleneck in our sync engine and this has improved sync run times. We are continuing to monitor the health of the system as syncs are scheduled, executed, and retried as necessary.
Posted Mar 21, 2023 - 16:55 UTC

Identified

We have identified a related issue that is also affecting sync speeds across the system. We are continuing our investigation. Syncs may be delayed or may need to be retried (this will be done automatically)
Posted Mar 21, 2023 - 16:39 UTC

Update

We have uncovered some additional delays and are continuing our investigation
Posted Mar 21, 2023 - 16:24 UTC

Monitoring

Sync scheduling has returned to normal and we are monitoring the scheduling service
Posted Mar 21, 2023 - 15:43 UTC

Investigating

We are investigating delays in scheduling new sync jobs in Census
Posted Mar 21, 2023 - 15:27 UTC
This incident affected: Sync Engine.