Previous incidents

The intermittent issues affecting Log Search and Inbox Search have been resolved. Both search experiences are operating normally.

We identified the root cause as a load-balancing issue between our PgBouncer instances at one of our infrastructure providers. Under high load, particularly during deployments, this caused intermittent message-processing delays. We have implemented mitigation measures to prevent this issue from recurring.

We have identified the source of the issue within one of our infrastructure providers. We are taking measures to improve resilience and mitigate the impact of similar provider-side disruptions in the future.

This incident has been resolved. Voice audio message delivery is operating normally again.

This incident has been resolved. Instant Setup pre-verified numbers are available again.

The incident lasted approximately 13 minutes and originated in asynchronous workflow processing. One transaction kept a workflow execution blocked while other jobs converged on it, creating a queue that exhausted the database connection pool. We restored service by terminating the blocking transaction and deployed no-wait locks, along with isolated connection pools for web and worker processes, to limit future contention.

A database configuration issue temporarily affected some connections, causing brief write failures for some projects. The issue has been resolved, and we are implementing measures to prevent this from recurring.

Log processing is back to normal. This incident has been resolved.