Incident with Webhooks and Actions
On November 9, 2026, from 09:23 to 11:40 UTC, one of our job queue clusters was degraded, delaying webhook deliveries, Actions workflow runs, and Dependabot and dependency graph updates. Until mitigation at 10:37 UTC, an average of 26% of webhook deliveries were delayed, peaking at 30%, and the backlog cleared by 11:40 UTC. Some deliveries were delayed by up to 3.5 hours, and delays for a small share began as early as 01:00 UTC. Actions skips events older than 30 minutes, so some workflow runs did not start. <br /><br />The incident was caused by a degraded host that had slowed one of the cluster's Kafka brokers since November 7. The broker stopped reporting the metrics our alerts rely on, so we did not identify it until it stalled the cluster on November 9. <br /><br />We mitigated the incident by removing the broker from service. We have since added an alert that pages us within 30 minutes when a host slows a broker this way, so we can remove the broker before it affects customers.
The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability.
Webhooks is experiencing degraded performance. We are continuing to investigate.
Webhooks is experiencing degraded availability. We are continuing to investigate.
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability.
We've applied a mitigation and are monitoring for recovery.
We're investigating delays in Actions runs start times and webhook deliveries.
We are investigating reports of degraded performance for Actions and Webhooks
關於這些資料
GitHub 的事件資料來自哪裡?
資料來自GitHub 官方狀態頁,每 5 分鐘同步,保留最近 90 天。詳情頁提供官方記錄連結。
如何解讀元件狀態列?
每格代表一天。綠色表示無異常記錄,黃色表示部分降級,紅色表示較大故障,灰色表示無資料。可用率統計涵蓋最近 90 天。
為什麼有些服務查不到歷史?
只收錄提供公開官方狀態頁的服務。
持續時間如何計算?
按官方標記的開始至復原時間計算。進行中的事件計算至目前時間。跨天事件會顯示在每個受影響的日期。
完整的服務清單與即時狀態見狀態總覽