跨多个服务的延迟问题

2026年7月23日，协调世界时07:08至09:39期间，多项服务出现延迟：8%的Actions工作流运行平均启动延迟10分钟，5%的Webhook投递超出服务等级目标，代码扫描、仓库、通知、议题及拉取请求在事件期间均经历了延迟增加。<br /><br />事件的根本原因是我们后台作业处理系统的一个节点在进入计划主机维护后未能恢复。通过识别问题分片并恢复其正确状态，事件得到缓解，随后队列积压得以清空，服务恢复正常。<br /><br />为加快缓解速度，我们已为维护操作后处于此不健康状态的节点添加了监控。为防止未来再次发生，我们正在调整生命周期自动化，以在计划重启后验证主机重新加入。

原文On July 23, 2026, between 07:08 and 09:39 UTC, several services experienced delays: 8% of actions workflow runs experienced an average run start delay of 10 minutes, 5% of webhook deliveries exceeded SLO, and code scanning, repos, notifications, issues and pull requests experienced increased latency over the life of the incident. <br /><br />The root cause of the incident was a node of our background job processing system which did not recover after entering scheduled host maintenance. The incident was mitigated by identifying the problematic shard and restoring its correct state, after which queue backlogs drained and services recovered. <br /><br />To speed mitigation, we have added monitors for nodes in this unhealthy state after maintenance operations. To prevent future recurrence, we are adapting our lifecycle automation to verify host rejoin after a scheduled reboot.

故障已缓解。我们正在监控以确保稳定性。

原文The degradation has been mitigated. We are monitoring to ensure stability.

Webhooks 运行正常。

原文Webhooks is operating normally.

影响Pull Requests的服务降级问题已得到缓解。我们正在持续监控以确保稳定性。

原文The degradation affecting Pull Requests has been mitigated. We are monitoring to ensure stability.

我们已定位影响多项服务的延迟源头并应用了修复。相关问题和操作正在恢复中，其余受影响服务随着处理积压的清除也在逐步改善。我们正积极监控所有服务的恢复情况。

原文We identified the source of latency affecting multiple services and applied a fix. Issues and Actions are recovering, and remaining affected services are seeing improvement as processing backlogs clear. We are actively monitoring recovery across all services.

影响Actions的服务降级问题已得到缓解。我们正在监控以确保稳定性。

原文The degradation affecting Actions has been mitigated. We are monitoring to ensure stability.

影响Issues的服务降级问题已得到缓解。我们正在监控以确保稳定性。

原文The degradation affecting Issues has been mitigated. We are monitoring to ensure stability.

我们目前正在调查多个服务的延迟问题。这可能导致Actions任务启动时间变长、Issues搜索返回过期结果，以及其他列出的服务受到类似影响。

原文We're currently investigating latency across multiple services. This can show as Actions jobs taking longer to start, Issues search serving stale results, and other listed services being similarly impacted.

Pull Requests 正经历性能下降。我们正在继续调查。

原文Pull Requests is experiencing degraded performance. We are continuing to investigate.

我们正在调查有关Actions、Issues和Webhooks可用性下降的报告。

原文We are investigating reports of degraded availability for Actions, Issues and Webhooks