Yesterday, packages.microsoft.com—the repository from which Microsoft serves software installers for Linux distributions including CentOS, Debian, Fedora, OpenSUSE, and more—went down hard, and it stayed down for around 18 hours. The outage impacted users trying to install .NET Core, Microsoft Teams, Microsoft SQL Server for Linux (yes, that's a thing) and more—as well as Azure's own devops pipelines.

We first became aware of the problem Wednesday evening when we saw 404 errors in the output of apt update on an Ubuntu workstation with Microsoft Teams installed. The outage is somewhat better-documented at this .NET Core issue report on Github, with many users from all around the world sharing their experiences and theories.

The short version is that the entire repository cluster that serves all Linux packages for Microsoft was completely down—issuing a range of HTTP 404 (content not found) and 500 (Internal Server Error) messages for any URL—for roughly 18 hours. Microsoft engineer Rahul Bhandari confirmed the outage roughly five hours after it was initially reported, with a cryptic comment about the infrastructure team "running into some space issues."

Eighteen hours after the issue was detailed, Bhandari said that the mirrors were once again available—although with temporarily degraded performance, likely due to cold caches. In this update, Bhandari said that the original cause of the outage was "a regression in [apt repositories] during some feature migration work that resulted in those packages becoming unavailable on the mirrors."

We're still waiting for a comprehensive incident report, since Bhandari's status updates provide clues but no real explanations. The good news: We can confirm that packages.microsoft.com is indeed up once again, and it is serving packages as it should.