By Lilly Schramm in KittyScan —

The Azure outage from the perspective of a Server Scanner

All times in GMT+1

As you have most likely noticed, yesterday Microsoft Azure had a large outage. This is the second event of this scale in just a few weeks, as just a week ago AWS had the same thing happen to them.

I do not want to open the discussion on whether it is good or bad that such outages cause halve the internet to collapse. Or the hyperreliance some companies have to these services. Instead I want to quickly detail how the outage looked and played out for me.

Since I am actively crawling Minecraft servers, my infrastructure and metrics are prominently linked to the Microsoft Azure where all official Minecraft services are hosted.

The first sign of the outage was from my Whitelist Scanners. These check if a given Server uses a Whitelist. Because an official Minecraft Account is needed to perform this check, they have to first authenticate with Microsoft and then perform the authentication handshake during the join process which requires communication with Mojang servers. At 16:41 I saw the first error message:

Post "https://sessionserver.mojang.com/session/minecraft/join": dial tcp 13.107.213.67:443: connect: connection refused

That itself is not out of the ordinary, as I get similar results from time to time, mostly when they update if I had to guess. But then it continued, from 16:44 onward every single join attempt resulted in the same error:

Post "https://sessionserver.mojang.com/session/minecraft/join": dial tcp 13.107.213.67:443: i/o timeout

My first instinct was that my IPs got blacklisted (again). While this should not happen anymore as I seem to have found a system which prevents the automatic protections to kick in, I could not exclude the possibility of them having made their filters more aggressive. So I shut the whitelist crawlers down for the day.

It wasn't until an hour later when I played around with Minestom that I noticed, that I could not authenticate. That is when I first noticed the big news.

So what do the metrics say? Let's first look at the "Online Players" graph. This graph details the amount of players that have been found in the player sample reported by servers within the last hour, that have been confirmed against the Mojang API. While this metric is far from perfect and should not be seen as an actual player count, it can still give an indication for player activity over time.

Graph titled "Oniline Players" X axis shows the time and the Y axis shows the player count, a range of the last ~3 days is selected. A regular pattern is disruped by a sharp drop.

Looking at this graph we can see that the player count reaches its daily peak around 22:00. The projection was just in line yesterday until 16:00 where it began to sharply drop, reaching its lowest point at 20:20. Shortly afterwards the player count recovered back to its usual count at 23:00.

The player count which is self-reported by the servers shows the same general trend, however it does not quite recover until today.

Graph titled "Oniline Player Distibution" X axis shows the time and the Y axis shows the amount of server with a given player count, a range of the last ~3 days is selected. A regular pattern is disruped by a sharp drop.

This was all I wanted to share. Thank you for reading and have a nice day!