Services too, not just the portal.","title":null,"type":"comment","url":null},{"author":"andhuman","children":[{"author":"andhuman","children":[],"created_at":"2025-10-29T16:40:44.000Z","created_at_i":1761756044,"id":45749374,"options":[],"parent_id":45748746,"points":null,"story_id":45748661,"text":"\u201c Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or sooner.
This message was last updated at 16:35 UTC on 29 October 2025\u201d","title":null,"type":"comment","url":null},{"author":"pbhjpbhj","children":[],"created_at":"2025-10-30T13:53:38.000Z","created_at_i":1761832418,"id":45760049,"options":[],"parent_id":45748746,"points":null,"story_id":45748661,"text":"That was my bet too, then I looked at ISC and noticed there were PoCs released for critical BIND9 vulns yesterday ... might be related?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:05:17.000Z","created_at_i":1761753917,"id":45748746,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I bet it\u2019s DNS.","title":null,"type":"comment","url":null},{"author":"joaomoreno","children":[],"created_at":"2025-10-29T16:07:43.000Z","created_at_i":1761754063,"id":45748789,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Yup, see it as well.","title":null,"type":"comment","url":null},{"author":"xuf","children":[],"created_at":"2025-10-29T16:07:51.000Z","created_at_i":1761754071,"id":45748792,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Down here too (region West Europe)","title":null,"type":"comment","url":null},{"author":"voidpointer2000","children":[],"created_at":"2025-10-29T16:08:24.000Z","created_at_i":1761754104,"id":45748797,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Down in Sweden Central as well (all our production systems are down)","title":null,"type":"comment","url":null},{"author":"chemodax","children":[{"author":"martini333","children":[{"author":"endianswap","children":[{"author":"Sharparam","children":[{"author":"12_throw_away","children":[],"created_at":"2025-10-29T18:12:40.000Z","created_at_i":1761761560,"id":45750821,"options":[],"parent_id":45749792,"points":null,"story_id":45748661,"text":"Heh, now it says Front Door and "Network Infrastructure" are down. That second one seems bad.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:06:15.000Z","created_at_i":1761757575,"id":45749792,"options":[],"parent_id":45749482,"points":null,"story_id":45748661,"text":"Only for the Azure Portal, despite Front Door also being down but showing as green on the status page.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:47:30.000Z","created_at_i":1761756450,"id":45749482,"options":[],"parent_id":45748839,"points":null,"story_id":45748661,"text":"It's red right now.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:11:02.000Z","created_at_i":1761754262,"id":45748839,"options":[],"parent_id":45748806,"points":null,"story_id":45748661,"text":"That status page is never red. Absolutely useless.
> There are currently no active events. Use Azure Service Health to view other issues that may be impacting your services.
Links to a page on Azure Portal which is down...","title":null,"type":"comment","url":null},{"author":"kylecazar","children":[],"created_at":"2025-10-29T16:29:05.000Z","created_at_i":1761755345,"id":45749163,"options":[],"parent_id":45748806,"points":null,"story_id":45748661,"text":"They added a message at the same time as your comment:
"We are investigating an issue with the Azure Portal where customers may be experiencing issues accessing the portal. More information will be provided shortly."","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:08:50.000Z","created_at_i":1761754130,"id":45748806,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"For me the same. It's very confusing that status page [1] is green
[1]: https://azure.status.microsoft/en-us/status","title":null,"type":"comment","url":null},{"author":"reid","children":[{"author":"etyhhgfff","children":[],"created_at":"2025-10-29T17:56:44.000Z","created_at_i":1761760604,"id":45750594,"options":[],"parent_id":45748826,"points":null,"story_id":45748661,"text":"It's always DNS, unless it's not DNS.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:10:14.000Z","created_at_i":1761754214,"id":45748826,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"This is impacting the Azure CDN at azureedge.net. DNS A records for azureedge.net tenants are taking 2-6 seconds and often return nothing.","title":null,"type":"comment","url":null},{"author":"uuuubbbb","children":[],"created_at":"2025-10-29T16:11:05.000Z","created_at_i":1761754265,"id":45748840,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Intune, Azure, Entra down in Switzerland","title":null,"type":"comment","url":null},{"author":"elFarto","children":[{"author":"sech8420","children":[],"created_at":"2025-10-29T16:14:17.000Z","created_at_i":1761754457,"id":45748897,"options":[],"parent_id":45748842,"points":null,"story_id":45748661,"text":"Same exact time for us as well.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:11:08.000Z","created_at_i":1761754268,"id":45748842,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"We saw all incoming traffic to our app drop to zero at about 15:45. I wonder how long this one will take to fix.","title":null,"type":"comment","url":null},{"author":"vincebowdren","children":[],"created_at":"2025-10-29T16:12:20.000Z","created_at_i":1761754340,"id":45748863,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"UK, and other regions too; our APAC installation in Australia is affected.","title":null,"type":"comment","url":null},{"author":"kierenj","children":[{"author":"ocdtrekkie","children":[{"author":"NetMageSCW","children":[],"created_at":"2025-10-29T17:02:36.000Z","created_at_i":1761757356,"id":45749727,"options":[],"parent_id":45749484,"points":null,"story_id":45748661,"text":"We are very dependent on Azure and Microsoft Authentication and Microsoft 365 and haven\u2019t had weekly or even monthly issues. I can think of maybe three issues this year.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:47:36.000Z","created_at_i":1761756456,"id":45749484,"options":[],"parent_id":45748903,"points":null,"story_id":45748661,"text":"I am still stunned people choose to do this, considering major Office 365 outages are basically a weekly thing now.","title":null,"type":"comment","url":null},{"author":"parliament32","children":[],"created_at":"2025-10-29T16:48:46.000Z","created_at_i":1761756526,"id":45749507,"options":[],"parent_id":45748903,"points":null,"story_id":45748661,"text":"SSO and 365 are working fine for us, but admin portals for Azure/365 are down. Our workloads in Azure don't seem to be impacted.","title":null,"type":"comment","url":null},{"author":"planewave","children":[],"created_at":"2025-10-29T17:13:29.000Z","created_at_i":1761758009,"id":45749902,"options":[],"parent_id":45748903,"points":null,"story_id":45748661,"text":"It is interesting to see the differential across different tenants in different geographies:
- on a US tenant I am unable to access login.microsoftonline.com and the login flow stalls on any SSO authentication attempt.
- on a European tenant, probably germany-west, I am able to login and access the Azure portal.","title":null,"type":"comment","url":null},{"author":"manbitesdog","children":[{"author":"btbuildem","children":[],"created_at":"2025-10-29T18:30:36.000Z","created_at_i":1761762636,"id":45751066,"options":[],"parent_id":45749907,"points":null,"story_id":45748661,"text":"It's Safe and Secure!","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:13:56.000Z","created_at_i":1761758036,"id":45749907,"options":[],"parent_id":45748903,"points":null,"story_id":45748661,"text":"Guess you have NASSO now (Not A Single Sign On)","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:14:39.000Z","created_at_i":1761754479,"id":45748903,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Ouch, and login.microsoftonline.com too - i.e. SSO using MS accounts. We'd just rolled that out across most (all?) of our internal systems...
And microsoft.com too - that's gotta hurt","title":null,"type":"comment","url":null},{"author":"patching-trowel","children":[{"author":"charles_f","children":[],"created_at":"2025-10-29T16:27:29.000Z","created_at_i":1761755249,"id":45749126,"options":[],"parent_id":45748927,"points":null,"story_id":45748661,"text":"It shows that some people have issues accessing the portal.","title":null,"type":"comment","url":null},{"author":"baconbrand","children":[],"created_at":"2025-10-29T16:29:44.000Z","created_at_i":1761755384,"id":45749177,"options":[],"parent_id":45748927,"points":null,"story_id":45748661,"text":"I have never noticed that page being updated in a timely manner.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:16:12.000Z","created_at_i":1761754572,"id":45748927,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"As of now Azure Status page still shows no incident. It must be manually updated, someone has to actively decide to acknowledge an issue, and they're just... not. It undermines confidence in that status page.","title":null,"type":"comment","url":null},{"author":"reid","children":[],"created_at":"2025-10-29T16:16:42.000Z","created_at_i":1761754602,"id":45748933,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Portal and Azure CDN are down here in the SF Bay Area. Tenant azureedge.net DNS A queries are taking 2-6 seconds and most often return nothing. I got a couple successful A response in the last 10 minutes.
Edit: As of 9:19 AM Pacific time, I'm now getting successful A responses but they can take several seconds. The web server at that address is not responding.","title":null,"type":"comment","url":null},{"author":"bronco21016","children":[],"created_at":"2025-10-29T16:17:03.000Z","created_at_i":1761754623,"id":45748937,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Unable to access the portal and any hit to SSO for other corporate accesses is also broken. Seems like there's something wrong in their Identity services.","title":null,"type":"comment","url":null},{"author":"blenderob","children":[{"author":"reid","children":[],"created_at":"2025-10-29T16:18:31.000Z","created_at_i":1761754711,"id":45748965,"options":[],"parent_id":45748946,"points":null,"story_id":45748661,"text":"You're looking at it. I couldn't find any discussion elsewhere yet...","title":null,"type":"comment","url":null},{"author":"sbergot","children":[{"author":"jeffrallen","children":[],"created_at":"2025-10-29T20:48:31.000Z","created_at_i":1761770911,"id":45752817,"options":[],"parent_id":45749033,"points":null,"story_id":45748661,"text":"I work for a cloud provider which is serious about transparency. Our customers know they are going to get the straight story from our status page.
When you find an honest vendor, cherish them. They are rare, and they work hard to earn and keep your confidence.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:22:39.000Z","created_at_i":1761754959,"id":45749033,"options":[],"parent_id":45748946,"points":null,"story_id":45748661,"text":"official status pages are useless most of the time.","title":null,"type":"comment","url":null},{"author":"sbergot","children":[],"created_at":"2025-10-29T16:23:54.000Z","created_at_i":1761755034,"id":45749056,"options":[],"parent_id":45748946,"points":null,"story_id":45748661,"text":"now there is an information about "Azure Portal Access Issues". No word about front door being down.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:17:36.000Z","created_at_i":1761754656,"id":45748946,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"https://azure.status.microsoft/en-us/status says everything's fine! Any place I can read more about this outage?","title":null,"type":"comment","url":null},{"author":"kryogen1c","children":[],"created_at":"2025-10-29T16:18:23.000Z","created_at_i":1761754703,"id":45748961,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"downdetector reports coincident cloudflare outage. is microsoft using cloudflare for management plane, or is there common infra? data center problem somewhere, maybe fiber backbone? BGP?","title":null,"type":"comment","url":null},{"author":"chemodax","children":[],"created_at":"2025-10-29T16:18:48.000Z","created_at_i":1761754728,"id":45748968,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"It seems Azure FrontDoor is affected, because our private VM works fine in different regions.","title":null,"type":"comment","url":null},{"author":"rluhar","children":[{"author":"zavec","children":[{"author":"estel","children":[{"author":"zavec","children":[],"created_at":"2025-10-29T16:59:15.000Z","created_at_i":1761757155,"id":45749671,"options":[],"parent_id":45749506,"points":null,"story_id":45748661,"text":"Definitely also a strong possibility. I wish I had paid more attention during the AWS one earlier to see what other things looked like on there at the time.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:48:45.000Z","created_at_i":1761756525,"id":45749506,"options":[],"parent_id":45749377,"points":null,"story_id":45748661,"text":"It's possibly more likely that people mis-attribute the cause of an outage to the wrong providers when they use downdetector.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:40:53.000Z","created_at_i":1761756053,"id":45749377,"options":[],"parent_id":45748979,"points":null,"story_id":45748661,"text":"Yeah the graph for that one looks exactly the same shape. I wonder if they were depending on some azure component somehow, or maybe there were things hosted on both and the azure failure made enough things failover to AWS that AWS couldn't cope? If that was the case I'd expect to see something similar with GCP too though.
Edit: nope looks like there's actually a spike on GCP as well","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:19:20.000Z","created_at_i":1761754760,"id":45748979,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Looks like AWS is also impacted?","title":null,"type":"comment","url":null},{"author":"kryogen1c","children":[{"author":"Mr_Bees69","children":[],"created_at":"2025-10-29T17:03:30.000Z","created_at_i":1761757410,"id":45749742,"options":[],"parent_id":45748991,"points":null,"story_id":45748661,"text":"nope, dont see any cf issues.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:20:07.000Z","created_at_i":1761754807,"id":45748991,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"downdetector reports coincident cloudflare outage. is microsoft using cloudflare for management plane, or is there common infra? data center problem somewhere, maybe fiber backbone? BGP?","title":null,"type":"comment","url":null},{"author":"kierenj","children":[],"created_at":"2025-10-29T16:20:23.000Z","created_at_i":1761754823,"id":45748997,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Sorry - my bad. I literally just connected an old XP VM to the internet to activate it.","title":null,"type":"comment","url":null},{"author":"ThatManulTheCat","children":[],"created_at":"2025-10-29T16:21:44.000Z","created_at_i":1761754904,"id":45749018,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Azure portal currently mostly not working (UK)... Downdetector reporting various Microsoft linked services are out (Minecraft, Microsoft 365, Xbox...)","title":null,"type":"comment","url":null},{"author":"mystcb","children":[],"created_at":"2025-10-29T16:22:07.000Z","created_at_i":1761754927,"id":45749027,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Updated 16:35 UTC
Azure Portal Access Issues
Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or sooner.
This message was last updated at 16:35 UTC on 29 October 2025
----
Azure Portal Access Issues
We are investigating an issue with the Azure Portal where customers may be experiencing issues accessing the portal. More information will be provided shortly.
This message was last updated at 16:18 UTC on 29 October 2025
-- From the Azure status page","title":null,"type":"comment","url":null},{"author":"NDizzle","children":[],"created_at":"2025-10-29T16:22:53.000Z","created_at_i":1761754973,"id":45749037,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"My best guess at the moment is something global like the CDN is having problems affecting things everywhere. I'm able to use a legacy application we have that goes directly to resources in uswest3, but I'm not able to use our more modern application which uses APIM/CDN networks at all.","title":null,"type":"comment","url":null},{"author":"mystcb","children":[{"author":"jdc0589","children":[{"author":"mystcb","children":[{"author":"chad_c","children":[],"created_at":"2025-10-29T16:36:57.000Z","created_at_i":1761755817,"id":45749309,"options":[],"parent_id":45749144,"points":null,"story_id":45748661,"text":"It was here https://news.ycombinator.com/item?id=45749054 but that comment has been deleted.","title":null,"type":"comment","url":null},{"author":"verst","children":[],"created_at":"2025-10-29T19:31:23.000Z","created_at_i":1761766283,"id":45751854,"options":[],"parent_id":45749144,"points":null,"story_id":45748661,"text":"It's a Sev 0 actually (as one would expect - this isn't a big secret). I was on the engineering bridge call earlier for a bit.\nThe Azure service I work on was minimally impacted (our customer facing dashboard could not load, but APIs and data layer were not impacted) but we found a workaround.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:28:07.000Z","created_at_i":1761755287,"id":45749144,"options":[],"parent_id":45749055,"points":null,"story_id":45748661,"text":"Yeah, I am guessing it's just a placeholder till they get more info. I thought I saw somewhere that internally within Microsoft it's seen as a "Sev 1" with "all hands on deck" - Annoyingly I can't remember where I saw it, so if someone spots it before I do, please credit that person :D
Edit: Typo!","title":null,"type":"comment","url":null},{"author":"daxfohl","children":[{"author":"jdc0589","children":[],"created_at":"2025-10-29T16:36:05.000Z","created_at_i":1761755765,"id":45749294,"options":[],"parent_id":45749250,"points":null,"story_id":45748661,"text":"yea I saw that, but im not sure on how accurate that is. a few large apps/companies I know to be 100% on AWS in us-east-1 are cranking along just fine.","title":null,"type":"comment","url":null},{"author":"NetMageSCW","children":[],"created_at":"2025-10-29T16:59:22.000Z","created_at_i":1761757162,"id":45749673,"options":[],"parent_id":45749250,"points":null,"story_id":45748661,"text":"AWS was performance issues and I believe is resolved.","title":null,"type":"comment","url":null},{"author":"rozenmd","children":[],"created_at":"2025-10-29T17:01:11.000Z","created_at_i":1761757271,"id":45749704,"options":[],"parent_id":45749250,"points":null,"story_id":45748661,"text":"From what I can tell, Downdetector just tracks traffic to their pages without actually checking if the site is down.
The other day during the AWS outage they "reported" OVH down too.","title":null,"type":"comment","url":null},{"author":"linhns","children":[],"created_at":"2025-10-29T18:12:17.000Z","created_at_i":1761761537,"id":45750814,"options":[],"parent_id":45749250,"points":null,"story_id":45748661,"text":"Not sure if this is true. I just login to the console with no glitch.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:33:45.000Z","created_at_i":1761755625,"id":45749250,"options":[],"parent_id":45749055,"points":null,"story_id":45748661,"text":"Downdetector says aws and gcp are down too. Might be in for a fun day.","title":null,"type":"comment","url":null},{"author":"planewave","children":[],"created_at":"2025-10-29T17:15:50.000Z","created_at_i":1761758150,"id":45749939,"options":[],"parent_id":45749055,"points":null,"story_id":45748661,"text":"yes, and it seems that at least for some login.microsoftonline.com is down too, which is part of the Entra login / SSO flow.","title":null,"type":"comment","url":null},{"author":"PeterCorless","children":[],"created_at":"2025-10-29T17:22:42.000Z","created_at_i":1761758562,"id":45750056,"options":[],"parent_id":45749055,"points":null,"story_id":45748661,"text":"Seems all Microsoft-related domains are impacted in some way.
\u2022 https://www.xbox.com/en-US also doesn't fully paint. Header comes up, but not the rest of the page.
\u2022 https://www.minecraft.net/en-us is extremely slow, but eventually came up.","title":null,"type":"comment","url":null},{"author":"bossyTeacher","children":[],"created_at":"2025-10-29T17:54:59.000Z","created_at_i":1761760499,"id":45750565,"options":[],"parent_id":45749055,"points":null,"story_id":45748661,"text":"It sure must be embarrassing for the website of the second richest company in the world to be down.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:23:49.000Z","created_at_i":1761755029,"id":45749055,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"yea its not just the portal. microsoft.com is down too","title":null,"type":"comment","url":null},{"author":"ThatManulTheCat","children":[],"created_at":"2025-10-29T16:38:11.000Z","created_at_i":1761755891,"id":45749332,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"DNS. Ofc.","title":null,"type":"comment","url":null},{"author":"8cvor6j844qw_d6","children":[{"author":"Insanity","children":[{"author":"kyrra","children":[],"created_at":"2025-10-29T16:57:28.000Z","created_at_i":1761757048,"id":45749640,"options":[],"parent_id":45749618,"points":null,"story_id":45748661,"text":"https://isitdns.com/","title":null,"type":"comment","url":null},{"author":"parliament32","children":[{"author":"RajT88","children":[],"created_at":"2025-10-29T17:25:07.000Z","created_at_i":1761758707,"id":45750112,"options":[],"parent_id":45749762,"points":null,"story_id":45748661,"text":"Needs STONITH","title":null,"type":"comment","url":null},{"author":"nijave","children":[],"created_at":"2025-10-29T23:02:15.000Z","created_at_i":1761778935,"id":45754231,"options":[],"parent_id":45749762,"points":null,"story_id":45748661,"text":">slightly increased latency
They didn't provide any details on latency. It could have been delayed an hour or a day and no one noticed","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:04:46.000Z","created_at_i":1761757486,"id":45749762,"options":[],"parent_id":45749618,"points":null,"story_id":45748661,"text":"That RCA was fun. A distributed system with members that don't know about each other, don't bother with leader elections, and basically all stomp all over each other updating the records. It "worked fine" until one of the members had slightly increased latency and everything cascade-failed down from there. I'm sure there was missing (internal) context but it did not sound like a well-architected system at all.","title":null,"type":"comment","url":null},{"author":"cdr420","children":[{"author":"tempest_","children":[{"author":"r_lee","children":[{"author":"chasd00","children":[],"created_at":"2025-10-29T19:51:19.000Z","created_at_i":1761767479,"id":45752133,"options":[],"parent_id":45751569,"points":null,"story_id":45748661,"text":"when a service goes down it's DNS when an entire nation or group of nations vanish it's BGP.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:09:35.000Z","created_at_i":1761764975,"id":45751569,"options":[],"parent_id":45750867,"points":null,"story_id":45748661,"text":"THIS is the real deal. Some say it's always DNS but many times it's some routing fuckup with BGP. two most cursed 3 letter acronym technologies out there","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:15:30.000Z","created_at_i":1761761730,"id":45750867,"options":[],"parent_id":45749776,"points":null,"story_id":45748661,"text":"It is a coin flip, heads DNS, tails BGP","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:05:17.000Z","created_at_i":1761757517,"id":45749776,"options":[],"parent_id":45749618,"points":null,"story_id":45748661,"text":"It's always DNS","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:56:07.000Z","created_at_i":1761756967,"id":45749618,"options":[],"parent_id":45749370,"points":null,"story_id":45748661,"text":"It's pretty unlikely. AWS published a public 'RCA' https://aws.amazon.com/message/101925/. A race condition in a DNS 'record allocator' causing all DNS records for DDB to be wiped out.
I'm simplifying a bit, but I don't think it's likely that Azure has a similar race condition wiping out DNS records on _one_ system than then propagates to all others. The similarity might just end at "it was DNS".","title":null,"type":"comment","url":null},{"author":"layer8","children":[{"author":"dotancohen","children":[],"created_at":"2025-10-29T23:16:03.000Z","created_at_i":1761779763,"id":45754382,"options":[],"parent_id":45750385,"points":null,"story_id":45748661,"text":"That's three of the hardest problems in CS ))","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:41:56.000Z","created_at_i":1761759716,"id":45750385,"options":[],"parent_id":45749370,"points":null,"story_id":45748661,"text":"DNS has both naming and cache invalidation, so no surprise it\u2019s among the hardest things to get right. ;)","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:40:35.000Z","created_at_i":1761756035,"id":45749370,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"I'll be interested in the incident writeup since DNS is mentioned. It will be interesting in a way if it is similar to what happened at AWS.","title":null,"type":"comment","url":null},{"author":"jjp","children":[{"author":"llama052","children":[{"author":"everfrustrated","children":[{"author":"guptadagger","children":[],"created_at":"2025-10-30T01:56:57.000Z","created_at_i":1761789417,"id":45755575,"options":[],"parent_id":45750440,"points":null,"story_id":45748661,"text":"i understood that to be a different third party that provided a CDN and was different than afd. https://learn.microsoft.com/en-us/azure/frontdoor/migrate-cd...","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:45:33.000Z","created_at_i":1761759933,"id":45750440,"options":[],"parent_id":45750179,"points":null,"story_id":45748661,"text":"Didn't the underlying vendor they used for Azure Front Door go bankrupt? It's probably on life support.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:28:53.000Z","created_at_i":1761758933,"id":45750179,"options":[],"parent_id":45749919,"points":null,"story_id":45748661,"text":"Please migrate off of front door. It's been a failure mode since it came out historically. Anything else is better at this point","title":null,"type":"comment","url":null},{"author":"progmetaldev","children":[],"created_at":"2025-10-29T22:00:38.000Z","created_at_i":1761775238,"id":45753649,"options":[],"parent_id":45749919,"points":null,"story_id":45748661,"text":"Currently even the Front Door landing page is only partially loading.
https://azure.microsoft.com/en-us/products/frontdoor","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:14:43.000Z","created_at_i":1761758083,"id":45749919,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"Whilst the status message acknowledge's the issue with Front Door (AFD), it seems as though the rest of the actions are about how to get Portal/internal services working without relying on AFD. For those of us using Front Door does that mean we're in for a long haul?","title":null,"type":"comment","url":null},{"author":"planewave","children":[{"author":"croemer","children":[{"author":"croemer","children":[],"created_at":"2025-10-29T20:32:31.000Z","created_at_i":1761769951,"id":45752653,"options":[],"parent_id":45750984,"points":null,"story_id":45748661,"text":"At this stage, we anticipate full mitigation within the next four hours as we continue to recover nodes. This means we expect recovery to happen by 23:20 UTC on 29 October 2025. We will provide another update on our progress within two hours, or sooner if warranted.
This message was last updated at 19:57 UTC on 29 October 2025","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:24:00.000Z","created_at_i":1761762240,"id":45750984,"options":[],"parent_id":45749996,"points":null,"story_id":45748661,"text":""We have initiated the deployment of our 'last known good' configuration. This is expected to be fully deployed in about 30 minutes from which point customers will start to see initial signs of recovery. Once this is completed, the next stage is to start to recover nodes while we route traffic through these healthy nodes."
"This message was last updated at 18:11 UTC on 29 October 2025"","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:18:47.000Z","created_at_i":1761758327,"id":45749996,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"Azure Network Availability Issues
Starting at approximately 16:00 UTC, we began experiencing Azure Front Door issues resulting in a loss of availability of some services. We suspect that an inadvertent configuration change as the trigger event for this issue. We are taking two concurrent actions where we are blocking all changes to the AFD services and at the same time rolling back to our last known good state.
We have failed the portal away from Azure Front Door (AFD) to mitigate the portal access issues. Customers should be able to access the Azure management portal directly.
We do not have an ETA for when the rollback will be completed, but we will update this communication within 30 minutes or when we have an update.
This message was last updated at 17:17 UTC on 29 October 2025","title":null,"type":"comment","url":null},{"author":"cyptus","children":[{"author":"RajT88","children":[{"author":"llama052","children":[{"author":"RajT88","children":[],"created_at":"2025-10-29T17:29:39.000Z","created_at_i":1761758979,"id":45750192,"options":[],"parent_id":45750117,"points":null,"story_id":45748661,"text":"That should go into the presentation you provide your CSAM with as well.
Storytelling is how issues get addressed. Help the CSAM tell the story to the higher ups.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:25:17.000Z","created_at_i":1761758717,"id":45750117,"options":[],"parent_id":45750063,"points":null,"story_id":45748661,"text":"I got a service health alert an hour after it started, saying the portal was having issues. Pretty useless and misleading.","title":null,"type":"comment","url":null},{"author":"cyptus","children":[],"created_at":"2025-10-29T17:41:26.000Z","created_at_i":1761759686,"id":45750372,"options":[],"parent_id":45750063,"points":null,"story_id":45748661,"text":"in many cases: no service health alerts, no status page updates and no confirmations from the support team in tickets.\nstill we can confirm these issues from different customers accross europe. Mostly the issues are regional dependent.","title":null,"type":"comment","url":null},{"author":"psunavy03","children":[{"author":"RajT88","children":[{"author":"riffic","children":[{"author":"RajT88","children":[{"author":"Noumenon72","children":[],"created_at":"2025-10-29T23:14:34.000Z","created_at_i":1761779674,"id":45754366,"options":[],"parent_id":45751321,"points":null,"story_id":45748661,"text":"That's an even 5:5 split between both meanings.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:51:37.000Z","created_at_i":1761763897,"id":45751321,"options":[],"parent_id":45751095,"points":null,"story_id":45748661,"text":"That's a funny criticism to make on a tech forum.
But, for future reference:
site:microsoft.com csam","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:32:27.000Z","created_at_i":1761762747,"id":45751095,"options":[],"parent_id":45750525,"points":null,"story_id":45748661,"text":"In general, plain language works so much better than throwing bowls of alphabet soup around.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:52:33.000Z","created_at_i":1761760353,"id":45750525,"options":[],"parent_id":45750504,"points":null,"story_id":45748661,"text":"Quite so. The acronym collision rate is high.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:50:38.000Z","created_at_i":1761760238,"id":45750504,"options":[],"parent_id":45750063,"points":null,"story_id":45748661,"text":"Some really unfortunate acronyms flying around the Microsoft ecosystem . . .","title":null,"type":"comment","url":null},{"author":"tomashubelbauer","children":[{"author":"linohh","children":[{"author":"andrewinardeer","children":[],"created_at":"2025-10-29T19:07:21.000Z","created_at_i":1761764841,"id":45751540,"options":[],"parent_id":45750621,"points":null,"story_id":45748661,"text":""Apply to become a CSAM mentor"","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:57:59.000Z","created_at_i":1761760679,"id":45750621,"options":[],"parent_id":45750560,"points":null,"story_id":45748661,"text":"Thank you, not going to google that shit.","title":null,"type":"comment","url":null},{"author":"ifwinterco","children":[{"author":"cyptus","children":[],"created_at":"2025-10-30T06:19:11.000Z","created_at_i":1761805151,"id":45756926,"options":[],"parent_id":45752474,"points":null,"story_id":45748661,"text":"haha :D","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:16:36.000Z","created_at_i":1761768996,"id":45752474,"options":[],"parent_id":45750560,"points":null,"story_id":45748661,"text":"Alternative f\u00fcr Deutschland was strange enough, when I saw CSAM I was really wondering what thread I had stumbled into","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:54:50.000Z","created_at_i":1761760490,"id":45750560,"options":[],"parent_id":45750063,"points":null,"story_id":45748661,"text":"CSAM apparently also means Customer Success Account Manager for those who might have gotten startled by this message like me.","title":null,"type":"comment","url":null},{"author":"cyberax","children":[{"author":"SAI_Peregrinus","children":[],"created_at":"2025-10-29T18:12:02.000Z","created_at_i":1761761522,"id":45750812,"options":[],"parent_id":45750631,"points":null,"story_id":45748661,"text":"They must really depend on their government contracts with this administration\u2026","title":null,"type":"comment","url":null},{"author":"mirekrusin","children":[{"author":"xp84","children":[{"author":"red-iron-pine","children":[],"created_at":"2025-10-30T13:02:46.000Z","created_at_i":1761829366,"id":45759588,"options":[],"parent_id":45752244,"points":null,"story_id":45748661,"text":"but that makes them sound like Managers, which is not what they are -- glorified sales people, really.
actual Managers hate that","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:59:10.000Z","created_at_i":1761767950,"id":45752244,"options":[],"parent_id":45751016,"points":null,"story_id":45748661,"text":"Most companies just call 'em CSMs","title":null,"type":"comment","url":null},{"author":"pndy","children":[{"author":"mirekrusin","children":[],"created_at":"2025-10-29T21:03:16.000Z","created_at_i":1761771796,"id":45752994,"options":[],"parent_id":45752477,"points":null,"story_id":45748661,"text":"...performed by cheap, open weight LLM.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:16:43.000Z","created_at_i":1761769003,"id":45752477,"options":[],"parent_id":45751016,"points":null,"story_id":45748661,"text":"Supervisor Customer Account Manager: a remote kind of job, paid occasionally with gift cards","title":null,"type":"comment","url":null},{"author":"tanseydavid","children":[{"author":"rightbyte","children":[],"created_at":"2025-10-30T09:32:00.000Z","created_at_i":1761816720,"id":45757990,"options":[],"parent_id":45753444,"points":null,"story_id":45748661,"text":"You could argue "pornography" is the euphemism?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:41:48.000Z","created_at_i":1761774108,"id":45753444,"options":[],"parent_id":45751016,"points":null,"story_id":45748661,"text":">> They should rename to Success Customer Account Manager.
No -- the one referencing crime should NEVER have be turned into an acronym.
Crimes should not be described in euphemistic terms (which is exactly what the acronym is)","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:26:50.000Z","created_at_i":1761762410,"id":45751016,"options":[],"parent_id":45750631,"points":null,"story_id":45748661,"text":"They should rename to Success Customer Account Manager.","title":null,"type":"comment","url":null},{"author":"RajT88","children":[{"author":"dotancohen","children":[],"created_at":"2025-10-29T23:14:05.000Z","created_at_i":1761779645,"id":45754357,"options":[],"parent_id":45751340,"points":null,"story_id":45748661,"text":"Didn't MS have three things called "link" at one time? They were all spelled differently, of course.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:52:54.000Z","created_at_i":1761763974,"id":45751340,"options":[],"parent_id":45750631,"points":null,"story_id":45748661,"text":"Definitely the most baffling acronym collision I have seen with Microsoft. I did one time count 4 different products abbreviated VSTS at one point.","title":null,"type":"comment","url":null},{"author":"codeduck","children":[{"author":"senderista","children":[{"author":"zemariagp","children":[{"author":"red-iron-pine","children":[],"created_at":"2025-10-30T13:04:34.000Z","created_at_i":1761829474,"id":45759602,"options":[],"parent_id":45756680,"points":null,"story_id":45748661,"text":"North American Zigbee Initiative","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T05:21:51.000Z","created_at_i":1761801711,"id":45756680,"options":[],"parent_id":45753158,"points":null,"story_id":45748661,"text":"Wait till you hear about the Keen Kubernetes Knowledge iniciative","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:17:00.000Z","created_at_i":1761772620,"id":45753158,"options":[],"parent_id":45752333,"points":null,"story_id":45748661,"text":""CSAM Ninja"","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:05:21.000Z","created_at_i":1761768321,"id":45752333,"options":[],"parent_id":45750631,"points":null,"story_id":45748661,"text":"Oh dear. Will make for an awkward thing to have on your resume.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:58:31.000Z","created_at_i":1761760711,"id":45750631,"options":[],"parent_id":45750063,"points":null,"story_id":45748661,"text":"> CSAM
Child Sex-Abuse Material?!? Well, a nice case of acronym collision.","title":null,"type":"comment","url":null},{"author":"nijave","children":[{"author":"RajT88","children":[{"author":"nijave","children":[],"created_at":"2025-11-07T13:43:49.000Z","created_at_i":1762523029,"id":45846267,"options":[],"parent_id":45760158,"points":null,"story_id":45748661,"text":"I think we weren't paying for support and it was standard Business Support they were pitching. At the time we were having pretty fundamental problems with Azure Single Server Postgres which was really just a terribly engineered solution which they admitted had some nasty issues (there was some bug that would cause the storage IO threads to deadlock causing Postgres to crash)","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T14:05:00.000Z","created_at_i":1761833100,"id":45760158,"options":[],"parent_id":45754181,"points":null,"story_id":45748661,"text":"Do you recall the kind of premium support? Azure Rapid Response?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:56:34.000Z","created_at_i":1761778594,"id":45754181,"options":[],"parent_id":45750063,"points":null,"story_id":45748661,"text":"Back when we used Azure the only outcome was them trying to upsell us on Premium Support","title":null,"type":"comment","url":null},{"author":"alias_neo","children":[],"created_at":"2025-10-30T09:49:21.000Z","created_at_i":1761817761,"id":45758109,"options":[],"parent_id":45750063,"points":null,"story_id":45748661,"text":"Where do these alerts supposedly come from? I started having issues around 4PM (GMT), couldn't access portal, and couldn't make AKV requests from the CLI, and initially asked our Ops guys but with no info and a vague "There may be issues with Portal" on their status page, that was me done for the day.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:23:01.000Z","created_at_i":1761758581,"id":45750063,"options":[],"parent_id":45750015,"points":null,"story_id":45748661,"text":"Spam those Azure tickets. If you have a CSAM, build them a nice powerpoint telling the story of all your AFD issues (that's what they are there for).
> In 50%+ the cases they just don\u2018t report it anywhere, even if its for 2h+.
I assume you mean publicly. Are you getting the service health alerts?","title":null,"type":"comment","url":null},{"author":"hallh","children":[],"created_at":"2025-10-29T17:28:42.000Z","created_at_i":1761758922,"id":45750175,"options":[],"parent_id":45750015,"points":null,"story_id":45748661,"text":"Same experience. We've recently migrated fully away from AFD due to how unreliable it is.","title":null,"type":"comment","url":null},{"author":"nevf1","children":[{"author":"cyptus","children":[],"created_at":"2025-10-29T19:05:32.000Z","created_at_i":1761764732,"id":45751518,"options":[],"parent_id":45751420,"points":null,"story_id":45748661,"text":"so true. instead of getting a fast feedback we are wasting time searching for our own issues first.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:58:38.000Z","created_at_i":1761764318,"id":45751420,"options":[],"parent_id":45750015,"points":null,"story_id":45748661,"text":"This is the single most frustrating thing about these incidents. As you're harmstrung on what you can do or how you can react until Microsoft officially acknowledges a problem. Took nearly 90mins both today and when it happened on 9th October.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:20:01.000Z","created_at_i":1761758401,"id":45750015,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"AFD is down quite often regionally in Europe for our services. In 50%+ the cases they just don\u2018t report it anywhere, even if its for 2h+.","title":null,"type":"comment","url":null},{"author":"jonathanlydall","children":[{"author":"out_sider","children":[{"author":"jonathanlydall","children":[{"author":"out_sider","children":[],"created_at":"2025-10-29T18:06:39.000Z","created_at_i":1761761199,"id":45750733,"options":[],"parent_id":45750531,"points":null,"story_id":45748661,"text":"Thanks :). We don't use Azure as our registrar. It seems I'll have to plan for this then, we also had another issue, AFD has a hard 500ms tls handshake timeout (doesn't matter how much you put on the origin timeout settings) which means if our server was slow for some reason we would get 504 origin timeout.","title":null,"type":"comment","url":null},{"author":"Figs","children":[{"author":"out_sider","children":[],"created_at":"2025-10-29T18:24:16.000Z","created_at_i":1761762256,"id":45750987,"options":[],"parent_id":45750780,"points":null,"story_id":45748661,"text":"I meant cloudfare","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:09:39.000Z","created_at_i":1761761379,"id":45750780,"options":[],"parent_id":45750531,"points":null,"story_id":45748661,"text":"CloudFlare != CloudFront","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:52:44.000Z","created_at_i":1761760364,"id":45750531,"options":[],"parent_id":45750144,"points":null,"story_id":45748661,"text":"Unless you pay for CloudFlare\u2019s Enterpise plan, you\u2019re required to have them host your DNS zone, you can use a different registrar as long as you just point your NS records to Cloudflare.
Be aware that if you\u2019re using Azure as your registrar, it\u2019s (probably still) impossible to change your NS records to point to CloudFlare\u2019s DNS server, at least it was for me about 6 months ago.
This also makes it impossible to transfer your domain to them either, as CloudFlare\u2019s domain transfer flow requires you set your NS records to point to them before their interface shows a transfer option.
In our case we had to transfer to a different registrar, we used Namecheap.
However, transferring a domain from Azure was also a nightmare. Their UI doesn\u2019t have any kind of transfer option, I eventually found an obscure document (not on their Learn website) which had an az command which would let you get a transfer code which I could give to Namecheap.
Then I had to wait over a week for the transfer timeout to occur because there is no way on Azure side that I could find to accept the transfer immediately.
I found CloudFlare\u2019s way of building rules quite easy to use, different from Front Door but I\u2019m not doing anything more complex than some redirects and reverse proxying.
I will say that Cloudflare\u2019s UI is super fast, with Front Door I always found it painfully slow when trying to do any kind of configuration.
Cloudflare also doesn\u2019t have the problem that Front Door has where it requires a manual process every 6 months or so to renew the APEX certificate.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:26:37.000Z","created_at_i":1761758797,"id":45750144,"options":[],"parent_id":45750017,"points":null,"story_id":45748661,"text":"we are considering the same but because our website uses APEX domain we would need to move all DNS resolver to cloudfront right ? Does it have as a nice "rule set builder" as azure ?","title":null,"type":"comment","url":null},{"author":"nosefrog","children":[],"created_at":"2025-10-29T20:11:37.000Z","created_at_i":1761768697,"id":45752407,"options":[],"parent_id":45750017,"points":null,"story_id":45748661,"text":"Front Door is not good.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:20:05.000Z","created_at_i":1761758405,"id":45750017,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"Yet another reason to move away from Front Door.
We already had to do it for large files served from Blob Storage since they would cap out at 2MB/s when not in cache of the nearest PoP. If you\u2019ve ever experienced slow Windows Store or Xbox downloads it\u2019s probably the same problem.
I had a support ticket open for months about this and in the end the agent said \u201cthis is to be expected and we don\u2019t plan on doing anything about it\u201d.
We\u2019ve moved to Cloudflare and not only is the performance great, but it costs less.
Only thing I need to move off Front Door is a static website for our docs served from Blob Storage, this incident will make us do it sooner rather than later.","title":null,"type":"comment","url":null},{"author":"eddie_catflap","children":[],"created_at":"2025-10-29T17:27:09.000Z","created_at_i":1761758829,"id":45750154,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"We saw issues before 16:00 UTC - approx 15:38","title":null,"type":"comment","url":null},{"author":"rconti","children":[],"created_at":"2025-10-29T17:55:51.000Z","created_at_i":1761760551,"id":45750573,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"Sounds like they need to move their portal to a region with more capacity for the desired instance type. /s","title":null,"type":"comment","url":null},{"author":"NDizzle","children":[{"author":"Aperocky","children":[{"author":"NDizzle","children":[],"created_at":"2025-10-29T19:24:46.000Z","created_at_i":1761765886,"id":45751759,"options":[],"parent_id":45751169,"points":null,"story_id":45748661,"text":"They re-added it once the site was accessible.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:38:02.000Z","created_at_i":1761763082,"id":45751169,"options":[],"parent_id":45750841,"points":null,"story_id":45748661,"text":"The statement is still there though on the status page though","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:14:11.000Z","created_at_i":1761761651,"id":45750841,"options":[],"parent_id":45749044,"points":null,"story_id":45748661,"text":"They briefly had a statement about using Traffic Manager to work with your AFD to work around this issue, with a link to learn.microsoft.com/...traffic-manager, and the link didn't work. Due to the same issue affecting everyone right now.
They quickly updated the message to REMOVE the link. Comical at this point.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:23:20.000Z","created_at_i":1761755000,"id":45749044,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Update 16:57 UTC:
Azure Portal Access Issues
Starting at approximately 16:00 UTC, we began experiencing Azure Front Door issues resulting in a loss of availability of some services. In addition. customers may experience issues accessing the Azure Portal. Customers can attempt to use programmatic methods (PowerShell, CLI, etc.) to access/utilize resources if they are unable to access the portal directly. We have failed the portal away from Azure Front Door (AFD) to attempt to mitigate the portal access issues and are continuing to assess the situation.
We are actively assessing failover options of internal services from our AFD infrastructure. Our investigation into the contributing factors and additional recovery workstreams continues. More information will be provided within 60 minutes or sooner.
This message was last updated at 16:57 UTC on 29 October 2025
---
Update: 16:35 UTC:
Azure Portal Access Issues
Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or sooner.
This message was last updated at 16:35 UTC on 29 October 2025
---
Azure Portal Access Issues
We are investigating an issue with the Azure Portal where customers may be experiencing issues accessing the portal. More information will be provided shortly.
This message was last updated at 16:18 UTC on 29 October 2025
---
Message from the Azure Status Page: https://azure.status.microsoft/en-gb/status","title":null,"type":"comment","url":null},{"author":"somerandomness","children":[],"created_at":"2025-10-29T16:23:26.000Z","created_at_i":1761755006,"id":45749048,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"yep having trouble logging into https://entra.microsoft.com/ as well","title":null,"type":"comment","url":null},{"author":"baconbrand","children":[],"created_at":"2025-10-29T16:23:40.000Z","created_at_i":1761755020,"id":45749053,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"All of our sites went down. This is my company\u2019s busiest time of year. Hooray.","title":null,"type":"comment","url":null},{"author":"a_f","children":[],"created_at":"2025-10-29T16:24:33.000Z","created_at_i":1761755073,"id":45749068,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Looks like MyGet is impacted too. Seems like they use Azure:
>What is required to be able to use MyGet? ... MyGet runs its operations from the Microsoft Azure in the West Europe region, near Amsterdam, the Netherlands.","title":null,"type":"comment","url":null},{"author":"Sharparam","children":[],"created_at":"2025-10-29T16:24:51.000Z","created_at_i":1761755091,"id":45749070,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The learning modules on https://learn.microsoft.com/ also seem to have a lot of issues properly loading.","title":null,"type":"comment","url":null},{"author":"alt227","children":[],"created_at":"2025-10-29T16:25:03.000Z","created_at_i":1761755103,"id":45749077,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Cant access certain banking websites in the UK, I am assuming it because of this.
https://www.natwest.com/","title":null,"type":"comment","url":null},{"author":"baconbrand","children":[],"created_at":"2025-10-29T16:25:16.000Z","created_at_i":1761755116,"id":45749080,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Our Azure DevOps site is still functioning and our Azure hosted databases are accessible. Everything else is cooked.","title":null,"type":"comment","url":null},{"author":"tyfon","children":[{"author":"hexbin010","children":[{"author":"bombcar","children":[],"created_at":"2025-10-30T09:42:23.000Z","created_at_i":1761817343,"id":45758056,"options":[],"parent_id":45749888,"points":null,"story_id":45748661,"text":"Someone somewhere thought that "national digital ID service" should absolutely rely on a cloud provider in and from another country.
What a time to be alive.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:12:42.000Z","created_at_i":1761757962,"id":45749888,"options":[],"parent_id":45749082,"points":null,"story_id":45748661,"text":"> Even the national digital id service is down.
Can't help but smirk as my country is ramming through "Digital ID" right now","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:25:20.000Z","created_at_i":1761755120,"id":45749082,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Seems to be down in Norway.
Even the national digital id service is down.","title":null,"type":"comment","url":null},{"author":"llama052","children":[],"created_at":"2025-10-29T16:25:28.000Z","created_at_i":1761755128,"id":45749086,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Just another day with microsoft. Honestly pretty tiring as something is always generally broken.","title":null,"type":"comment","url":null},{"author":"ZeroConcerns","children":[{"author":"taeric","children":[{"author":"graemep","children":[{"author":"taeric","children":[{"author":"graemep","children":[{"author":"taeric","children":[],"created_at":"2025-10-30T15:01:28.000Z","created_at_i":1761836488,"id":45760803,"options":[],"parent_id":45757844,"points":null,"story_id":45748661,"text":"That isn't a free pass. You have no data showing how many people did go to competitors over this. You are asserting it is zero, but why do you think that? Going on the talks here, you can find plenty of folks that opted not to go with or stay on them.
You are further asserting that these outages prove they are not still more reliable than home spun. Is that the case? More than a few people aren't ready for a single hard drive to crash on the stuff they are doing.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T09:06:03.000Z","created_at_i":1761815163,"id":45757844,"options":[],"parent_id":45750264,"points":null,"story_id":45748661,"text":"I would say you are explaining why they get a free pass so they still get one - they are bad but their main competitors are even worse!
I thought one of the major selling points of the big cloud providers was that they were more reliable than running your own stuff (by which i mean anything from a VPS to multiple data centres depending on your scale. Compared to those alternatives they seem to be less reliable in practice!
The solution is to have a multi-region, or even multi-cloud setup, but then bang goes the "they do all the work for you" argument (which i doubt anyway).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:34:13.000Z","created_at_i":1761759253,"id":45750264,"options":[],"parent_id":45750012,"points":null,"story_id":45748661,"text":"Apologies, but this just reads like a low effort critique of big things.
To be clear, they should get criticism. They should be held liable for any damage they cause.
But that they remain the biggest cloud offering out there isn't something you'd expect to change from a few outages that, by most all evidence, potential replacements have, as well? More, a lot of the outages potential replacements have are often more global in nature.","title":null,"type":"comment","url":null},{"author":"prmoustache","children":[],"created_at":"2025-10-29T17:54:10.000Z","created_at_i":1761760450,"id":45750554,"options":[],"parent_id":45750012,"points":null,"story_id":45748661,"text":"Well, they have successfully locked their customers captive thanks to huge egress fees.","title":null,"type":"comment","url":null},{"author":"philipallstar","children":[],"created_at":"2025-10-29T17:57:22.000Z","created_at_i":1761760642,"id":45750614,"options":[],"parent_id":45750012,"points":null,"story_id":45748661,"text":"Have people left GitHub due to the multiple post-acquisition outages? That is a pass if you don't judge it the same way.","title":null,"type":"comment","url":null},{"author":"arccy","children":[],"created_at":"2025-10-29T23:20:20.000Z","created_at_i":1761780020,"id":45754424,"options":[],"parent_id":45750012,"points":null,"story_id":45748661,"text":"customers like us are certainly looking at expanding from just multi region into instead being multi cloud...","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:19:54.000Z","created_at_i":1761758394,"id":45750012,"options":[],"parent_id":45749286,"points":null,"story_id":45748661,"text":"Have repeated outages lost them customers? has it lost them any money in any way?
That is a pass.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:35:47.000Z","created_at_i":1761755747,"id":45749286,"options":[],"parent_id":45749093,"points":null,"story_id":45748661,"text":"I didn't realize AWS got a pass?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:25:40.000Z","created_at_i":1761755140,"id":45749093,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Oh, well, I'm sure Azure will be given the same pass that AWS got here recently when they had their 12-hour outage...","title":null,"type":"comment","url":null},{"author":"thewisenerd","children":[],"created_at":"2025-10-29T16:27:43.000Z","created_at_i":1761755263,"id":45749130,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"they recently had an incident with front door reachability, wonder if it's back.
QNBQ-5W8","title":null,"type":"comment","url":null},{"author":"djeastm","children":[{"author":"nartaczact","children":[],"created_at":"2025-10-29T17:03:55.000Z","created_at_i":1761757435,"id":45749747,"options":[],"parent_id":45749142,"points":null,"story_id":45748661,"text":"Sounds like Shrodinger's Deploy","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:28:02.000Z","created_at_i":1761755282,"id":45749142,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I'm mid-deployment, but thankfully it seems to be running ok so far. Just the portal is not working so my visibility is not good.","title":null,"type":"comment","url":null},{"author":"siva7","children":[{"author":"barpol","children":[],"created_at":"2025-10-29T16:47:06.000Z","created_at_i":1761756426,"id":45749474,"options":[],"parent_id":45749158,"points":null,"story_id":45748661,"text":"still down","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:28:49.000Z","created_at_i":1761755329,"id":45749158,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"auth services are down","title":null,"type":"comment","url":null},{"author":"Jamie452","children":[{"author":"SoftTalker","children":[{"author":"withinboredom","children":[{"author":"computerdork","children":[],"created_at":"2025-10-29T17:04:15.000Z","created_at_i":1761757455,"id":45749752,"options":[],"parent_id":45749620,"points":null,"story_id":45748661,"text":"Anyone remember Bob's number?? Bob?! Oh the humanity! We're all gonna be canned!","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:56:12.000Z","created_at_i":1761756972,"id":45749620,"options":[],"parent_id":45749516,"points":null,"story_id":45748661,"text":"I knew an old guy in the '00s who specialized in cobal/fortran for working on tiller software. Guess he retired and they couldn't maintain it","title":null,"type":"comment","url":null},{"author":"tcmart14","children":[],"created_at":"2025-10-29T18:57:41.000Z","created_at_i":1761764261,"id":45751403,"options":[],"parent_id":45749516,"points":null,"story_id":45748661,"text":"You can, but it's all about risk mitigation. Most processors have some form of store and forward (and it can have limitations like only X number of transactions). Some even have controls to limit the amount you can store-and-forward (for instance, only charges under $50). But ultimately, it's still risk mitigation. You can store-and-forward, but you're trusting that the card/account has the funds. If it doesn't, you loose and ain't shit you can do about it. If you can't tolerate any risk, you don't turn on store and forward systems and then you can't process cards offline.
Its not the we are not capable. Its, is the business willing to assume the risk?","title":null,"type":"comment","url":null},{"author":"bombcar","children":[],"created_at":"2025-10-30T09:37:49.000Z","created_at_i":1761817069,"id":45758030,"options":[],"parent_id":45749516,"points":null,"story_id":45748661,"text":"Most retailers trust their cashiers a bit less than they trust the customers. They'd rather shut down during a power/Internet failure than give any autonomy to the worker drones.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:49:30.000Z","created_at_i":1761756570,"id":45749516,"options":[],"parent_id":45749162,"points":null,"story_id":45748661,"text":"Mind-boggling that any retailer would not have the capability to at least run the checkout stations offline.","title":null,"type":"comment","url":null},{"author":"chasd00","children":[{"author":"ryandrake","children":[{"author":"ElevenLathe","children":[{"author":"xboxnolifes","children":[],"created_at":"2025-10-29T18:39:01.000Z","created_at_i":1761763141,"id":45751186,"options":[],"parent_id":45750073,"points":null,"story_id":45748661,"text":"It's hit or miss. My (brand new) bank card and chase credit card are raised. But my other credit cards are flat.","title":null,"type":"comment","url":null},{"author":"rkomorn","children":[],"created_at":"2025-10-29T18:40:59.000Z","created_at_i":1761763259,"id":45751208,"options":[],"parent_id":45750073,"points":null,"story_id":45748661,"text":"> kachunk-kachunk credit card machines
How aptly descriptive.","title":null,"type":"comment","url":null},{"author":"progmetaldev","children":[],"created_at":"2025-10-29T23:17:47.000Z","created_at_i":1761779867,"id":45754396,"options":[],"parent_id":45750073,"points":null,"story_id":45748661,"text":"My card tied to my main financial institution have the raised digits, but most cards you'd sign up for online now no longer have the raised digits (and often allow you to select art to appear on the card face).","title":null,"type":"comment","url":null},{"author":"hinkley","children":[],"created_at":"2025-10-30T18:36:35.000Z","created_at_i":1761849395,"id":45763551,"options":[],"parent_id":45750073,"points":null,"story_id":45748661,"text":"My credit union has been behind for a while. I think I had an embossed one until about nine years ago. Six at the latest. Still doesn\u2019t have NFC in it.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:23:13.000Z","created_at_i":1761758593,"id":45750073,"options":[],"parent_id":45749986,"points":null,"story_id":45748661,"text":"The kachunk-kachunk credit card machines need raised digits on the cards, and I don't think most banks have been issuing those for years at this point. Mine have been smooth for at least 10 years.","title":null,"type":"comment","url":null},{"author":"BenjiWiebe","children":[{"author":"hinkley","children":[],"created_at":"2025-10-30T18:33:40.000Z","created_at_i":1761849220,"id":45763511,"options":[],"parent_id":45750371,"points":null,"story_id":45748661,"text":"After having my credit card locked by stupid fraud prevention algorithms on my honeymoon, I had a long chat with them before going overseas a second time.
And that was the day Visa had a full on outage. We would walk into one shop, try to buy stuff, get declined, then go into the next and get accepted because they were running in offline mode.
Got a nice big bill from my cellphone carrier for making the call to visa to ask them wtf as well.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:41:26.000Z","created_at_i":1761759686,"id":45750371,"options":[],"parent_id":45749986,"points":null,"story_id":45748661,"text":"I think a lot of payment terminals have an option to record transactions offline and upload them later, but apparently it's not enabled by default - probably because it increases your risk that someone pays with a bad card.","title":null,"type":"comment","url":null},{"author":"voidmain0001","children":[],"created_at":"2025-10-29T17:48:59.000Z","created_at_i":1761760139,"id":45750494,"options":[],"parent_id":45749986,"points":null,"story_id":45748661,"text":"If they used standalone merchant terminals, then those typically use the local LAN which can rollover to cellular or PoT in the event of a network outage. The store can process a card transaction with the merchant terminal and then reconcile with the end of day chit. This article from 2008 describes their PoS https://www.retailtouchpoints.com/topics/store-operations/ca...","title":null,"type":"comment","url":null},{"author":"Spooky23","children":[{"author":"progmetaldev","children":[],"created_at":"2025-10-29T23:21:50.000Z","created_at_i":1761780110,"id":45754438,"options":[],"parent_id":45751353,"points":null,"story_id":45748661,"text":"These stores appear everywhere, even in areas with high income. You'd be surprised, but often people with those high incomes shop for certain products at very low rates, and that's how they keep their savings. A good example is garbage bags. Most people don't care too much about the quality of their garbage bags, unless they rip on the way to the bin.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:53:33.000Z","created_at_i":1761764013,"id":45751353,"options":[],"parent_id":45749986,"points":null,"story_id":45748661,"text":"It\u2019s family dollar. They don\u2019t care about customer satisfaction and the cost of reliability is cost.
The stores are in the hood or middle of nowhere. The customers don\u2019t have many options.","title":null,"type":"comment","url":null},{"author":"wat10000","children":[],"created_at":"2025-10-29T19:22:32.000Z","created_at_i":1761765752,"id":45751725,"options":[],"parent_id":45749986,"points":null,"story_id":45748661,"text":"Many businesses don't lose revenue from short outages, it just gets shifted.","title":null,"type":"comment","url":null},{"author":"Finnucane","children":[],"created_at":"2025-10-29T20:46:39.000Z","created_at_i":1761770799,"id":45752790,"options":[],"parent_id":45749986,"points":null,"story_id":45748661,"text":"Then they would need to get the little booklets of invalid numbers to keep by the register to check (yes, I am old).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:18:03.000Z","created_at_i":1761758283,"id":45749986,"options":[],"parent_id":45749765,"points":null,"story_id":45748661,"text":"You'd think any SeriousBusiness would have a backup way to take customers' money. This is the one thing you always want to be able to do: accept payment. If they made it so they can't do that, they deserve the hit to their revenue. People should just walk out of the store with the goods if they're not being charged.
Why doesn't someone in the store at least have one of those manual kachunk-kachunk carbon copy card readers in the back that they can resuscitate for a few days until the technology is turned back on? Did they throw them all away?","title":null,"type":"comment","url":null},{"author":"jrodom","children":[{"author":"consp","children":[],"created_at":"2025-10-29T17:30:18.000Z","created_at_i":1761759018,"id":45750202,"options":[],"parent_id":45750027,"points":null,"story_id":45748661,"text":"2-3%, bit higher on perishables. Though i'd just ask lump sum payments in cash since it likely has to no go through corporate (as in, avoid the corporation).","title":null,"type":"comment","url":null},{"author":"chasd00","children":[{"author":"gwbas1c","children":[],"created_at":"2025-10-29T19:58:57.000Z","created_at_i":1761767937,"id":45752240,"options":[],"parent_id":45751884,"points":null,"story_id":45748661,"text":"They're all run on a shoestring:
1: I doubt they're "with it" enough to put together a backup arrangement for internet.
2: Their internet problems are probably due to a cheapo router, loose wire, ect.
3: The employees probably like the break.","title":null,"type":"comment","url":null},{"author":"hinkley","children":[{"author":"chasd00","children":[],"created_at":"2025-10-30T13:31:00.000Z","created_at_i":1761831060,"id":45759843,"options":[],"parent_id":45752680,"points":null,"story_id":45748661,"text":"i'm not a businessman, just a computer nerd ( swe turned tech. consultant )
It seems like an easy problem to fix and a retail store being closed for a whole weekday because of inet access sounds crazy to me.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:35:21.000Z","created_at_i":1761770121,"id":45752680,"options":[],"parent_id":45751884,"points":null,"story_id":45748661,"text":"I think the point people are trying and failing to make is that asking for half of means sales is half of revenue not half of net and that you\u2019re out of your goddamned mind if you think a store with razor thin margins would sell at a massive loss rather than just close due to connectivity problems.
Your responses imply that you think people are questioning whether you would lose money on the deal while we are instead saying you\u2019ll get laughed out of the store, or possibly asked never to come back.","title":null,"type":"comment","url":null},{"author":"progmetaldev","children":[],"created_at":"2025-10-29T23:15:22.000Z","created_at_i":1761779722,"id":45754373,"options":[],"parent_id":45751884,"points":null,"story_id":45748661,"text":"Unfortunately they are largely corporate, which is how they can sell items for such a cheap price. The store manager probably has zero say in nearly anything. Even if they wanted to "break the rules," I doubt they could make use of your connection as a backup, but I've also worked for smaller companies that were able to sell internet access to individual locations like Denny's and various large hotels in the US. Being able to somehow share sales would be the difficult part, since all sales are reported back to corporate.
Good luck if you make this work for you, it would be exciting to hear about if you're able to get them to work with you.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:33:04.000Z","created_at_i":1761766384,"id":45751884,"options":[],"parent_id":45750027,"points":null,"story_id":45748661,"text":"It's Family Dollar, margin has to be almost nothing and sales per day is probably < $1k. That's why I said 50% of sales and not profit.
I go there daily because it's a nice 30min round trip walk and I wfh. I go up there to get a diet coke or something else just to get out of the house. It amazes me when i see a handwritten sign on the door "closed, system is down". I've gotten to know the cashiers so I asked and it's because the internet connection goes down all the time. That store has to one of the most poorly run things i've ever seen yet it stays in business somehow.","title":null,"type":"comment","url":null},{"author":"OptionOfT","children":[],"created_at":"2025-10-29T19:41:13.000Z","created_at_i":1761766873,"id":45752015,"options":[],"parent_id":45750027,"points":null,"story_id":45748661,"text":"In that case the other 50%.","title":null,"type":"comment","url":null},{"author":"jiveturkey","children":[{"author":"hinkley","children":[],"created_at":"2025-10-29T20:40:17.000Z","created_at_i":1761770417,"id":45752721,"options":[],"parent_id":45752177,"points":null,"story_id":45748661,"text":"All my limited knowledge about retail is that losing money in Q4 means you\u2019re dead. Are they fundamentally different than retail?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:54:36.000Z","created_at_i":1761767676,"id":45752177,"options":[],"parent_id":45750027,"points":null,"story_id":45748661,"text":"it's retail. the margin is 30-50% for sure.
EDIT: their last quarterly was 36%. they lost $3.7bn in 24Q4 -- the christmas quarter. sold to PE in Q1.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:21:08.000Z","created_at_i":1761758468,"id":45750027,"options":[],"parent_id":45749765,"points":null,"story_id":45748661,"text":"50% of sales? what do you think the gross margin is on average for each item sold?","title":null,"type":"comment","url":null},{"author":"BenjiWiebe","children":[],"created_at":"2025-10-29T17:42:41.000Z","created_at_i":1761759761,"id":45750394,"options":[],"parent_id":45749765,"points":null,"story_id":45748661,"text":"Pretty sure it'd be a lot better deal for them to have no sales than to pay out 50% of sales on stuff with single digit margins.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:04:52.000Z","created_at_i":1761757492,"id":45749765,"options":[],"parent_id":45749162,"points":null,"story_id":45748661,"text":"There's a Family Dollar by my house that is down at least 2 full days per month because of bad inet connectivity. I live close enough that with a small tower on my roof i can get line of sight to theirs. I've thought about offering them a backup link off my home inet if they give me 50% of sales whenever its in use. It would be a pretty good deal for them, better some sales when their inet is down vs none.","title":null,"type":"comment","url":null},{"author":"david422","children":[{"author":"cyberax","children":[],"created_at":"2025-10-29T18:05:25.000Z","created_at_i":1761761125,"id":45750721,"options":[],"parent_id":45749805,"points":null,"story_id":45748661,"text":"I remember that banks will try to honor the transactions, even if the customer's balance/credit limit is exhausted. It doesn't apply only to some gift cards.","title":null,"type":"comment","url":null},{"author":"tcmart14","children":[{"author":"fy20","children":[],"created_at":"2025-10-30T04:19:26.000Z","created_at_i":1761797966,"id":45756402,"options":[],"parent_id":45751360,"points":null,"story_id":45748661,"text":"In many places it's also possibly just a left over feature from older times. I worked at a major UK supermarket in the mid-00s, and their checkout system had this feature. But it was like that because that's how it was originally built, it wasn't a 'feature' they added.
Credit card information would be recorded by the POS, synced to a mini-server in the back office (using store-and-forward to handle network issues) and then in a batch process overnight, sent to HQ where the payment was processed.
It wasn't until chip-and-PIN was rolled out that they started supporting "online" (i.e. processed then and there) card transactions, and even then the old method still worked if there was a network issues or power failure (all POSes has their own UPS).
The only real risk at the time was that someone tried to pay with a cancelled credit card - the bank would always honour the payment otherwise. But that was pretty uncommon back then, as you'd have to phone your bank to do it, not just press a button in an app.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:54:16.000Z","created_at_i":1761764056,"id":45751360,"options":[],"parent_id":45749805,"points":null,"story_id":45748661,"text":"Yea, good old store and forward. We implemented it in our PoS system. Now, we do non PCI integrations so we arn't in PCI scope, but depending on the processor, it can come with some limitations. Like, you can do store and forward, but only up to X number of transactions. I think for one integration, it's 500-ish store wide (it uses a local gateway that store and forwards to the processors gateway). The other integration we have, its 250, but store and forward on device, per device.","title":null,"type":"comment","url":null},{"author":"onionisafruit","children":[{"author":"progmetaldev","children":[],"created_at":"2025-10-29T23:09:16.000Z","created_at_i":1761779356,"id":45754303,"options":[],"parent_id":45751435,"points":null,"story_id":45748661,"text":"They still capture the name on the card, so I would be careful about trying this, unless you can make use of a prepaid card.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:59:39.000Z","created_at_i":1761764379,"id":45751435,"options":[],"parent_id":45749805,"points":null,"story_id":45748661,"text":"What I gather from this is to always try a dead card first just in case the store is in offline mode","title":null,"type":"comment","url":null},{"author":"reaperducer","children":[{"author":"joshstrange","children":[],"created_at":"2025-10-30T12:21:01.000Z","created_at_i":1761826861,"id":45759230,"options":[],"parent_id":45753072,"points":null,"story_id":45748661,"text":"The POS I work on also has this feature. Line busters take the order and payment but we have a toggle where you can immediately \u201capprove\u201d and queue it up. If the payment fails then the person handing you your food will see it on the order and ask you for alternative payment. It helps prevent loss and speeds up the line overall.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:10:14.000Z","created_at_i":1761772214,"id":45753072,"options":[],"parent_id":45749805,"points":null,"story_id":45748661,"text":"IIRC, the grocery chain I worked for used to have an offline mode to move customers out the door.
Chick-fil-a has this.
One of the tech people there was on HN a few years ago describing their system. Credit card approval slows down the line, so the cards are automatically "approved" at the terminal, and the transaction is added to a queue.
The loss from fraudulent transactions turns out to be less than the loss from customers choosing another restaurant because of the speed of the lines.","title":null,"type":"comment","url":null},{"author":"ransom1538","children":[],"created_at":"2025-10-30T02:07:56.000Z","created_at_i":1761790076,"id":45755656,"options":[],"parent_id":45749805,"points":null,"story_id":45748661,"text":"I was shopping at a mall with a visa vanilla card once. I got it as a gift and didn't know the limit. No matter what I bought the card kept going -- and I never got a balance of what was on the card. Eventually, later that day it stopped. I called customer support and asked how much was left on the balance. They told me they had no idea my balance - but everything I bought was mine.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:07:13.000Z","created_at_i":1761757633,"id":45749805,"options":[],"parent_id":45749162,"points":null,"story_id":45748661,"text":"IIRC, the grocery chain I worked for used to have an offline mode to move customers out the door. But it meant that when the system came back online, if the customers card was denied, the customer got free groceries.","title":null,"type":"comment","url":null},{"author":"pndy","children":[],"created_at":"2025-10-29T20:32:04.000Z","created_at_i":1761769924,"id":45752649,"options":[],"parent_id":45749162,"points":null,"story_id":45748661,"text":"I remember last mechanical cash registers in my country in 90s and when these got replaced by early electronic ones with blue vacuum fluorescent tubes. Then everything got smaller and smaller. Now I'm pestered to "add the item to the cart" by software.
Last week I couldn't pay for flowers for grandma's grave because smartphone-sized card terminal refused to work - it stuck on charging-booting loop so I had to get cash. Tho my partner thinks she actually wanted to get cash without a receipt for herself excluding taxes","title":null,"type":"comment","url":null},{"author":"Jamie452","children":[],"created_at":"2025-10-29T20:33:52.000Z","created_at_i":1761770032,"id":45752666,"options":[],"parent_id":45749162,"points":null,"story_id":45748661,"text":"Just to add - this particular supermarket wasn\u2019t fully down, it took ages for them to press \u201csub total\u201d and then pick the payment method. I suspect it was slow waiting for a request to timeout perhaps","title":null,"type":"comment","url":null},{"author":"reaperducer","children":[],"created_at":"2025-10-29T21:02:46.000Z","created_at_i":1761771766,"id":45752984,"options":[],"parent_id":45749162,"points":null,"story_id":45748661,"text":"Currently standing in a half closed supermarket because the tills are down and they cant take payments
There's a fairly large supermarket near me that has both kinds of outages.
Occasionally it can't take cards because the (fiber? cable?) internet is down, so it's cash only.
Occasionally it can't take cash because the safe has its own cellular connection, and the cell tower is down.
I was at Frank's Pizza in downtown Houston a few weeks ago and they were giving slices of pizza away because the POS terminal died, and nobody knew enough math to take cash. I tried to give them a $10 and told them to keep the change, but "keep the change" is an unknown phrase these days. They simply couldn't wrap their brains around it. But hey, free pizza!","title":null,"type":"comment","url":null},{"author":"thisOtterBeGood","children":[{"author":"bombcar","children":[{"author":"hinkley","children":[],"created_at":"2025-10-30T18:42:04.000Z","created_at_i":1761849724,"id":45763638,"options":[],"parent_id":45758017,"points":null,"story_id":45748661,"text":"The odd thing is that the US has been teaching math using the \u201cin your head\u201d heuristic (New Math) for almost 20 years and yet young adults cannot make change without the machine to save their lives.
God help me if I hand someone $25 for a $14.75 total. I\u2019m getting small bills back.
I wonder what they teach in Germany.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T09:35:38.000Z","created_at_i":1761816938,"id":45758017,"options":[],"parent_id":45757454,"points":null,"story_id":45748661,"text":"It's sad the number of stores I've seen where they just shut down when they can't use the checkout machines; the clerks aren't allowed to do math even if they could.
Whereas the smaller, owner-run stores have more leeway; the local tiny grocery "sold" all freezer/refrigerator food for cheap/free during a power failure. The big Walmart closed and threw everything away the next day.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T07:59:45.000Z","created_at_i":1761811185,"id":45757454,"options":[],"parent_id":45749162,"points":null,"story_id":45748661,"text":"In Germany many stores still accept cash and some even only accept cash and we are ridiculed for this... Seems like one of the rare instances where this is useful :D","title":null,"type":"comment","url":null},{"author":"the_black_hand","children":[],"created_at":"2025-10-31T06:15:41.000Z","created_at_i":1761891341,"id":45768926,"options":[],"parent_id":45749162,"points":null,"story_id":45748661,"text":"why tf would a supermarket depend on Azure? Payment processing isn't their thing","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:29:01.000Z","created_at_i":1761755341,"id":45749162,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Currently standing in a half closed supermarket because the tills are down and they cant take payments","title":null,"type":"comment","url":null},{"author":"twodave","children":[{"author":"NDizzle","children":[],"created_at":"2025-10-29T16:34:31.000Z","created_at_i":1761755671,"id":45749266,"options":[],"parent_id":45749186,"points":null,"story_id":45748661,"text":"Yeah, I have non prod environments that don't use FD that are functioning. Routing through FD does not work. And a different app, nonprod doesn't use FD (and is working) but loads assets from the CDN (which is not working).
FD and CDN are global resources and are experiencing issues. Probably some other global resources as well.
Hate to say it, but DNS is looking like it's still the undisputed champ.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:30:09.000Z","created_at_i":1761755409,"id":45749186,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Appears to be an issue in Front Door. Our back end stuff is fine but FD is bouncing everything.","title":null,"type":"comment","url":null},{"author":"borg16","children":[{"author":"aurumque","children":[{"author":"Mr_Bees69","children":[],"created_at":"2025-10-29T16:59:20.000Z","created_at_i":1761757160,"id":45749672,"options":[],"parent_id":45749613,"points":null,"story_id":45748661,"text":"Yeah, im regretting my decision to buy an xbox now. Every once in a while, everything goes down.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:55:42.000Z","created_at_i":1761756942,"id":45749613,"options":[],"parent_id":45749204,"points":null,"story_id":45748661,"text":"Azure goes down all the time. On Friday we had an entire regional service down all day. Two weeks ago same thing different region. You only hear about it when it's something everyone uses like the portal, because in general nobody uses Azure unless they're held hostage.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:31:17.000Z","created_at_i":1761755477,"id":45749204,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"i guess folks in azure wanted to show some solidarity with aws brethren
(couldn't resist adding it. i acknowledge this comment adds no value to the discussion)","title":null,"type":"comment","url":null},{"author":"8cvor6j844qw_d6","children":[{"author":"conroydave","children":[],"created_at":"2025-10-29T16:32:40.000Z","created_at_i":1761755560,"id":45749228,"options":[],"parent_id":45749205,"points":null,"story_id":45748661,"text":"cost cutting attempts","title":null,"type":"comment","url":null},{"author":"iAMkenough","children":[{"author":"shepherdjerred","children":[{"author":"Capricorn2481","children":[],"created_at":"2025-10-29T16:44:46.000Z","created_at_i":1761756286,"id":45749434,"options":[],"parent_id":45749386,"points":null,"story_id":45748661,"text":"I don't think it's meant to be serious. It's a comment on Microsoft laying off their staff and stuffing their Azure and Dotnet teams with AI product managers.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:41:27.000Z","created_at_i":1761756087,"id":45749386,"options":[],"parent_id":45749302,"points":null,"story_id":45748661,"text":"Do you have any insight or do you just dislike AI? Incidents like this happened long before AI generated code","title":null,"type":"comment","url":null},{"author":"whynotminot","children":[],"created_at":"2025-10-29T16:48:00.000Z","created_at_i":1761756480,"id":45749493,"options":[],"parent_id":45749302,"points":null,"story_id":45748661,"text":"Yeah, these things never happened when humans were trusted without sufficient review and oversight of changes to production.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:36:31.000Z","created_at_i":1761755791,"id":45749302,"options":[],"parent_id":45749205,"points":null,"story_id":45748661,"text":"Trusting AI without sufficient review and oversight of changes to production.","title":null,"type":"comment","url":null},{"author":"stuff4ben","children":[],"created_at":"2025-10-29T16:47:03.000Z","created_at_i":1761756423,"id":45749472,"options":[],"parent_id":45749205,"points":null,"story_id":45748661,"text":"It's always freakin DNS...","title":null,"type":"comment","url":null},{"author":"jcims","children":[{"author":"sanskarix","children":[{"author":"jcims","children":[],"created_at":"2025-10-30T07:52:54.000Z","created_at_i":1761810774,"id":45757427,"options":[],"parent_id":45756920,"points":null,"story_id":45748661,"text":"I don't think I would recommend multi-cloud right out of the gate unless you already have a lot of experience in the space or there is a strong demand from your customers. There's a tremendous amount of overhead with security/compliance, incident management, billing, tooling, entitlements, etc. There are a number of external factors that drove our decision to do it, resiliency is just one of them. But we are a pretty big shop, spending ~$10M/mo on cloud infra and have ~100 people in the platform management department.
I would recommend focusing on multi-region within a single CSP instead (both for workloads AND your tooling), which covers the vast majority of incidents and lays some of the architectural foundation for multi-cloud down the road. Develop failover plans for each service in your architecture (eg. planned/tested runbooks to migrate to Traffic Manager in the event AFD goes down)
Also choose your provider wisely. We experience 3-5x the number of service-impacting incidents on Azure that we do on AWS. I'm sure others have different experiences, but I would never personally start a company on Azure. AWS has its own issues, of course, but reliability has not been a major one (relatively speaking) over the past 10 years. Last week's incident with DynamoDB in us-east-1 had zero impact on our AWS workloads in other regions.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T06:18:26.000Z","created_at_i":1761805106,"id":45756920,"options":[],"parent_id":45749784,"points":null,"story_id":45748661,"text":"This is the eternal tension for early-stage builders, isn't it? Multi-cloud gives you resilience, but adds so much complexity that it can actually slow down shipping features and iterating.
I'm curious\u2014at what point did you decide the overhead was worth it? Was it after experiencing an outage, or did you architect for it from day one?
As someone launching a product soon (more on the builder/product side than infra-engineer), I keep wrestling with this. The pragmatist in me says "start simple, prove the concept, then layer in resilience." But then you see events like this week and think "what if this happens during launch?"
How did you handle the operational complexity? Did you need dedicated DevOps folks, or are there patterns/tools that made it manageable for a smaller team?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:05:50.000Z","created_at_i":1761757550,"id":45749784,"options":[],"parent_id":45749205,"points":null,"story_id":45748661,"text":"We're multi-cloud and it really saved a few workloads last week with the AWS issue.
It's not easy though.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:31:21.000Z","created_at_i":1761755481,"id":45749205,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Quite close to the recent AWS outage. Let me take a look if its a major one similar to AWS.
Any guess on what's causing it?
In hindsight, I guess the foresight of some organizations to go multi-cloud was correct after all.","title":null,"type":"comment","url":null},{"author":"tartieret","children":[{"author":"llama052","children":[{"author":"drjasonharrison","children":[],"created_at":"2025-10-29T17:31:30.000Z","created_at_i":1761759090,"id":45750220,"options":[],"parent_id":45749292,"points":null,"story_id":45748661,"text":"https://www.cbc.ca/news/investigates/tesla-grok-mom-9.695693...
This mom\u2019s son was asking Tesla\u2019s Grok AI chatbot about soccer. It told him to send nude pics, she says
xAI, the company that developed Grok, responds to CBC: 'Legacy Media Lies'","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:36:02.000Z","created_at_i":1761755762,"id":45749292,"options":[],"parent_id":45749206,"points":null,"story_id":45748661,"text":"Always fun when you can't trust the main status page but have to go to some opinionated social medial website to see the actual problem.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:31:22.000Z","created_at_i":1761755482,"id":45749206,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Microsoft posted an update on X:\nhttps://x.com/AzureSupport/status/1983569891379835372?ref_sr...
"We\u2019re investigating an issue impacting Azure Front Door services. Customers may experience intermittent request failures or latency. Updates will be provided shortly."","title":null,"type":"comment","url":null},{"author":"vanviegen","children":[{"author":"AznHisoka","children":[],"created_at":"2025-10-29T16:40:28.000Z","created_at_i":1761756028,"id":45749369,"options":[],"parent_id":45749215,"points":null,"story_id":45748661,"text":"Luckily?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:31:59.000Z","created_at_i":1761755519,"id":45749215,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Many (all?) LinkedIn profiles are also down for me. Luckily the frontpage still works. ;-)
Go cloud!","title":null,"type":"comment","url":null},{"author":"SoftTalker","children":[{"author":"jeffdn","children":[{"author":"rcarmo","children":[{"author":"jansper39","children":[{"author":"rcarmo","children":[],"created_at":"2025-10-29T16:55:13.000Z","created_at_i":1761756913,"id":45749606,"options":[],"parent_id":45749552,"points":null,"story_id":45748661,"text":"I'd say DNS/Front Door (or some carrier interconnect) is the thing affected, since I can auth just fine in a few places. (I'm at MS, but not looped into anything operational these days, so I'm checking my personal subscription).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:51:43.000Z","created_at_i":1761756703,"id":45749552,"options":[],"parent_id":45749404,"points":null,"story_id":45748661,"text":"They don't, however authentication for those services relies on Entra ID which seems to be affected.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:42:50.000Z","created_at_i":1761756170,"id":45749404,"options":[],"parent_id":45749280,"points":null,"story_id":45748661,"text":"Are you absolutely sure?","title":null,"type":"comment","url":null},{"author":"RajT88","children":[],"created_at":"2025-10-29T17:33:19.000Z","created_at_i":1761759199,"id":45750254,"options":[],"parent_id":45749280,"points":null,"story_id":45748661,"text":"They definitely do run on Azure. Probably not 100%, but at least some footprint of those services do.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:35:29.000Z","created_at_i":1761755729,"id":45749280,"options":[],"parent_id":45749229,"points":null,"story_id":45748661,"text":"They don't run on Azure!","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:32:41.000Z","created_at_i":1761755561,"id":45749229,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"We're on Office 365 and so far it's still responding. At least Outlook and Teams is.","title":null,"type":"comment","url":null},{"author":"ape4","children":[{"author":"drewnick","children":[{"author":"dylan604","children":[{"author":"daveguy","children":[{"author":"dylan604","children":[],"created_at":"2025-10-29T17:43:18.000Z","created_at_i":1761759798,"id":45750406,"options":[],"parent_id":45750287,"points":null,"story_id":45748661,"text":"really should stop skimming the comment when i find a part to comment on <facepalm>","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:36:06.000Z","created_at_i":1761759366,"id":45750287,"options":[],"parent_id":45749988,"points":null,"story_id":45748661,"text":"(and on to real DCs later) would imply their bare metal is now located in a data center.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:18:07.000Z","created_at_i":1761758287,"id":45749988,"options":[],"parent_id":45749738,"points":null,"story_id":45748661,"text":"Are you still on a dual T1? that's gotta be expensive","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:03:14.000Z","created_at_i":1761757394,"id":45749738,"options":[],"parent_id":45749237,"points":null,"story_id":45748661,"text":"I've been doing it since 1998 in my bedroom with a dual T1 (and on to real DCs later). While I've had some outages for sure it makes me feel better I am not that divergent in uptime in the long run vs big clouds.","title":null,"type":"comment","url":null},{"author":"Aperocky","children":[{"author":"hshdhdhehd","children":[],"created_at":"2025-10-29T21:28:03.000Z","created_at_i":1761773283,"id":45753296,"options":[],"parent_id":45752006,"points":null,"story_id":45748661,"text":"At least YOLD is possible. Is there capacity in the world for everyone to ditch clouds.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:40:41.000Z","created_at_i":1761766841,"id":45752006,"options":[],"parent_id":45749237,"points":null,"story_id":45748661,"text":"I'd predict the year of linux desktop instead.","title":null,"type":"comment","url":null},{"author":"0xbadcafebee","children":[{"author":"reaperducer","children":[],"created_at":"2025-10-29T21:21:36.000Z","created_at_i":1761772896,"id":45753209,"options":[],"parent_id":45752485,"points":null,"story_id":45748661,"text":"TIL it's called Nirvana Fallacy
We used to call it "The grass is always greener on the other side of the fence."","title":null,"type":"comment","url":null},{"author":"hshdhdhehd","children":[],"created_at":"2025-10-29T21:26:10.000Z","created_at_i":1761773170,"id":45753269,"options":[],"parent_id":45752485,"points":null,"story_id":45748661,"text":"The upside of keep moving it acts as a hardener and chaos monkey. You shake out any crufty service no one knows how to build let alone deploy.","title":null,"type":"comment","url":null},{"author":"t0lo","children":[],"created_at":"2025-10-29T23:57:06.000Z","created_at_i":1761782226,"id":45754694,"options":[],"parent_id":45752485,"points":null,"story_id":45748661,"text":"Just experienced this with moving around multiple states and universities in the past year :) Grass really was greener in my hometown","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:17:27.000Z","created_at_i":1761769047,"id":45752485,"options":[],"parent_id":45749237,"points":null,"story_id":45748661,"text":"2027: the year of migrating from your own metal to a managed provider
2028: the year of migrating from a managed provider to the cloud
2029: the year of migrating from the cloud to your own metal in a rack
People keep thinking the solution to their problems is to do something new (that they don't fully understand).
TIL it's called Nirvana Fallacy","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:32:55.000Z","created_at_i":1761755575,"id":45749237,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"2026: the year of your own metal in a rack","title":null,"type":"comment","url":null},{"author":"majnata","children":[],"created_at":"2025-10-29T16:34:09.000Z","created_at_i":1761755649,"id":45749259,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The Azure API is still working though.","title":null,"type":"comment","url":null},{"author":"m_fayer","children":[],"created_at":"2025-10-29T16:34:34.000Z","created_at_i":1761755674,"id":45749269,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"And there goes https://www.microsoft.com/","title":null,"type":"comment","url":null},{"author":"rawgabbit","children":[{"author":"ctoth","children":[{"author":"onraglanroad","children":[],"created_at":"2025-10-29T17:56:17.000Z","created_at_i":1761760577,"id":45750580,"options":[],"parent_id":45749554,"points":null,"story_id":45748661,"text":"Now that is actually funny!","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:51:49.000Z","created_at_i":1761756709,"id":45749554,"options":[],"parent_id":45749346,"points":null,"story_id":45748661,"text":"Layoffs will continue until uptime improves!","title":null,"type":"comment","url":null},{"author":"FeteCommuniste","children":[{"author":"ryandrake","children":[],"created_at":"2025-10-29T17:14:03.000Z","created_at_i":1761758043,"id":45749909,"options":[],"parent_id":45749814,"points":null,"story_id":45748661,"text":"Like most technology initiative these tech CEOs dream up: You're going to get it and swallow it, whether you want it or not.","title":null,"type":"comment","url":null},{"author":"mring33621","children":[{"author":"binarymax","children":[],"created_at":"2025-10-29T17:18:20.000Z","created_at_i":1761758300,"id":45749992,"options":[],"parent_id":45749924,"points":null,"story_id":45748661,"text":"The mondAIs","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:14:53.000Z","created_at_i":1761758093,"id":45749924,"options":[],"parent_id":45749814,"points":null,"story_id":45748661,"text":"Sounds like someone has a case of the 'Mondays'...","title":null,"type":"comment","url":null},{"author":"gcanyon","children":[],"created_at":"2025-10-29T17:26:43.000Z","created_at_i":1761758803,"id":45750147,"options":[],"parent_id":45749814,"points":null,"story_id":45748661,"text":"Real life Pluribus https://en.wikipedia.org/wiki/Pluribus_(TV_series)","title":null,"type":"comment","url":null},{"author":"bostik","children":[],"created_at":"2025-10-29T17:38:03.000Z","created_at_i":1761759483,"id":45750316,"options":[],"parent_id":45749814,"points":null,"story_id":45748661,"text":"You'll have to find another planet.
Although judging by the available transports it will likely be colonized by nazis.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:07:31.000Z","created_at_i":1761757651,"id":45749814,"options":[],"parent_id":45749346,"points":null,"story_id":45748661,"text":"> [Satya Nadella] said that the company\u2019s future opportunity was to bring AI to all eight billion people on the planet.
But what if I don't want AI brought to me?","title":null,"type":"comment","url":null},{"author":"the_af","children":[],"created_at":"2025-10-29T17:17:35.000Z","created_at_i":1761758255,"id":45749977,"options":[],"parent_id":45749346,"points":null,"story_id":45748661,"text":"I especially like how Nadella speaks of layoffs as some kind of uncontrollable natural disaster, like a hurricane, caused by no-one in particular. A kind of "God works in mysterious ways".
> \u201cMicrosoft is being recognized and rewarded at levels never seen before,\u201d Nadella wrote. \u201cAnd yet, at the same time, we\u2019ve undergone layoffs. This is the enigma of success in an industry that has no franchise value.\u201d\n \n > Nadella explained the disconnect between thriving financials and layoffs by stating that \u201cprogress isn\u2019t linear\u201d and that it is \u201csometimes dissonant, and always demanding.\u201d\n > These decisions are among the most difficult we have to make. They affect people we\u2019ve worked alongside, learned from, and shared countless moments with\u2014our colleagues, teammates, and friends.\n> An inadvertent tenant configuration change within Azure Front Door (AFD) triggered a widespread service disruption affecting both Microsoft services and customer applications dependent on AFD for global content delivery. The change introduced an invalid or inconsistent configuration state that caused a significant number of AFD nodes to fail to load properly, leading to increased latencies, timeouts, and connection errors for downstream services.
> As unhealthy nodes dropped out of the global pool, traffic distribution across healthy nodes became imbalanced, amplifying the impact and causing intermittent availability even for regions that were partially healthy. We immediately blocked all further configuration changes to prevent additional propagation of the faulty state and began deploying a \u2018last known good\u2019 configuration across the global fleet. Recovery required reloading configurations across a large number of nodes and rebalancing traffic gradually to avoid overload conditions as nodes returned to service. This deliberate, phased recovery was necessary to stabilize the system while restoring scale and ensuring no recurrence of the issue.
> The trigger was traced to a faulty tenant configuration deployment process. Our protection mechanisms, to validate and block any erroneous deployments, failed due to a software defect which allowed the deployment to bypass safety validations. Safeguards have since been reviewed and additional validation and rollback controls have been immediately implemented to prevent similar issues in the future.
So, so far they're saying it's a combination of bad config + their config-validator had a bug. Would love more details.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:41:49.000Z","created_at_i":1761756109,"id":45749390,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Looking forward to the post mortem.","title":null,"type":"comment","url":null},{"author":"rcarmo","children":[{"author":"tgv","children":[],"created_at":"2025-10-29T18:46:58.000Z","created_at_i":1761763618,"id":45751274,"options":[],"parent_id":45749398,"points":null,"story_id":45748661,"text":"I tried to look some things up on their support pages before 1600Z, and it timed-out. The Dutch railways are also affected (they're an MS shop, IIRC).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:42:27.000Z","created_at_i":1761756147,"id":45749398,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Not seeing it. I have VMs in US East and Netherlands and they're up.","title":null,"type":"comment","url":null},{"author":"giantg2","children":[],"created_at":"2025-10-29T16:43:01.000Z","created_at_i":1761756181,"id":45749406,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Compare the comments and news coverage on this compared to the AWS outage... pretty telling.","title":null,"type":"comment","url":null},{"author":"Uehreka","children":[{"author":"hypeatei","children":[{"author":"Hamuko","children":[],"created_at":"2025-10-29T16:58:36.000Z","created_at_i":1761757116,"id":45749659,"options":[],"parent_id":45749639,"points":null,"story_id":45748661,"text":"Gonna build my application to be multicloud so that it requires multiple cloud platforms to be online at the same time. The RAID 0 of cloud computing.","title":null,"type":"comment","url":null},{"author":"andoma","children":[],"created_at":"2025-10-29T16:59:48.000Z","created_at_i":1761757188,"id":45749680,"options":[],"parent_id":45749639,"points":null,"story_id":45748661,"text":"Go multi-cloud they said...","title":null,"type":"comment","url":null},{"author":"SoftTalker","children":[{"author":"mring33621","children":[],"created_at":"2025-10-29T17:13:29.000Z","created_at_i":1761758009,"id":45749903,"options":[],"parent_id":45749688,"points":null,"story_id":45748661,"text":"you wouldn't believe some of the crap enterprise bigco mgmt put in place for disaster recovery.
they think that they are 'eliminating a single point of failure', but in reality, they end up adding multiple, complicated points of mostly failure.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:00:08.000Z","created_at_i":1761757208,"id":45749688,"options":[],"parent_id":45749639,"points":null,"story_id":45748661,"text":"They are multi-cloud --- vulnerable to all outages!","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:57:23.000Z","created_at_i":1761757043,"id":45749639,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"Starbucks mobile was down during the AWS outage too...","title":null,"type":"comment","url":null},{"author":"pants2","children":[{"author":"parliament32","children":[],"created_at":"2025-10-29T17:07:30.000Z","created_at_i":1761757650,"id":45749813,"options":[],"parent_id":45749715,"points":null,"story_id":45748661,"text":"Always has been.","title":null,"type":"comment","url":null},{"author":"lysace","children":[{"author":"jjice","children":[],"created_at":"2025-10-29T18:01:09.000Z","created_at_i":1761760869,"id":45750668,"options":[],"parent_id":45749980,"points":null,"story_id":45748661,"text":"Yeah looks like they're back on M5.
dang saying it's temporary: https://news.ycombinator.com/item?id=32031136
$ dig news.ycombinator.com\n\n ; <<>> DiG 9.10.6 <<>> news.ycombinator.com\n ;; global options: +cmd\n ;; Got answer:\n ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54819\n ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1\n\n ;; OPT PSEUDOSECTION:\n ; EDNS: version: 0, flags:; udp: 512\n ;; QUESTION SECTION:\n ;news.ycombinator.com. IN A\n\n ;; ANSWER SECTION:\n news.ycombinator.com. 1 IN A 209.216.230.207\n\n ;; Query time: 79 msec\n ;; SERVER: 100.100.100.100#53(100.100.100.100)\n ;; WHEN: Wed Oct 29 13:59:29 EDT 2025\n ;; MSG SIZE rcvd: 65\nhttps://news.ycombinator.com/item?id=32030400","title":null,"type":"comment","url":null},{"author":"dang","children":[{"author":"airstrike","children":[{"author":"Cthulhu_","children":[],"created_at":"2025-10-30T15:03:13.000Z","created_at_i":1761836593,"id":45760830,"options":[],"parent_id":45750648,"points":null,"story_id":45748661,"text":"My plan B was going to the Stack Exchange homepage for some interesting threads but it got repetitive.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:59:51.000Z","created_at_i":1761760791,"id":45750648,"options":[],"parent_id":45750608,"points":null,"story_id":45748661,"text":"I love that "Ask HN: What'd you do while HN was down?" was a thing","title":null,"type":"comment","url":null},{"author":"gogusrl","children":[],"created_at":"2025-10-29T19:41:12.000Z","created_at_i":1761766872,"id":45752014,"options":[],"parent_id":45750608,"points":null,"story_id":45748661,"text":"I got burnt by this bug on freakin' Christmas Eve 2020 ( https://forum.hddguru.com/viewtopic.php?f=10&t=40766 ). There was some data loss and a lot of lessons learned.","title":null,"type":"comment","url":null},{"author":"hinkley","children":[],"created_at":"2025-10-29T20:11:21.000Z","created_at_i":1761768681,"id":45752402,"options":[],"parent_id":45750608,"points":null,"story_id":45748661,"text":"I\u2019ve seen this up close twice and I\u2019m surprised it\u2019s only twice. Between March and September one year, 6 people on one team had to get new hard drives in their thinkpads and rebuild their systems. All from the same PO but doled out over the course of a project rampup. That was the first project where the onboarding docs were really really good, since we got a lot of practice in a short period of time.
Long before that, the first raid array anyone set up for my (teams\u2019) usage, arrived from Sun with 2 dead drives out of 10. They RMA\u2019d us 2 more drives and one of those was also DOA. That was a couple years after Sun stopped burning in hardware for cost savings, which maybe wasn\u2019t that much of a savings all things considered.","title":null,"type":"comment","url":null},{"author":"praccu","children":[],"created_at":"2025-10-30T02:27:58.000Z","created_at_i":1761791278,"id":45755776,"options":[],"parent_id":45750608,"points":null,"story_id":45748661,"text":"Many years ago (13?), I was around when Amazon moved SABLE from RAM to SSDs. A whole rack came from a single batch, and something like 128 disks went out at once.
I was an intern but everyone seemed very stressed.","title":null,"type":"comment","url":null},{"author":"Cthulhu_","children":[],"created_at":"2025-10-30T15:02:10.000Z","created_at_i":1761836530,"id":45760814,"options":[],"parent_id":45750608,"points":null,"story_id":45748661,"text":"Man I hit something like that once, a SSD had a firmware bug where it would stop working at an exact number of hours.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:57:08.000Z","created_at_i":1761760628,"id":45750608,"options":[],"parent_id":45749715,"points":null,"story_id":45748661,"text":"Just don't use genetically identical hardware:
https://news.ycombinator.com/item?id=32031639
https://news.ycombinator.com/item?id=32032235
Edit: wow, I can't believe we hadn't put https://news.ycombinator.com/item?id=32031243 in https://news.ycombinator.com/highlights. Fixed now.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:01:52.000Z","created_at_i":1761757312,"id":45749715,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"Good thing HN is hosted on a couple servers in a basement. Much more reliable than cloud, it seems!","title":null,"type":"comment","url":null},{"author":"sergiotapia","children":[{"author":"iso1631","children":[{"author":"peanut-walrus","children":[{"author":"bobro","children":[{"author":"DaSHacka","children":[{"author":"munchlax","children":[],"created_at":"2025-10-30T08:19:26.000Z","created_at_i":1761812366,"id":45757573,"options":[],"parent_id":45757547,"points":null,"story_id":45748661,"text":"And ridiculously expensive","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T08:15:06.000Z","created_at_i":1761812106,"id":45757547,"options":[],"parent_id":45756675,"points":null,"story_id":45748661,"text":"I agree, but because the coffee is crap","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T05:20:26.000Z","created_at_i":1761801626,"id":45756675,"options":[],"parent_id":45752957,"points":null,"story_id":45748661,"text":"Or maybe we should throw them in jail.","title":null,"type":"comment","url":null},{"author":"bombcar","children":[],"created_at":"2025-10-30T09:25:03.000Z","created_at_i":1761816303,"id":45757957,"options":[],"parent_id":45752957,"points":null,"story_id":45748661,"text":"It's absolutely batshit that an in-person transaction with cash becomes impossible when the computers are down.
I've seen it multiple times at various stores; only once did I see them taking cash and writing things down (probably to enter into the system later when it came back up).","title":null,"type":"comment","url":null},{"author":"iso1631","children":[],"created_at":"2025-10-30T10:37:47.000Z","created_at_i":1761820667,"id":45758409,"options":[],"parent_id":45752957,"points":null,"story_id":45748661,"text":"If I were a Starbucks shareholder I wouldn't be happy that my company is throwing away revenue because of the CTO's decision to outsource accountability
Time and time again it's shown that AWS is far more expensive than other solutions, just easier for the Execs to offshore the blame.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:00:09.000Z","created_at_i":1761771609,"id":45752957,"options":[],"parent_id":45752002,"points":null,"story_id":45748661,"text":"Why? Starbucks is not providing a critical service. Spending less money and resources and just accepting the risk that occasionally you won't be able to sell coffee for a few hours is a completely valid decision from both management and engineering pov.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:40:36.000Z","created_at_i":1761766836,"id":45752002,"options":[],"parent_id":45749773,"points":null,"story_id":45748661,"text":"You'd think that Starbucks execs would be held accountable for the fragile system they have put in place.
But they won't be.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:05:10.000Z","created_at_i":1761757510,"id":45749773,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"Wow I just left a Starbucks drivethru line because it was just not moving. I guess it was because of this.","title":null,"type":"comment","url":null},{"author":"01284a7e","children":[],"created_at":"2025-10-29T18:27:01.000Z","created_at_i":1761762421,"id":45751017,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"Ha, maybe rethink the I AM NOTHING BUT A HUGE CLOUD CONSUMER thing on some fundamental levels? Like food?","title":null,"type":"comment","url":null},{"author":"Theodores","children":[],"created_at":"2025-10-29T19:36:43.000Z","created_at_i":1761766603,"id":45751943,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"My inner Nelson-from-the-Simpsons wishes I was on your team today, able to flaunt my flask of tea and homemade packed sandwiches. I would tease you by saying 'ha ha!' as your efforts to order coffee with IP packets failed.
I always go everywhere adequately prepared for beverages and food. Thanks to your comment, I have a new reason to do so. Take out coffees are actually far from guaranteed. Payment systems could go down, my bank account could be hacked or maybe the coffee shop could be randomly closed. Heck, I might even have an accident crossing the road. Anything could happen. Hence, my humble flask might not have the top beverage in it but at least it works.
We all design systems with redundancy, backups and whatnot, but few of us apply this thinking to our food and drink. Maybe get a kettle for the office and a backup kettle, in case the first one fails?","title":null,"type":"comment","url":null},{"author":"port11","children":[{"author":"pasc1878","children":[{"author":"port11","children":[],"created_at":"2025-10-31T12:21:37.000Z","created_at_i":1761913297,"id":45771235,"options":[],"parent_id":45760560,"points":null,"story_id":45748661,"text":"It could be local-first. I don't mind the cross-device sync being done centrally, of course, but the app specifically asks for access to Home and Local Network. I wonder if Home Assistant could deal with blackouts\u2026","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T14:42:18.000Z","created_at_i":1761835338,"id":45760560,"options":[],"parent_id":45752384,"points":null,"story_id":45748661,"text":"Same here for netatmo - ironically I replied to an incident report with netatmo saying all was OK when the whole system was falling over.
However netatmo does need to have a server to store data as you need to consolidate acreoss devices plus you can query gfor a year's data and that won't and can't be held locally.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:09:13.000Z","created_at_i":1761768553,"id":45752384,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"I noticed it when my Netatmo rigamajig stopped notifying me of bad indoor air quality. Lovely. Why does it need to go through the cloud if the data is right there in the home network\u2026","title":null,"type":"comment","url":null},{"author":"jeffrallen","children":[{"author":"0_____0","children":[],"created_at":"2025-10-30T02:23:29.000Z","created_at_i":1761791009,"id":45755754,"options":[],"parent_id":45752562,"points":null,"story_id":45748661,"text":"I was at a McDs a couple months back and I'm pretty sure you had to use the kiosk to order. Some places are deprecating the cashier entirely.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:24:24.000Z","created_at_i":1761769464,"id":45752562,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"You know you can talk to your barista and ask for a bagel, right? If you're lucky they still take cash... if you still _have_ cash. :)","title":null,"type":"comment","url":null},{"author":"Havoc","children":[],"created_at":"2025-10-29T21:24:36.000Z","created_at_i":1761773076,"id":45753252,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"The sysadmin subreddit tends to beat hn on outage reports by an hour+ in my experience.
Bunch of on-call peeps over there that definitely know the instant something major goes down","title":null,"type":"comment","url":null},{"author":"garbagewoman","children":[],"created_at":"2025-10-30T07:14:25.000Z","created_at_i":1761808465,"id":45757220,"options":[],"parent_id":45749410,"points":null,"story_id":45748661,"text":"Service culture is so hollow","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:43:21.000Z","created_at_i":1761756201,"id":45749410,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I noticed that Starbucks mobile ordering was down and thought \u201cwelp, I guess I\u2019ll order a bagel and coffee on Grubhub\u201d, then GrubHub was down. My next stop was HN to find the common denominator, and y\u2019all did not disappoint.","title":null,"type":"comment","url":null},{"author":"avgDev","children":[{"author":"ThatManulTheCat","children":[],"created_at":"2025-10-29T16:49:46.000Z","created_at_i":1761756586,"id":45749524,"options":[],"parent_id":45749411,"points":null,"story_id":45748661,"text":"Free upgrade","title":null,"type":"comment","url":null},{"author":"Mr_Bees69","children":[],"created_at":"2025-10-29T16:57:53.000Z","created_at_i":1761757073,"id":45749648,"options":[],"parent_id":45749411,"points":null,"story_id":45748661,"text":"Might be a failsafe, if you cant get a license status, and you're aware that MS is down, just default to the highest tier.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:43:22.000Z","created_at_i":1761756202,"id":45749411,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I am having a bunch of issues. It looks like their sites and azure are both affected.
I also got weird notification in VS2022 that my license key was upgraded to Enterprise, but we did not purchase anything.","title":null,"type":"comment","url":null},{"author":"vs4vijay","children":[{"author":"ipsum2","children":[{"author":"charv","children":[{"author":"karateka01","children":[{"author":"millzlane","children":[],"created_at":"2025-10-29T17:46:05.000Z","created_at_i":1761759965,"id":45750452,"options":[],"parent_id":45749721,"points":null,"story_id":45748661,"text":"It begs the question from a noob like me... Where should they host the status page? Surely it shouldn't be on the same infra that it's supposed to be monitoring. Am I correct in thinking that?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:02:16.000Z","created_at_i":1761757336,"id":45749721,"options":[],"parent_id":45749612,"points":null,"story_id":45748661,"text":"status page being affected by the same issue is so lame","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:55:31.000Z","created_at_i":1761756931,"id":45749612,"options":[],"parent_id":45749438,"points":null,"story_id":45748661,"text":"oh the irony, the status link being down too","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:44:59.000Z","created_at_i":1761756299,"id":45749438,"options":[],"parent_id":45749413,"points":null,"story_id":45748661,"text":"Status page (first link) is down for me. Second one works","title":null,"type":"comment","url":null},{"author":"aftergibson","children":[],"created_at":"2025-10-29T16:45:51.000Z","created_at_i":1761756351,"id":45749449,"options":[],"parent_id":45749413,"points":null,"story_id":45748661,"text":"Looks like the status page is overloaded...","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:43:34.000Z","created_at_i":1761756214,"id":45749413,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Service Status: https://status.cloud.microsoft/ and https://azure.status.microsoft/en-us/status","title":null,"type":"comment","url":null},{"author":"mythz","children":[{"author":"bad_haircut72","children":[{"author":"yabones","children":[],"created_at":"2025-10-29T17:27:16.000Z","created_at_i":1761758836,"id":45750156,"options":[],"parent_id":45749453,"points":null,"story_id":45748661,"text":"DO has been shockingly reliable for me. I shut down a neglected box almost 900 days uptime the other day. In that time AWS has randomly dropped many of my boxes with no warning requiring a manual stop/start action to recover them... But everybody keeps telling me that DO isn't "as reliable" as the big three are.","title":null,"type":"comment","url":null},{"author":"robotnikman","children":[],"created_at":"2025-10-29T17:53:47.000Z","created_at_i":1761760427,"id":45750548,"options":[],"parent_id":45749453,"points":null,"story_id":45748661,"text":"Same here, I run a few droplets for personal projects and never had any issues with then.","title":null,"type":"comment","url":null},{"author":"ipdashc","children":[],"created_at":"2025-10-29T19:11:05.000Z","created_at_i":1761765065,"id":45751586,"options":[],"parent_id":45749453,"points":null,"story_id":45748661,"text":"To be fair, in the AWS/Azure outages, I don't think any individual (already created) boxes went down, either. In AWS' case you couldn't start up new EC2 instances, and presumably same for Azure (unless you bypass the management portal, I guess). And obviously services like DynamoDB and Front Door, respectively, went down. Hetzner/DO don't offer those, right? Or at least they're not very popular.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:46:01.000Z","created_at_i":1761756361,"id":45749453,"options":[],"parent_id":45749437,"points":null,"story_id":45748661,"text":"Same with DigitalOcean. I run one box and it hasnt gone down for like 2 years","title":null,"type":"comment","url":null},{"author":"bongodongobob","children":[{"author":"NDizzle","children":[{"author":"bongodongobob","children":[{"author":"Jarwain","children":[{"author":"out_sider","children":[],"created_at":"2025-10-29T17:17:39.000Z","created_at_i":1761758259,"id":45749979,"options":[],"parent_id":45749859,"points":null,"story_id":45748661,"text":"we use front door (as does miccrosoft.com) and our website was down, I was able to change the DNS records to point directly to our server and will leave it like that for a few hours until everything is green","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:10:23.000Z","created_at_i":1761757823,"id":45749859,"options":[],"parent_id":45749575,"points":null,"story_id":45748661,"text":"Do you use on front door? Our VMs that don't are working fine, but our app services that do aren't.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:53:29.000Z","created_at_i":1761756809,"id":45749575,"options":[],"parent_id":45749504,"points":null,"story_id":45748661,"text":"Interesting, everything else is working just fine for us. Offices across the US.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:48:27.000Z","created_at_i":1761756507,"id":45749504,"options":[],"parent_id":45749488,"points":null,"story_id":45748661,"text":"It's absolutely not only the admin portal.
It's CDN and FrontDoor at least.","title":null,"type":"comment","url":null},{"author":"Foobar8568","children":[],"created_at":"2025-10-29T17:38:08.000Z","created_at_i":1761759488,"id":45750321,"options":[],"parent_id":45749488,"points":null,"story_id":45748661,"text":"Plenty of sites are down and/or login not available. It's just really a mess.","title":null,"type":"comment","url":null},{"author":"_vqpz","children":[],"created_at":"2025-10-29T18:01:21.000Z","created_at_i":1761760881,"id":45750673,"options":[],"parent_id":45749488,"points":null,"story_id":45748661,"text":"The bank I work at is reporting all Power Apps applications are down.","title":null,"type":"comment","url":null},{"author":"12_throw_away","children":[],"created_at":"2025-10-29T18:03:09.000Z","created_at_i":1761760989,"id":45750700,"options":[],"parent_id":45749488,"points":null,"story_id":45748661,"text":"Nope, more than the portal. For instance, I just searched for "Azure Front Door" because I hadn't heard of it before (I now know it's a CDN), and neither the product page itself [1] nor the technical docs [2] are coming up for me.
[1] https://azure.microsoft.com/en-us/products/frontdoor
[2] https://learn.microsoft.com/en-us/azure/frontdoor/front-door...","title":null,"type":"comment","url":null},{"author":"pocketman","children":[],"created_at":"2025-10-29T18:06:36.000Z","created_at_i":1761761196,"id":45750732,"options":[],"parent_id":45749488,"points":null,"story_id":45748661,"text":"It looks like it is just the 365 admin panels for us. Admittedly, we don't currently host any other services on Azure though.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:47:43.000Z","created_at_i":1761756463,"id":45749488,"options":[],"parent_id":45749437,"points":null,"story_id":45748661,"text":"It's just the admin portal.","title":null,"type":"comment","url":null},{"author":"graemep","children":[],"created_at":"2025-10-29T17:19:00.000Z","created_at_i":1761758340,"id":45749998,"options":[],"parent_id":45749437,"points":null,"story_id":45748661,"text":"Ostensible reason.
The real reason is that outages are not your fault. Its the new version of "nobody ever got fired for buying IBM" - later it became MS, and now its any big cloud provider.","title":null,"type":"comment","url":null},{"author":"jmaker","children":[],"created_at":"2025-10-29T17:45:29.000Z","created_at_i":1761759929,"id":45750439,"options":[],"parent_id":45749437,"points":null,"story_id":45748661,"text":"For one it\u2019s statistics - Hetzner simply runs far fewer major services than hyperscalers. And the services they run are also more affluent, with larger customer bases, so downtimes are systemically critical. Therefore it\u2019s louder.
On the merits though, I agree, haven\u2019t had any serious issues with Hetzner.","title":null,"type":"comment","url":null},{"author":"prmoustache","children":[{"author":"zavec","children":[],"created_at":"2025-10-29T17:54:16.000Z","created_at_i":1761760456,"id":45750555,"options":[],"parent_id":45750524,"points":null,"story_id":45748661,"text":"It's like back in school when there was a snow day!","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:52:22.000Z","created_at_i":1761760342,"id":45750524,"options":[],"parent_id":45749437,"points":null,"story_id":45748661,"text":"I think the biggest features of the big cloud vendors is that when they are down, not only you but your customers and your competitors usually have issues at the same time so everybody just shrug and have a lazy/off day at the same time. Even on call teams reall just have to wait and stay on standby because there is very little they can do. Doing a failover can be slower than waiting for the recovery, not help at all if outage is spanned accross several region, or bring aditional risks.
And more importantly nobody lose any reputation except AWS/Azure/Google.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:44:56.000Z","created_at_i":1761756296,"id":45749437,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"High availability is touted as a reason for their high prices, but I swear I read about major cloud outages far more than I experience any outages at Hetzner.","title":null,"type":"comment","url":null},{"author":"dlcarrier","children":[{"author":"m_fayer","children":[{"author":"shishcat","children":[],"created_at":"2025-10-29T17:00:15.000Z","created_at_i":1761757215,"id":45749691,"options":[],"parent_id":45749536,"points":null,"story_id":45748661,"text":"Who?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:50:35.000Z","created_at_i":1761756635,"id":45749536,"options":[],"parent_id":45749468,"points":null,"story_id":45748661,"text":"And if they don't, we'll know who the culprit is.","title":null,"type":"comment","url":null},{"author":"Insanity","children":[{"author":"xenolithis","children":[],"created_at":"2025-10-29T17:12:31.000Z","created_at_i":1761757951,"id":45749885,"options":[],"parent_id":45749678,"points":null,"story_id":45748661,"text":"fairly certain they had a significant multi region outage within the past few years. I'll try to find some details to link.
Few customers....few voices to complain as well.","title":null,"type":"comment","url":null},{"author":"Mr_Bees69","children":[],"created_at":"2025-10-29T17:17:12.000Z","created_at_i":1761758232,"id":45749967,"options":[],"parent_id":45749678,"points":null,"story_id":45748661,"text":"as a victim of xbox, azure is down 'bout as often as its up","title":null,"type":"comment","url":null},{"author":"luhn","children":[],"created_at":"2025-10-29T17:23:11.000Z","created_at_i":1761758591,"id":45750070,"options":[],"parent_id":45749678,"points":null,"story_id":45748661,"text":"They had a pretty massive one earlier this year. https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S...
This isn't GCP's fault, but the outage ended up taking down Cloudflare too, so in total impact I think that takes the cake.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:59:46.000Z","created_at_i":1761757186,"id":45749678,"options":[],"parent_id":45749468,"points":null,"story_id":45748661,"text":"Maybe they are and no one realized yet.. :P
That said, I don't hear about GCP outages all that often. I do think AWS might be leading in outages, but that's a gut feeling, I didn't look up numbers.","title":null,"type":"comment","url":null},{"author":"briffle","children":[],"created_at":"2025-10-29T17:07:48.000Z","created_at_i":1761757668,"id":45749818,"options":[],"parent_id":45749468,"points":null,"story_id":45748661,"text":"here's hoping its Oracle's cloud instead....","title":null,"type":"comment","url":null},{"author":"gtowey","children":[{"author":"dkdcio","children":[],"created_at":"2025-10-29T17:25:18.000Z","created_at_i":1761758718,"id":45750118,"options":[],"parent_id":45749906,"points":null,"story_id":45748661,"text":"throwback to when they deleted a customer's entire account! https://arstechnica.com/gadgets/2024/05/google-cloud-acciden...","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:13:55.000Z","created_at_i":1761758035,"id":45749906,"options":[],"parent_id":45749468,"points":null,"story_id":45748661,"text":"This is because Azure just copies everything AWS does. Google is a bit more innovative, they will have something else unexpected happen.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:46:48.000Z","created_at_i":1761756408,"id":45749468,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Yesterday Amazon, today Microsoft. Are Google's cloud services going down tomorrow?","title":null,"type":"comment","url":null},{"author":"flumpcakes","children":[{"author":"kierenj","children":[{"author":"reknih","children":[],"created_at":"2025-10-29T16:52:43.000Z","created_at_i":1761756763,"id":45749567,"options":[],"parent_id":45749505,"points":null,"story_id":45748661,"text":"Front Door is down for us (as Azure\u2018s Twitter account confirms)","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:48:36.000Z","created_at_i":1761756516,"id":45749505,"options":[],"parent_id":45749477,"points":null,"story_id":45748661,"text":"FWIW, all of our databases, VMs, AKS clusters, services, jobs etc - are all working fine. Which services are down for you, maybe we can build a list?","title":null,"type":"comment","url":null},{"author":"parliament32","children":[{"author":"flumpcakes","children":[],"created_at":"2025-10-29T17:18:34.000Z","created_at_i":1761758314,"id":45749993,"options":[],"parent_id":45749621,"points":null,"story_id":45748661,"text":"We're using Application Gateway for ingress, that seems to be effected.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:56:13.000Z","created_at_i":1761756973,"id":45749621,"options":[],"parent_id":45749477,"points":null,"story_id":45748661,"text":"All of our Azure workloads are up, but we don't use Azure Front Door. That seems to be the only impacted product, apart from the management portal.","title":null,"type":"comment","url":null},{"author":"jayw_lead","children":[],"created_at":"2025-10-29T17:14:16.000Z","created_at_i":1761758056,"id":45749913,"options":[],"parent_id":45749477,"points":null,"story_id":45748661,"text":"Same playbook for AWS. When they admitted that Dynamo was inaccessible, they failed to provide context that their internal services are heavily dependent on Dynamo
It's only after the fact they are transparent about the impact","title":null,"type":"comment","url":null},{"author":"wbsun","children":[],"created_at":"2025-10-29T21:32:03.000Z","created_at_i":1761773523,"id":45753328,"options":[],"parent_id":45749477,"points":null,"story_id":45748661,"text":"Does their status page depend on something that is down already, so the page just fails static now hence no new updates?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:47:14.000Z","created_at_i":1761756434,"id":45749477,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Pretty much all Azure services seem to be down. Their status page says it's only the portal since 16:00. It would be nice if these mega-companies could update their status page when they take down a large fraction of the Internet and thousands of services that use them.","title":null,"type":"comment","url":null},{"author":"vpears87","children":[{"author":"CommanderData","children":[],"created_at":"2025-10-29T16:56:45.000Z","created_at_i":1761757005,"id":45749627,"options":[],"parent_id":45749481,"points":null,"story_id":45748661,"text":"Likely behind Azure Front Door.
Much of Xbox is behind that too.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:47:29.000Z","created_at_i":1761756449,"id":45749481,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"At least MSFT is consistent: https://www.microsoft.com/en-us/ is down as well","title":null,"type":"comment","url":null},{"author":"dlcarrier","children":[{"author":"shagie","children":[{"author":"MiguelHudnandez","children":[],"created_at":"2025-10-29T17:56:39.000Z","created_at_i":1761760599,"id":45750590,"options":[],"parent_id":45750066,"points":null,"story_id":45748661,"text":"When you look at the scale of the reports, you find they are much lower than Azure's. seeing a bunch of 24-hour sparkline type graphs next to each other can make it look like they are equally impacted, but AWS has 500 reports and Azure has 20,000. The scale is hidden by the choice of graph.
In other words, people reporting outages at AWS are probably having trouble with microsoft-run DNS services or caching proxies. It's not that the issues aren't there, it's that the internet is full of intermingled complexity. Just that amount of organic false-positives can make it look like an unrelated major service is impacted.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:23:05.000Z","created_at_i":1761758585,"id":45750066,"options":[],"parent_id":45749494,"points":null,"story_id":45748661,"text":"Like AWS or GCP? https://downdetector.com/status/aws-amazon-web-services/ - https://downdetector.com/status/google-cloud/","title":null,"type":"comment","url":null},{"author":"Insanity","children":[],"created_at":"2025-10-29T17:23:12.000Z","created_at_i":1761758592,"id":45750071,"options":[],"parent_id":45749494,"points":null,"story_id":45748661,"text":"Multi cloud is really hard to get right at scale, and honestly not worth the effort for the majority of companies and use-case.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:48:00.000Z","created_at_i":1761756480,"id":45749494,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"We're quickly learning who's relying on a single cloud provider.","title":null,"type":"comment","url":null},{"author":"macshome","children":[{"author":"chokolad","children":[],"created_at":"2025-10-29T17:04:59.000Z","created_at_i":1761757499,"id":45749768,"options":[],"parent_id":45749500,"points":null,"story_id":45748661,"text":"Majority of actual Xbox services are working fine, xbox.com itself is busted.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:48:15.000Z","created_at_i":1761756495,"id":45749500,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I just tried to check the Xbox services status page and it never even loaded.","title":null,"type":"comment","url":null},{"author":"Jarwain","children":[],"created_at":"2025-10-29T16:48:17.000Z","created_at_i":1761756497,"id":45749501,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"On our end, our VMs are still working, so our gitlab instance is still up. Our services using Azure App Services are available through their provided url. However, Front Door is failing to resolve any domains that it was responsible for.","title":null,"type":"comment","url":null},{"author":"ThatManulTheCat","children":[],"created_at":"2025-10-29T16:49:20.000Z","created_at_i":1761756560,"id":45749512,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Yudkowsky's feared Superintellignece holding Azure hostage","title":null,"type":"comment","url":null},{"author":"LaserToy","children":[],"created_at":"2025-10-29T16:49:39.000Z","created_at_i":1761756579,"id":45749521,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Azure portal still insists the issue is jsut with Console.
We had to bypass the Frontdoor","title":null,"type":"comment","url":null},{"author":"ksec","children":[],"created_at":"2025-10-29T16:51:04.000Z","created_at_i":1761756664,"id":45749543,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":">Last week AWS, now this.
This is not the first or second time this happened, multiple Hyperscaler failed one by one.","title":null,"type":"comment","url":null},{"author":"cbovis","children":[],"created_at":"2025-10-29T16:51:46.000Z","created_at_i":1761756706,"id":45749553,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Looks to be affecting our pipelines that rely on Playwright as they download images from Azure e.g. https://playwright.azureedge.net/builds/chromium/1124/chromi... which aren't currently resolving.","title":null,"type":"comment","url":null},{"author":"kierenj","children":[],"created_at":"2025-10-29T16:53:13.000Z","created_at_i":1761756793,"id":45749571,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"microsoft.com is back -
edit: it worked once, then died again. So I guess - some resolvers, or FD servers may be working!","title":null,"type":"comment","url":null},{"author":"speckx","children":[{"author":"Boxersteavee","children":[],"created_at":"2025-10-29T17:13:28.000Z","created_at_i":1761758008,"id":45749901,"options":[],"parent_id":45749657,"points":null,"story_id":45748661,"text":"503 Service Unavailable","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:58:35.000Z","created_at_i":1761757115,"id":45749657,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"FYI: https://status.cloud.microsoft/","title":null,"type":"comment","url":null},{"author":"speckx","children":[{"author":"chuckadams","children":[],"created_at":"2025-10-29T17:02:38.000Z","created_at_i":1761757358,"id":45749728,"options":[],"parent_id":45749660,"points":null,"story_id":45748661,"text":"Which itself is^H^H was down. Wow.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:58:37.000Z","created_at_i":1761757117,"id":45749660,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"FYI: https://status.cloud.microsoft/","title":null,"type":"comment","url":null},{"author":"jacquesm","children":[],"created_at":"2025-10-29T17:00:55.000Z","created_at_i":1761757255,"id":45749701,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"It is much more than azure. One of my kids needs a key for their laptop and can't reach that either. Great excuse though, 'Azure ate my homework'. What a ridiculous world we are building. Fuck MS and their account requirements for windows.","title":null,"type":"comment","url":null},{"author":"improbableinf","children":[],"created_at":"2025-10-29T17:01:50.000Z","created_at_i":1761757310,"id":45749713,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"What a time to be alive!","title":null,"type":"comment","url":null},{"author":"thimkerbell","children":[],"created_at":"2025-10-29T17:03:04.000Z","created_at_i":1761757384,"id":45749736,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Does (should, could) DownDetector also say what customer-facing services are down, when some infrastructure is unworking? Or is that the info that the malefactors are seeking?","title":null,"type":"comment","url":null},{"author":"improbableinf","children":[{"author":"jasonjmcghee","children":[],"created_at":"2025-10-29T17:08:31.000Z","created_at_i":1761757711,"id":45749835,"options":[],"parent_id":45749769,"points":null,"story_id":45748661,"text":"Don't visit this address.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:05:01.000Z","created_at_i":1761757501,"id":45749769,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"According to downtector.com - both AWS and GCP are down as well. Interesting","title":null,"type":"comment","url":null},{"author":"bernardo786","children":[],"created_at":"2025-10-29T17:05:03.000Z","created_at_i":1761757503,"id":45749770,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"now aws down again?","title":null,"type":"comment","url":null},{"author":"pred8er","children":[],"created_at":"2025-10-29T17:05:21.000Z","created_at_i":1761757521,"id":45749778,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"things seem to be coming back up now","title":null,"type":"comment","url":null},{"author":"MangoCoffee","children":[{"author":"kube-system","children":[{"author":"Mr_Bees69","children":[],"created_at":"2025-10-29T17:13:47.000Z","created_at_i":1761758027,"id":45749905,"options":[],"parent_id":45749826,"points":null,"story_id":45748661,"text":"Yeah, but MyChart is down.","title":null,"type":"comment","url":null},{"author":"acedTrex","children":[],"created_at":"2025-10-29T17:16:12.000Z","created_at_i":1761758172,"id":45749946,"options":[],"parent_id":45749826,"points":null,"story_id":45748661,"text":"big, if true","title":null,"type":"comment","url":null},{"author":"kbelder","children":[],"created_at":"2025-10-29T18:32:15.000Z","created_at_i":1761762735,"id":45751092,"options":[],"parent_id":45749826,"points":null,"story_id":45748661,"text":"I need to drop AWS and start passing data through encrypted HN posts.","title":null,"type":"comment","url":null},{"author":"chasd00","children":[],"created_at":"2025-10-29T20:01:37.000Z","created_at_i":1761768097,"id":45752273,"options":[],"parent_id":45749826,"points":null,"story_id":45748661,"text":"When AWS was down we were talking about it here, now Azure is down and we're still talking about it here. Where does HN actually live?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:08:11.000Z","created_at_i":1761757691,"id":45749826,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"It still is very decentralized. We are discussing this via the internet right now.","title":null,"type":"comment","url":null},{"author":"alt227","children":[{"author":"jslaby","children":[],"created_at":"2025-10-29T17:55:42.000Z","created_at_i":1761760542,"id":45750570,"options":[],"parent_id":45749853,"points":null,"story_id":45748661,"text":"Not everyone has moved over, but I'm sure there have been thoughts or plans to.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:09:54.000Z","created_at_i":1761757794,"id":45749853,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"Thats the whole point, big players like AWS and MS can go down, but here we are still talking on the internet.
Decentralisation is winning it seems.","title":null,"type":"comment","url":null},{"author":"mrinterweb","children":[{"author":"gwbas1c","children":[{"author":"JoBrad","children":[],"created_at":"2025-10-29T17:53:55.000Z","created_at_i":1761760435,"id":45750549,"options":[],"parent_id":45749985,"points":null,"story_id":45748661,"text":"It\u2019s the Heisenberg cloud principal.","title":null,"type":"comment","url":null},{"author":"bossyTeacher","children":[{"author":"gwbas1c","children":[{"author":"bossyTeacher","children":[{"author":"gwbas1c","children":[{"author":"gwbas1c","children":[],"created_at":"2025-10-31T15:02:26.000Z","created_at_i":1761922946,"id":45772743,"options":[],"parent_id":45770721,"points":null,"story_id":45748661,"text":"AND: A lot of shops use tools like Terraform / tofu, which means they manage their environments with scripts instead of point-and-click.","title":null,"type":"comment","url":null}],"created_at":"2025-10-31T11:11:34.000Z","created_at_i":1761909094,"id":45770721,"options":[],"parent_id":45765935,"points":null,"story_id":45748661,"text":"You were able to manage everything through the command line the entire time.
Now I will admit I am more of a point-and-click person; but if I had to I could have figured out how to use the command line.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T22:03:56.000Z","created_at_i":1761861836,"id":45765935,"options":[],"parent_id":45750937,"points":null,"story_id":45748661,"text":"The portal was down for most of the day and accessing any resources from the portal once the portal was up was not possible.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:20:35.000Z","created_at_i":1761762035,"id":45750937,"options":[],"parent_id":45750629,"points":null,"story_id":45748661,"text":"> no way whatsoever to access most/all of your services
I work on a product hosted on Azure. That's not the case. Except for front door, everything else is running fine. (Front door is a reverse proxy for static web sites.)
The product itself (an iot stormwater management system) is running, but our customers just can't access the website. If they need to do something, they can go out to the sites or call us and we can "rub two sticks together" and bypass the website. (We could also bypass front door if someone twisted our arms.)
Most customers only look at the website a few times a year.
---
That being said, our biggest point of failure is a completely different iot vendor who you probably won't hear about on Hacker News when they, or their data networks, have downtime.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:58:24.000Z","created_at_i":1761760704,"id":45750629,"options":[],"parent_id":45749985,"points":null,"story_id":45748661,"text":"Not sure how the current situation is better. Being stranded with no way whatsoever to access most/all of your services sounds way more terrifying than regular issues limited to a couple of services at a time","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:18:02.000Z","created_at_i":1761758282,"id":45749985,"options":[],"parent_id":45749866,"points":null,"story_id":45748661,"text":"Ahh, but you forget what it used to be like. Sites used to go down all the time.
Now, they go down a lot less frequently, but when they do, it's more widespread.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:11:03.000Z","created_at_i":1761757863,"id":45749866,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"A lot of money and years of marketing the cloud as the responsible business decision led us here. Now that the cloud providers have vendor lock-in, few will leave, and customers will continue to wildly overpay for cloud services.","title":null,"type":"comment","url":null},{"author":"pphysch","children":[{"author":"SoKamil","children":[],"created_at":"2025-10-29T17:30:00.000Z","created_at_i":1761759000,"id":45750198,"options":[],"parent_id":45749871,"points":null,"story_id":45748661,"text":"Is there a theorem that models this behavior? Capital feels like a mass that attracts more mass the larger it becomes, like gravity.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:11:26.000Z","created_at_i":1761757886,"id":45749871,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"Consolidation is the inevitable outcome of free unregulated markets.
In our highly interconnected world, decentralization paradoxically requires a central authority to enforce decentralization by restricting M&A, cartels, etc.","title":null,"type":"comment","url":null},{"author":"nzach","children":[],"created_at":"2025-10-29T17:23:28.000Z","created_at_i":1761758608,"id":45750078,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"> How did we get here?
I think the response lies in the surrounding ecosystem.
If you have a company it's easier to scale your team if you use AWS (or any other established ecosystem). It's way easier to hire 10 engineers that are competent with AWS tools than it is to hire 10 engineers that are competent with the IBM tools.
And from the individuals perspective it also make sense to bet on larger platforms. If you want to increase your odds of getting a new job, learning the AWS tools gives you a better ROI than learning the IBM tools.","title":null,"type":"comment","url":null},{"author":"SecretDreams","children":[],"created_at":"2025-10-29T17:23:42.000Z","created_at_i":1761758622,"id":45750084,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"> How did we get here?
Stonks","title":null,"type":"comment","url":null},{"author":"AndrewKemendo","children":[],"created_at":"2025-10-29T17:38:03.000Z","created_at_i":1761759483,"id":45750317,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"A natural monopoly is a monopoly in an industry in which high infrastructure costs and other barriers to entry relative to the size of the market give the largest supplier in an industry, often the first supplier in a market, an overwhelming advantage over potential competitors. Specifically, an industry is a natural monopoly if a single firm can supply the entire market at a lower long-run average cost than if multiple firms were to operate within it. In that case, it is very probable that a company (monopoly) or a minimal number of companies (oligopoly) will form, providing all or most of the relevant products and/or services.
https://en.wikipedia.org/wiki/Natural_monopoly","title":null,"type":"comment","url":null},{"author":"anonymars","children":[{"author":"deathanatos","children":[],"created_at":"2025-10-29T21:51:45.000Z","created_at_i":1761774705,"id":45753561,"options":[],"parent_id":45750326,"points":null,"story_id":45748661,"text":"Maybe in a perfect world, or in a free market.
But the cloud compute market is basically centralized into 2.5 companies at this point. The point of paying companies like Azure here is that they've in theory centralized the knowledge and know-how of running multiple, distributed datacenters, so as to be resilient.
But that we keep seeing outages encompassing more than a failure domain, then it should be fair game for engineers / customers to ask "what am I paying for, again?"
Moreover, this seems to be a classic case of large barriers to entry (the huge capital costs associated with building out a datacenter) barring new entrants into the market, coupled with "nobody ever got fired for buying IBM" level thinking. Are outages like these truly factored into the napkin math that says externalizing this is worth it?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:38:20.000Z","created_at_i":1761759500,"id":45750326,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"Efficiency (aka cost) <---> Resiliency/redundancy
Pick your point on the scale","title":null,"type":"comment","url":null},{"author":"codethief","children":[],"created_at":"2025-10-29T17:50:06.000Z","created_at_i":1761760206,"id":45750500,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"Meredith Whittaker (of Signal) addressed your question the other day: https://mastodon.world/@Mer__edith/115445701583902092","title":null,"type":"comment","url":null},{"author":"deaux","children":[],"created_at":"2025-10-29T18:21:24.000Z","created_at_i":1761762084,"id":45750949,"options":[],"parent_id":45749788,"points":null,"story_id":45748661,"text":"From today [0].
> Big Tech lobbying is riding the EU\u2019s deregulation wave by spending more, hiring more, and pushing more, according to a new report by NGO\u2019s Corporate Europe Observatory and LobbyControl on Wednesday (29 October).
> Based on data from the EU\u2019s transparency register, the NGOs found that tech companies spend the most on lobbying of any sector, spending \u20ac151m a year on lobbying \u2014 a 33 percent increase from \u20ac113m in 2023.
Gee whizz, I really do wonder how they end up having all the power!
[0] https://news.ycombinator.com/item?id=45744973","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:06:07.000Z","created_at_i":1761757567,"id":45749788,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The Internet is supposed to be decentralized. The big three seem to have all the power now (Amazon, Microsoft, and Google) plus Cloudflare/Oracle.
How did we get here? Is it because of scale? Going to market in minutes by using someone else's computers instead of building out your own, like co-location or dedicated servers, like back in the day.","title":null,"type":"comment","url":null},{"author":"AdmiralAsshat","children":[{"author":"dboreham","children":[{"author":"alt227","children":[],"created_at":"2025-10-29T17:29:56.000Z","created_at_i":1761758996,"id":45750196,"options":[],"parent_id":45750104,"points":null,"story_id":45748661,"text":"I may or may not have been known to temporarily take a database down in the past to make a point to management about how unreliable some old software is.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:24:43.000Z","created_at_i":1761758683,"id":45750104,"options":[],"parent_id":45749790,"points":null,"story_id":45748661,"text":"This is funny but also possibly true because: business/MBA types see these outages as a way to prove how critical some services are, leading to investors deciding to load up on the vendor's stock.","title":null,"type":"comment","url":null},{"author":"Telemakhos","children":[{"author":"embedding-shape","children":[{"author":"terminalshort","children":[{"author":"cruffle_duffle","children":[],"created_at":"2025-10-29T18:08:03.000Z","created_at_i":1761761283,"id":45750754,"options":[],"parent_id":45750437,"points":null,"story_id":45748661,"text":"\u201cVoluntold\u201d","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:45:15.000Z","created_at_i":1761759915,"id":45750437,"options":[],"parent_id":45750313,"points":null,"story_id":45748661,"text":"PRISM wasn't voluntary. Also there are 3 levels here:
1. Mandatory
2. "Voluntary"
3. Voluntary
And I suspect that very little of what the NSA does falls into category 3. As Sen Chuck Schumer put it "you take on the intelligence community, they have six ways from Sunday at getting back at you"","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:37:53.000Z","created_at_i":1761759473,"id":45750313,"options":[],"parent_id":45750178,"points":null,"story_id":45748661,"text":"Do Microsoft still say "If the government has a broader voluntary national security program to gather customer data, we don't participate in it" today (which PRISM proved very false), or are they at least acknowledging they're participating in whatever NSA has deployed today?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:28:52.000Z","created_at_i":1761758932,"id":45750178,"options":[],"parent_id":45749790,"points":null,"story_id":45748661,"text":"Or, the NSA needed to upgrade their access at both.","title":null,"type":"comment","url":null},{"author":"jrochkind1","children":[],"created_at":"2025-10-29T17:30:08.000Z","created_at_i":1761759008,"id":45750200,"options":[],"parent_id":45749790,"points":null,"story_id":45748661,"text":"I was gonna say that obv AWS hacked em to even things up.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:06:10.000Z","created_at_i":1761757570,"id":45749790,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Some exec at Microsoft told the Azure guys to ape everything Amazon does and they took it literally.","title":null,"type":"comment","url":null},{"author":"almosthere","children":[{"author":"reaperducer","children":[],"created_at":"2025-10-29T17:56:28.000Z","created_at_i":1761760588,"id":45750587,"options":[],"parent_id":45749817,"points":null,"story_id":45748661,"text":"Reports of Azure and AWS down on the same day? Infrastructure terrorism?
> We have confirmed that an inadvertent configuration change as the trigger event for this issue.
Save the speculation for Reddit. HN is better than that.","title":null,"type":"comment","url":null},{"author":"12_throw_away","children":[],"created_at":"2025-10-29T18:20:32.000Z","created_at_i":1761762032,"id":45750935,"options":[],"parent_id":45749817,"points":null,"story_id":45748661,"text":"> Infrastructure terrorism?
Unless that's a euphemism for "vibe coding", no.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:07:46.000Z","created_at_i":1761757666,"id":45749817,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Reports of Azure and AWS down on the same day? Infrastructure terrorism?","title":null,"type":"comment","url":null},{"author":"vachina","children":[{"author":"0xbadcafebee","children":[],"created_at":"2025-10-29T17:22:25.000Z","created_at_i":1761758545,"id":45750048,"options":[],"parent_id":45749820,"points":null,"story_id":45748661,"text":"That specific subdomain has issues with propagation: https://dnschecker.org/#A/answers.microsoft.com (only four resolvers return records)
The root zone and www. do not: https://dnschecker.org/#A/microsoft.com (all resolvers return records)
And querying https://www.microsoft.com/ results in HTTP 200 on the root document, but the page elements return errors (a 504 on the .css/.js documents, a 404 on some fonts, Name Not Resolved on scripts.clarity.ms, Connection Timed Out on wcpstatic.microsoft.com and mem.gfx.ms). That many different kinds of errors is actually kind of impressive.
I'm gonna say this was a networking/routing issue. The CDN stayed up, but everything else non-CDN became unroutable, and different requests traveled through different paths/services, but each eventually hit the bad network path, and that's what created all the different responses. Could also have been a bad deploy or a service stopped running and there's different things trying to access that service in different ways, leading to the weird responses... but that wouldn't explain the failed DNS propagation.","title":null,"type":"comment","url":null},{"author":"Aperocky","children":[],"created_at":"2025-10-29T17:43:16.000Z","created_at_i":1761759796,"id":45750405,"options":[],"parent_id":45749820,"points":null,"story_id":45748661,"text":"wow, right after AWS suffered a similar thing.
I wonder if this is microsoft "learning" to "prevent" such an issue and instead triggered it...
"One often meets his destiny on the path he takes to avoid it" -- Master Oogway","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:07:55.000Z","created_at_i":1761757675,"id":45749820,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"microsoft.com and some subdomains (answers.microsoft.com) has no A and AAA records. They screwed up big time.
https://archive.is/Q4izZ","title":null,"type":"comment","url":null},{"author":"zelias","children":[],"created_at":"2025-10-29T17:09:20.000Z","created_at_i":1761757760,"id":45749846,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Anyone have betting odds on when Google will go down next? Are we looking at all 3 providers having outages in the span of 3 weeks?","title":null,"type":"comment","url":null},{"author":"pred8er","children":[],"created_at":"2025-10-29T17:10:09.000Z","created_at_i":1761757809,"id":45749855,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"looks like MS completed a failover and things are be recovering slowly","title":null,"type":"comment","url":null},{"author":"Mr_Bees69","children":[],"created_at":"2025-10-29T17:11:39.000Z","created_at_i":1761757899,"id":45749875,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"MS website seems to be up but really slow. Think xbox might still be down, Bing works for some reason tho!?","title":null,"type":"comment","url":null},{"author":"opengrass","children":[],"created_at":"2025-10-29T17:12:00.000Z","created_at_i":1761757920,"id":45749878,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Github Actions and Codespaces degraded.","title":null,"type":"comment","url":null},{"author":"tecleandor","children":[],"created_at":"2025-10-29T17:12:20.000Z","created_at_i":1761757940,"id":45749881,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"LinkedIn has been acting funny for an hour or so, and some pages in the learn.microsoft.com domain have been failing for me too...","title":null,"type":"comment","url":null},{"author":"philipallstar","children":[],"created_at":"2025-10-29T17:14:48.000Z","created_at_i":1761758088,"id":45749922,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Can't get to microsoft.com even.","title":null,"type":"comment","url":null},{"author":"gianpaj","children":[{"author":"robotnikman","children":[],"created_at":"2025-10-29T18:50:32.000Z","created_at_i":1761763832,"id":45751311,"options":[],"parent_id":45749957,"points":null,"story_id":45748661,"text":"Also cant do anything right now with the repo's we have in Azure Devops, how lovely...","title":null,"type":"comment","url":null},{"author":"loopduplicate","children":[],"created_at":"2025-10-29T20:19:50.000Z","created_at_i":1761769190,"id":45752507,"options":[],"parent_id":45749957,"points":null,"story_id":45748661,"text":"get vscodium then","title":null,"type":"comment","url":null},{"author":"progmetaldev","children":[],"created_at":"2025-10-29T23:39:56.000Z","created_at_i":1761781196,"id":45754567,"options":[],"parent_id":45749957,"points":null,"story_id":45748661,"text":"I have had intermittent issues with winget today. I use UniGetUI for a front-end, and anything tied to Microsoft has failed for me. Judging by the logs, it's mostly retrieving the listing of versions (I assume similar to what 'apt-get update' does, I'm fairly new to using winget for Windows package management).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:16:43.000Z","created_at_i":1761758203,"id":45749957,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Can't download VSCode :D
Error: visual-studio-code: Download failed on Cask 'visual-studio-code' with message: Download failed: https://update.code.visualstudio.com/1.105.1/darwin-arm64/st...","title":null,"type":"comment","url":null},{"author":"ApolloFortyNine","children":[{"author":"consp","children":[],"created_at":"2025-10-29T17:24:55.000Z","created_at_i":1761758695,"id":45750107,"options":[],"parent_id":45749961,"points":null,"story_id":45748661,"text":"> as having no issues on their status page
Unsubstantiated idea: So the support contract likely says there is a window between each reporting step and the status page is the last one and the one in the legal documents giving them several more hours before the clauses trigger.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:16:47.000Z","created_at_i":1761758207,"id":45749961,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"They admit in their update blurb azure front door is having issues but still report azure front door as having no issues on their status page.
And it's very clear from these updates that they're more focused on the portal than the product, their updates haven't even mentioned fixing it yet, just moving off of it, as if it's some third party service that's down.","title":null,"type":"comment","url":null},{"author":"empath75","children":[],"created_at":"2025-10-29T17:17:00.000Z","created_at_i":1761758220,"id":45749963,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Friend of mine at MSFT says it's a Sev-0 outage and they can't even get to the ticket tracking system.","title":null,"type":"comment","url":null},{"author":"jimmyl02","children":[{"author":"jonathanlydall","children":[],"created_at":"2025-10-29T17:34:25.000Z","created_at_i":1761759265,"id":45750266,"options":[],"parent_id":45750014,"points":null,"story_id":45748661,"text":"Our Azure hosted dotnet App Service is working fine, but our docs site served via Front Door went down. Can\u2019t access anything through the Portal.","title":null,"type":"comment","url":null},{"author":"layer8","children":[],"created_at":"2025-10-29T17:38:43.000Z","created_at_i":1761759523,"id":45750335,"options":[],"parent_id":45750014,"points":null,"story_id":45748661,"text":"Maybe they need a downtime tracker. ;)","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:20:00.000Z","created_at_i":1761758400,"id":45750014,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"pretty interesting how datadog's uptime tracker (https://updog.ai/) says all the sites are fully available.
if that's true then it's a sign that Azure's control / data plane separation is doing it's job! at least for now","title":null,"type":"comment","url":null},{"author":"syntaxing","children":[{"author":"monkaiju","children":[{"author":"1718627440","children":[],"created_at":"2025-10-29T18:26:34.000Z","created_at_i":1761762394,"id":45751013,"options":[],"parent_id":45750162,"points":null,"story_id":45748661,"text":"More visibility for the general person to see how brittle software is?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:27:41.000Z","created_at_i":1761758861,"id":45750162,"options":[],"parent_id":45750057,"points":null,"story_id":45748661,"text":"If true then what "utility" is there?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:22:46.000Z","created_at_i":1761758566,"id":45750057,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I absolutely love the utility aspect of LLMs but part of me is curious if moving faster by using AI is going to make these sorts of failure more and more often.","title":null,"type":"comment","url":null},{"author":"givemeethekeys","children":[],"created_at":"2025-10-29T17:23:54.000Z","created_at_i":1761758634,"id":45750088,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Surely more vibecoding will fix this problem. Time to fire more staff","title":null,"type":"comment","url":null},{"author":"agency","children":[{"author":"kurttheviking","children":[],"created_at":"2025-10-29T17:56:00.000Z","created_at_i":1761760560,"id":45750576,"options":[],"parent_id":45750140,"points":null,"story_id":45748661,"text":"I am unable to load this article...presumably for related reasons","title":null,"type":"comment","url":null},{"author":"Shuddown","children":[{"author":"sodafountan","children":[],"created_at":"2025-10-29T21:18:52.000Z","created_at_i":1761772732,"id":45753177,"options":[],"parent_id":45751993,"points":null,"story_id":45748661,"text":"I was wondering the same thing","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:39:54.000Z","created_at_i":1761766794,"id":45751993,"options":[],"parent_id":45750140,"points":null,"story_id":45748661,"text":"Pretty much every single Microsoft domain I've tried to access loads for a looooong time before giving me some bare html. I wonder if someone can explain why that's happening.","title":null,"type":"comment","url":null},{"author":"MangoCoffee","children":[],"created_at":"2025-10-29T19:53:30.000Z","created_at_i":1761767610,"id":45752163,"options":[],"parent_id":45750140,"points":null,"story_id":45748661,"text":""BREAKING: Alaska Airlines' website, app impacted amid Microsoft Azure outage"
https://www.youtube.com/watch?v=YJVkLP57yvM","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:26:17.000Z","created_at_i":1761758777,"id":45750140,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"So that's why I can't check in for my Alaska Airlines flight... https://news.microsoft.com/source/features/digital-transform...","title":null,"type":"comment","url":null},{"author":"okokwhatever","children":[],"created_at":"2025-10-29T17:26:33.000Z","created_at_i":1761758793,"id":45750143,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"This cannot be a coincidence","title":null,"type":"comment","url":null},{"author":"hypeatei","children":[{"author":"randomsofr","children":[{"author":"hypeatei","children":[],"created_at":"2025-10-29T17:31:48.000Z","created_at_i":1761759108,"id":45750226,"options":[],"parent_id":45750194,"points":null,"story_id":45748661,"text":"The portal is up for me and their status page confirms they did a failover for it. Definitely not disputing that its reach is wide, but a lot of smaller setups probably aren't using Front Door.","title":null,"type":"comment","url":null},{"author":"rahkiin","children":[],"created_at":"2025-10-29T17:33:21.000Z","created_at_i":1761759201,"id":45750256,"options":[],"parent_id":45750194,"points":null,"story_id":45748661,"text":"Both work for me in the Netherlands","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:29:42.000Z","created_at_i":1761758982,"id":45750194,"options":[],"parent_id":45750148,"points":null,"story_id":45748661,"text":"SSO is down, Azure Portal Down and more, seems like a major outage. Already a lot of services seem to be affected: banks, airlines, consumer apps, etc.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:26:44.000Z","created_at_i":1761758804,"id":45750148,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"All of my employers things are hosted on Azure and running just fine and didn't go down at all. Portal access has been fixed.
Doesn't seem to be too bad of an outage unless you were relying on Azure Front Door.","title":null,"type":"comment","url":null},{"author":"gmassman","children":[{"author":"alt227","children":[],"created_at":"2025-10-29T17:56:30.000Z","created_at_i":1761760590,"id":45750589,"options":[],"parent_id":45750150,"points":null,"story_id":45748661,"text":"All the clouds hav had major outages this year.
At this point I dont believe that any one of them is any better or reliable than the others.","title":null,"type":"comment","url":null},{"author":"btmiller","children":[],"created_at":"2025-10-29T21:31:38.000Z","created_at_i":1761773498,"id":45753325,"options":[],"parent_id":45750150,"points":null,"story_id":45748661,"text":"Never let a good disaster go to waste ;)","title":null,"type":"comment","url":null},{"author":"not_a_bot_4sho","children":[],"created_at":"2025-10-30T06:25:33.000Z","created_at_i":1761805533,"id":45756961,"options":[],"parent_id":45750150,"points":null,"story_id":45748661,"text":"> I feel pretty justified in my previous decisions to move away from Azure
I felt this way about AWS last week","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:26:46.000Z","created_at_i":1761758806,"id":45750150,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I\u2019ve been migrating our services off of Azure slowly for the past couple of years. The last internet facing things remaining are a static assets bucket and an analytics VM running Matomo. Working with Front Door has been an abysmal experience, and today was the push I needed to finally migrate our assets to Cloudflare.
I feel pretty justified in my previous decisions to move away from Azure. Using it feels like building on quicksand\u2026","title":null,"type":"comment","url":null},{"author":"amluto","children":[],"created_at":"2025-10-29T17:29:03.000Z","created_at_i":1761758943,"id":45750183,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"vscode.dev appears to be down. I think this will be my excuse to find an alternative -- I never really liked vscode.dev anyway.
(Coder is currently at the top of the experiment list. Any other suggestions?)","title":null,"type":"comment","url":null},{"author":"wingless_angel","children":[],"created_at":"2025-10-29T17:29:09.000Z","created_at_i":1761758949,"id":45750186,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Please sort it out, I'll be out of a job tomorrow.","title":null,"type":"comment","url":null},{"author":"I_am_tiberius","children":[],"created_at":"2025-10-29T17:30:38.000Z","created_at_i":1761759038,"id":45750209,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Shouldn't regions be completely independent?","title":null,"type":"comment","url":null},{"author":"glzone1","children":[{"author":"mikestew","children":[],"created_at":"2025-10-29T19:33:42.000Z","created_at_i":1761766422,"id":45751892,"options":[],"parent_id":45750217,"points":null,"story_id":45748661,"text":"Could be DNS, I'm seeing SERVFAIL trying to resolve what look to be MS servers when I'm hitting (just one example) mygoodtogo.com (trying to pay a road toll bill, and failing).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:31:23.000Z","created_at_i":1761759083,"id":45750217,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Wasn't the saying "It's always DNS" floating around somewhere?
Be interesting to understand cause here. Pretty big impact on services we use","title":null,"type":"comment","url":null},{"author":"glzone1","children":[{"author":"alt227","children":[],"created_at":"2025-10-29T19:43:25.000Z","created_at_i":1761767005,"id":45752042,"options":[],"parent_id":45750236,"points":null,"story_id":45748661,"text":"That saying is just as alive today as it ever was.
https://isitdns.com/","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:32:05.000Z","created_at_i":1761759125,"id":45750236,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I remember the saying "It's always DNS". I'm old.
Kind of mindboggling it's still sometimes DNS maybe.","title":null,"type":"comment","url":null},{"author":"0000000000100","children":[{"author":"alt227","children":[],"created_at":"2025-10-29T17:53:04.000Z","created_at_i":1761760384,"id":45750535,"options":[],"parent_id":45750309,"points":null,"story_id":45748661,"text":"... and DNS caching, and browser file cache, and sessions...
Moving a website quickly is never fun.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:37:26.000Z","created_at_i":1761759446,"id":45750309,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Yeah just took down the prod site for one of our clients since we host the front-end out of their CDN. Just got wrapped up panic hosting it somewhere else for the past hour, very quickly reminds you about the pain of cookies...","title":null,"type":"comment","url":null},{"author":"anon025","children":[{"author":"I_am_tiberius","children":[],"created_at":"2025-10-29T17:56:22.000Z","created_at_i":1761760582,"id":45750583,"options":[],"parent_id":45750357,"points":null,"story_id":45748661,"text":"Why are Azure App Services still working?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:40:51.000Z","created_at_i":1761759651,"id":45750357,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"It's the DNS https://dnschecker.org/#A/get.helm.sh is unreachable","title":null,"type":"comment","url":null},{"author":"howard941","children":[],"created_at":"2025-10-29T17:40:57.000Z","created_at_i":1761759657,"id":45750360,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Took out the archive.ph and .is sites too?","title":null,"type":"comment","url":null},{"author":"LouisLazaris","children":[{"author":"codethief","children":[],"created_at":"2025-10-29T17:57:16.000Z","created_at_i":1761760636,"id":45750613,"options":[],"parent_id":45750367,"points":null,"story_id":45748661,"text":"https://www.microsoft.com works for me (with the www subdomain).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:41:11.000Z","created_at_i":1761759671,"id":45750367,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The VS Code website is down: https://code.visualstudio.com/
And so is Microsoft: http://www.microsoft.com/","title":null,"type":"comment","url":null},{"author":"bossyTeacher","children":[{"author":"Ylpertnodi","children":[],"created_at":"2025-10-29T18:49:46.000Z","created_at_i":1761763786,"id":45751299,"options":[],"parent_id":45750401,"points":null,"story_id":45748661,"text":">How can one of the richest companies in the world not offer a better service?
Better service costs money.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T17:42:55.000Z","created_at_i":1761759775,"id":45750401,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I noticed issues on Azure so I went to the status page. It said everything was fine even though the Azure Portal was down. It took more than 10 minutes for that status page to update.
How can one of the richest companies in the world not offer a better service?","title":null,"type":"comment","url":null},{"author":"everfrustrated","children":[],"created_at":"2025-10-29T17:43:01.000Z","created_at_i":1761759781,"id":45750403,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"GitHub runners (specifically the "larger" runner types) are all down for us. These are known to be hosted on Azure.","title":null,"type":"comment","url":null},{"author":"pred8er","children":[],"created_at":"2025-10-29T17:44:07.000Z","created_at_i":1761759847,"id":45750415,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"on the line with msft, they said 4 hours is what they are thinking. a workaround they are saying is to use traffic manager,","title":null,"type":"comment","url":null},{"author":"zzake","children":[],"created_at":"2025-10-29T17:46:52.000Z","created_at_i":1761760012,"id":45750455,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Portal is now accessible, bypassing FDN","title":null,"type":"comment","url":null},{"author":"ApolloFortyNine","children":[],"created_at":"2025-10-29T17:58:05.000Z","created_at_i":1761760685,"id":45750625,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Two hours after the initial outage, they have finally updated the Front Door status on their status page.","title":null,"type":"comment","url":null},{"author":"port11","children":[],"created_at":"2025-10-29T18:00:25.000Z","created_at_i":1761760825,"id":45750652,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"So much of Belgium runs on Azure\u2026 it's honestly baffling how many services are down, there's no resilience built into (even large) companies anymore.","title":null,"type":"comment","url":null},{"author":"move-on-by","children":[],"created_at":"2025-10-29T18:07:12.000Z","created_at_i":1761761232,"id":45750741,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Instead of cyber security awareness month, we should rename it to cloud availability awareness month.","title":null,"type":"comment","url":null},{"author":"martijnvds","children":[],"created_at":"2025-10-29T18:07:41.000Z","created_at_i":1761761261,"id":45750749,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"This probably explains why paying for street parking in Cologne by phone/web didn't work (eternal spinner) then","title":null,"type":"comment","url":null},{"author":"ukblewis","children":[],"created_at":"2025-10-29T18:08:41.000Z","created_at_i":1761761321,"id":45750764,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"GitHub also seems to be having trouble for me","title":null,"type":"comment","url":null},{"author":"jacquesclouseau","children":[{"author":"croemer","children":[],"created_at":"2025-10-29T18:27:55.000Z","created_at_i":1761762475,"id":45751027,"options":[],"parent_id":45750778,"points":null,"story_id":45748661,"text":"They already announced that.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:09:23.000Z","created_at_i":1761761363,"id":45750778,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"My bet is on a bad config change.","title":null,"type":"comment","url":null},{"author":"hedayet","children":[{"author":"navane","children":[{"author":"marcosdumay","children":[{"author":"alt227","children":[],"created_at":"2025-10-29T19:41:49.000Z","created_at_i":1761766909,"id":45752025,"options":[],"parent_id":45751600,"points":null,"story_id":45748661,"text":"I think he was implying that those companies think they are so important that it doesnt matter they are down, they wont loose any customers over it because they are too big and important.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:12:30.000Z","created_at_i":1761765150,"id":45751600,"options":[],"parent_id":45750926,"points":null,"story_id":45748661,"text":"What do you mean? That IT isn't important for Microsoft and Amazon?
That's certainly not the right conclusion.","title":null,"type":"comment","url":null},{"author":"cyberax","children":[],"created_at":"2025-10-29T19:14:21.000Z","created_at_i":1761765261,"id":45751616,"options":[],"parent_id":45750926,"points":null,"story_id":45748661,"text":"So we can look forward to "accidental" cloud outages just to show their importance?
I guess the GCP is next.","title":null,"type":"comment","url":null},{"author":"Arrath","children":[],"created_at":"2025-10-29T19:53:06.000Z","created_at_i":1761767586,"id":45752158,"options":[],"parent_id":45750926,"points":null,"story_id":45748661,"text":""They'll learn their lesson and be rock solid after this! I better invest now!"","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:19:40.000Z","created_at_i":1761761980,"id":45750926,"options":[],"parent_id":45750795,"points":null,"story_id":45748661,"text":"Look how important we are, is what these failures show","title":null,"type":"comment","url":null},{"author":"locusofself","children":[{"author":"jlarocco","children":[],"created_at":"2025-10-29T19:18:29.000Z","created_at_i":1761765509,"id":45751671,"options":[],"parent_id":45751078,"points":null,"story_id":45748661,"text":"Because it shows how much lock-in they have.
You know nobody is migrating off of AWS or Azure because of these.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:31:06.000Z","created_at_i":1761762666,"id":45751078,"options":[],"parent_id":45750795,"points":null,"story_id":45748661,"text":"AMZN went up almost 4 percent between the day of the outage and the day after. Crazy market.","title":null,"type":"comment","url":null},{"author":"AtNightWeCode","children":[],"created_at":"2025-10-29T18:52:18.000Z","created_at_i":1761763938,"id":45751330,"options":[],"parent_id":45750795,"points":null,"story_id":45748661,"text":"I looked into this before and the stocks of these large corps simply does not move when outages happens. Maybe intra-day, I don't have that data, but in general no effect.","title":null,"type":"comment","url":null},{"author":"iamtheworstdev","children":[],"created_at":"2025-10-29T19:49:38.000Z","created_at_i":1761767378,"id":45752109,"options":[],"parent_id":45750795,"points":null,"story_id":45748661,"text":"well, at this point, 90% of the market cap of FAANGS plus Microsoft is... OMG AI LLM hype","title":null,"type":"comment","url":null},{"author":"bigstrat2003","children":[],"created_at":"2025-10-29T23:00:01.000Z","created_at_i":1761778801,"id":45754211,"options":[],"parent_id":45750795,"points":null,"story_id":45748661,"text":"That's a good thing. Stock prices shouldn't go down because of rare incidents which don't accurately represent how successful a company is likely to be in the future.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:11:03.000Z","created_at_i":1761761463,"id":45750795,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The sad thing is - $MSFT isn't even down by 1%. And IIRC, $AMZN actually went up during their previous outage.
So if we look at these companies' bottom lines, all those big wigs are actually doing something right. Sales and lobbying capacity is way more effective than reliability or good engineering (at least in the short term).","title":null,"type":"comment","url":null},{"author":"rsolva","children":[],"created_at":"2025-10-29T18:11:33.000Z","created_at_i":1761761493,"id":45750805,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"So that's why all of our municipality's digital services are down ... utter chaos at the political meeting I attended just now.","title":null,"type":"comment","url":null},{"author":"irusensei","children":[],"created_at":"2025-10-29T18:17:30.000Z","created_at_i":1761761850,"id":45750892,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I was working when I saw the portal page showing only resource groups and lots of items missing. I thought it was a weird browser cache issue.
The actual stuff I was working on (App Insights, Function App) that was still open was operational.","title":null,"type":"comment","url":null},{"author":"AtNightWeCode","children":[],"created_at":"2025-10-29T18:18:04.000Z","created_at_i":1761761884,"id":45750904,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Earnings report today. A coincidence?
I can at least login to Azure. But several MS sites are down.","title":null,"type":"comment","url":null},{"author":"tonyhart7","children":[],"created_at":"2025-10-29T18:18:48.000Z","created_at_i":1761761928,"id":45750916,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Wtf happen with US east????","title":null,"type":"comment","url":null},{"author":"bob1029","children":[{"author":"redwood","children":[],"created_at":"2025-10-29T18:30:26.000Z","created_at_i":1761762626,"id":45751063,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"I read "Microsoft shop" as "Microsoft slop". Fitting. But at least they open source wash themselves so much they're practically a charity right?","title":null,"type":"comment","url":null},{"author":"xmcp123","children":[{"author":"WorldMaker","children":[{"author":"Insanity","children":[{"author":"high_priest","children":[],"created_at":"2025-10-29T19:01:15.000Z","created_at_i":1761764475,"id":45751459,"options":[],"parent_id":45751396,"points":null,"story_id":45748661,"text":"Imagine OG Xbox menus, or the PS3/PSP menus.","title":null,"type":"comment","url":null},{"author":"WorldMaker","children":[],"created_at":"2025-10-29T20:12:16.000Z","created_at_i":1761768736,"id":45752418,"options":[],"parent_id":45751396,"points":null,"story_id":45748661,"text":"Think Niri [0], but worse, embedded in a web browser tab, and without keyboard navigation.
Here's a somewhat ancient Stack Overflow screenshot I found: https://i.sstatic.net/yCseI.png
(I think that's from near the transition because it has full "windowing" controls of minimize/maximize/close buttons. I recall a period with only close buttons.)
All that blue space you could keep filling with more "blades" as you clicked on things until the entire page started scrolling horizontally to switch between "blades". Almost everything you could click opened in a new blade rather than in place in the existing blade. (Like having "Open in New Window" as your browser default.)
It was trying to merge the needs of a configurable Dashboard and a "multi-window experience". You could save collections of blades (a bit like Niri workspaces) as named Dashboards. Overall it was somewhere between overkill and underthought.
(Also someone reminded me that many "blades" still somewhat exist in the modern Portal, because, of course, Microsoft backwards compatibility. Some of the pages are just "maximized Blades" and you can accidentally unmaximize them and start horizontally scrolling into new blades.)
[0] https://github.com/YaLTeR/niri","title":null,"type":"comment","url":null},{"author":"csydas","children":[],"created_at":"2025-10-29T20:13:54.000Z","created_at_i":1761768834,"id":45752438,"options":[],"parent_id":45751396,"points":null,"story_id":45748661,"text":"azure likes to open new sections on the same tab / page as opposed to reloading or opening a new page / tab (overlays? modals? I'm lost on graphic terms)
depending on the resource you're accessing, you can get 5+ sections each with their own ui/ux on the same page/tab and it can be confusing to understand where you're at in your resources
if you're having trouble visualizing it, imagine an url where each new level is a different application with its own ui/ux and purpose all on the same webpage","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:57:01.000Z","created_at_i":1761764221,"id":45751396,"options":[],"parent_id":45751207,"points":null,"story_id":45748661,"text":"Not sure what to imagine with this given I didn't use Azure at the time. Is this like the Windows XP style task menu?","title":null,"type":"comment","url":null},{"author":"btown","children":[{"author":"xnorswap","children":[{"author":"WorldMaker","children":[{"author":"btown","children":[{"author":"WorldMaker","children":[],"created_at":"2025-10-30T19:05:21.000Z","created_at_i":1761851121,"id":45763964,"options":[],"parent_id":45762954,"points":null,"story_id":45748661,"text":"> By bringing those eyeballs onto your cloud console, you're creating infinitely more opportunities for branded interaction and discovery of your other cloud products - you could even quantify these eyeballs as you would ad inventory! There should have been an arms race for each cloud provider to have the best log-tailing and log-searching and log-aggregation system imaginable. OTel could have been killed before it began, because Honeycomb and its other originators would have been acquired years ago and made specific locked-in value-adds for each cloud.
I think that's what Application Insights has always been, Azure's free-to-start, suggest-out-of-the-box Honeycomb. App Insights had a long slow road away from Microsoft-specific log and metrics ingesters that weren't OTel, but it is hard to argue that standard ingestors are a bad idea. App Insights still downplays that it can be "just a Honeycomb" using only OTel sources and still encourages "secret sauce" ingestors in addition to OTel ones. App Insights is a small moat (around a data lake; to mix metaphors). That said, it's also a standards-supporting tool now as well.
It's not been as clear of an arms race because AWS and GCP didn't invest in it in a similar way and it mostly impacted what are often called "dark matter" teams (Microsoft shops doing "boring" stuff that rarely makes HN headlines), but I have worked in teams that absolutely favored Azure over AWS/GCP with one of the reasons being Application Insights was an easy install and powerful first-party supported tool rather than an extra third party vendor relationship like Grafana/Honeycomb/Dynatrace/etc.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T17:55:19.000Z","created_at_i":1761846919,"id":45762954,"options":[],"parent_id":45761409,"points":null,"story_id":45748661,"text":"I love that OTel exists, but I've always been confused why OTel even existed. If I were running a cloud provider, I'd put a "skunkworks" team on all things observability and log management a decade ago, and make their product free to use. It's not critical-path on infrastructure, so it's not a correlated risk other than the investment in manpower, but it dramatically changes how user-friendly your web interface is, and how likely people are to use it daily vs. managing things over command line or with third-party interfaces.
By bringing those eyeballs onto your cloud console, you're creating infinitely more opportunities for branded interaction and discovery of your other cloud products - you could even quantify these eyeballs as you would ad inventory! There should have been an arms race for each cloud provider to have the best log-tailing and log-searching and log-aggregation system imaginable. OTel could have been killed before it began, because Honeycomb and its other originators would have been acquired years ago and made specific locked-in value-adds for each cloud.
But nobody had this foresight, and thus comments like yours are absolutely correct. OTel is a blessing and I love the tools coming out. But from a cloud provider's perspective, it's a massive missed opportunity that continues to be missed.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T15:52:00.000Z","created_at_i":1761839520,"id":45761409,"options":[],"parent_id":45757337,"points":null,"story_id":45748661,"text":"A lot of Azure Insights is a value add on top of OTel. You can use "just" OTel feeding Azure Insights (and that's what the modern Aspire-influenced defaults mostly do) and possibly avoid that feeling of vendor lockin. That perspective might also give you ideas of other "modern" vendors to audition such as Grafana if you wanted to see what other people are doing with OTel rather than Event Logs and file system logs.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T07:36:29.000Z","created_at_i":1761809789,"id":45757337,"options":[],"parent_id":45751628,"points":null,"story_id":45748661,"text":"I still feel lost just trying to view my application logs.
I don't want to pay for or lock myself into, "Azure Insights".
I just want to see the logging, that I know if I can remember the right buttons to click, are available.
The worst place to try is "Monitoring > Logs", this is where you get faced up front with a query designer. I've never worked out how to do a simple "list by time" on that query designer, but it doesn't matter, because if you suffer through that UX, you find out that's not actually where the logs are anyway.
You have to go down a different path. Don't be distracted by "Log Stream", that's not it either, it sounds useful but it's not. By default it doesn't log anything. If you do configure it to log, then it still doesn't actually log everything.
What you have to actually do, and I've had to open the portal to check this, is click "Diagnose and Solve Problems" and then look for "Diagnostic tools" and then a small link to "Application Event Logs".
Finally you get to your logs, although it's still a bad way to try to view logs, it's at least marginally better than the real windows event viewer, an application that feels like it hasn't been updated since NT4. ( Although some might suggest that's a good thing. )","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:15:55.000Z","created_at_i":1761765355,"id":45751628,"options":[],"parent_id":45751207,"points":null,"story_id":45748661,"text":"Azure to me has always suffered from a belief that \u201cUI innovations can solve UX complexity if you just try hard enough.\u201d
Like, AWS, and GCP to a lesser extent, has a principled approach where simple click-ops goals are simple. You can access the richer metadata/IAM object model at any time, but the wizards you see are dumb enough to make easy things easy.
With Azure, those blades allow tremendously complex \u201cyou need to build an X Container and a Container Bucket to be able to add an X\u201d flows to coexist on the same page. While this exposes the true complexity, and looks cool/works well for power users, it is exceedingly unintuitive. Inline documentation doesn\u2019t solve this problem.
I sometimes wonder if this is by design: like QuickBooks, there\u2019s an entire economy of consultants who need to be Certified and thus will promote your product for their own benefit! Making the interface friendly to them and daunting to mere mortals is a feature, not a bug.
But in Azure\u2019s case it\u2019s hard to tell how much this is intentional.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:40:48.000Z","created_at_i":1761763248,"id":45751207,"options":[],"parent_id":45751077,"points":null,"story_id":45748661,"text":"The "Blades" experience [0] where instead of navigating between pages it just kept opening things to the side and expanding horizontally?
Yeah, that had some fun ideas but was way more confusing than it needed to be. But also that was quite a few years back now. The Portal ditched that experience relatively quickly. Just long enough to leave a lot of awful first impressions, but not long enough for it to be much more than a distant memory at this point, several redesigns later.
[0] The name "Blades" for that came from the early years of the Xbox 360, maybe not the best UX to emulate for a complex control panel/portal.","title":null,"type":"comment","url":null},{"author":"nazgulsenpai","children":[],"created_at":"2025-10-29T18:59:00.000Z","created_at_i":1761764340,"id":45751425,"options":[],"parent_id":45751077,"points":null,"story_id":45748661,"text":"Count your blessings. You could have to use Azure SSO through Oracle Cloud..... ; ;","title":null,"type":"comment","url":null},{"author":"lovich","children":[],"created_at":"2025-10-29T19:14:07.000Z","created_at_i":1761765247,"id":45751614,"options":[],"parent_id":45751077,"points":null,"story_id":45748661,"text":"I found it intuitive but admittedly it felt a lot like their Xbox UI which I used a lot during my formative years","title":null,"type":"comment","url":null},{"author":"jeffrallen","children":[],"created_at":"2025-10-29T19:20:03.000Z","created_at_i":1761765603,"id":45751690,"options":[],"parent_id":45751077,"points":null,"story_id":45748661,"text":"Amazon: here's two buttons, some check boxes and a random popup.
MSFT : Hold my beer...","title":null,"type":"comment","url":null},{"author":"foresterre","children":[{"author":"brap","children":[{"author":"conductr","children":[],"created_at":"2025-10-29T20:03:24.000Z","created_at_i":1761768204,"id":45752303,"options":[],"parent_id":45751846,"points":null,"story_id":45748661,"text":"I still find it much easier to just self host than learn cloud and I\u2019ve tried a few times but it just seems overly complex for the sake of complexity. It seems they tie in all their services to jack up charges, eg. I came for S3 but now I\u2019m paying for 5 other things just to get it working.
Any time something is that unintuitive to get started, I automatically assume that if I encounter a problem that I\u2019ll be unable to solve it. That thought alone leads me to bounce every time.","title":null,"type":"comment","url":null},{"author":"reddalo","children":[],"created_at":"2025-10-29T20:47:31.000Z","created_at_i":1761770851,"id":45752804,"options":[],"parent_id":45751846,"points":null,"story_id":45748661,"text":"100% agree. I've been working in the industry for almost 20 years, I'm a full stack developer and I manage my servers. I've tried signing up for AWS and I noped out.
AWS Is a complete mess. Everything is obscured behind other products, and they're all named in the most confusing way possible.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:30:48.000Z","created_at_i":1761766248,"id":45751846,"options":[],"parent_id":45751717,"points":null,"story_id":45748661,"text":"It\u2019s more than just the UI itself (which is horrible), it\u2019s the whole thing that is very hostile to new users even if they\u2019re experienced. It\u2019s such an incoherent mess. The UI, the product names, the entire product line itself, with seemingly overlapping or competing products\u2026 and now it\u2019s AI this and AI that. If you don\u2019t know exactly what you\u2019re looking for, good luck finding it. It\u2019s like they\u2019re deliberately trying to make things as confusing as possible.
For some reason this applies to all AWS, GCP and Azure. Seems like the result of dozens of acquisitions.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:21:52.000Z","created_at_i":1761765712,"id":45751717,"options":[],"parent_id":45751077,"points":null,"story_id":45748661,"text":"AWS' UI is similarly messy, and to this day. They regularly remove useful data from the UI, or change stuff like the default sort order of database snapshots from last created to initial instance created date.
I never understood why a clear and consistent UI and improved UX isn't more of a priority for the big three cloud providers. Even though you talk mostly via platform SDK's, I would consider better UI especially initially, a good way to bind new customers and pick your platform over others.
I guess with their bottom line they don't need it (or cynically, you don't want to learn and invest in another cloud if you did it once).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:31:05.000Z","created_at_i":1761762665,"id":45751077,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"Many years back was the first time I used Azure, evaluating it for a client.
I remember I at one point had expanded enough menus that it covered the entirety of the screen.
Never before have I felt so lost in a cloud product.","title":null,"type":"comment","url":null},{"author":"aftbit","children":[{"author":"bob1029","children":[{"author":"thewebguyd","children":[],"created_at":"2025-10-29T19:08:13.000Z","created_at_i":1761764893,"id":45751551,"options":[],"parent_id":45751487,"points":null,"story_id":45748661,"text":"I've found that lift and shifting to EC2 is also generally cheaper than the equivalent VMs on Azure.
Microsoft really wants you to use their PaaS offerings, and so things on Azure are priced accordingly. A Microsoft shop just wanting to lift-and-shift, Azure isn't the best choice unless the org has that "nobody ever got fired for buying Microsoft" attitude.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:03:26.000Z","created_at_i":1761764606,"id":45751487,"options":[],"parent_id":45751161,"points":null,"story_id":45748661,"text":"I moved a 100% MS shop to AWS circa 2015. We ran our DCs on EC2 instances just as if they were on prem. At some point we installed the AAD connector and bridged some stuff to Azure for office/mail/etc., but it was all effectively in AWS. We were selling software to banks so we had a lot of due diligence to suffer. AWS Artifact did much of the heavy lifting for us. We started with Amazon's compliance documentation and provided our own feedback on top where needed.
I feel like compliance is the entire point of using these cloud providers. You get a huge head start. Maintaining something like PCI-DSS when you own the real estate is a much bigger headache than if it's hosted in a provider who is already compliant up through the physical/hardware/networking layers. Getting application-layer checkboxes ticked off is trivial compared to "oops we forgot to hire an armed security team". I just took a look and there are currently 316 certifications and attestations listed under my account.
https://aws.amazon.com/artifact/faq/","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:37:23.000Z","created_at_i":1761763043,"id":45751161,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"The problem is that in some industries, Microsoft is the only option. Many of these regulated industries are just now transitioning from the data center to the cloud, and they've barely managed to get approval for that with all of the Microsoft history in their organization. AWS or GCloud are complete non-starters.","title":null,"type":"comment","url":null},{"author":"arccy","children":[{"author":"foresterre","children":[{"author":"lebski88","children":[],"created_at":"2025-10-29T19:19:38.000Z","created_at_i":1761765578,"id":45751680,"options":[],"parent_id":45751475,"points":null,"story_id":45748661,"text":"At least some bits of it do. I was writing something to pull logs the other day and the redirect was to an azure bucket. It also returned a 401 with the valid temporary authed redirect in the header. I was a bit worried I'd found a massive security hole but it appears after some testing it just returned the wrong status code.","title":null,"type":"comment","url":null},{"author":"OptionOfT","children":[],"created_at":"2025-10-29T19:40:46.000Z","created_at_i":1761766846,"id":45752008,"options":[],"parent_id":45751475,"points":null,"story_id":45748661,"text":"https://www.githubstatus.com/incidents/4jxdz4m769gy
> They're stating they're working with the Azure teams, so I suspect this is related.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:02:39.000Z","created_at_i":1761764559,"id":45751475,"options":[],"parent_id":45751337,"points":null,"story_id":45748661,"text":"GitHub doesn't use Azure yet, but has just published their migration path to azure a few days ago.
I would link to that article, but that one does seem down ;)","title":null,"type":"comment","url":null},{"author":"abhinavk","children":[],"created_at":"2025-10-30T08:03:28.000Z","created_at_i":1761811408,"id":45757474,"options":[],"parent_id":45751337,"points":null,"story_id":45748661,"text":"I only noticed it because VSCode's updater failed.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:52:35.000Z","created_at_i":1761763955,"id":45751337,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"The only reason you'd notice MS was down was if Github was down....","title":null,"type":"comment","url":null},{"author":"otterdude","children":[{"author":"WD-42","children":[],"created_at":"2025-10-29T19:17:06.000Z","created_at_i":1761765426,"id":45751648,"options":[],"parent_id":45751454,"points":null,"story_id":45748661,"text":"Preach","title":null,"type":"comment","url":null},{"author":"lazystar","children":[{"author":"otterdude","children":[],"created_at":"2025-10-30T11:30:10.000Z","created_at_i":1761823810,"id":45758836,"options":[],"parent_id":45751786,"points":null,"story_id":45748661,"text":"All I'm saying Its a complete 360 from the marketing they did to establish their brand... you know, save the world kind of stuff, especially from google.
From 2000-2016 most tech marketing\\branding was aimed at some kind of social benefit.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:26:31.000Z","created_at_i":1761765991,"id":45751786,"options":[],"parent_id":45751454,"points":null,"story_id":45748661,"text":"> they do not care about anything other than the bottom line.
if all companies focused on fixing each and every social issue that exists in the world, how would they make any money?","title":null,"type":"comment","url":null},{"author":"amnesty6249","children":[],"created_at":"2025-10-29T19:51:55.000Z","created_at_i":1761767515,"id":45752142,"options":[],"parent_id":45751454,"points":null,"story_id":45748661,"text":"These were my thoughts exactly. I may have my tinfoil hat on, but outages these close together between the largest cloud providers amid social unrest, my wonder is the government / tech companies implementing some update that adds additional spyware / blackout functionality.
I really hope this pushes the internet back to how it used to be, self hosted, privacy, anonymity. I truly hope that's where we're headed, but the masses seem to just want to stay comfortable as long as their show is on TV","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:00:59.000Z","created_at_i":1761764459,"id":45751454,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"What Amazon, Azure, and Google are showing with their platform crashes amid layoffs, while they supports governments that are Oppressing's Citizens and Ignoring the Law, is that they do not care about anything other than the bottom line.
They think they have the market captured, but I think what their dwindling quality and ethics are really going to drive is adoption of self hosting, distributed computing frameworks. Nerds are the ones who drove adoption of these platforms, and we can eventually end if we put in the work.
Seriously with container technology, and a bit more work / adoption on distributed compute systems and file storage (IPFS,FileCoin) there is a future where we dont have to use big brothers compute platform. Fuck these guys.","title":null,"type":"comment","url":null},{"author":"danielovichdk","children":[],"created_at":"2025-10-29T19:16:54.000Z","created_at_i":1761765414,"id":45751644,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"What did you do when AWS was down last week?","title":null,"type":"comment","url":null},{"author":"WD-42","children":[],"created_at":"2025-10-29T19:18:13.000Z","created_at_i":1761765493,"id":45751668,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"Azure outages don\u2019t faze anyone because nobody notices when it happens.","title":null,"type":"comment","url":null},{"author":"multiplegeorges","children":[{"author":"bpye","children":[{"author":"antonkochubey","children":[{"author":"nflekkhnnn","children":[],"created_at":"2025-10-30T10:51:29.000Z","created_at_i":1761821489,"id":45758519,"options":[],"parent_id":45753127,"points":null,"story_id":45748661,"text":"ACI is the corresponding service in our favorite cloud.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:14:23.000Z","created_at_i":1761772463,"id":45753127,"options":[],"parent_id":45752245,"points":null,"story_id":45748661,"text":"Isn\u2019t ECS Fargate pretty much the same thing?","title":null,"type":"comment","url":null},{"author":"vrosas","children":[],"created_at":"2025-10-30T01:06:43.000Z","created_at_i":1761786403,"id":45755223,"options":[],"parent_id":45752245,"points":null,"story_id":45748661,"text":"Cloud Run is incredible. It\u2019s one of those things I wish more devs knew about. Even at work where we use GCP all the \u201csmart\u201d devs insist on GKE for their \u201cwebscale\u201d services that get dozens of requests a second. Dozens!","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:59:13.000Z","created_at_i":1761767953,"id":45752245,"options":[],"parent_id":45751777,"points":null,"story_id":45748661,"text":"I haven't used much of GCP, but I have had a good experience with Cloud Run and really haven't found a comparable offering from the other clouds.","title":null,"type":"comment","url":null},{"author":"macintux","children":[{"author":"kccqzy","children":[],"created_at":"2025-10-30T01:40:28.000Z","created_at_i":1761788428,"id":45755477,"options":[],"parent_id":45752669,"points":null,"story_id":45748661,"text":"That's generally speaking a good practice anyways. My Amazon shopping account has a different email than my Amazon Web Services account. I intuited that they need to be different from the get go.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:34:00.000Z","created_at_i":1761770040,"id":45752669,"options":[],"parent_id":45751777,"points":null,"story_id":45748661,"text":"I know for some people the prospect of losing their Google Cloud access due to an automated terms of service violation on some completely unrelated service is worrisome.
https://cloud.google.com/resource-manager/docs/project-suspe...
I'd hope you can create a Google Cloud account under a completely different email address, but I do as little business with Google as I can get away with, so I have no idea.","title":null,"type":"comment","url":null},{"author":"archon810","children":[{"author":"a012","children":[],"created_at":"2025-10-30T09:45:09.000Z","created_at_i":1761817509,"id":45758077,"options":[],"parent_id":45757054,"points":null,"story_id":45748661,"text":"GCP console is not the best but as a long term multicloud user, I can assure you that GCP is much better than Azure portal and/or Azure APIs which is fucking hell","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T06:44:14.000Z","created_at_i":1761806654,"id":45757054,"options":[],"parent_id":45751777,"points":null,"story_id":45748661,"text":">I've used AWS, Azure, and recently GCP. You do NOT want to use GCP.
>TBH, GCP is very good! More people should use it.
These takes couldn't be further apart. Gotta love HN comments.","title":null,"type":"comment","url":null},{"author":"the_black_hand","children":[],"created_at":"2025-10-31T06:13:44.000Z","created_at_i":1761891224,"id":45768912,"options":[],"parent_id":45751777,"points":null,"story_id":45748661,"text":"have used both. I prefer Azure. Much easier to use. found GCP unintuitive.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:25:59.000Z","created_at_i":1761765959,"id":45751777,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"> At this point I'd rather use GCP over Azure and I have zero seconds of experience with it.
TBH, GCP is very good! More people should use it.","title":null,"type":"comment","url":null},{"author":"Nemo_bis","children":[],"created_at":"2025-10-29T21:46:04.000Z","created_at_i":1761774364,"id":45753488,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"Microsoft is better at regulatory capture, so Azure has many customers in the public sector. So an Azure outage probably affects the public sector more (see example above about trains).","title":null,"type":"comment","url":null},{"author":"karel-3d","children":[],"created_at":"2025-10-30T05:36:32.000Z","created_at_i":1761802592,"id":45756738,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"Microsoft has the regulatory capture. All the European privacy and regulatory laws are good for Azure. That's why your average European government or baking app runs most likely on Azure. (or Oracle, but more likely Azure)","title":null,"type":"comment","url":null},{"author":"issafram","children":[],"created_at":"2025-10-30T06:23:22.000Z","created_at_i":1761805402,"id":45756949,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"I've used AWS, Azure, and recently GCP. You do NOT want to use GCP.","title":null,"type":"comment","url":null},{"author":"major505","children":[],"created_at":"2025-10-30T08:47:31.000Z","created_at_i":1761814051,"id":45757731,"options":[],"parent_id":45750953,"points":null,"story_id":45748661,"text":"I like azure. I think they are more intuitive tua aws and good, and they have good prices for startups ( essentially free for a whole year )","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:21:30.000Z","created_at_i":1761762090,"id":45750953,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"For some reason an Azure outage does not faze me in the same way that an AWS outage does.
I have never had much confidence in Azure as a cloud provider. The vertical integration of all the things for a Microsoft shop was initially very compelling. I was ready to fight that battle. But, this fantasy was quickly ruined by poor execution on Microsoft's part. They were able to convince me to move back to AWS by simply making it difficult to provision compute resources. Their quota system & availability issues are a nightmare to deal with compared to EC2.
At this point I'd rather use GCP over Azure and I have zero seconds of experience with it. The number of things Microsoft gets right in 2025 can be counted single-handedly. The things they do get right are quite good, but everything else tends to be extremely awful.","title":null,"type":"comment","url":null},{"author":"AtNightWeCode","children":[],"created_at":"2025-10-29T18:23:28.000Z","created_at_i":1761762208,"id":45750975,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"From Azure status page: "Customers can consider implementing failover strategies with Azure Traffic Manager, to fail over from Azure Front Door to your origins".
What a terrible advise.","title":null,"type":"comment","url":null},{"author":"amir734jj","children":[],"created_at":"2025-10-29T18:25:27.000Z","created_at_i":1761762327,"id":45751001,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"It's DNS","title":null,"type":"comment","url":null},{"author":"worik","children":[],"created_at":"2025-10-29T18:27:31.000Z","created_at_i":1761762451,"id":45751023,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"An important quality of the cloud is that it is always available.
Except that it is not!
Interesting times...","title":null,"type":"comment","url":null},{"author":"btbuildem","children":[],"created_at":"2025-10-29T18:29:07.000Z","created_at_i":1761762547,"id":45751043,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"https://login.microsoftonline.com/ is down, so that's fun","title":null,"type":"comment","url":null},{"author":"basfo","children":[{"author":"Aperocky","children":[{"author":"basfo","children":[],"created_at":"2025-10-29T19:38:33.000Z","created_at_i":1761766713,"id":45751974,"options":[],"parent_id":45751953,"points":null,"story_id":45748661,"text":"And it was HUGE, the microsoft logo was like 50% of the screen.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:37:08.000Z","created_at_i":1761766628,"id":45751953,"options":[],"parent_id":45751059,"points":null,"story_id":45748661,"text":"Outages are one thing, but having your content polluted seems like a more serious problem? Unless you subscribed to microsoft banners somehow.","title":null,"type":"comment","url":null},{"author":"qiller","children":[],"created_at":"2025-10-29T21:02:22.000Z","created_at_i":1761771742,"id":45752978,"options":[],"parent_id":45751059,"points":null,"story_id":45748661,"text":"We battled https://learn.microsoft.com/en-us/answers/questions/1331370/... for over a year, and finally decided to move off since there was no any resolution. Unfortunately our API servers were still behind AFD so they were affected by today's stuff...","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:30:03.000Z","created_at_i":1761762603,"id":45751059,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"We\u2019re 100% on Azure but so far there\u2019s no impact for us.
Luckily, we moved off Azure Front Door about a year ago. We\u2019d had three major incidents tied to Front Door and stopped treating it as a reliable CDN.
They weren\u2019t global outages, more like issues triggered by new deployments. In one case, our homepage suddenly showed a huge Microsoft banner about a \u201cpost-quantum encryption algorithm\u201d or something along those lines.
Kinda wild that a company that big can be so shaky on a CDN, which should be rock solid.","title":null,"type":"comment","url":null},{"author":"perks_12","children":[],"created_at":"2025-10-29T18:31:16.000Z","created_at_i":1761762676,"id":45751079,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Thank you. I was wondering what was going on at a company whose web app I need to access. I just checked with BuiltWith and it seems they are on Azure.","title":null,"type":"comment","url":null},{"author":"redwood","children":[],"created_at":"2025-10-29T18:32:24.000Z","created_at_i":1761762744,"id":45751094,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Is it Cosmos DB? If so the symmetry with AWS/Dynamo would be very eerie.","title":null,"type":"comment","url":null},{"author":"chrisgeleven","children":[{"author":"unethical_ban","children":[{"author":"jeffrallen","children":[],"created_at":"2025-10-29T20:43:47.000Z","created_at_i":1761770627,"id":45752756,"options":[],"parent_id":45751158,"points":null,"story_id":45748661,"text":"Front Door (tm), with Back Door access for the FBI included free with your subscription! ;)","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:37:02.000Z","created_at_i":1761763022,"id":45751158,"options":[],"parent_id":45751100,"points":null,"story_id":45748661,"text":"I wonder if many Germans are eager to sign up for AFD.
But seriously I thought it would be the console, not a CDN.","title":null,"type":"comment","url":null},{"author":"oliyoung","children":[],"created_at":"2025-10-29T21:48:34.000Z","created_at_i":1761774514,"id":45753521,"options":[],"parent_id":45751100,"points":null,"story_id":45748661,"text":"We should've never let marketing in the door honestly, all of the product names for the big three are awful.
Microsoft CDN
There, that's it. You're selling it to (hopefully) technical people","title":null,"type":"comment","url":null},{"author":"joquarky","children":[],"created_at":"2025-10-29T22:51:50.000Z","created_at_i":1761778310,"id":45754135,"options":[],"parent_id":45751100,"points":null,"story_id":45748661,"text":"It so strongly implies a counterpart.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:32:48.000Z","created_at_i":1761762768,"id":45751100,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":""Front Door" has to be the worst product name for a CDN I've ever heard of. I used to work for a CDN too.","title":null,"type":"comment","url":null},{"author":"montague27","children":[],"created_at":"2025-10-29T18:33:20.000Z","created_at_i":1761762800,"id":45751110,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Guess when/who has the next outage!","title":null,"type":"comment","url":null},{"author":"mattdecker100","children":[],"created_at":"2025-10-29T18:33:53.000Z","created_at_i":1761762833,"id":45751117,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Unable to use Ona's GitPod through VSCode SSH - Unable to download code server from https://update.code.visualstudio.com","title":null,"type":"comment","url":null},{"author":"smithkl42","children":[],"created_at":"2025-10-29T18:34:49.000Z","created_at_i":1761762889,"id":45751124,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The iron law of uptime: "The mandatory single point of failure in every possible system is configuration."","title":null,"type":"comment","url":null},{"author":"aftbit","children":[],"created_at":"2025-10-29T18:36:16.000Z","created_at_i":1761762976,"id":45751144,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I still can't log into Azure Gov Cloud with
https://microsoft.com/deviceloginus
Seems like they migrated the non-Gov login but not the Gov one. C'mon Microsoft, I've got a deadline in a few days.","title":null,"type":"comment","url":null},{"author":"alt227","children":[],"created_at":"2025-10-29T18:37:05.000Z","created_at_i":1761763025,"id":45751159,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Microsoft have started putting customer status pages up on windows.net, so it must be really really bad!
For example when I try to log into our payroll provider Brightpay, it sends me here:
https://bpuk1prod1environment.blob.core.windows.net/host-pro...","title":null,"type":"comment","url":null},{"author":"qmr","children":[],"created_at":"2025-10-29T18:37:51.000Z","created_at_i":1761763071,"id":45751166,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Always in these large provider outages you see people who have forgotten the old ways.","title":null,"type":"comment","url":null},{"author":"udev4096","children":[],"created_at":"2025-10-29T18:43:10.000Z","created_at_i":1761763390,"id":45751228,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Luckily, no one uses azure and it's fully expected from azure to go down all the time! Keep it up!","title":null,"type":"comment","url":null},{"author":"CKMo","children":[],"created_at":"2025-10-29T18:43:59.000Z","created_at_i":1761763439,"id":45751238,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Reasons to not use hyperscalers, exhibit 654
There's a lot of outages this month!","title":null,"type":"comment","url":null},{"author":"Imustaskforhelp","children":[{"author":"TiredOfLife","children":[{"author":"kentonv","children":[],"created_at":"2025-10-29T19:45:32.000Z","created_at_i":1761767132,"id":45752069,"options":[],"parent_id":45752018,"points":null,"story_id":45748661,"text":"It was Workers KV (an optional storage add-on to Workers), and we fixed it, it no longer depends on GCP:
https://blog.cloudflare.com/rearchitecting-workers-kv-for-re...","title":null,"type":"comment","url":null},{"author":"Imustaskforhelp","children":[],"created_at":"2025-10-29T19:50:24.000Z","created_at_i":1761767424,"id":45752119,"options":[],"parent_id":45752018,"points":null,"story_id":45748661,"text":"Hm it seemed that they hosted a critical service for cloudflare kv on google itself, but I wonder about the update.
Personally I just trust cloudflare more than google, given how their focus is on security whereas google feels googly...
I have heard some good things about google cloud run and the google's interface feels the best out of AWS,Azure,GCloud but I still would just prefer cloudflare/hetzner iirc
Another question: Has there ever been a list of all major cloud outages, like I am interested how many times google cloud and all cloud providers went majorly down I guess y'know? is there a website/git project that tracks this?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:41:25.000Z","created_at_i":1761766885,"id":45752018,"options":[],"parent_id":45751245,"points":null,"story_id":45748661,"text":"One of recent (4 months ago) Cloudflare outages (I think it was even workers) was caused by Google Cloud being down and Cloudflare hosting an essential service there","title":null,"type":"comment","url":null},{"author":"hshdhdhehd","children":[],"created_at":"2025-10-29T21:20:16.000Z","created_at_i":1761772816,"id":45753194,"options":[],"parent_id":45751245,"points":null,"story_id":45748661,"text":"Are you after nines? Maybe do multi provider?","title":null,"type":"comment","url":null},{"author":"Havoc","children":[{"author":"dijit","children":[],"created_at":"2025-10-29T23:59:42.000Z","created_at_i":1761782382,"id":45754717,"options":[],"parent_id":45753273,"points":null,"story_id":45748661,"text":"Probably not, but at least you don\u2019t delude yourself into thinking reliability is a solved problem just because you\u2019re paying through the nose for compute and storage.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:26:22.000Z","created_at_i":1761773182,"id":45753273,"options":[],"parent_id":45751245,"points":null,"story_id":45748661,"text":"Hetzner seems sound, but I doubt they play in the same reliability league as google","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:44:18.000Z","created_at_i":1761763458,"id":45751245,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Google cloud run or cloudflare workers it is.
Personally I am thinking more and more about hetzner, yes I know its not an apples to orange comparison. But its honestly so good
Someone had created a video where they showed the underlying hardware etc., I am wondering if there is something like https://vpspricetracker.com/ but with geek-benchmarks as well.
This video was affiliated with scalahosting but still I don't think that there was too much bias of them and they showed at around 3:37 a graph comparison with prices https://www.youtube.com/watch?v=9dvuBH2Pc1g
Now it shows how contabo has better hardware but I am pretty sure that there might be some other issues, and honestly I feel a sense of trust with hetzner I am not sure about others.
Either hetzner or self hosting stuff personally or just having a very cheap vps and going to hetzner if need be but hetzner already is pretty cheap or I might use some free service that I know of are good as well.","title":null,"type":"comment","url":null},{"author":"sedatk","children":[{"author":"thewebguyd","children":[],"created_at":"2025-10-29T19:05:20.000Z","created_at_i":1761764720,"id":45751510,"options":[],"parent_id":45751257,"points":null,"story_id":45748661,"text":"People feel it, but usually not general consumers like they do when AWS goes down.
If Azure goes down, it's mostly affecting internal stuff at big old enterprises. Jane in accounting might notice, but the customers don't. Contrast with AWS which runs most of the world's SaaS products.
People not being able to do their jobs internally for a day tends not to make headlines like "100 popular internet services down for everyone" does.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T18:45:04.000Z","created_at_i":1761763504,"id":45751257,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The paradox of cloud provider crashes is that if the provider goes down and takes the whole world with it, it's actually good advertisement. Because, that means so many things rely on it, it's critically important, and has so many big customers. That might be why Amazon stock went up after AWS crash.
If Azure goes down and nobody feels it, does Azure really matter?","title":null,"type":"comment","url":null},{"author":"whalesalad","children":[],"created_at":"2025-10-29T18:49:54.000Z","created_at_i":1761763794,"id":45751301,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Yikes, http://schemas.xmlsoap.org/soap/encoding/ is running on Azure and it's down. So any SOAP/WSDL api's are dead in the water.
![https://i.sstatic.net/yCseI.png](\"https://i.sstatic.net/yCseI.png\"){kind=link}
HTTPSConnectionPool(host='schemas.xmlsoap.org', port=443): Max retries exceeded with url: /soap/encoding/ (Caused by SSLError(CertificateError("hostname 'schemas.xmlsoap.org' doesn't match '*.azureedge.net'")))\n160k+ results on GitHub: https://github.com/search?q=http%3A%2F%2Fschemas.xmlsoap.org...","title":null,"type":"comment","url":null},{"author":"delf","children":[],"created_at":"2025-10-29T18:53:28.000Z","created_at_i":1761764008,"id":45751352,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The outage impacted GitSocial minor version bump release: https://marketplace.visualstudio.com/items?itemName=GitSocia...
There's no way to tell, and after about 30 minutes, the release process on VS Code Marketplace failed with a cryptic message: "Repository signing for extension file failed.". And there's no way to restart/resume it.","title":null,"type":"comment","url":null},{"author":"zingababba","children":[],"created_at":"2025-10-29T18:54:24.000Z","created_at_i":1761764064,"id":45751363,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"This brings to mind this -> https://thenewstack.io/github-will-prioritize-migrating-to-a...","title":null,"type":"comment","url":null},{"author":"amaccuish","children":[],"created_at":"2025-10-29T19:02:46.000Z","created_at_i":1761764566,"id":45751480,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Seeing users having issues with the "Modern Outlook", specifically empty accounts. Switching back to the "Legacy Outlook" which functions largely without the help of the cloud fixes the issue. How ironic.","title":null,"type":"comment","url":null},{"author":"_andrei_","children":[],"created_at":"2025-10-29T19:11:15.000Z","created_at_i":1761765075,"id":45751589,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"https://www.reddit.com/r/cscareerquestions/comments/1ojbebq/...","title":null,"type":"comment","url":null},{"author":"user3939382","children":[{"author":"ycombinator_acc","children":[{"author":"user3939382","children":[],"created_at":"2025-10-30T01:03:18.000Z","created_at_i":1761786198,"id":45755198,"options":[],"parent_id":45751893,"points":null,"story_id":45748661,"text":"I\u2019ll roll it out","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:33:43.000Z","created_at_i":1761766423,"id":45751893,"options":[],"parent_id":45751730,"points":null,"story_id":45748661,"text":"Aww man you got me curious for a sec there.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:22:49.000Z","created_at_i":1761765769,"id":45751730,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I know how to fix this but this community is too close minded and argumentative egocentric sensitive pedantic threatened angry etc to bother discussing it","title":null,"type":"comment","url":null},{"author":"tpl","children":[],"created_at":"2025-10-29T19:27:52.000Z","created_at_i":1761766072,"id":45751807,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Part of this outage involves outlook hanging and then blaming random addins. Pretty terrible practice by Microsoft to blame random vendors for their own outage.","title":null,"type":"comment","url":null},{"author":"foresterre","children":[{"author":"conductr","children":[{"author":"klardotsh","children":[{"author":"vanviegen","children":[{"author":"jfengel","children":[{"author":"conductr","children":[{"author":"iAMkenough","children":[{"author":"conductr","children":[{"author":"jfengel","children":[],"created_at":"2025-10-29T22:50:53.000Z","created_at_i":1761778253,"id":45754122,"options":[],"parent_id":45753535,"points":null,"story_id":45748661,"text":"Not everyone does. It varies from state to state. Red states in particular have little to no early voting.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:49:49.000Z","created_at_i":1761774589,"id":45753535,"options":[],"parent_id":45753213,"points":null,"story_id":45748661,"text":"That's all happening too, but it's honestly a different topic altogether. We have the ability to vote early. Whether you trust it or politicians are trying to undermine your trust in it, etc.... whole other can of worms","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:22:10.000Z","created_at_i":1761772930,"id":45753213,"options":[],"parent_id":45752784,"points":null,"story_id":45748661,"text":"We're on year five of one of the two parties telling voters to not trust early voting. Their choice is because of the Fear, Uncertainty, and Doubt created by the propaganda they are fed.
Here's the President of the United States on Sunday: https://truthsocial.com/@realDonaldTrump/posts/1154418712892...
"No mail-in or 'Early' Voting, Yes to Voter ID! Watch how totally dishonest the California Prop Vote is! Millions of Ballots being 'shipped.' GET SMART REPUBLICANS, BEFORE IT IS TOO LATE!!!"","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:45:48.000Z","created_at_i":1761770748,"id":45752784,"options":[],"parent_id":45752554,"points":null,"story_id":45748661,"text":"We have early voting, nobody has to wait, they choose to wait","title":null,"type":"comment","url":null},{"author":"tags2k","children":[],"created_at":"2025-10-29T20:50:26.000Z","created_at_i":1761771026,"id":45752836,"options":[],"parent_id":45752554,"points":null,"story_id":45748661,"text":"You have early voting, some choose not to trust the early voting system.","title":null,"type":"comment","url":null},{"author":"christkv","children":[{"author":"esseph","children":[],"created_at":"2025-10-29T22:51:30.000Z","created_at_i":1761778290,"id":45754131,"options":[],"parent_id":45753054,"points":null,"story_id":45748661,"text":"Have not for me. I mark on a paper ballot that then gets fed into a machine to be recorded. That leaves a paper copy and a digital voting record.","title":null,"type":"comment","url":null},{"author":"jfengel","children":[],"created_at":"2025-10-29T22:54:03.000Z","created_at_i":1761778443,"id":45754156,"options":[],"parent_id":45753054,"points":null,"story_id":45748661,"text":"Not as much as hanging chads do.","title":null,"type":"comment","url":null},{"author":"vitorgrs","children":[],"created_at":"2025-10-30T04:55:34.000Z","created_at_i":1761800134,"id":45756577,"options":[],"parent_id":45753054,"points":null,"story_id":45748661,"text":"At least in Brazil, that's not the case. You get there to vote, and it doesn't take longer than 5 minutes to leave the place.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:07:57.000Z","created_at_i":1761772077,"id":45753054,"options":[],"parent_id":45752554,"points":null,"story_id":45748661,"text":"Voting machines slow down voting from what I understand","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:23:52.000Z","created_at_i":1761769432,"id":45752554,"options":[],"parent_id":45752439,"points":null,"story_id":45748661,"text":"In the US, hours-long lines are routine. Not everywhere, but poorer places tend to have fewer voting machines and longer lines.
We've been closing a lot of polling places recently:
https://abcnews.go.com/US/protecting-vote-1-5-election-day-p...","title":null,"type":"comment","url":null},{"author":"conductr","children":[{"author":"Boltgolt","children":[{"author":"conductr","children":[{"author":"gowld","children":[{"author":"conductr","children":[],"created_at":"2025-10-29T21:53:52.000Z","created_at_i":1761774832,"id":45753579,"options":[],"parent_id":45753433,"points":null,"story_id":45748661,"text":"Mild curiosity, no idea whether it would be statistically relevant but asking the question is the first step. If you knew the answer, you might want to extend the voting window even if it wouldn't effect an elections outcome it would be a quantified number of people excluded from the democratic process for simply having bad luck at the wrong time.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:40:35.000Z","created_at_i":1761774035,"id":45753433,"options":[],"parent_id":45752956,"points":null,"story_id":45748661,"text":"What percent of the electorate is incapacitated on voting day?
What is the that group's deviation from the general voting population's preferences?
What are the margins of the votes on those ballot questions?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:00:08.000Z","created_at_i":1761771608,"id":45752956,"options":[],"parent_id":45752876,"points":null,"story_id":45748661,"text":"How is that an acceptable response? Honestly. You\u2019re in the hospital, in pain, likely having a minor surgery, and having someone cast your vote for you is going to be on your mind too? Do you have your voting card in your pocket just in case this were to play out?
That\u2019s just ridiculous in my opinion. Makes me wonder how many well intentioned would be voters end up missing out each election cause shit happens and voting is pretty optional","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:53:57.000Z","created_at_i":1761771237,"id":45752876,"options":[],"parent_id":45752768,"points":null,"story_id":45748661,"text":"If hospitalized on that specific day: Sign the back of the voting card and give your ID to a family member, they can cast your vote","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:44:40.000Z","created_at_i":1761770680,"id":45752768,"options":[],"parent_id":45752439,"points":null,"story_id":45748661,"text":"So, if you have a minor emergency, like a kidney stone and hospitalized for the day - you just miss your chance to vote in that election?
If so, I see a lot to dislike. As the point I was making is you can\u2019t anticipate what might come up. Just because it\u2019s worked thus far doesn\u2019t mean it\u2019s designed for resilience. There\u2019s a lot of ways you could miss out in that type of situation. I seems silly to make sure everything else is redundant and fault tolerant in the name of democracy when the democratic process itself isn\u2019t doing the same.","title":null,"type":"comment","url":null},{"author":"Sleaker","children":[{"author":"vanviegen","children":[],"created_at":"2025-10-29T23:26:37.000Z","created_at_i":1761780397,"id":45754474,"options":[],"parent_id":45753490,"points":null,"story_id":45748661,"text":"Oh, I know. I'm just saying it can be done properly on a single day. It is a pretty challenging and expensive logistical operation though.","title":null,"type":"comment","url":null},{"author":"yellow_postit","children":[{"author":"wyre","children":[],"created_at":"2025-10-30T02:29:50.000Z","created_at_i":1761791390,"id":45755784,"options":[],"parent_id":45755582,"points":null,"story_id":45748661,"text":"Rule of thumb: if Republicans are against it, it\u2019s probably a good thing for everyone else, like mail-in voting.
So excited to see how the right-wing pedants here disagree with this.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T01:57:45.000Z","created_at_i":1761789465,"id":45755582,"options":[],"parent_id":45753490,"points":null,"story_id":45748661,"text":"Don\u2019t forget you can\u2019t dare offer water or food to those stuck in lines else that\u2019s considered tampering in many (all?) locales in the US.
Mail in voting is just better all around for a geographically diverse place as the US and I wish would be adopted by all states.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:46:09.000Z","created_at_i":1761774369,"id":45753490,"options":[],"parent_id":45752439,"points":null,"story_id":45748661,"text":"Please lookup US voting poll overflow issues that come up every election cycle. Just because you experience a well streamlined process doesn't mean that it's the norm everywhere.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:13:54.000Z","created_at_i":1761768834,"id":45752439,"options":[],"parent_id":45752167,"points":null,"story_id":45748661,"text":"As a Dutchman, I have to go vote in person on a specific day. But to be honest: I really don't mind doing so. If you live in a town or city, there'll usually be multiple voting locations you can choose from within 10 minutes walking distance. I've never experienced waiting times more than a couple of minutes. Opening times are pretty good, from 7:30 til 21:00. The people there are friendly. What's not to like? (Except for some of the candidates maybe, but that's a whole different story. :-))","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:53:51.000Z","created_at_i":1761767631,"id":45752167,"options":[],"parent_id":45752108,"points":null,"story_id":45748661,"text":"Washington State having full vote-by-mail (there is technically a layer of in-person voting as a fallback for those who need it for accessibility reasons or who missed the registration deadline) has spoiled me rotten, I couldn't imagine having to go back to synchronous on-site voting on a single day like I did in Illinois. Awful. Being able to fill my ballot at my leisure, at home, where I can have all the research material open, and drive it to a ballot drop box whenever is convenient in a 2-3 week window before 20:00 on election night, is a game-changer for democracy. Of course this also means that people who serve to benefit from disenfranchising voters and making it more difficult to vote, absolutely hate our system and continually attack it for one reason or another.","title":null,"type":"comment","url":null},{"author":"DontBreakAlex","children":[{"author":"jampekka","children":[{"author":"silversmith","children":[{"author":"ndom91","children":[],"created_at":"2025-10-29T20:32:44.000Z","created_at_i":1761769964,"id":45752655,"options":[],"parent_id":45752615,"points":null,"story_id":45748661,"text":"Germany has mail-in voting, not sure if that counts as advanced voting though","title":null,"type":"comment","url":null},{"author":"generalspecific","children":[{"author":"alborzb","children":[{"author":"embedding-shape","children":[],"created_at":"2025-10-29T22:41:19.000Z","created_at_i":1761777679,"id":45754038,"options":[],"parent_id":45753704,"points":null,"story_id":45748661,"text":"I think they meant "don't have it" as in except in special circumstances, and that form says:
> You may use this form to apply for a postal vote if, due to the circumstances of your work/service or your full-time study in the State, you cannot go to your polling station on polling day.
Which seems to indicate that's only for people who can't go to the polling station, otherwise you do have to go there.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:06:02.000Z","created_at_i":1761775562,"id":45753704,"options":[],"parent_id":45752781,"points":null,"story_id":45748661,"text":"That's not true (as somebody who had to do this last year in 2024 because I was traveling in another country for work on election day)
Here is the form to register for postal voting in the Republic of Ireland - https://www.dublincity.ie/sites/default/files/2024-01/pv4-wo...
Instructions on how to submit the form / register for mail-in votes is on page 4.
Hope that helps anyone else out who needs in Ireland","title":null,"type":"comment","url":null},{"author":"esperent","children":[],"created_at":"2025-10-30T00:59:28.000Z","created_at_i":1761785968,"id":45755174,"options":[],"parent_id":45752781,"points":null,"story_id":45748661,"text":"I think that a lot of Ireland's voting practices come from having a small population but a huge diaspora. I imagine the percentage of people living outside Ireland what would be eligible to vote in many other countries is significant enough to effect elections, certainly if they are close.
As someone who spent the first 30 years of my life in Ireland but is now part of that diaspora, it's frustrating but I get it. I don't get to vote, but neither do thousands of plastic paddys who have very little genuine connection to Ireland.
That said, I'm sure they could expand the voting window to a couple of days at least without too much issue.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:45:37.000Z","created_at_i":1761770737,"id":45752781,"options":[],"parent_id":45752615,"points":null,"story_id":45748661,"text":"Ireland doesn't have it.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:28:50.000Z","created_at_i":1761769730,"id":45752615,"options":[],"parent_id":45752397,"points":null,"story_id":45748661,"text":"Off the top of my head, I can't think of an EU country that does not have some form of advance voting.
Here in Latvia the "election day" is usually (always?) on weekend, but the polling stations are open for some (and different!) part of every weekday leading up. Something like couple hours on monday morning, couple hours on tuesday evening, couple around midday wednesday, etc. In my opinion, it's a great system. You have to have a pretty convoluted schedule for at least one window not to line up for you.","title":null,"type":"comment","url":null},{"author":"mrighele","children":[],"created_at":"2025-10-29T22:27:05.000Z","created_at_i":1761776825,"id":45753907,"options":[],"parent_id":45752397,"points":null,"story_id":45748661,"text":"Italy has mail-in vote only for citizen residing abroad. The rest vote on the election Sunday (and Monday morning in some cases, at least in the past).","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:10:43.000Z","created_at_i":1761768643,"id":45752397,"options":[],"parent_id":45752186,"points":null,"story_id":45748661,"text":"Many countries in Europe have advance voting.","title":null,"type":"comment","url":null},{"author":"tempestn","children":[{"author":"vasco","children":[{"author":"samtp","children":[{"author":"arsome","children":[{"author":"trevoragilbert","children":[{"author":"maxdamantus","children":[{"author":"bee_rider","children":[],"created_at":"2025-10-30T03:24:37.000Z","created_at_i":1761794677,"id":45756110,"options":[],"parent_id":45754269,"points":null,"story_id":45748661,"text":"I\u2019m not sure what\u2019s so special in Oregon\u2019s ballot boxes. But, tampering that is detected (don\u2019t need much special to detect a burning box I guess!) is not a complete failure for a system. If any elections were close enough for a box to matter, they could have rerun them.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:06:14.000Z","created_at_i":1761779174,"id":45754269,"options":[],"parent_id":45754054,"points":null,"story_id":45748661,"text":"Doesn't look difficult: https://www.fbi.gov/wanted/seeking-info/ballot-box-fires\n(yes, that's in Oregon)","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:42:50.000Z","created_at_i":1761777770,"id":45754054,"options":[],"parent_id":45753887,"points":null,"story_id":45748661,"text":"You should research what\u2019s inside the boxes in Oregon before just assuming they\u2019re easier to tamper with.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:24:27.000Z","created_at_i":1761776667,"id":45753887,"options":[],"parent_id":45753804,"points":null,"story_id":45748661,"text":"One of these things is much easier to burn or otherwise tamper with.","title":null,"type":"comment","url":null},{"author":"maxdamantus","children":[{"author":"samtp","children":[],"created_at":"2025-11-06T20:09:23.000Z","created_at_i":1762459763,"id":45839773,"options":[],"parent_id":45754167,"points":null,"story_id":45748661,"text":"> I'm sure I've seen reports in the US of people burning postboxes around election time
Yeah that happened once in OR then got re-plastered all over the news dozens of times. I'm sure you can find way more incidents of intimidation, fighting, long lines and other issues for in-person voting. But individual incidents does not mean that there is anything wrong with a system that has worked for decades in multiple states.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:55:08.000Z","created_at_i":1761778508,"id":45754167,"options":[],"parent_id":45753804,"points":null,"story_id":45748661,"text":"Here in NZ when I've been to vote, there are usually a couple of party affiliates at the voting location, doing what one of the parent posts described:
> You can stay there and wait for the count at the end of the day if you want to.
And if you watch the election night news, you'll see footage of multiple people counting the votes from the ballot boxes, again with various people observing to check that nothing dodgy is going on.
Having everyone just put their ballots in a postbox seems like a good way remove public trust from the electoral system, because noone's standing around waiting for the postie to collect the mail, or looking at what happens in the mail truck, or the rest of the mail distribution process.
I'm sure I've seen reports in the US of people burning postboxes around election time. Things like this give more excuses to treat election results as illegitimate, which I believe has been an issue over there.
(Yes, we do also have advanced voting in NZ, but I think they're considered "special votes" and are counted separately .. the elections are largely determined on the day by in-person votes, with the special votes being confirmed some days later)","title":null,"type":"comment","url":null},{"author":"belorn","children":[],"created_at":"2025-10-30T00:14:14.000Z","created_at_i":1761783254,"id":45754820,"options":[],"parent_id":45753804,"points":null,"story_id":45748661,"text":"In Sweden, mail/early votes get sent through the postal system to the official ballot box for those votes. In 2018, a local election had to be redone because the post delivered votes late. Mail delivery occasionally have packaged delayed or lost, and votes are note immune to this problem. In one case the post also gave the votes to an unauthorized person, through the votes did end up at the right place.
It is a small but distinct difference between mail/early voting and putting the votes directly into the ballot box.","title":null,"type":"comment","url":null},{"author":"_heimdall","children":[],"created_at":"2025-10-30T01:29:00.000Z","created_at_i":1761787740,"id":45755388,"options":[],"parent_id":45753804,"points":null,"story_id":45748661,"text":"How is that the same?
The description of voting in the Netherlands is that you can see your ballot physically go into a clear box and stay to see that exact box be opened and all ballots tallied.
Dropping a ballot in a box in tour neighborhood helps ensure nothing with regards to the actually ballot count.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:16:13.000Z","created_at_i":1761776173,"id":45753804,"options":[],"parent_id":45753393,"points":null,"story_id":45748661,"text":"Well "mail in voting" in Washington state pretty much means you drop off your ballot in a drop box in your neighborhood. Which is pretty much the same thing as putting it in a ballot box.","title":null,"type":"comment","url":null},{"author":"esseph","children":[{"author":"ceejayoz","children":[{"author":"panarky","children":[{"author":"esseph","children":[],"created_at":"2025-10-30T04:18:25.000Z","created_at_i":1761797905,"id":45756396,"options":[],"parent_id":45754631,"points":null,"story_id":45748661,"text":"My point is, you don't actually know that.
Sure you got a notification! That doesn't mean anything. Even with human counted ballots or electronic ballots.
Following the chain of custody from vote to verification, in some way, would be nice.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:48:46.000Z","created_at_i":1761781726,"id":45754631,"options":[],"parent_id":45754223,"points":null,"story_id":45748661,"text":"My county sends me a text message when they've counted my ballot.","title":null,"type":"comment","url":null},{"author":"kaashif","children":[],"created_at":"2025-10-30T01:12:52.000Z","created_at_i":1761786772,"id":45755271,"options":[],"parent_id":45754223,"points":null,"story_id":45748661,"text":"You have to trust that whole system. Maybe you do, I don't know the details of how any of that works.
When I vote in person, I know all the officials there from various parties are just like...looking at the box for the whole day to make sure everything is counted. It's much easier to understand and trust.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:01:30.000Z","created_at_i":1761778890,"id":45754223,"options":[],"parent_id":45754093,"points":null,"story_id":45748661,"text":"My county lets you look up if it was received. You can vote on Election Day in person if they don\u2019t.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:48:21.000Z","created_at_i":1761778101,"id":45754093,"options":[],"parent_id":45753393,"points":null,"story_id":45748661,"text":"Seeing your ballot drop in a box is no indicator the vote is actually recorded in the grand tally, or what was recorded for your vote.","title":null,"type":"comment","url":null},{"author":"hahajk","children":[{"author":"kaashif","children":[{"author":"throwaway7783","children":[],"created_at":"2025-10-30T01:33:08.000Z","created_at_i":1761787988,"id":45755426,"options":[],"parent_id":45755246,"points":null,"story_id":45748661,"text":"Even with ballot boxes you still need to trust what happens after ballot enters the box.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T01:09:50.000Z","created_at_i":1761786590,"id":45755246,"options":[],"parent_id":45755171,"points":null,"story_id":45748661,"text":"That doesn't seem at all like the same thing as literally seeing the ballot enter the box in the presence of observers from all parties.
There's so much more you have to trust.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T00:59:13.000Z","created_at_i":1761785953,"id":45755171,"options":[],"parent_id":45753393,"points":null,"story_id":45748661,"text":"I mail in to Florida and I can log in and see that they received it and it was counted. So, close to seeing it enter the box.","title":null,"type":"comment","url":null},{"author":"bee_rider","children":[{"author":"parliament32","children":[{"author":"bee_rider","children":[],"created_at":"2025-10-30T04:09:46.000Z","created_at_i":1761797386,"id":45756344,"options":[],"parent_id":45756233,"points":null,"story_id":45748661,"text":"Hmm, good point.
Googling around a bit, it sounds like there are systems that let you verify that your ballot made it, but not necessarily that it was counted correctly. (For this reason, I guess?)","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T03:48:43.000Z","created_at_i":1761796123,"id":45756233,"options":[],"parent_id":45756163,"points":null,"story_id":45748661,"text":"Voting systems require that there be no way to prove that you voted a certain way, otherwise it opens the market for vote-selling.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T03:34:17.000Z","created_at_i":1761795257,"id":45756163,"options":[],"parent_id":45753393,"points":null,"story_id":45748661,"text":"It could be possible to have a system like:
If you wish, you can write a phrase on your ballot. The phrases and their corresponding vote are broadcast (on tv, internet, etc). So if you want to validate that your vote was tallied correctly, write a unique phrase. Or you could pick a random 30 digit number, collisions should be zero-probability, right?
I mean, this would be annoying because people would write slurs and advertisements, and the government would have to broadcast them. But, it seems pretty robust.
I\u2019d suggest the state handle the number issuing, but then they could record who they issues which numbers to, and the winning party could go about rounding up their opposition, etc.","title":null,"type":"comment","url":null},{"author":"heroic","children":[{"author":"vasco","children":[],"created_at":"2025-10-30T06:12:30.000Z","created_at_i":1761804750,"id":45756887,"options":[],"parent_id":45756450,"points":null,"story_id":45748661,"text":"You can see electrons or what do you mean?","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T04:30:54.000Z","created_at_i":1761798654,"id":45756450,"options":[],"parent_id":45753393,"points":null,"story_id":45748661,"text":"In India we have electronic voting and we get to see our vote going in the ballot box.","title":null,"type":"comment","url":null},{"author":"edoceo","children":[{"author":"vincnetas","children":[{"author":"edoceo","children":[],"created_at":"2025-10-30T22:14:01.000Z","created_at_i":1761862441,"id":45766037,"options":[],"parent_id":45758232,"points":null,"story_id":45748661,"text":"That's just that they got my ballot. How to ensure they allocated my specific vote to the specific candidate/measure.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T10:10:08.000Z","created_at_i":1761819008,"id":45758232,"options":[],"parent_id":45756564,"points":null,"story_id":45748661,"text":"yes there is. Check double envelope mail in voting mechanics.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T04:53:27.000Z","created_at_i":1761800007,"id":45756564,"options":[],"parent_id":45753393,"points":null,"story_id":45748661,"text":"Is it possible to trace your own vote after? There has to be a technical solution to ensure that your own vote was counted","title":null,"type":"comment","url":null},{"author":"mulmen","children":[],"created_at":"2025-10-30T07:12:13.000Z","created_at_i":1761808333,"id":45757206,"options":[],"parent_id":45753393,"points":null,"story_id":45748661,"text":"In Washington you can track your ballot return status: https://www.sos.wa.gov/elections/data-research/ballot-status...","title":null,"type":"comment","url":null},{"author":"vincnetas","children":[],"created_at":"2025-10-30T10:09:08.000Z","created_at_i":1761818948,"id":45758227,"options":[],"parent_id":45753393,"points":null,"story_id":45748661,"text":"With proper mail voting you have a way to verify that your mailed in vote is counted.
(AI generated explanation)\nHow the double-envelope system works
Inner \u201csecrecy\u201d envelope
You mark your ballot, fold it, and slip it into an unmarked inner envelope.\nNo name or identifying info is on this envelope, so your choices stay anonymous.\nOuter declaration envelope
The inner envelope goes inside a larger outer envelope that carries:\n\u2013 A ballot ID/barcode unique to you.\n\u2013 A signature line that must match the one on file with your election office.\nIn many states, a detachable privacy flap or perforated strip hides the signature until election officials open the outer envelope, keeping the ballot secret.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:37:47.000Z","created_at_i":1761773867,"id":45753393,"options":[],"parent_id":45752422,"points":null,"story_id":45748661,"text":"They both share the fact that you don't see your vote enter a ballot box.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:12:37.000Z","created_at_i":1761768757,"id":45752422,"options":[],"parent_id":45752186,"points":null,"story_id":45748661,"text":"Electronic voting and mail voting are very different things though.","title":null,"type":"comment","url":null},{"author":"ed_elliott_asc","children":[{"author":"xnorswap","children":[],"created_at":"2025-10-29T21:33:11.000Z","created_at_i":1761773591,"id":45753346,"options":[],"parent_id":45752753,"points":null,"story_id":45748661,"text":"With the option to do a postal vote, or vote-by-proxy.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:43:20.000Z","created_at_i":1761770600,"id":45752753,"options":[],"parent_id":45752186,"points":null,"story_id":45748661,"text":"UK is a one day affair with voting booths typically open like 6 am to 10 pm","title":null,"type":"comment","url":null},{"author":"speakfreely","children":[{"author":"Sammi","children":[{"author":"fancyswimtime","children":[{"author":"konimex","children":[{"author":"ehnto","children":[{"author":"Sammi","children":[],"created_at":"2025-10-30T12:37:13.000Z","created_at_i":1761827833,"id":45759335,"options":[],"parent_id":45756554,"points":null,"story_id":45748661,"text":"Any id can be correlated.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T04:52:04.000Z","created_at_i":1761799924,"id":45756554,"options":[],"parent_id":45754906,"points":null,"story_id":45748661,"text":"The blockchain doesn't require your ID, just the voting station ID.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T00:23:44.000Z","created_at_i":1761783824,"id":45754906,"options":[],"parent_id":45754801,"points":null,"story_id":45748661,"text":"Not really. Your ballot should be secret, which goes against blockchain, I guess.","title":null,"type":"comment","url":null},{"author":"kaibee","children":[],"created_at":"2025-10-30T00:25:12.000Z","created_at_i":1761783912,"id":45754917,"options":[],"parent_id":45754801,"points":null,"story_id":45748661,"text":"Anonymous voting means that you can't sell your vote. Like, if I pay you $5 to vote for X, but I can't actually verify that you voted for X and not Y, then I wouldn't bother trying. Or if I'm your boss and I want you to vote for X... etc.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T00:11:36.000Z","created_at_i":1761783096,"id":45754801,"options":[],"parent_id":45754713,"points":null,"story_id":45748661,"text":"wouldn't that be a feature in this case?","title":null,"type":"comment","url":null},{"author":"johnsonelephant","children":[],"created_at":"2025-10-30T00:31:04.000Z","created_at_i":1761784264,"id":45754976,"options":[],"parent_id":45754713,"points":null,"story_id":45748661,"text":"Monero demonstrates a solution (https://en.wikipedia.org/wiki/Ring_signature)","title":null,"type":"comment","url":null},{"author":"ehnto","children":[],"created_at":"2025-10-30T04:51:06.000Z","created_at_i":1761799866,"id":45756551,"options":[],"parent_id":45754713,"points":null,"story_id":45748661,"text":"You don't have to attribute any name to the transaction, just a voting booth ID and the vote. The actual benefit is just that it is hard to tamper and easy to trace where tampering happened.
But I still prefer the paper vote and I usually a blockchain apathetic.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:59:25.000Z","created_at_i":1761782365,"id":45754713,"options":[],"parent_id":45754637,"points":null,"story_id":45748661,"text":"Nope. Blockchain has no anonymity.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:49:34.000Z","created_at_i":1761781774,"id":45754637,"options":[],"parent_id":45752186,"points":null,"story_id":45748661,"text":"Voting seems like one of the few problems that blockchain is actually the solution for.","title":null,"type":"comment","url":null},{"author":"wodenokoto","children":[],"created_at":"2025-10-30T04:43:32.000Z","created_at_i":1761799412,"id":45756512,"options":[],"parent_id":45752186,"points":null,"story_id":45748661,"text":"We do mail voting from embassies or consulates when abroad.","title":null,"type":"comment","url":null},{"author":"w3ll_w3ll_w3ll","children":[],"created_at":"2025-10-30T06:55:43.000Z","created_at_i":1761807343,"id":45757110,"options":[],"parent_id":45752186,"points":null,"story_id":45748661,"text":"In Italy we typically vote for two days, usually Sunday and Monday or Saturday and Sunday.","title":null,"type":"comment","url":null},{"author":"chrisandchris","children":[],"created_at":"2025-10-30T14:22:07.000Z","created_at_i":1761834127,"id":45760323,"options":[],"parent_id":45752186,"points":null,"story_id":45748661,"text":"Europe, expect in the middle of it in Switzerland where at least I know nobody that actually goes to the voting place. We do it by mail.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:55:08.000Z","created_at_i":1761767708,"id":45752186,"options":[],"parent_id":45752108,"points":null,"story_id":45748661,"text":"In europe, voting typically happens in one day, where everyone physically goes to their designated voting place and puts papers in a transparent box. You can stay there and wait for the count at the end of the day if you want to. Tom Scott has a very good video about why we don't want electronic/mail voting: https://www.youtube.com/watch?v=w3_0x6oaDmI","title":null,"type":"comment","url":null},{"author":"mc32","children":[{"author":"20kleagues","children":[],"created_at":"2025-10-29T20:32:24.000Z","created_at_i":1761769944,"id":45752651,"options":[],"parent_id":45752519,"points":null,"story_id":45748661,"text":"Voting in India is staggered over multiple phases over multiple days/weeks. Only the vote count happens on a single day at the end.","title":null,"type":"comment","url":null},{"author":"sampo","children":[{"author":"Freedom2","children":[{"author":"stevenwoo","children":[{"author":"Freedom2","children":[],"created_at":"2025-10-30T04:11:22.000Z","created_at_i":1761797482,"id":45756348,"options":[],"parent_id":45755772,"points":null,"story_id":45748661,"text":"That makes it difficult, but the original comment said it wasn't 'possible'. I'm failing to see the impossibility still.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T02:27:37.000Z","created_at_i":1761791257,"id":45755772,"options":[],"parent_id":45753664,"points":null,"story_id":45748661,"text":"It takes a lot of people (redundancy and to keep shift hours low to increase count accuracy) to accurately count by hand. https://verifiedvoting.org/election-system/hand-counted-pape...","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:02:27.000Z","created_at_i":1761775347,"id":45753664,"options":[],"parent_id":45753431,"points":null,"story_id":45748661,"text":"How is it not possible? It's just additional votes, there isn't anything actually stopping counting by hand, is there? How was it counted historically without voting machines?","title":null,"type":"comment","url":null},{"author":"konimex","children":[],"created_at":"2025-10-30T00:46:08.000Z","created_at_i":1761785168,"id":45755081,"options":[],"parent_id":45753431,"points":null,"story_id":45748661,"text":"Say, how many voting stations are there in a typical city/county in the US?
Here in Indonesia, in a city of 2 million people there are over 7000 voting stations. While we vote for 5 ballots (President, Legislative (National, Province, and City/Regency), we still use paper ballots and count them by hand.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:40:31.000Z","created_at_i":1761774031,"id":45753431,"options":[],"parent_id":45752519,"points":null,"story_id":45748661,"text":"> If India can have voters vote and tally all the votes in one day, then so can everyone else.
In most countries, in the elections you vote or the member of parliament you want. Presidential elections, and city council elections are held separately, but are also equally simple. But in one election you cast your vote for one person, and that's it.
With this kind of elections, many countries manage to hold the elections on paper ballots, count them all by hand, and publish results by midnight.
But on an American ballot, you vote for, for example:
- US president\n - US senator \n - US member of congress \n - state governor \n - state senator \n - state member of congress \n - several votes for several different state judge positions \n - several other state officer positions \n - several votes for several local county officers \n - local sheriff \n - local school board member \n - several yes/no votes for several proposed laws, whether they should be passed or not\nhttps://ballotpedia.org/Official_sample_ballots,_2020","title":null,"type":"comment","url":null},{"author":"platevoltage","children":[{"author":"trenchpilgrim","children":[],"created_at":"2025-10-30T06:00:57.000Z","created_at_i":1761804057,"id":45756836,"options":[],"parent_id":45754458,"points":null,"story_id":45748661,"text":"In particular India has a law that no one shall be made to walk more than 2km to vote. The Indian military will literally deploy a voting booth into the jungle so that a single caretaker of an old temple can vote.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:25:04.000Z","created_at_i":1761780304,"id":45754458,"options":[],"parent_id":45752519,"points":null,"story_id":45748661,"text":"If it's not a national holiday where the vast majority of people don't have to work, and if there aren't polling places reasonably near every voting age citizen, it's voter suppression.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:20:37.000Z","created_at_i":1761769237,"id":45752519,"options":[],"parent_id":45752108,"points":null,"story_id":45748661,"text":"If India can have voters vote and tally all the votes in one day, then so can everyone else. It\u2019s the best way to avoid fraud and people going with whoever is ahead. I am sympathetic with emergency protocols for deadly pandemics, but for all else, in-person on a given day.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:49:31.000Z","created_at_i":1761767371,"id":45752108,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"Is voting there a one day only event? If not, I feel the solution to that particular problem is quite clear. There\u2019s a million things that could go wrong causing you to miss something when you try to do it in a narrow time range (today after work before polls close)
If it\u2019s a multi day event, it\u2019s probably that way for a reason. Partially the same as the solution to above.","title":null,"type":"comment","url":null},{"author":"barrenko","children":[{"author":"gowld","children":[{"author":"barrenko","children":[],"created_at":"2025-10-29T22:19:59.000Z","created_at_i":1761776399,"id":45753836,"options":[],"parent_id":45753455,"points":null,"story_id":45748661,"text":"Nope :)","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:42:28.000Z","created_at_i":1761774148,"id":45753455,"options":[],"parent_id":45752132,"points":null,"story_id":45748661,"text":"Why would you put Microsoft above this?","title":null,"type":"comment","url":null},{"author":"platevoltage","children":[],"created_at":"2025-10-29T23:26:52.000Z","created_at_i":1761780412,"id":45754479,"options":[],"parent_id":45752132,"points":null,"story_id":45748661,"text":"I wouldn't put Texas above this either.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:51:18.000Z","created_at_i":1761767478,"id":45752132,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"> wouldn't put China or Russia above this","title":null,"type":"comment","url":null},{"author":"alt227","children":[{"author":"onionisafruit","children":[{"author":"dijit","children":[],"created_at":"2025-10-30T01:31:34.000Z","created_at_i":1761787894,"id":45755406,"options":[],"parent_id":45753412,"points":null,"story_id":45748661,"text":"yeah, those are the two options. and before the cloud there was no transportation possible except via foot.
Horses were famously tamed in 2007 after AWS released S3 to the public, this is the best of times.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:39:28.000Z","created_at_i":1761773968,"id":45753412,"options":[],"parent_id":45752137,"points":null,"story_id":45748661,"text":"It is a great age to live in where we have transportation infrastructure beyond foot paths.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:51:36.000Z","created_at_i":1761767496,"id":45752137,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"Wow thats crazy! National transport infrastructure being so fragile. What a great age we live in.","title":null,"type":"comment","url":null},{"author":"hinkley","children":[{"author":"Archelaos","children":[{"author":"Hamuko","children":[],"created_at":"2025-10-29T21:32:27.000Z","created_at_i":1761773547,"id":45753334,"options":[],"parent_id":45752750,"points":null,"story_id":45748661,"text":"Same in Finland. And even if you work Sundays, there's a week's worth of early voting so you can take your pick.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:42:49.000Z","created_at_i":1761770569,"id":45752750,"options":[],"parent_id":45752336,"points":null,"story_id":45748661,"text":"In Germany it is always a Sunday.","title":null,"type":"comment","url":null},{"author":"tmtvl","children":[{"author":"brendoelfrendo","children":[],"created_at":"2025-10-29T21:09:00.000Z","created_at_i":1761772140,"id":45753064,"options":[],"parent_id":45752918,"points":null,"story_id":45748661,"text":"In the US, where I assume a lot of the griping comes from, election day is not a national holiday, nor is it on a weekend (in fact, by law it is defined as "the Tuesday next after the first Monday in November"), and even though it is acknowledged as an important civic duty, only about half of the states have laws on the books that require employers provide time off to vote. There are no federal laws to that effect, so it's left entirely to states to decide.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:57:21.000Z","created_at_i":1761771441,"id":45752918,"options":[],"parent_id":45752336,"points":null,"story_id":45748661,"text":"Here in Belgium voting is usually done during the weekend, although it shouldn't matter because voting is a civic duty (unless you have a good reason you have to go vote or you'll be fined), so those who work during the weekend have a valid reason to come in late or leave early.","title":null,"type":"comment","url":null},{"author":"hshdhdhehd","children":[{"author":"wanderingmind","children":[],"created_at":"2025-10-30T03:15:57.000Z","created_at_i":1761794157,"id":45756068,"options":[],"parent_id":45753172,"points":null,"story_id":45748661,"text":"From mandatory voting to preferential voting, australia seems to have figured out what this works best for democracy.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:18:35.000Z","created_at_i":1761772715,"id":45753172,"options":[],"parent_id":45752336,"points":null,"story_id":45748661,"text":"In Australia there are so many places to vote, it is almost popping out to get milk level if convenience. Just detour your dog walk slightly. Always at the weekend.","title":null,"type":"comment","url":null},{"author":"hshdhdhehd","children":[{"author":"dullcrisp","children":[{"author":"hshdhdhehd","children":[{"author":"dullcrisp","children":[{"author":"hshdhdhehd","children":[{"author":"dullcrisp","children":[],"created_at":"2025-10-30T21:53:31.000Z","created_at_i":1761861211,"id":45765832,"options":[],"parent_id":45765758,"points":null,"story_id":45748661,"text":"No it is strange.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T21:45:49.000Z","created_at_i":1761860749,"id":45765758,"options":[],"parent_id":45755441,"points":null,"story_id":45748661,"text":"America seems so strange from outside... but Im sure that applies in reverse.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T01:35:36.000Z","created_at_i":1761788136,"id":45755441,"options":[],"parent_id":45755399,"points":null,"story_id":45748661,"text":"There are gas stations but I\u2019m not sure I\u2019d trust the milk there.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T01:30:31.000Z","created_at_i":1761787831,"id":45755399,"options":[],"parent_id":45754512,"points":null,"story_id":45748661,"text":"No convenience stores?","title":null,"type":"comment","url":null},{"author":"johnfn","children":[],"created_at":"2025-10-30T01:41:07.000Z","created_at_i":1761788467,"id":45755480,"options":[],"parent_id":45754512,"points":null,"story_id":45748661,"text":"Yes, or you do it on your drive back from work, and it takes 3 minutes.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:31:58.000Z","created_at_i":1761780718,"id":45754512,"options":[],"parent_id":45753183,"points":null,"story_id":45748661,"text":"In the US getting milk involves driving multiple miles, finding parking, walking to the store, finding a shopping cart, finding the grocery department, navigating the aisles to the dairy section, finding the milk, waiting in line to check out, returning the cart if you\u2019re courteous, and driving back. Could take an hour or so.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:19:12.000Z","created_at_i":1761772752,"id":45753183,"options":[],"parent_id":45752336,"points":null,"story_id":45748661,"text":"In Australia there are so many places to vote, it is almost popping out to get milk level if convenience. (At least in urbia and suburbia) Just detour your dog walk slightly. Always at the weekend.","title":null,"type":"comment","url":null},{"author":"sleepybrett","children":[{"author":"hinkley","children":[],"created_at":"2025-10-30T00:58:34.000Z","created_at_i":1761785914,"id":45755166,"options":[],"parent_id":45754842,"points":null,"story_id":45748661,"text":"You have a mail-in voting system... for now.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T00:17:17.000Z","created_at_i":1761783437,"id":45754842,"options":[],"parent_id":45752336,"points":null,"story_id":45748661,"text":"In washington we have a 100% mail-in voting system (for all intents and purposes). I can put my ballot back in the mail or drop at any number of drop-boxes throughout the city (less dropboxes in rural areas i'm sure). I think there are some allowances for in-person voting but I don't think they are often used.
There is a ballot tracking system as well, I can see and be notified as my ballot moves through the counting system. It's pretty cool.
I actually just got back from dropping off my local elections ballot 15m ago, quick bike trip maybe a mile or so away and back.
Of course, because it makes it easy for people to vote, the republicans want to do away with it. If you have to stand in line for several hours (which seems to be very normal in most cities) and potentially miss work to do it that's going to all but guarantee that working people and the less motivated will not vote.
So yes in places that only do in person voting, national or state holiday.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:05:30.000Z","created_at_i":1761768330,"id":45752336,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"Voting days should be a national holiday.","title":null,"type":"comment","url":null},{"author":"j45","children":[{"author":"esseph","children":[{"author":"j45","children":[],"created_at":"2025-11-01T15:18:44.000Z","created_at_i":1762010324,"id":45782328,"options":[],"parent_id":45754144,"points":null,"story_id":45748661,"text":"Especially today with competing tools taking turns leading or new ones popping up.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T22:53:22.000Z","created_at_i":1761778402,"id":45754144,"options":[],"parent_id":45752483,"points":null,"story_id":45748661,"text":"Hybrid was always the way. Use different tools to solve different problems.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:17:10.000Z","created_at_i":1761769030,"id":45752483,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"Organizations who had their own datacenters were chided for being resistant to modernizing, and now they modernized to use someone else's shared computers and they stopped working.
I really do feel the only viable future for clouds is hybrid or agnostic clouds.","title":null,"type":"comment","url":null},{"author":"tmtvl","children":[],"created_at":"2025-10-29T20:54:36.000Z","created_at_i":1761771276,"id":45752883,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"The Flemish bus company (de Lijn) uses Azure and I couldn't activate my ticket when I came home after training a couple of hours ago. I should probably start using physical tickets again, because at least those work properly. It's just stupid that there's so much stuff being moved to digital only (often even only being accessible through an Android or iOS app, despite the parent companies of those two being utterly atrocious) when the physical alternatives are more reliable.","title":null,"type":"comment","url":null},{"author":"stefs","children":[{"author":"myself248","children":[],"created_at":"2025-10-29T21:42:05.000Z","created_at_i":1761774125,"id":45753450,"options":[],"parent_id":45753430,"points":null,"story_id":45748661,"text":"How ever did buses run before The Cloud\u2122? What a weird world that must have been.","title":null,"type":"comment","url":null},{"author":"dotancohen","children":[{"author":"baby_souffle","children":[{"author":"Tostino","children":[{"author":"dotancohen","children":[{"author":"Tostino","children":[{"author":"dotancohen","children":[],"created_at":"2025-11-01T10:41:08.000Z","created_at_i":1761993668,"id":45780637,"options":[],"parent_id":45767127,"points":null,"story_id":45748661,"text":"Yes, for some use cases proxying is fine. For larger resources or heavily loaded servers (servers I don't want to scale) I prefer to avoid it.
Thank you.","title":null,"type":"comment","url":null}],"created_at":"2025-10-31T00:39:06.000Z","created_at_i":1761871146,"id":45767127,"options":[],"parent_id":45764866,"points":null,"story_id":45748661,"text":"The way I solved auth in my case was just proxying everything through my backend and having that do the auth. I have my own URL scheme and the users never see the URL for the file in S3.
Another way you can do it is generating pre-signed URLs in your backend on each request to download something... but the URL that is generated when you do that is only valid for some small time period, so not a stable URL at all.
In my use case, I needed stable URLs, so I went the proxy route.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T20:21:28.000Z","created_at_i":1761855688,"id":45764866,"options":[],"parent_id":45760002,"points":null,"story_id":45748661,"text":"Even on non-AWS projects, I still use S3. I haven't really explored the other options, but if you have opinions or advice I'd love to hear them.
One thing very important, is that I can authorise specific web clients (users) to access specific resources from S3. Such as a document that he can download, but others with the link should not be able to download.
Thank you!","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T13:47:49.000Z","created_at_i":1761832069,"id":45760002,"options":[],"parent_id":45754342,"points":null,"story_id":45748661,"text":"There are plenty of compatible S3-like offerings. That's one of the lesser things that tie me to a cloud.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:12:43.000Z","created_at_i":1761779563,"id":45754342,"options":[],"parent_id":45754302,"points":null,"story_id":45748661,"text":"It's absolutely doable if you design for it.
The moment you choose to use S3 instead of hosting your own object store, though, you either use AWS because S3 and IAM already have you or spend more time on the care and feeding of your storage system as opposed to actually doing the thing you customers are paying you to do.
It's not impossible, just complicated and difficult for any moderately complex architecture.","title":null,"type":"comment","url":null},{"author":"zharknado","children":[],"created_at":"2025-10-30T06:00:55.000Z","created_at_i":1761804055,"id":45756835,"options":[],"parent_id":45754302,"points":null,"story_id":45748661,"text":"\u201cprecipitates\u201d ha! Wonderfully evocative.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T23:09:12.000Z","created_at_i":1761779352,"id":45754302,"options":[],"parent_id":45753430,"points":null,"story_id":45748661,"text":"I run non trivial services on EC2, using that service as a VPS. My deploy script works just as well on provisioned Digital Ocean services and on docker containers using docker-compose.
I do need a human to provision a few servers and configure e.g. load balancing and when to spin up additional servers under load. But that is far less of a PITA than having my systems tied to a specific provider or down whenever a cloud precipitates.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T21:40:30.000Z","created_at_i":1761774030,"id":45753430,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"i'm not sure this is an easily solvable problem. i remember reading an article arguing that your cloud provider is part of your tech stack and it's close to impossible/a huge PITA to make a non-trivial service provider-agnostic. they'd have to run their own openstack in different datacenters, which would be costly and have their own points of failure.","title":null,"type":"comment","url":null},{"author":"bironran","children":[],"created_at":"2025-10-29T21:57:56.000Z","created_at_i":1761775076,"id":45753620,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"Yet... deploy on two clouds and you'll get tax payers scream at you for "wasting money" preparing for a black swan event. Can't have both, either reliability or lower cost.","title":null,"type":"comment","url":null},{"author":"alliao","children":[],"created_at":"2025-10-29T22:43:13.000Z","created_at_i":1761777793,"id":45754059,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"dang even zealand didn't survive! new zealand got some soul searching with this outage which took down government person ID service, it's called RealME and it can be used to file your taxes apply for passport etc","title":null,"type":"comment","url":null},{"author":"fHr","children":[{"author":"qrios","children":[],"created_at":"2025-10-30T03:49:26.000Z","created_at_i":1761796166,"id":45756238,"options":[],"parent_id":45754954,"points":null,"story_id":45748661,"text":"Is that you? The same guy with the comment "hahahhahaha"[1] on "Women dating safety app 'Tea' breached, users' IDs posted to 4chan"[2]?
[1] https://news.ycombinator.com/item?id=44689366
[2] https://news.ycombinator.com/item?id=44684373","title":null,"type":"comment","url":null},{"author":"vanviegen","children":[],"created_at":"2025-10-30T09:27:18.000Z","created_at_i":1761816438,"id":45757967,"options":[],"parent_id":45754954,"points":null,"story_id":45748661,"text":"When voting remotely, how can we be sure that your vote was not bought or coerced?","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T00:29:21.000Z","created_at_i":1761784161,"id":45754954,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"can't believe it's 2025 and some still need to go to some place to vote. I can vote since I can remember(at least 20 years) by mail for anything, we also vote multiple times a year(4-6 times), we just get 1 Month before the things to vote by mail and then mail in back votes. Hope we can soon vote online to get rid of the paper overhead.","title":null,"type":"comment","url":null},{"author":"varispeed","children":[],"created_at":"2025-10-30T00:46:50.000Z","created_at_i":1761785210,"id":45755087,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"Wasn't cloud sold based on a premise to prevent the very thing that is happening? Sounds like a massive fail of the whole concept.","title":null,"type":"comment","url":null},{"author":"bethekidyouwant","children":[{"author":"vachina","children":[{"author":"fylo","children":[],"created_at":"2025-10-30T07:10:50.000Z","created_at_i":1761808250,"id":45757194,"options":[],"parent_id":45756428,"points":null,"story_id":45748661,"text":"How?","title":null,"type":"comment","url":null},{"author":"isbvhodnvemrwvn","children":[],"created_at":"2025-10-30T07:16:40.000Z","created_at_i":1761808600,"id":45757238,"options":[],"parent_id":45756428,"points":null,"story_id":45748661,"text":"No you don't, lol.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T04:27:21.000Z","created_at_i":1761798441,"id":45756428,"options":[],"parent_id":45756082,"points":null,"story_id":45748661,"text":"At least I get to control when the 0.01 happens.","title":null,"type":"comment","url":null},{"author":"bombcar","children":[],"created_at":"2025-10-30T09:30:12.000Z","created_at_i":1761816612,"id":45757981,"options":[],"parent_id":45756082,"points":null,"story_id":45748661,"text":"You can get way more 9s if you roll your own procedures - and each step has a way of handling failure of the other steps.
Old trains had paper tickets, the locomotive was its own power source, the conductor had a flashlight, and the conductor could sell tickets for cash.
And if everything else failed, the conductor would just let you ride for free.
Now everything's so interconnected that any one part failing brings everything to a halt.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T03:19:18.000Z","created_at_i":1761794358,"id":45756082,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"You are not getting more 9\u2019s rolling your own","title":null,"type":"comment","url":null},{"author":"Mr_Minderbinder","children":[],"created_at":"2025-10-30T07:59:14.000Z","created_at_i":1761811154,"id":45757452,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"Finance is increasingly reliant on it too, my bank moved their entire system to AWS. The amount of power being handed over to these cloud companies in exchange for \u201cconvenience\u201d is astonishing.","title":null,"type":"comment","url":null},{"author":"tuukkah","children":[{"author":"foresterre","children":[{"author":"tuukkah","children":[],"created_at":"2025-10-30T10:29:26.000Z","created_at_i":1761820166,"id":45758360,"options":[],"parent_id":45757952,"points":null,"story_id":45748661,"text":"Thanks! Perhaps one of the sites that track train delays can give a statistic?
This list doesn't have anything that looks relevant: https://www.rijdendetreinen.nl/en/disruptions/archive?date_b...
The day does not appear as an outlier in the monthly statistics: https://www.rijdendetreinen.nl/en/statistics/2025/10
I don't find a detailed statistic on the overall delays, but the per-station statistics for Amsterdam Centraal say 5% of trains were cancelled and 17% were delayed by 5 minutes or more (mostly by 10 minutes): https://www.rijdendetreinen.nl/en/train-archive/2025-10-29/a...","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T09:24:00.000Z","created_at_i":1761816240,"id":45757952,"options":[],"parent_id":45757777,"points":null,"story_id":45748661,"text":"Here is an article in English:
https://nltimes.nl/2025/10/29/ns-hit-microsoft-cloud-outage-...
It should be noted that the article isn't complete: while the travel planner and ticket machines were the first to fail, trains were cancelled soon after; it took a few hours before everything restarted.
Based on what the conductors said, I would speculate that the train drivers digital schedule was not operative, so they didn't know where to go next.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T08:56:08.000Z","created_at_i":1761814568,"id":45757777,"options":[],"parent_id":45752029,"points":null,"story_id":45748661,"text":"Could you provide a reference about the train disruption? I tried but couldn't find anything in English.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:42:27.000Z","created_at_i":1761766947,"id":45752029,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"It still surprises me how much essential services like public transport are completely reliant on cloud providers, and don't seem to have backups in place.
Here in The Netherlands, almost all trains were first delayed significantly, and then cancelled for a few hours because of this, which had real impact because today is also the day we got to vote for the next parlement (I know some who can't get home in time before the polls close, and they left for work before they opened).","title":null,"type":"comment","url":null},{"author":"bragma","children":[],"created_at":"2025-10-29T19:51:03.000Z","created_at_i":1761767463,"id":45752128,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"They suggest to use Traffic Manager to route around failing CDNs. But DNS is not working too, making the suggestion another fail.","title":null,"type":"comment","url":null},{"author":"bragma","children":[{"author":"asciii","children":[],"created_at":"2025-10-29T19:56:38.000Z","created_at_i":1761767798,"id":45752205,"options":[],"parent_id":45752140,"points":null,"story_id":45748661,"text":"Yeah they're suggesting to use CLI but then my Frontdoor deployment failed. Welp.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T19:51:52.000Z","created_at_i":1761767512,"id":45752140,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"They suggest to use Traffic Manager to router around failing FrontDoor CDN, but DNS is failing too, making the suggestion another failure.","title":null,"type":"comment","url":null},{"author":"rodolphoarruda","children":[],"created_at":"2025-10-29T20:03:03.000Z","created_at_i":1761768183,"id":45752296,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I could not access MS Clarity the entire day.","title":null,"type":"comment","url":null},{"author":"Shuddown","children":[],"created_at":"2025-10-29T20:03:17.000Z","created_at_i":1761768197,"id":45752300,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Github Codespaces (for the 5 people that use them) are also still down.","title":null,"type":"comment","url":null},{"author":"_pdp_","children":[],"created_at":"2025-10-29T20:10:03.000Z","created_at_i":1761768603,"id":45752393,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"With all the recent outages considered, it is time to move off the cloud.","title":null,"type":"comment","url":null},{"author":"seinecle","children":[],"created_at":"2025-10-29T20:11:38.000Z","created_at_i":1761768698,"id":45752408,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Can't connect to Claude","title":null,"type":"comment","url":null},{"author":"tonymet","children":[],"created_at":"2025-10-29T20:16:40.000Z","created_at_i":1761769000,"id":45752476,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Hello fellow boomers!
I noticed that winget is also down eg.
winget upgrade fabric\n Failed in attempting to update the source: winget\n An unexpected error occurred while executing the command:\n InternetOpenUrl() failed.\n 0x80072ee7 : unknown errorInstitutional knowledge matters. Just has to be the right institution is all.","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T20:23:09.000Z","created_at_i":1761769389,"id":45752545,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Based on the delay in resolving the issue, it appears MC attempted to rehire some of the DevOps engineers whom AI had previously replaced.","title":null,"type":"comment","url":null},{"author":"m_a_g","children":[],"created_at":"2025-10-29T20:27:15.000Z","created_at_i":1761769635,"id":45752597,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"It\u2019s not DNS
There is no way it\u2019s DNS
It was DNS","title":null,"type":"comment","url":null},{"author":"eeasss","children":[],"created_at":"2025-10-29T20:45:43.000Z","created_at_i":1761770743,"id":45752782,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Deglobalization in geopolitics should be followed by deglobalization in cloud providers as well. Viva la local vendors.","title":null,"type":"comment","url":null},{"author":"senderista","children":[],"created_at":"2025-10-29T21:13:30.000Z","created_at_i":1761772410,"id":45753120,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Even if the cloud providers have much better reliability than most on-prem infra, the failure correlation they induce negates much of the benefit.","title":null,"type":"comment","url":null},{"author":"progmetaldev","children":[],"created_at":"2025-10-29T21:37:00.000Z","created_at_i":1761773820,"id":45753383,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"I was having issues a few hours ago. I'm now able to access the portal, although I get lots of errors in the browser console, and things are loading slowly. I have services in the US-East region.
I have been having issues with GitHub and the winget tool for updates throughout the day as well. I imagine things are pulling from the same locations on Azure for some of the software I needed to update (NPM dependencies, and some .NET tooling).","title":null,"type":"comment","url":null},{"author":"tonymet","children":[],"created_at":"2025-10-29T21:38:07.000Z","created_at_i":1761773887,"id":45753397,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Any healthcare IT admins care to chime in? A predominantly MS industry with critical workloads.","title":null,"type":"comment","url":null},{"author":"acd","children":[],"created_at":"2025-10-29T21:47:44.000Z","created_at_i":1761774464,"id":45753511,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Putting all your eggs software in one basket","title":null,"type":"comment","url":null},{"author":"zbowling","children":[],"created_at":"2025-10-29T22:10:47.000Z","created_at_i":1761775847,"id":45753748,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Alaska Airlines is redircting folks to their slimmed down international site and you can't check in on mobile.","title":null,"type":"comment","url":null},{"author":"ycombinatornews","children":[],"created_at":"2025-10-29T22:11:30.000Z","created_at_i":1761775890,"id":45753757,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"So that\u2019s why CapitalOne is out today. Even though their (incorrect) status page says all systems operational.","title":null,"type":"comment","url":null},{"author":"journal","children":[],"created_at":"2025-10-29T22:41:58.000Z","created_at_i":1761777718,"id":45754043,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"one day these outages will cause a starvation.","title":null,"type":"comment","url":null},{"author":"udfalkso","children":[],"created_at":"2025-10-29T22:53:40.000Z","created_at_i":1761778420,"id":45754148,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"OpenAI Clip python library fails because the model download is a hardcoded azure cdn url :(","title":null,"type":"comment","url":null},{"author":"ChuckMcM","children":[],"created_at":"2025-10-29T23:38:26.000Z","created_at_i":1761781106,"id":45754559,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":""On Prem" is looking better and better :-).","title":null,"type":"comment","url":null},{"author":"jasonthorsness","children":[],"created_at":"2025-10-30T00:55:29.000Z","created_at_i":1761785729,"id":45755140,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Ahh it got me, Alaska air web site has an Azure outage banner","title":null,"type":"comment","url":null},{"author":"jmspring","children":[{"author":"bombcar","children":[],"created_at":"2025-10-30T09:43:11.000Z","created_at_i":1761817391,"id":45758062,"options":[],"parent_id":45756079,"points":null,"story_id":45748661,"text":"I hate these failures because you end up with things that keep working fine because the login credentials are cached, etc; but if you restart or otherwise refresh, you're doomed.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T03:18:50.000Z","created_at_i":1761794330,"id":45756079,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"The outage was really weird. For me, parts of the portal worked, other parts didn't. I had access to a couple of resource groups, but no resources visible in those groups. Azure Devops Pipelines that needed do download from packages.microsoft.com didn't work.
The Microsoft status page mostly referenced the portal outage, but it was more than that.","title":null,"type":"comment","url":null},{"author":"xer0x","children":[{"author":"croemer","children":[],"created_at":"2025-10-30T05:48:19.000Z","created_at_i":1761803299,"id":45756781,"options":[],"parent_id":45756325,"points":null,"story_id":45748661,"text":"Not officially - status page says all healthy","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T04:06:54.000Z","created_at_i":1761797214,"id":45756325,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Wow, they are still down 12 hours later. :/","title":null,"type":"comment","url":null},{"author":"nextworddev","children":[],"created_at":"2025-10-30T04:17:35.000Z","created_at_i":1761797855,"id":45756389,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Fascinating timing given the APEC summit ;)","title":null,"type":"comment","url":null},{"author":"croemer","children":[{"author":"onionisafruit","children":[{"author":"not_a_bot_4sho","children":[{"author":"onionisafruit","children":[],"created_at":"2025-10-30T11:53:43.000Z","created_at_i":1761825223,"id":45759032,"options":[],"parent_id":45756937,"points":null,"story_id":45748661,"text":"I think you\u2019re right. I missed that subtlety on first reading.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T06:20:44.000Z","created_at_i":1761805244,"id":45756937,"options":[],"parent_id":45756843,"points":null,"story_id":45748661,"text":"I read it as the second investigation being specific to AFD. The first more general.","title":null,"type":"comment","url":null},{"author":"ssss11","children":[{"author":"ozim","children":[{"author":"normie3000","children":[],"created_at":"2025-10-30T11:02:18.000Z","created_at_i":1761822138,"id":45758603,"options":[],"parent_id":45757567,"points":null,"story_id":45748661,"text":"One crash after another","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T08:17:59.000Z","created_at_i":1761812279,"id":45757567,"options":[],"parent_id":45757493,"points":null,"story_id":45748661,"text":"Load some carbs with chocolate chip cookies as well, that\u2019s what I would do.
You don\u2019t want to debug stuff with low sugar.","title":null,"type":"comment","url":null},{"author":"red-iron-pine","children":[],"created_at":"2025-10-30T12:59:55.000Z","created_at_i":1761829195,"id":45759563,"options":[],"parent_id":45757493,"points":null,"story_id":45748661,"text":"burn a smoko and take a leak","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T08:06:44.000Z","created_at_i":1761811604,"id":45757493,"options":[],"parent_id":45756843,"points":null,"story_id":45748661,"text":"Quick coffee run before we get stuck in mate","title":null,"type":"comment","url":null},{"author":"taco_emoji","children":[],"created_at":"2025-10-30T16:56:02.000Z","created_at_i":1761843362,"id":45762197,"options":[],"parent_id":45756843,"points":null,"story_id":45748661,"text":"
16:04 Started running around screaming\n 16:15 Sat down & looked at logsMe: "How do I connect [X] to [Y] using [Z]?"
Copilot: "Please select the AKS cluster you'd like to delete"","title":null,"type":"comment","url":null},{"author":"nflekkhnnn","children":[],"created_at":"2025-10-30T10:42:43.000Z","created_at_i":1761820963,"id":45758442,"options":[],"parent_id":45757992,"points":null,"story_id":45748661,"text":"Actually one of the inventors of k8s was the project lead for copilot in the azure portal, and deployed it over a year ago.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T09:32:10.000Z","created_at_i":1761816730,"id":45757992,"options":[],"parent_id":45757614,"points":null,"story_id":45748661,"text":"Save it for when they stick Copilot into Azure portal.","title":null,"type":"comment","url":null},{"author":"dijit","children":[],"created_at":"2025-10-30T09:34:26.000Z","created_at_i":1761816866,"id":45758007,"options":[],"parent_id":45757614,"points":null,"story_id":45748661,"text":"Not to put too fine a point on it, but if I have a dark passenger in my tech life it is almost entirely caused by what Microsoft wants to inflict on humanity - and more importantly; how successful they are at doing it.","title":null,"type":"comment","url":null},{"author":"amelius","children":[],"created_at":"2025-10-30T10:41:52.000Z","created_at_i":1761820912,"id":45758437,"options":[],"parent_id":45757614,"points":null,"story_id":45748661,"text":"In commenter's defense, their comment makes no sense.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T08:26:22.000Z","created_at_i":1761812782,"id":45757614,"options":[],"parent_id":45757500,"points":null,"story_id":45748661,"text":"If you call that defending microsoft, I'd hate to see what attacking them looks like :)","title":null,"type":"comment","url":null},{"author":"madjam002","children":[{"author":"macintux","children":[{"author":"Hilift","children":[],"created_at":"2025-10-30T09:48:43.000Z","created_at_i":1761817723,"id":45758102,"options":[],"parent_id":45758039,"points":null,"story_id":45748661,"text":"Ironically, the GitHub Desktop Windows app is quite nice.","title":null,"type":"comment","url":null},{"author":"dspillett","children":[],"created_at":"2025-10-30T10:30:30.000Z","created_at_i":1761820230,"id":45758365,"options":[],"parent_id":45758039,"points":null,"story_id":45748661,"text":"Yes. But the point is compared to Azure in places the statement was very much the pot commenting on the kettles sooty arse. And git makes no particular pretence to be particularly friendly, just that it does a particular job efficiently.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T09:39:45.000Z","created_at_i":1761817185,"id":45758039,"options":[],"parent_id":45758021,"points":null,"story_id":45748661,"text":"Isn\u2019t it?","title":null,"type":"comment","url":null},{"author":"lawgimenez","children":[{"author":"ac2u","children":[{"author":"lawgimenez","children":[],"created_at":"2025-10-31T01:07:20.000Z","created_at_i":1761872840,"id":45767289,"options":[],"parent_id":45759157,"points":null,"story_id":45748661,"text":"Yeah Windows Phone's first releases were decent. I have developed apps for Windows actually using Window's UWP framework but there weren't enough users on their platform sadly.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T12:12:49.000Z","created_at_i":1761826369,"id":45759157,"options":[],"parent_id":45758863,"points":null,"story_id":45748661,"text":"Feel like I have to defend windows phone here, I liked it! Although I swore off the platform after the hardware I bought wasn\u2019t eligible for the windows phone 8 upgrade even though the hardware was less than two years old. They punished early adopters","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T11:33:02.000Z","created_at_i":1761823982,"id":45758863,"options":[],"parent_id":45758021,"points":null,"story_id":45748661,"text":"Sounds like he\u2019s describing Windows phone.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T09:36:10.000Z","created_at_i":1761816970,"id":45758021,"options":[],"parent_id":45757500,"points":null,"story_id":45748661,"text":"My favourite was the Azure CTO complaining that Git was unintuitive, clunky and difficult to use","title":null,"type":"comment","url":null},{"author":"sfn42","children":[{"author":"voidfunc","children":[{"author":"ukblewis","children":[],"created_at":"2025-10-30T17:00:12.000Z","created_at_i":1761843612,"id":45762253,"options":[],"parent_id":45758868,"points":null,"story_id":45748661,"text":"Some are much worse than others\u2026","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T11:33:32.000Z","created_at_i":1761824012,"id":45758868,"options":[],"parent_id":45758068,"points":null,"story_id":45748661,"text":"They are all broken, weird, and expensive in their own ways. Its nothing unique to Azure.","title":null,"type":"comment","url":null},{"author":"lokar","children":[{"author":"sfn42","children":[{"author":"lokar","children":[],"created_at":"2025-10-30T22:00:55.000Z","created_at_i":1761861655,"id":45765907,"options":[],"parent_id":45762298,"points":null,"story_id":45748661,"text":"Pretty much anything","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T17:04:11.000Z","created_at_i":1761843851,"id":45762298,"options":[],"parent_id":45761461,"points":null,"story_id":45748661,"text":"Worse at what?","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T15:56:02.000Z","created_at_i":1761839762,"id":45761461,"options":[],"parent_id":45758068,"points":null,"story_id":45748661,"text":"For something fairly small, they are about the same.
At a large scale, azure is dramatically worse then Aws.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T09:44:21.000Z","created_at_i":1761817461,"id":45758068,"options":[],"parent_id":45757500,"points":null,"story_id":45748661,"text":"I've only used Azure, to me it seems fine ish. Some things are rather overcomplicated and it's far from perfect but I assumed the other providers were similarly complicated and imperfect.
Can't say I've experienced many bugs in there either. It definitely is overpriced but I assume they all are?","title":null,"type":"comment","url":null},{"author":"sofixa","children":[{"author":"hinkley","children":[],"created_at":"2025-10-30T22:05:37.000Z","created_at_i":1761861937,"id":45765945,"options":[],"parent_id":45758904,"points":null,"story_id":45748661,"text":"Given how much time I spent on my first real multi-tenant project, dealing with the consequences of architecture decisions meant to prevent these sorts of issues, I can see clearly the temptation to avoid dealing with them.
But what we do when things are easy is not who we are. That's a fiction. It's how we show up when we are in the shit that matters. It's discipline that tells you to voluntarily go into all of the multi-tenant mitigations instead of waiting for your boss to notice and move the goalposts you should have moved on your own.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T11:37:54.000Z","created_at_i":1761824274,"id":45758904,"options":[],"parent_id":45757500,"points":null,"story_id":45748661,"text":"> In Microsoft's defense, Azure has always been a complete joke. It's extremely developer unfriendly, buggy and overpriced.
Don't forget extremely insecure. There is a quarterly critical cross-tenant CVE with trivial exploitation for them, and it has been like that for years.","title":null,"type":"comment","url":null},{"author":"rk06","children":[],"created_at":"2025-10-31T10:37:05.000Z","created_at_i":1761907025,"id":45770483,"options":[],"parent_id":45757500,"points":null,"story_id":45748661,"text":"Hmm, isn't that the same argument we use in defense of windows and ms teams?","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T08:08:15.000Z","created_at_i":1761811695,"id":45757500,"options":[],"parent_id":45757288,"points":null,"story_id":45748661,"text":"In Microsoft's defense, Azure has always been a complete joke. It's extremely developer unfriendly, buggy and overpriced.","title":null,"type":"comment","url":null},{"author":"sbergot","children":[{"author":"ambentzen","children":[],"created_at":"2025-10-30T09:09:35.000Z","created_at_i":1761815375,"id":45757863,"options":[],"parent_id":45757582,"points":null,"story_id":45748661,"text":"There might have been, but they didn't know because they couldn't access it. Could have been something totally unrelated.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T08:20:51.000Z","created_at_i":1761812451,"id":45757582,"options":[],"parent_id":45757288,"points":null,"story_id":45748661,"text":"and for a while the status was "there might be issues on azure portal".","title":null,"type":"comment","url":null},{"author":"infaloda","children":[{"author":"Xss3","children":[{"author":"hinkley","children":[],"created_at":"2025-10-30T22:10:32.000Z","created_at_i":1761862232,"id":45765997,"options":[],"parent_id":45759107,"points":null,"story_id":45748661,"text":"It's 19 minutes until active engagement by staff. And planned rolling restarts can trigger alerts if you don't set thresholds of time instead of just thresholds of count.
It would be nice though if alert systems made it easy to wire up CD to turn down sensitivity during observed actions. Sort of like how the immune system turns down a bit while you're eating.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T12:05:09.000Z","created_at_i":1761825909,"id":45759107,"options":[],"parent_id":45757639,"points":null,"story_id":45748661,"text":"That does not say it took 19 minutes for alerts to appear. Following could mean any amount of time.","title":null,"type":"comment","url":null},{"author":"hinkley","children":[{"author":"tremon","children":[{"author":"hinkley","children":[],"created_at":"2025-10-31T00:11:58.000Z","created_at_i":1761869518,"id":45766961,"options":[],"parent_id":45766886,"points":null,"story_id":45748661,"text":"That\u2019s some very carefully chosen phrasing.
I think if you really wanted to do on call right to avoid gaps you\u2019d want no more than 6 hours on primary per day per shift, and you want six, not four, shifts per day. So you\u2019re only alone for four hours in the middle of your shift and have plenty of time to hand off.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T23:59:58.000Z","created_at_i":1761868798,"id":45766886,"options":[],"parent_id":45765978,"points":null,"story_id":45748661,"text":"I'd like to think that a company the size of Microsoft can afford to have incident response teams in enough time zones to cover basic operations without relying on night shifts.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T22:08:34.000Z","created_at_i":1761862114,"id":45765978,"options":[],"parent_id":45757639,"points":null,"story_id":45748661,"text":"10 minutes to alert, to avoid flapping false positives. 10 minute response window for first responders. Or, 5 minute window before failing over to backup alerts, and 4 minutes to wake up, have coffee, and open the appropriate windows.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T08:31:27.000Z","created_at_i":1761813087,"id":45757639,"options":[],"parent_id":45757288,"points":null,"story_id":45748661,"text":"More importantly\n`15:45 UTC on 29 October 2025 \u2013 Customer impact began.
16:04 UTC on 29 October 2025 \u2013 Investigation commenced following monitoring alerts being triggered.\n`\nA 19-minute delay in alert is a joke.","title":null,"type":"comment","url":null},{"author":"imglorp","children":[],"created_at":"2025-10-30T10:39:15.000Z","created_at_i":1761820755,"id":45758419,"options":[],"parent_id":45757288,"points":null,"story_id":45748661,"text":"That's about how long it took to bubble up three levels of management and then go past the PR and legal teams for approvals.","title":null,"type":"comment","url":null},{"author":"thayne","children":[{"author":"ape4","children":[],"created_at":"2025-10-30T15:07:05.000Z","created_at_i":1761836825,"id":45760874,"options":[],"parent_id":45760820,"points":null,"story_id":45748661,"text":"Perhaps they could set the time to when it really started after executive approval.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T15:02:30.000Z","created_at_i":1761836550,"id":45760820,"options":[],"parent_id":45757288,"points":null,"story_id":45748661,"text":"Unfortunately,that is also typical. I've seen it take longer than that for AWS to update their status page.
The reason is probably because changes to the status page require executive approval, because false positives could lead to bad publicity, and potentially having to reimburse customers for failing to meet SLAs.","title":null,"type":"comment","url":null},{"author":"HeavyStorm","children":[],"created_at":"2025-10-30T15:30:33.000Z","created_at_i":1761838233,"id":45761132,"options":[],"parent_id":45757288,"points":null,"story_id":45748661,"text":"I've been on bridges where people _forgot_ to send comms for dozens of minutes. Too many inexperienced people around these days.","title":null,"type":"comment","url":null},{"author":"schainks","children":[],"created_at":"2025-10-30T17:19:54.000Z","created_at_i":1761844794,"id":45762491,"options":[],"parent_id":45757288,"points":null,"story_id":45748661,"text":"AWS either is \u201con it\u201d or you they will say something somewhere between 60-90 minutes after impact.
We should be lucky MSFT is so consistent!
Hug ops to the Azure team, since management is shredding up talent over there.","title":null,"type":"comment","url":null},{"author":"campbel","children":[{"author":"hinkley","children":[],"created_at":"2025-10-30T22:06:49.000Z","created_at_i":1761862009,"id":45765958,"options":[],"parent_id":45763093,"points":null,"story_id":45748661,"text":"I see Amazon using a subset of the same sorts of obfuscations that Microsoft was infamous for. They just chopped off the crusts so it's less obvious that it's the same shit sandwich.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T18:07:13.000Z","created_at_i":1761847633,"id":45763093,"options":[],"parent_id":45757288,"points":null,"story_id":45748661,"text":"As a technologist, you should always avoid MS. Even if they have a best-in-class solution for some domain, they will use that to leverage you into their absolute worst-in-class ecosystem.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T07:26:59.000Z","created_at_i":1761809219,"id":45757288,"options":[],"parent_id":45756772,"points":null,"story_id":45748661,"text":"33 minutes from impact to status page for a complete outage is a joke.","title":null,"type":"comment","url":null},{"author":"oofbey","children":[],"created_at":"2025-10-30T15:17:07.000Z","created_at_i":1761837427,"id":45760975,"options":[],"parent_id":45756772,"points":null,"story_id":45748661,"text":"\u201cOur protection mechanisms, to validate and block any erroneous deployments, failed due to a software defect which allowed the deployment to bypass safety validations.\u201d
Very circular way of saying \u201cthe validator didn\u2019t do its job\u201d. This is AFAICT a pretty fundamental root cause of the issue.
It\u2019s never good enough to have a validator check the content and hope that finds all the issues. Validators are great and can speed a lot of things up. But because they are independent code paths they will always miss something. For critical services you have to assume the validator will be wrong, and be prepared to contain the damage WHEN it is wrong.","title":null,"type":"comment","url":null},{"author":"notorandit","children":[{"author":"hinkley","children":[],"created_at":"2025-10-30T22:17:31.000Z","created_at_i":1761862651,"id":45766069,"options":[],"parent_id":45762248,"points":null,"story_id":45748661,"text":"I've seen sensitivity get tuned down to avoid false positives during deployments or rolling restarts for host updates. And to a lesser extent for autoscaling noise. It can be hard to get right.
I think it's perhaps a gap in the tools. We apply the same alert criteria at 2 am that we do while someone is actively running deployment or admin tasks and there's a subset that should stay the same, like request failure rate, and others that should be tuned down, like overall error rate and median response times.
And it means one thing if the failure rate for one machine is 90% and something else if the cluster failure rate is 5%, but if you've only got 18 boxes it's hard to discern the difference. And which is the higher priority error may change from one project to another.","title":null,"type":"comment","url":null},{"author":"deadbolt","children":[],"created_at":"2025-10-31T00:52:38.000Z","created_at_i":1761871958,"id":45767211,"options":[],"parent_id":45762248,"points":null,"story_id":45748661,"text":"Just what you want in a cloud provider, right?","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T16:59:55.000Z","created_at_i":1761843595,"id":45762248,"options":[],"parent_id":45756772,"points":null,"story_id":45748661,"text":"What puzzles me too is the time it took to recognize an outage.
Looks like there was no monitoring and no alerts.
Which is kinda weird.","title":null,"type":"comment","url":null},{"author":"neop1x","children":[],"created_at":"2025-10-31T08:58:25.000Z","created_at_i":1761901105,"id":45769814,"options":[],"parent_id":45756772,"points":null,"story_id":45748661,"text":">> We began the investigation and started to examine configuration changes within AFD.
Troubleshooting has completed
Troubleshooting was unable to automatically fix all of the issues found. You can find more details below.
>> We initiated the deployment of our \u2018last known good\u2019 configuration.
System Restore can help fix problems that might be making your computer run slowly or stop responding.
System Restore does not affect any of your documents, pictures, or other personal data. Recently installed programs and drivers might be uninstalled.
Confirm your restore point
Your computer will be restored to the state it was in before the event in the Description field below.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T05:45:53.000Z","created_at_i":1761803153,"id":45756772,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Preliminary post incident review: https://azure.status.microsoft/en-gb/status/history/
Timeline
15:45 UTC on 29 October 2025 \u2013 Customer impact began.
16:04 UTC on 29 October 2025 \u2013 Investigation commenced following monitoring alerts being triggered.
16:15 UTC on 29 October 2025 \u2013 We began the investigation and started to examine configuration changes within AFD.
16:18 UTC on 29 October 2025 \u2013 Initial communication posted to our public status page.
16:20 UTC on 29 October 2025 \u2013 Targeted communications to impacted customers sent to Azure Service Health.
17:26 UTC on 29 October 2025 \u2013 Azure portal failed away from Azure Front Door.
17:30 UTC on 29 October 2025 \u2013 We blocked all new customer configuration changes to prevent further impact.
17:40 UTC on 29 October 2025 \u2013 We initiated the deployment of our \u2018last known good\u2019 configuration.
18:30 UTC on 29 October 2025 \u2013 We started to push the fixed configuration globally.
18:45 UTC on 29 October 2025 \u2013 Manual recovery of nodes commenced while gradual routing of traffic to healthy nodes began after the fixed configuration was pushed globally.
23:15 UTC on 29 October 2025 - PowerApps mitigation of dependency, and customers confirm mitigation.
00:05 UTC on 30 October 2025 \u2013 AFD impact confirmed mitigated for customers.","title":null,"type":"comment","url":null},{"author":"razodactyl","children":[],"created_at":"2025-10-30T06:20:30.000Z","created_at_i":1761805230,"id":45756936,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"AWS, now Azure - wasn't this a plot point in Terminator where SkyNet was causing computer systems to have issues much before it finally become self-aware?
Funnily enough, AI has been training on its own data as generated by users writing AI conversations back to the internet - there's a feedback loop at play.","title":null,"type":"comment","url":null},{"author":"major505","children":[],"created_at":"2025-10-30T08:43:05.000Z","created_at_i":1761813785,"id":45757714,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Somewhere, an ex microsoft engineer that where layoff during the last week, is saying to himself \u201cthank god, this shit is not my problem anymore\u201d","title":null,"type":"comment","url":null},{"author":"zimpenfish","children":[],"created_at":"2025-10-30T09:12:55.000Z","created_at_i":1761815575,"id":45757884,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":""Microsoft Azure will serve as the backbone of Asda\u2019s digital infrastructure"[0]
Oh, that'll be why Scan & Go was down yesterday evening. I thought it was another instance of an iOS 26 update breaking their crappy code.
[0] https://corporate.asda.com/newsroom/2025/22/09/asda-announce...","title":null,"type":"comment","url":null},{"author":"Aldipower","children":[],"created_at":"2025-10-30T09:55:05.000Z","created_at_i":1761818105,"id":45758147,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Hetzner, Netcup, OVH,\nBunnyCDN, \nClouDNS,\nPostmark
You name them. Other good providers you have experience with?
There is no reason for an expensive cloud. Never has been, but decision makers tried to keep their pants dry.","title":null,"type":"comment","url":null},{"author":"buttscicles","children":[{"author":"tartieret","children":[],"created_at":"2025-10-30T14:38:42.000Z","created_at_i":1761835122,"id":45760514,"options":[],"parent_id":45758247,"points":null,"story_id":45748661,"text":"I was a little puzzled as we got notified our apps were down, and then I tried to login in the Azure portal with no success. But the Azure status page reported no incident, so I posted here and quickly confirmed that others were impacted! They did a pretty bad job with their status page as the front door service was shown green all along","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T10:12:29.000Z","created_at_i":1761819149,"id":45758247,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Interesting that everybody knows when AWS goes down but Azure needs a "Tell HN" :)
Best of luck to the teams responding to this incident.","title":null,"type":"comment","url":null},{"author":"jammo","children":[{"author":"dspillett","children":[],"created_at":"2025-10-30T10:44:21.000Z","created_at_i":1761821061,"id":45758450,"options":[],"parent_id":45758254,"points":null,"story_id":45748661,"text":"> We all need to move away from these big cloud providers.
But then who do we blame when things are down? If we manage our own infrastructure we have to stay late to fix it when it breaks instead of saying \u201csorry, Microsoft, nothing we can do\u201d and magically our clients accepting that\u2026","title":null,"type":"comment","url":null},{"author":"stbenjam","children":[],"created_at":"2025-10-30T10:55:15.000Z","created_at_i":1761821715,"id":45758546,"options":[],"parent_id":45758254,"points":null,"story_id":45748661,"text":"Ah yes, let's put our multibillion dollar ecommerce site on... checks notes Hetzner.
Lol","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T10:13:30.000Z","created_at_i":1761819210,"id":45758254,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"We all need to move away from these big cloud providers. Two medium size smaller providers is enough.
-Cloudflare for R2 (object storage) and CDN (Fastly+backblaze also available).\n-Two VPS/Server providers with a decent reputation and mid-size (using a comparison site like https://serversearcher.com or look directly into people like Hetzner or latitude)\n-PlanetScale or Neon for database if you don't co-locate it, though better to use someone like digital ocean, vultr or latitude who offer databases too)","title":null,"type":"comment","url":null},{"author":"nflekkhnnn","children":[],"created_at":"2025-10-30T10:45:44.000Z","created_at_i":1761821144,"id":45758463,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Shut the front door!","title":null,"type":"comment","url":null},{"author":"DeathArrow","children":[{"author":"mnau","children":[],"created_at":"2025-10-30T12:57:37.000Z","created_at_i":1761829057,"id":45759545,"options":[],"parent_id":45758491,"points":null,"story_id":45748661,"text":"It doesn't matter whether you actually are safe or not. What matters is that you are in compliance.","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T10:49:13.000Z","created_at_i":1761821353,"id":45758491,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Buy cloud because you're always safe! Until you aren't.","title":null,"type":"comment","url":null},{"author":"kure256","children":[],"created_at":"2025-10-30T11:15:41.000Z","created_at_i":1761822941,"id":45758725,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"We\u2019ve been experimenting with multi-cluster failover for Kubernetes workloads, and one open-source project that actually works really well is k8gb\n.
It acts as a GSLB controller inside Kubernetes \u2014 doing DNS-level health checks, region awareness, and automatic failover between clusters when one goes down.
It integrates with ExternalDNS and supports multiple DNS providers (Infoblox, Route53, Azure DNS, NS1, etc.), so it can handle failover across both on-prem and cloud clusters.
It\u2019s not a silver bullet for every architecture, but it\u2019s one of the few OSS projects that make multi-region failover actually manageable in practice.","title":null,"type":"comment","url":null},{"author":"tartieret","children":[],"created_at":"2025-10-30T14:31:09.000Z","created_at_i":1761834669,"id":45760413,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"it took a good half hour after we detected the problem to see a notification on the Azure status page. Thanks to those who responded to my question as it validated the issue was global and we contacted our users t\nright away","title":null,"type":"comment","url":null},{"author":"zaoui_amine","children":[],"created_at":"2025-10-30T17:11:45.000Z","created_at_i":1761844305,"id":45762379,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Yeah, Azure is a mess today. Can't do anything without the portal.","title":null,"type":"comment","url":null},{"author":"zaoui_amine","children":[{"author":"altcognito","children":[],"created_at":"2025-10-30T17:14:14.000Z","created_at_i":1761844454,"id":45762416,"options":[],"parent_id":45762397,"points":null,"story_id":45748661,"text":"wrong thread","title":null,"type":"comment","url":null}],"created_at":"2025-10-30T17:13:04.000Z","created_at_i":1761844384,"id":45762397,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Language models aren't perfect; they can still generate similar outputs. Invertibility is a stretch.","title":null,"type":"comment","url":null},{"author":"FrostKiwi","children":[],"created_at":"2025-11-02T10:47:55.000Z","created_at_i":1762080475,"id":45789353,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"Surprised to see the situation getting worse, what the hell.
Had some Frontdoor operations timing out, but now I'm straight up denied with "Message: All Changes to Azure Frondoor Configuration are blocked currently."
What a mess.","title":null,"type":"comment","url":null},{"author":"shivenigma","children":[],"created_at":"2025-11-03T06:16:31.000Z","created_at_i":1762150591,"id":45796326,"options":[],"parent_id":45748661,"points":null,"story_id":45748661,"text":"what's happening? self hosting advocate groups attacking all cloud to prove their point?","title":null,"type":"comment","url":null}],"created_at":"2025-10-29T16:01:18.000Z","created_at_i":1761753678,"id":45748661,"options":[],"parent_id":null,"points":885,"story_id":45748661,"text":"Azure is down for us, we can't even access the azure portal. Are other experiencing this? Our services are located in Canada/Central and US-East 2
https://downdetector.ca/status/windows-azure/
https://azure.status.microsoft/en-gb/status","title":"Tell HN: Azure outage","type":"story","url":null}