The advice, which is specifically for virtual machines using Azure, shows that sometimes the solution to a catastrophic failure is turn it off and on again. And again.
  • bamboo@lemmy.blahaj.zoneEnglish
    47·
    5 months ago

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

    Have you tried turning it off and back on again?

      • EtzBetz@feddit.de
        9·
        5 months ago

        Hey I rebooted 14 times now, just as you told me, but it’s still not working.*

        :D

    • Kissaki@beehaw.orgEnglish
      10·
      5 months ago

      No no.

      Have you tried turning it off and on and off and on and off and on and off and on and off and on and off and on and off and on and off and on and off and on and off and on and off and on and off and on and off and on and off and on and off and on again?

  • 30p87@feddit.de
    23·
    5 months ago

    Well in the time until Windows rebooted 15 times Windows 12 will be out.

  • zephorah@lemm.ee
    10·
    5 months ago

    We did get 7 computers back by 1am last night just by constantly rebooting.

    That said, 40 out of 47 never came back. So clearly something more is needed.

  • gwindli@lemy.lol
    8·
    5 months ago

    there’s an easy fix. it could be done with a single boot attempt if M$ hadnt made it so needlessly difficult to enter safe mode

    • Norgur@fedia.io
      7·
      5 months ago

      Many of the machines in question will have safe mode walled off for security reasons anyway.

      • gwindli@lemy.lol
        1·
        5 months ago

        fair enough. i can see that disabling safe mode would be a decent security measure. but by the time that kind of exploit is used, you’ve already got bad actors inside your network and there are much easier methods available to pivot to other devices and accounts.

          • Scary le Poo@beehaw.org
            2·
            5 months ago

            Well then obviously you could opt to restrict safe mode on laptops only, or laptops and desktops allowing you to get your server infrastructure up quickly so at least the back end is running properly.

            Ffs.

            • jarfil@beehaw.org
              1·
              5 months ago

              Servers with KVM access, could have it compromised, letting bad actors enter safe mode.

                • jarfil@beehaw.org
                  1·
                  5 months ago

                  Doesn’t need to be fully compromised, but it isn’t unusual for the access credentials to some portion, to be stored on an easier to compromise system. Disabling safe mode on a server, prevents stuff like a single compromised laptop, from becoming a full server compromise.

  • Jimmybander@champserver.net
    5·
    5 months ago

    Most of our machines at my office run Win 10 or 11 and we haven’t had the blue screen. I was wondering why we hadn’t experienced this. Still don’t know.

    • zabadoh@ani.social
      5·
      5 months ago

      Azure is MS’s cloud computing. As long as you weren’t using MS OneDrive, or 365 Office, or something else that relied on MS cloud, you’re good.

      • Midnitte@beehaw.orgEnglish
        23·
        5 months ago

        Actually it’s due to whether your company uses CrowdStrike or not.

        The issue is not being caused by Microsoft but by third-party CrowdStrike software that’s widely used by many businesses worldwide for managing the security of Windows PCs and servers.

        Supposedly, one of the fixes (aside from rebooting and hoping it grabs the update fire) is to delete a single file in the CrowdStrike directory after booting into safe mode.

        • nutlink@beehaw.orgEnglish
          6·
          5 months ago

          I just spent the morning doing this with my help desk team, although we just do it via command prompt at the recovery screen. We’ve had a 100% success rate so far at 93 devices and counting. I’m glad our organization practices read only Friday, at least.

          • EtzBetz@feddit.de
            3·
            5 months ago

            Tbh, I would then also not update anything on Thursdays (which does maybe do overnight procedures) since it may be breaking over night then, leaving you just little time to fix before the weekend :D

            This kinda can be extended up until Monday, I know, but, at least in Germany, on Fridays people go home way sooner than other days.

        • Saik0@lemmy.saik0.comEnglish
          2·
          5 months ago

          Yes, but Azure platform itself was using it. So many of those systems were down overnight (and there’s probably still stragglers). The guy you responded to specifically called out Azure-based services.

          • Midnitte@beehaw.orgEnglish
            4·
            5 months ago

            Sure, but the OP of the thread didn’t.

            Most of our machines at my office run Win 10 or 11 and we haven’t had the blue screen. I was wondering why we hadn’t experienced this. Still don’t know.

            So it isn’t whether you’re using Azure, it’s whether you’re using CrowdStrike (Azure related or not)

            • Saik0@lemmy.saik0.comEnglish
              2·
              5 months ago

              So it isn’t whether you’re using Azure, it’s whether you’re using CrowdStrike (Azure related or not)

              No. Azure platform is using Crowdstrike on their hypervisors. So simply using Azure could be sufficient to hurt you in this case even if your Azure host isn’t using Crowdstrike itself. But yes, otherwise it’s a mix of Windows+Crowdstrike.

              • Kissaki@beehaw.orgEnglish
                2·
                5 months ago

                Can you source your claim, that Azure hypervisor uses CrowdStrike? Because a Microsoft spokesperson told Ars that that issue was unrelated to the CrowdStrike update.

                […] cited as “a backend cluster management workflow [that] deployed a configuration change causing backend access to be blocked between a subset of Azure Storage clusters and compute resources in the Central US region.”

                A spokesperson for Microsoft told Ars in a statement Friday that the CrowdStrike update was not related to its July 18 Azure outage. “That issue has fully recovered,” the statement read.

          • Kissaki@beehaw.orgEnglish
            2·
            5 months ago

            Microsoft services were, in a seemingly terrible coincidence, also down overnight Thursday into Friday. […]

            A spokesperson for Microsoft told Ars in a statement Friday that the CrowdStrike update was not related to its July 18 Azure outage. “That issue has fully recovered,” the statement read.

            from https://arstechnica.com/information-technology/2024/07/major-outages-at-crowdstrike-microsoft-leave-the-world-with-bsods-and-confusion/

            They were not “using it”. And there’s no “stragglers still”.

    • Flax@feddit.ukEnglish
      3·
      5 months ago

      Why bother encrypting passwords? Just store them in plaintext, preferably on a web server that’s publicly accessible so other services can easily access them.