DeepSeek R1 reproduced for $30: Berkeley researchers replicate DeepSeek R1 for $30—casting doubt on H100 claims and controversy
techstartups.com
    • vrighter@discuss.tchncs.deEnglish
      12·
      7 小时前

      open weights is not open source. If it were, then nobody would have to work on trying to reproduce it. They could just run the build script.

  • reallykindasorta@slrpnk.netEnglish
    12·
    18 小时前

    Non-techie requesting a laymen explanation if anyone has time!

    After reading a couple of”what makes nvidias h100 chips so special” articles I’m gathering that they were supposed to have a significant amount more computational capability than their competitors (which I’m taking to mean more computations per second). So the question with deepseek and similar is something like ‘how are they able to get the same results with less computations?’ and the answer is speculated to be more efficient code/instructions for the AI model so it can make the same conclusions with less computations overall, potentially reducing the need for special jacked up cpus to run it?

    • fallowseed@lemmy.worldEnglish
      62·
      11 小时前

      i read that that the chinese made alterations to the cards, as well-- they dismantled them to access the chips themselves and were able to do more precise micromanagement that cuda doesn’t support, for instance… basically they took the training wheels off and used a more fine-tuned and hands-on approach that gave them some serious advantages

          • froztbyte@awful.systemsEnglish
            5·
            5 小时前

            okay so that post’s core supposition (“using ptx instead of cuda”) is just fucking wrong fucking weird and I’m not going to spend time on it, but it links to this tweet which has this:

            DeepSeek customized parts of the GPU’s core computational units, called SMs (Streaming Multiprocessors), to suit their needs. Out of 132 SMs, they allocated 20 exclusively for server-to-server communication tasks instead of computational tasks

            this still reads more like simply tuning allocation than outright scheduler and execution control (which your post alluded to)

            [x] doubt

            e: original wording because cuda still uses ptx anyway, whereas this post looks like it’s saying “they steered ptx directly”. at first I read the tweet more like “asm vs python” but it doesn’t appear to be what that part meant to convey. still doubting the core hypothesis tho

            • froztbyte@awful.systemsEnglish
              3·
              5 小时前

              sidebar: I definitely wouldn’t be surprised if it comes to this overall being a case of “a shop optimised by tuning, and then it suddenly turns out the entire industry has never tried to tune a thing ever”

              because why try hard when the money taps are open and flowing free? velocity over everything! this is the bayfucker way.

            • fallowseed@lemmy.worldEnglish
              12·
              5 小时前

              well you’re always free to doubt and do your own research-- as i mentioned- it is something i read and between believing what the US tech bros are saying when all their money and hegemony is on the line vs what the chinese have given up for free-use, i am going to go out on a limb and trust the chinese. you’re free to make your own decisions in this regard and kudos for having your own mind.

              • froztbyte@awful.systemsEnglish
                3·
                5 小时前

                mine isn’t a “USA v China: Jelly Wrestling Deluxe” comment and you’re not really understanding the point

                • fallowseed@lemmy.worldEnglish
                  12·
                  5 小时前

                  what is your point? i thought i was giving a “explain like i’m 5” answer to a guy asking for one… you came along asking me to show sources… now this?

    • justOnePersistentKbinPlease@fedia.io
      18·
      17 小时前

      From a technical POV, from having read into it a little:

      Deepseek devs worked in a very low level language called Assembly. This language is unlike relatively newer languages like C in that it provides no guardrails at all and is basically CPU instructions in extreme shorthand. An “if” statement would be something like BEQ 1000, where it goes to a specific memory location(in this case address 1000 if two CPU registers are equal.)

      The advantage of using it is that it is considerably faster than C. However, it also means that the code is mostly locked to that specific hardware. If you add more memory or change CPUs you have to refactor. This is one of the reasons the language was largely replaced with C and other languages.

      Edit: to expound on this: “modern” languages are even slower, but more flexible in terms of hardware. This would be languages like Python, Java and C#

      • V0ldek@awful.systemsEnglish
        1·
        1 小时前

        This is a really weird comment. Assembly is not faster than C, that’s a nonsensical statement, C compiles down to assembly. LLVM’s optimizations will most likely outperform or directly match whatever hand-crafted assembly you write. Why would BEQ 1000 be “considerably faster” than if (x == 1000) goto L_1000;? This collapses even further if you consider any application larger than a few hundred lines of code, any sensible compiler is going to beat you on optimizations if you try to write hand-crafted assembly. Try loading up assembly code and manually performing intraprocedural optimizations, lol, there’s a reason every compiled language goes through an intermediate representation.

        Saying that C# is slower than C is also non-sensical, especially now that C# has built-in PGO it’s very likely it could outperform an application written in C. C#'s JIT compiler is not somehow slower because it’s flexible in terms of hardware, if anything that’s what makes it fast. For example you can write a vectorized loop that will be JIT-compiled to the ideal fastest instruction set available on the CPU running the program, whereas in C or assembly you’d have to manually write a version for each. There’s no reason to think that manual implementation would be faster than what the JIT comes up with at runtime, though, especially with PGO.

        It’s kinda like you’re saying that a V12 engine is faster than a Ferrari and that they are both faster than a spaceship because the spaceship doesn’t have wheels.

        I know you’re trying to explain this to a non-technical person but what you said is so terribly misleading I cannot see educational value in it.

        • blakestacey@awful.systemsEnglish
          14·
          16 小时前

          And I’m sure that your snide remark will both tell them what to simplify and explain how to do so.

          Enjoy your free trip to the egress.