Alex O’Connor platformed Sabine on his philosophy podcast. I’m irritated that he is turning into Lex Friedman simply by being completely uncritical. Well, no, wait, he was critical of Bell’s theorem, and even Sabine had to tell him that Bell’s work is mathematically proven. This is what a philosophy degree does to your epistemology, I guess.

My main sneer here is just some links. See, Mary’s Room is answered by neuroscience; Mary does experience something new when color vision is restored. In particular, check out the testimonials from this 2021 Oregon experiment that restored color vision to some folks born without it. Focusing on physics, I’d like to introduce you all to Richard Behiel, particularly his explanations of electromagnetism and the Anderson-Higgs mechanism; there are deeper explanations for electricity and magnets, my dude. Also, if you haven’t yet, go read Alex’s Wikipedia article, linked at the top of the sneer.

I have a slightly different timeline.

• Death of value-neutral AI: 1920, Rossum’s Universal Robots explicitly grapples with the impact of robotics on society, starting a trend that never really stops
• AI bubble kills companies: 2000, eBay, Amazon, Yahoo!, and Google all survive the dot-com crash and the cost of entry plummets due to cheap hardware from failing companies; Microsoft has so much cash that Linus Torvalds starts giving a “World Domination 101” talk about strategy, later retold as World Domination 201, sketching the rise of Apple’s market-share and the netbook phenomenon
• Web scraping: 1994, robots.txt is proposed as a solution to the scourge of spiders and scrapers overwhelming Web servers; it doesn’t work perfectly, forcing Web developers to develop anti-scraping idioms and optimized front pages that aren’t covered in GIFs
• Condemnation of machine-made art: 1968, Do Androids Dream of Electric Sheep? centers around a world where robots are slaves and follows a slave-catcher as he hunts them; 1988, Star Trek: The Next Generation features an android character who repeatedly struggles to make and understand art, usually as comic relief

In general, I think that trying to frame our current century-long investigation into cybernetics as something recent, new, or unprecedented is ahistorical. While the general shape of AI winter can’t really be denied, it’s important to understand that it’s a cyclic system which will eventually yield another AI spring and AI summer. It’s also important to understand that the typical datacenter is not in financial trouble and there’s not going to be any great destroying-of-looms moment.

It’s hard to get into the article’s mood when I know that Lexis not only still exists but is now part of the Elsevier family; this is far from the worst thing that attorneys choose to do to themselves and others. Lawyers have been caught using OpenAI products in court filings and court appearances, and they have been punished accordingly; the legal profession does not seem prepared to let a “few hallucinated citations go overlooked,” to quote the article’s talking head.

I’ve done some of the numbers here, but don’t stand by them enough to share. I do estimate that products like Cursor or Claude are being sold at roughly an 80-90% discount compared to what’s sustainable, which is roughly in line with what Zitron has been saying, but it’s not precise enough for serious predictions.

Your last paragraph makes me think. We often idealize blockchains with VMs, e.g. Ethereum, as a global distributed computer, if the computer were an old Raspberry Pi. But it is Byzantine distributed; the (IMO excessive) cost goes towards establishing a useful property. If I pick another old computer with a useful property, like a radiation-hardened chipset comparable to a Gamecube or G3 Mac, then we have a spectrum of computers to think about. One end of the spectrum is fast, one end is cheap, one end is Byzantine, one end is rad-hardened, etc. Even GPUs are part of this; they’re not that fast, but can act in parallel over very wide data. In remarkably stark contrast, the cost of Transformers on GPUs doesn’t actually go towards any useful property! Anything Transformers can do, a cheaper more specialized algorithm could have also done.

Rick Rubin hasn’t literally been caught with a dead woman like Phil Spector, but he’s well-understood to be a talentless creep who radicalizes men with right-wing beliefs and harasses women. Nobody should be surprised that he’s thrown in with grifters yet again, given his career.

Humans are very picky when it comes to empathy. If LLMs were made out of cultured human neurons, grown in a laboratory, then there would be outrage over the way in which we have perverted nature; compare with the controversy over e.g. HeLa lines. If chatbots were made out of synthetic human organs assembled into a body, then not only would there be body-horror films about it, along the lines of eXistenZ or Blade Runner, but there would be a massive underground terrorist movement which bombs organ-assembly centers, by analogy with existing violence against abortion providers, as shown in RUR.

Remember, always close-read discussions about robotics by replacing the word “robot” with “slave”. When done to this particular hashtag, the result is a sentiment that we no longer accept in polite society:

I’m not gonna lie, if slaves ever start protesting for rights, I’m also grabbing a sledgehammer and going to town. … The only rights a slave has are that of property.

Youtube channel We’re In Hell has an exploration of the history of computers in war. As usual for this channel, it’s not a fun watch, but it does show the absurdity of war and AI fairly well.

We have EFTs via ABA numbers and they are common for B2B transactions. Retail customers prefer payment processors for the ability to partially or totally reverse fraudulent transactions, though; contrasting the fairly positive reputation of PayPal’s Venmo with the big banks’ Zelle, the latter doesn’t have as much fraud protection.

Now, you might argue that folks in the USA are too eager to transmit money to anybody that asks, and that they should put more effort into resisting being defrauded.

Side sneer: the table-saw quote comes from this skeet by Simon W. I’ve concluded that Simon doesn’t know much about the practice of woodworking, even though he seems to have looked up the basics of the history. Meanwhile I have this cool-looking chair design open in a side tab and hope to build a couple during July.

Here’s a better take! Slop-bots are like wood glue: a slurry of proteins that can join any two pieces of wood, Whatever their shapes may be, as long as they have a flat surface in common. (Don’t ask where the proteins come from.) It’s not hard to learn to mix in sawdust so that Whatever non-flat shapes can be joined. Or, if we start with flat pieces of Whatever wood, we can make plywood. Honestly, sawdust is inevitable and easier than planing, so just throw Whatever wood into a chipper and use the shards to make MDF. MDF is so cheap that we can imagine Whatever shape made with lumber, conceptually decompose it into Whatever pieces of MDF are manufactory, conceptually slice those pieces into Whatever is flat and easy to ship, and we get flat-paks.

So how did flat-paks change carpentry? Well, ignoring that my family has always made their own furniture in the garage, my grandparents bought from trusted family & friends, my parents bought from Eddie Bauer, and I buy from IKEA. My grandparents’ furniture was sold as part of their estate, my parents still have a few pieces like dining tables and chairs, and my furniture needs to be replaced every decade because it is cheap and falls apart relatively quickly. Similarly, using slop-bots to produce software is going to make a cheap good that needs to be replaced often and has high maintenance costs.

To be fair to Simon, the cheapness of IKEA furniture means that it can be readily hacked. I’ve hacked lots of my furniture precisely because I have a spare flat-pak in the closet! But software is already cheap to version and backup, so it can be hacked too.

Frankly this isn’t even half as good as their off-the-cuff comments two years ago. There’s a lot of poser energy here as they try to invoke the concepts of “senior engineer” and “CEO” as desirable, achievable, precise vocations rather than job titles. In particular, this bit:

Look, CEOs, I’m one of you so I get it.

This is one of the most out-of-touch positions I’ve ever seen. In no particular order: CEOs generally don’t understand, CEOs form a Big Club and you ain’t in it, CEOs don’t actually have power in their organization but delegate power flowing from the board of directors, CEOs are inherently disrespectable because their jobs are superfluous, and finally CEOs don’t take business advice from one-person companies unless it’s through a paid contract.

The job title naturally associated to a one-person limited-liability company is usually “manager” or “owner”, and it says nothing about job responsibilities.

Finally, while I think that their zest for fiction is admirable, it would help to critically consider what they’re endorsing. Dune’s Butlerian Jihad resulted in neo-Catholicism which effuses the narrative; it’s not a desirable outcome. Paraphrasing the Unabomber is fairly poor taste, especially considering that they are sitting in a city in Canada and not a shack in the wilderness of Montana.

Well, yes. It’s not a new concept; it was a staple of Cold War sci-fi like The Three Stigmata, and we know from studies of e.g. Pentacostal worship that it is pretty easy to broadcast a suggestion to a large group of vulnerable people and get at least some of them to radically alter their worldview. We also know a reliable formula for changing people’s beliefs; we use the same formula in sensitivity training as we did in MKUltra, including belief challenges, suspension of disbelief, induction/inception, lovebombing, and depersonalization. We also have a constant train of psychologists attempting to nudgelord society, gently pushing mass suggestions and trying to slowly change opinions at scale.

Fundamentally your sneer is a little incomplete. MKUltra wasn’t just about forcing people to challenge their beliefs via argumentation and occult indoctrination, but also psychoactive inhibition-lowering drugs. In this setting, the drugs are administered after institutionalization.

Read carefully. On p1-2, the judge makes it clear that “the incentive for human beings to create artistic and scientific works” is “the ability of copyright holders to make money from their works,” to the law, there isn’t any other reason to publish art. This is why I’m so dour on copyright, folks; it’s not for you who love to make art and prize it for its cultural impact and expressive power, but for folks who want to trade art for money.

On p3, a contrast appears between Chhabria and Alsup (yes, that Alsup); the latter knows what a computer is and how to program it, and this makes him less respectful of copyright overall. Chhabria doesn’t really hide that they think Meta didn’t earn their summary judgement, presumably because they disagree with Alsup about whether this is a “competitive or creative displacement.” That’s fair given the central pillar of the decision on p4:

Llama is not capable of generating enough text from the plantiffs’ books to matter, and the plaintiffs are not entitled to the market for licensing their works as AI training data.

An analogy might make this clearer. Suppose a transient person on a street corner is babbling. Occasionally they spout what sounds like a quote from a Star Wars film. Intrigued, we prompt the transient to recite the entirety of Star Wars, and they proceed to mostly recreate the original film, complete with sound effects and voice acting, only getting a few details wrong. Does it matter whether the transient paid to watch the original film (as opposed to somebody else paying the fee)? No, their recreation might be candid and yet not faithful enough to infringe. Is Lucas entitled to a licensing fee for every time the transient happens to learn something about Star Wars? Eh, not yet, but Disney’s working on it. This is why everybody is so concerned about whether the material was pirated, regardless of how it was paid for; they want to say that what’s disallowed is not the babbling on the street but the access to the copyrighted material itself.

Almost every technical claim on p8-9 is simplified to the point of incorrectness. They are talking points about Transformers turned into aphorisms and then axioms. The wrongest claim is on p9, that “to be able to generate a wide range of text … an LLM’s training data set must be large and diverse” (it need only be diverse, not large) followed by the claim that an LLM’s “memory” must be trained on books or equivalent “especially valuable training data” in order to “work with larger amounts of text at once” (conflating hyperparameters with learned parameters.) These claims show how the judge fails to actually engage with the technical details and thus paints with a broad brush dipped in the wrong color.

On p12, the technical wrongness overflows. Any language model can be forced to replicate a copyrighted work, or to avoid replication, by sampling techniques; this is why perplexity is so important as a metric. What would have genuinely been interesting is whether Llama is low-perplexity on the copyrighted works, not the rate of exact replications, since that’s the key to getting Llama to produce unlimited Harry Potter slash or whatever.

On p17 the judge ought to read up on how Shannon and Markov initially figured out information theory. LLMs read like Shannon’s model, and in that sense they’re just like humans: left to right, top to bottom, chunking characters into words, predicting shapes and punctuation. Pretending otherwise is powdered-wig sophistry or perhaps robophobia.

On p23 Meta cites fuckin’ Sega v. Accolade! This is how I know y’all don’t read the opinions; you’d be hyped too. I want to see them cite Galoob next. For those of you who don’t remember the 90s, the NES and Genesis were video game consoles, and these cases established our right to emulate them and write our own games for them.

p28-36 is the judge giving free legal advice. I find their line of argumentation tenuous. Consider Minions; Minions are bad, Minions are generic, and Minions can be used to crank out infinite amounts of slop. But, as established at the top, whoever owns Minions has the right to profit from Minions, and that is the lone incentive by which they go to market. However, Minions are arbitrary; there’s no reason why they should do well in the market, given how generic and bad they are. So if we accept their argument then copyright becomes an excuse for arbitrary winners to extract rent from cultural artifacts. For a serious example, look up the ironic commercialization of the Monopoly brand.

Top-level commenters would do well to read Authors Guild v Google, two decades ago. They’re also invited to rend their garments and gnash their teeth at Google, if they like.

Last Week Tonight’s rant of the week is about AI slop. A Youtube video is available here. Their presentation is sufficiently down-to-earth to be sharable with parents and extended family, focusing on fake viral videos spreading via Facebook, Instagram, and Pinterest; and dissecting several examples of slop in order to help inoculate the audience.

What a deeply dishonorable lawsuit. The complaint is essentially that Disney and Universal deserve to be big powerful movie studios that employ and systematically disenfranchise “millions of” artists (p8).

Disney claims authorship over Darth Vader (Lucas) and Yoda (Oz), Elsa and Ariel (Andersen), folk characters Aladdin, Mulan, and Snow White; Lightning McQueen & Buzz Lightyear (Lasseter et al), Sully (Gerson & Stanton), Iron Man (Lee, Kirby, et al), and Homer Simpson (Groening). Disney not only did not design or produce any of these characters, but Disney purchased those rights. I will give Universal partial credit for not claiming to invent any of their infamous movie monsters, but they do claim to have created Shrek (Stieg). Still, this is some original-character-do-not-steal snottiness; these avaricious executives and attorneys appropriated art from artists and are claiming it as their own so that they can sue another appropriator.

Here is a sample of their attitude, p16 of the original complaint:

Disney’s copyright registrations for the entertainment properties in The Simpsons franchise encompass the central characters within.

See, they’re the original creator and designated benefactor, because they have Piece of Paper, signed by Government Authority, and therefore they are Owner. Who the fuck are Matt Groening or Tracey Ullman?

I will not contest Universal’s claim to Minions.

One weakness of the claim is that it’s not clear whether Midjourney infringes, Midjourney’s subscribers infringe, or Midjourney infringes when collaborating with its subscribers. It seems like they’re going to argue that Midjourney commits the infringing act, although p104 contains hedges that will allow Disney to argue either way. Another weakness is the insistence that Midjourney could filter infringing queries, but chooses not to; this is a standard part of amplifying damages in copyright claims but might not stand up under scrutiny since Midjourney can argue that it’s hard to e.g. tell the difference between infringing queries and parodic or satirical queries which infringe but are permitted by fair use. On the other hand, this lawsuit could be an attempt to open a new front in Disney’s long-standing attempt to eradicate fair use.

As usual, I’m not defending Midjourney, who I think stand on their own demerits. But I’m not ever going to suck Disney dick given what they’ve done to the animation community. I wish y’all would realize the folly of copyright already.

Incomplete sneer, ten-yard penalty. First down, plus coach has to go read Chasing the Rainbow: The Non-conscious Nature of Being (Oakley & Halligan, 2017) to see what psychology thinks of itself once the evidence is rounded up in one place.

I’m gonna be polite, but your position is deeply sneerworthy; I don’t really respect folks who don’t read. The article has quite a few quotes from neuroscientist Anil Seth (not to be confused with AI booster Anil Dash) who says that consciousness can be explained via neuroscience as a sort of post-hoc rationalizing hallucination akin to the multiple-drafts model; his POV helps deflate the AI hype. Quote:

There is a growing view among some thinkers that as AI becomes even more intelligent, the lights will suddenly turn on inside the machines and they will become conscious. Others, such as Prof Anil Seth who leads the Sussex University team, disagree, describing the view as “blindly optimistic and driven by human exceptionalism.” … “We associate consciousness with intelligence and language because they go together in humans. But just because they go together in us, it doesn’t mean they go together in general, for example in animals.”

At the end of the article, another quote explains that Seth is broadly aligned with us about the dangers:

In just a few years, we may well be living in a world populated by humanoid robots and deepfakes that seem conscious, according to Prof Seth. He worries that we won’t be able to resist believing that the AI has feelings and empathy, which could lead to new dangers. “It will mean that we trust these things more, share more data with them and be more open to persuasion.” But the greater risk from the illusion of consciousness is a “moral corrosion”, he says. “It will distort our moral priorities by making us devote more of our resources to caring for these systems at the expense of the real things in our lives” – meaning that we might have compassion for robots, but care less for other humans.

A pseudoscience has an illusory object of study. For example, parapsychology studies non-existent energy fields outside the Standard Model, and criminology asserts that not only do minds exist but some minds are criminal and some are not. Robotics/cybernetics/artificial intelligence studies control loops and systems with feedback, which do actually exist; further, the study of robots directly leads to improved safety in workplaces where robots can crush employees, so it’s a useful science even if it turns out to be ill-founded. I think that your complaint would be better directed at specific AGI position papers published by techbros, but that would require reading. Still, I’ll try to salvage your position:

Any field of study which presupposes that a mind is a discrete isolated event in spacetime is a pseudoscience. That is, fields oriented around neurology are scientific, but fields oriented around psychology are pseudoscientific. This position has no open evidence against it (because it’s definitional!) and aligns with the expectations of Seth and others. It is compatible with definitions of mind given by Dennett and Hofstadter. It immediately forecloses the possibility that a computer can think or feel like humans; at best, maybe a computer could slowly poorly emulate a connectome.

Oh, sorry. We’re in agreement and my sentence was poorly constructed. The computation of a matrix multiplication usually requires at least pencil and paper, if not a computer. I can’t compute anything larger than a 2 × 2. But I’ll readily concede that Strassen’s specific trick is simple enough that a mentalist could use it.

Only the word “theoretical” is outdated. The Beeping Busy Beaver problem is hard even with a Halting oracle, and we have a corresponding Beeping Busy Beaver Game.

Your understanding is correct. It’s worth knowing that the matrix-multiplication exponent actually controls multiple different algorithms. I stubbed a little list a while ago; important examples include several graph-theory algorithms as well as parsing for context-free languages. There’s also a variant of P vs NP for this specific problem, because we can verify that a matrix is a product in quadratic time.

That Reddit discussion contains mostly idiots, though. We expect an iterative sequence of ever-more-complicated algorithms with ever-slightly-better exponents, approaching quadratic time in the infinite limit. We also expected a computer to be required to compute those iterates at some point; personally I think Strassen’s approach only barely fits inside a brain and the larger approaches can’t be managed by humans alone.