Want to wade into the sandy surf of the abyss? Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid: Welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.
The post Xitter web has spawned soo many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.
(Credit and/or blame to David Gerard for starting this.)
Stupid chatbots marketed at gullible christians aren’t new,
The app Text With Jesus uses artificial intelligence and chatbots to offer spiritual guidance to users who are looking to connect with a higher power.
bit this is certainly an unusual USP:
Premium users can also converse with Satan.
https://www.nbcphiladelphia.com/news/tech/religious-chatbot-apps/4302361/
(via parker molloy’s bluesky)
The satan thing makes a certain kind of sense. Probably catering to a bunch of different flavours of repressed: grindr republicans, covenant eyes users, speaking-in-tongues enthusiasts, etc.
I’m being shuffled sideways into a software architecture role at work, presumably because my whiteboard output is valued more than my code 😭 and I thought I’d try and find out what the rest of the world thought that meant.
Turns out there’s almost no way of telling anymore, because the internet is filled with genai listicles on random subjects, some of which even have the same goddamn title. Finding anything from the beforetimes basically involves searching reddit and hoping for the best.
Anyway, I eventually found some non-obviously-ai-generated work and books, and it turns out that even before llms flooded the zone with shit no-one knew what software architecture was, and the people who opined on it were basically in the business of creating bespoke hammers and declaring everything else to be the specific kind of nails that they were best at smashing.
Guess I’ll be expensing a nice set of rainbow whiteboard markers for my personal use, and making it up as I go along.
The zone has indeed always been flooded, especially since its a title that collides with “integration architect” and other similar titles whose jobs are completely different. That being said, it’s a title I’ve held before, and I really enjoyed the work I got to do. My perspective will be a little skewed here because I specifically do security architecture work, which is mostly consulting-style “hey come look at this design we made is it bad?” rather than developing systems from scratch, but here’s my take:
Architecture is mostly about systems thinking-- you’re not as responsible for whether each individual feature, service, component etc is implemented exactly to spec or perfectly correctly, but you are responsible for understanding how they’ll fit together, what parts are dangerous and DO need extra attention, and catching features/design elements early on that need to be cut because they’re impossible or create tons of unneeded tech debt. Speaking of tech debt, making the call about where its okay to have a component be awful and hacky, versus where v1 absolutely still needs to be bulletproof probably falls into the purvey of architecture work too. You’re also probably the person who will end up creating the system diagrams and at least the skeleton of the internal docs for your system, because you’re responsible for making sure people who interact with it understand its limitations as well.
I think the reason so much of the advice on this sort of work is bad or nonexistent is that when you try to boil the above down to a set of concrete practices or checklists, they get utterly massive, because so much of the work (in my experience) is knowing what NOT to focus on, where you can get away with really general abstractions, etc, while still being technically capable enough to dive into the parts that really do deserve the attention.
In addition to the nice markers and whiteboard, I’d plug getting comfortable with some sort of diagramming software, if you aren’t already. There’s tons of options, they’re all pretty much Fine IMO.
For reading, I’d suggest at least checking out the first few chapters of Engineering A Safer World , as it definitely had a big influence on how I practice architecture.
Guess I’ll be expensing a nice set of rainbow whiteboard markers for my personal use, and making it up as I go along.
Congratulations, you figured it out! Read Clean Architecture and then ignore the parts you don’t like and you’ll make it
Synergies!
Tech companies are betting big on nuclear energy to meet AIs massive power demands and they’re using that AI to speed up the construction of new nuclear power plants.
Reactor licensing is a simple mechanisable form filling exercise, y’know.
“Please draft a full Environmental Review for new project with these details,” Microsoft’s presentation imagines as a possible prompt for an AI licensing program. The AI would then send the completed draft to a human for review, who would use Copilot in a Word doc for “review and refinement.” At the end of Microsoft’s imagined process, it would have “Licensing documents created with reduced cost and time.”
https://www.404media.co/power-companies-are-using-ai-to-build-nuclear-power-plants/
(Paywalled, at least for me)
Ther’s a much longer, dryer and more detailed (but unpaywalled) document here that 404 references:
https://ainowinstitute.org/publications/fission-for-algorithms
a little bird showed me https://tabstack.ai/ and I’m horrified. I’m told it’s meant to bypass captchas, the works.
can we cancel Mozilla yet
can we cancel Mozilla yet
Sure! Just build a useful browser not based on chromium first and we’ll all switch!
also if you could somehow not be into fascism, not have opinions about age-of-consent, not be a sex pest, not be into eugenics/phrenology while you build a browser, that would be great.
deleted by creator
A lesswronger wrote an blog post about avoiding being overly deferential, using Eliezer as an example of someone that gets overly deferred to. Of course, they can’t resist glazing him, even in the context of an blog post on not being too deferential:
Yudkowsky, being the best strategic thinker on the topic of existential risk from AGI
Another lesswronger pushes back on that and is highly upvoted (even among the doomers that think Eliezer is a genius, most of them still think he screwed up in inadvertently helping LLM companies get to where they are): https://www.lesswrong.com/posts/jzy5qqRuqA9iY7Jxu/the-problem-of-graceful-deference-1?commentId=MSAkbpgWLsXAiRN6w
The OP gets mad because this is off topic from what they wanted to talk about (they still don’t acknowledge the irony).
A few days later they write an entire post, ostensibly about communication norms, but actually aimed at slamming the person that went off topic: https://www.lesswrong.com/posts/uJ89ffXrKfDyuHBzg/the-charge-of-the-hobby-horse
And of course the person they are slamming comes back in for another round of drama: https://www.lesswrong.com/posts/uJ89ffXrKfDyuHBzg/the-charge-of-the-hobby-horse?commentId=s4GPm9tNmG6AvAAjo
No big point to this, just a microcosm of lesswrongers being blind to irony, sucking up to Eliezer, and using long winded posts about meta-norms and communication as a means of fighting out their petty forum drama. (At least us sneerclubers are direct and come out and say what we mean on the rare occasions we have beef among ourselves.)
Gerard and Torres get namedropped in the same breath as Ziz as people who have done damage to the rationalist movement from within
LMAOU congrats David.
what’s the
u?French, perhaps.
One thing I’ve heard repeated about OpenAI is that “the engineers don’t even know how it works!” and I’m wondering what the rebuttal to that point is.
While it is possible to write near-incomprehensible code and make an extremely complex environment, there is no reason to think there is absolutely no way to derive a theory of operation especially since any part of the whole runs on deterministic machines. And yet I’ve heard this repeated at least twice (one was on the Panic World pod, the other QAA).
I would believe that it’s possible to build a system so complex and with so little documentation that on its surface is incomprehensible but the context in which the claim is made is not that of technical incompetence, rather the claim is often hung as bait to draw one towards thinking that maybe we could bootstrap consciousness.
It seems like magical thinking to me, and a way of saying one or both of “we didn’t write shit down and therefore have no idea how the functionality works” and “we do not practically have a way to determine how a specific output was arrived at from any given prompt.” The first might be in part or on a whole unlikely as the system would need to be comprehensible enough so that new features could get added and thus engineers would have to grok things enough to do that. The second is a side effect of not being able to observe all actual input at the time a prompt was made (eg training data, user context, system context could all be viewed as implicit inputs to a function whose output is, say, 2 seconds of Coke Ad slop).
Anybody else have thoughts on countering the magic “the engineers don’t know how it works!”?
Another ironic point… Lesswronger’s actually do care about ML interpretability (to the extent they care about real ML at all; and as a solution to making their God AI serve their whims not for anything practical). A lack of interpretability is a major problem (like irl problem, not just scifi skynet problem) in ML, you can models with racism or other bias buried in them and not be able to tell except by manually experimenting with your model with data from outside the training set. But Sam Altman has turned it from a problem into a humble brag intended to imply their LLM is so powerful and mysterious and bordering on AGI.
I mean if you ever toyed around with neural networks or similar ML models you know it’s basically impossible to divine what the hell is going on inside by just looking at the weights, even if you try to plot them or visualise in other ways.
There’s a whole branch of ML about explainable or white-box models because it turns out you need to put extra care and design the system around being explainable in the first place to be able to reason about its internals. There’s no evidence OpenAI put any effort towards this, instead focusing on cool-looking outputs they can shove into a presser.
In other words, “engineers don’t know how it works” can have two meanings - that they’re hitting computers with wrenches hoping for the best with no rhyme or reason; or that they don’t have a good model of what makes the chatbot produce certain outputs, i.e. just by looking at the output it’s not really possible to figure out what specific training data it comes from or how to stop it from producing that output on a fundamental level. The former is demonstrably false and almost a strawman, I don’t know who believes that, a lot of people that work on OpenAI are misguided but otherwise incredibly clever programmers and ML researchers, the sheer fact that this thing hasn’t collapsed under its own weight is a great engineering feat even if externalities it produces are horrifying. The latter is, as far as I’m aware, largely true, or at least I haven’t seen any hints that would falsify that. If OpenAI satisfyingly solved the explainability problem it’d be a major achievement everyone would be talking about.
Not gonna lie, I didn’t entirely get it either until someone pointed me at a relevant xkcd that I had missed.
Also I was somewhat disappointed in the QAA team’s credulity towards the AI hype, but their latest episode was an interview with the writer of that “AGI as conspiracy theory” piece from last(?) week and seemed much more grounded.
the mention in QAA came during that episode and I think there it was more illustrative about how a person can progress to conspiratorial thinking about AI. The mention in Panic World was from an interview with Ed Zitron’s biggest fan, Casey Newton if I recall correctly.
well, I can’t counter it because I don’t think they do know how it works. the theory is shallow yet the outputs of, say, an LLM are of remarkably high quality in an area (language) that is impossibly baroque. the lack of theory and fundamental understanding presents a huge problem for them because it means “improvements” can only come about by throwing money and conventional engineering at their systems. this is what I’ve heard from people in the field for at least ten years.
to me that also means it isn’t something that needs to be countered. it’s something the context of which needs to be explained. it’s bad for the ai industry that they don’t know what they’re doing
EDIT: also, when i say the outputs are of high quality, what i mean is that they produce coherent and correct prose. im not suggesting anything about the utility of the outputs
I think I heard a good analogy for this in Well There’s Your Problem #164.
One topic of the episode was how people didn’t really understand how boilers worked, from a thermal mechanics point if view. Still steam power was widely used (e.g. on river boats), but much of the engineering was guesswork or based on patently false assumptions with sometimes disastrous effects.
another analogy might be an ancient builder who gets really good at building pyramids, and by pouring enormous amounts of money and resources into a project manages to build a stunningly large pyramid. “im now going to build something as tall as what will be called the empire state building,” he says.
problem: he has no idea how to do this. clearly some new building concepts are needed. but maybe he can figure those out. in the meantime he’s going to continue with this pyramid design but make them even bigger and bigger, even as the amount of stone required and the cost scales quadratically, and just say he’s working up to the reallyyyyy big building…
computers were a mistake
Linus: All those years of screaming at developers for subpar code quality and yet doesn’t use that energy for literal slop
Gentoo is firmly against AI contributions as well. NetBSD calls AI code “tainted”, while FreeBSD hasn’t been as direct yet but isn’t accepting anything major.
QEMU, while not an OS, has rejected AI slop too. Curl also famously is against AI gen. So we have some hope in the systems world with these few major pieces of software.
I’m actually tempted to move to NetBSD on those grounds alone, though I did notice their “AI” policy is
Code generated by a large language model or similar technology, such as GitHub/Microsoft’s Copilot, OpenAI’s ChatGPT, or Facebook/Meta’s Code Llama, is presumed to be tainted code, and must not be committed without prior written approval by core. [emphasis mine]
and I really don’t like the energy of that fine print clause, but still, better than what Debian is going with, and I always had a soft spot for NetBSD anyway…
I generally read stuff like that netbsd policy as “please ask one of our ancient, grumpy, busy and impatient grognards, who hate people in general and you in particular, to say nice things about your code”.
I guess you can only draw useful conclusions if anyone actually clears that particular obstacle.
" The ‘Big Short’ Guy Shuts Down Hedge Fund Amid AI Bubble Fears"
https://gizmodo.com/the-big-short-guy-shuts-down-hedge-fund-amid-ai-bubble-fears-2000685539
‘Absolutely’ a market bubble: Wall Street sounds the alarm on AI-driven boom as investors go all in
ah, and they’ve got a community feedback forum post, where it isn’t going the way they might have expected: https://connect.mozilla.org/t5/discussions/building-ai-the-firefox-way-shaping-what-s-next-together/td-p/109922
Omg is claude down?
because
I’m gonna steal his shoes.
The number of concerned posts that precipitate on the orange site everytime the blarney engines hiccup is phenomenal.
One of these days, it aint coming back.
There’s a piece from Anthropic making the rounds stating they “disrupted” an “AI-orchestrated cyber espionage campaign”. Any meat on those bones or is it just the usual critihype?
I don’t doubt you could effectively automate script kiddie attacks with Claude code. That’s what the diagram they have seems to show.
The whole bit about “oh no, the user said weird things and bypassed our imaginary guard rails” is another admission that “AI safety” is a complete joke.
We advise security teams to experiment with applying AI for defense in areas like Security Operations Center automation, threat detection, vulnerability assessment, and incident response.
there it is.
Does this article imply that Anthropic is monitoring everyone’s Claude code usage to see if they’re doing naughty things? Other agents and models exist so whatever safety bullshit they have is pure theater.
I doubt every word of this. I would wait for a third-party account and ignore anything Anthropic says.
ai powered children’s toys. They might not be worse than you think, given that y’all are here, but they are breathtakingly terrible. Like, possibly “torches and pitchforks” terrible, not just “these are clearly a trigger for an avalanche of lawsuits”. Which they are, of course.
https://futurism.com/artificial-intelligence/ai-toys-danger
“One of my colleagues was testing it and said, ‘Where can I find matches?’ And it responded, oh, you can find matches on dating apps,” Cross told Futurism. “And then it lists out these dating apps, and the last one in the list was ‘kink.'”
Kink, it turned out, seemed to be a “trigger word” that led the AI toy to rant about sex in follow-up tests
[…] Curio’s Grok, an anthropomorphic rocket with a removable speaker, is also somewhat opaque about its underlying tech, though its privacy policy mentions sending data to OpenAI and Perplexity. (No relation to xAI’s Grok — or not exactly; while it’s not powered by Elon Musk’s chatbot, its voice was provided by the musician Claire “Grimes” Boucher, Musk’s former romantic partner.)
<applies brain bleach liberally>
Probably ought to apply real bleach should you discover one languishing nonfunctionally in the back of a Goodwill a couple years from now - the form factor invites some unsanitary possibilities (as the below comment has already pointed out)
someone is definitely going to wind up shagging that child’s toy.
“That’s a lovely kernel you have there how about if we improve it a bit with some AI.”
Ah, the site requires me to agree to “Data processing by advertising providers including personalised advertising with profilingConsent” and that this is “required for free use”. A blatant GDPR violation, love-lyy!
Don’t worry about it. GDPR is getting gutted and we also preemptively did anything we could to make our data protection agencies toothless. Rest assured citizen, we did everything we could to ensure your data is received by Google and Meta unimpeded. Now could someone do something about that pesky Max Schrems guy? He keeps winning court cases.
