Lately I have been using AI more and more in my codebase and that’s been a bit of a hit and miss if you ask me.
I reckon it’s an amazing tool that allows developers to truly optimise their workflow, however at times laziness take over and code reviews are not as frequent as they should be.
I really REALLY wanted to build something without using AI after having spent months and months getting a bit too complacent. And this is the first idea that came to mind. Of course to celebrate this occasion I went for a technology I hadn’t used in almost a decade: jQuery!!!
You can find the repo here: https://github.com/fccview/ackchyually-ai
You can play around with the tool on fccview.github.io/ackchyually-ai
Few disclaimers:
- I am a full stack javascript developer (with extensive knowledge of php and a very good grasp of shell). This means majority of the detection will be revolving around javascript, if you develop in a different language and you feel like you know AI patterns in that language PLEASE do create a pull request so we can make the detection even more accurate for other type of syntax!
- The tool uses MY OWN metrics, this is doing what I usually do to see if something is (in my opinion) been made with AI and barely reviewed by a human. It’s not gospel, it’s not a scientific method, it’s most definitely going to be flawed, so keep it with a grain of salt, it can always be improved and collaboration/community support will definitely help with it
- I don’t hate AI, on contrary I enjoy using it and I find value in not needing to “google it” whenever I hit a blocker, more times than not, AI has a valid solution that can be used, refined and applied properly. What I have mostly an issue with is people randomly generating code, not reviewing it and trying to sell you the moon when they barely understand the architecture of what they built
- Please do not make this post a fight between pro/against AI individuals, that’s not the aim of it
All the patterns I am using to identify possible AI code are here: https://github.com/fccview/ackchyually-ai/tree/main/data if you come up with more patterns/ideas and have more ideas on what to look for please reply here, open an issue and/or make a pull request, but at ONE CONDITION, do NOT use AI for this. It’d go against the core principle of this one pure little app I decided to build <3
I was trying to solidify a joke about beating up people who are pro-AI, but just can’t get the timing right.
But I do have a question for someone who can apparently see things from both sides. Do you think that people’s perceptions and engagement would be different if “AI” was marketed as natural language processing coding / programming expert system.
I feel like some of the visceral pushback against “AI” would be reduced if we didn’t all have to pretend there was actual intelligence in operation. Plus it would seem more like the successor to context aware code completion, rather than the successor to the individual sat at the keyboard.
natural language processing coding
I believe the hype would be basically non existant and most of it would be bashing against it because it get’s most things wrong (basically a StackOverflow copy bot)
Maybe.
I think the pushback stems from a bunch of different things.
It’s genuinely bad at some things. asking it to make a clock out of CSS and HTML is mostly awful.
Historically, it’s been really bad at everything. So if someone hasn’t done a serious dive on it recently, they’re going to have the impression that it’s even worse than it really is.
A lot of people don’t understand how to use it, a lot of times it’s like working with a monkey’s paw. You’ve got to pre-guess all the things that could go wrong and keep adding detail until it has no choice but to do it right. And even then, you have to come back and do iterations sometimes.
It’s making a bunch of oligarchs extremely wealthy, for no good reason, on the backs of the working class, while we can barely buy RAM. At the same time, they’re burning through a hell of a lot of natural resources.
They’re shoving options and features down our throats and making us pay for them even if we don’t want to use them.
Some people are genuinely scared that corporations will use it to replace skiled labor with unskilled labor, which they are.
I have seen advanced versions rewrite an entire cross-platform basic interpreter in a couple of tries.
I lost a rather complicated Python program I wrote to manage projectors for my Halloween display. I had it make a framework. I went through all of my different options and modes one at a time and explained exactly how they needed to work. I recreated a couple of weeks of work in a couple of hours and added a significant number of features.
It’s crap like make that admin page look good on a cell phone that’s absolutely bananas. That’s a feature I would never have the time to sit down and work on because it’s not that big of a deal. But it would literally be a day of trial and error on multiple test devices for me to write it myself.
Would it be better received if it were marketed differently? Probably a little bit. But not beyond the things that I wrote about. It would be a subtle improvement in visibility I feel.
100% it would! I think the biggest issue around the AI hate is a total misunderstand of how it works, paired with people using it for the dumbest reasons, actually draining important resources when there’s absolutely no need for it.
I think eventually it’ll be inevitably regulated as the actual shortage of water we’re seeing in the US is unmanageable, and once it does get regulated things will start working way better
p.s. i’m not talking about local models, I doubt these would ever be regulated and they SHOULDN’T, I’m talking about how many resources a company that allows AI usage should be able to utilise, mostly.
I think eventually it’ll be inevitably regulated
I know some fear regulation, or over regulation, but right now it’s the wild wild west. Reminiscent of when the internet first became available to the general public. Regulation is necessary, as much as I sometimes chafe at the thought. Corporate penny pinchers are noticing the increased bandwidth u$age and there is no quid pro quo for them.
Regulation is absolutely needed, anyone saying otherwise is just deluded at this point 😅
@shads @selfhosted Agreed. Words carry baggage and #AI has a lot of baggage.
I wish it want thrown around like it was AGI.
AI for ML systems has always kinda been a misleading term after it was co-oped by science fiction
AI is an evolutionary dead end and will be the death of humanity as we know it.
That’s pretty neat, will play around with it whenever I get the chance later today
I share some of the same views you do about AI. I do use AI to help me with code. Not to develop and sell, just for my own personal use. So I tested out your ackchyually-ai. I had Grok create a docker compose to combine BorgBackup and BorgUI.
Result:
spoiler
######################### # Services ######################### services: # ------------------------------------------------- # Borg repository (data only) # ------------------------------------------------- borg: image: ghcr.io/borgbackup/borg:latest container_name: borg_repo restart: always command: ["sleep", "infinity"] # keep container alive volumes: - ./repo:/var/borg/repo # persistent repo - ./ssh:/root/.ssh:ro # optional SSH keys (read‑only) environment: # Uncomment if you want the repo to be encrypted by default # BORG_PASSPHRASE: "${BORG_PASSPHRASE}" networks: - backup_net # ------------------------------------------------- # Borgmatic – scheduled backup runner # ------------------------------------------------- borgmatic: image: ghcr.io/borgmatic-collective/borgmatic:latest container_name: borgmatic restart: always depends_on: - borg volumes: - ./repo:/var/borg/repo # same repo as above - ./ssh:/root/.ssh:ro # SSH keys (read‑only) - ./borgmatic/config.yaml:/etc/borgmatic/config.yaml:ro - ./borgmatic/source_dirs:/source_dirs:ro # directories to back up environment: - BORG_PASSPHRASE=${BORG_PASSPHRASE} # Run borgmatic every day at 02:30 am using the built‑in cron of the image command: ["cron", "-f"] # If you prefer a host‑side cron, replace the command with “tail -f /dev/null” # and schedule `docker exec borgmatic borgmatic run` from the host. networks: - backup_net # ------------------------------------------------- # BorgWeb – Flask UI for browsing the repo # ------------------------------------------------- borgweb: image: ghcr.io/borgbackup/borgweb:latest container_name: borgweb restart: always depends_on: - borg environment: - BORGWEB_REPO=/var/borg/repo - BORG_PASSPHRASE=${BORG_PASSPHRASE} # Optional basic‑auth (handled by the tiny built‑in Flask auth) - BORGWEB_BASIC_AUTH_USER=${BORGWEB_BASIC_AUTH_USER} - BORGWEB_BASIC_AUTH_PASS=${BORGWEB_BASIC_AUTH_PASS} volumes: - ./repo:/var/borg/repo # Expose the UI directly; you can map it to any host port you like. # 8080 = plain HTTP (for testing only) # 8443 = HTTPS if you put an external TLS terminator in front. ports: - "48443:8080" networks: - backup_net # ------------------------------------------------- # OPTIONAL: Minimal NGINX reverse‑proxy for TLS & auth # (If you already have a front‑end proxy you can skip this block) # ------------------------------------------------- nginx: image: nginx:alpine container_name: nginx_tls restart: always depends_on: - borgweb volumes: - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro # If you have your own cert/key files, mount them here: # - ./certs/fullchain.pem:/etc/ssl/certs/fullchain.pem:ro # - ./certs/privkey.pem:/etc/ssl/private/privkey.pem:ro ports: - "443:443" networks: - backup_netThen I ran it through your checker.
spoiler
Results:
Result breakdown
Overall score
This is entirely made up by ME, it’s not an official metric, so please don’t take it as gospel.
There’s a lot of AI generated code here, it’s getting dangerously close to be sloppy. The ratio of comment to code is over 20%, this automatically pushes the score to 100% Comment ratio The amount of comments in AI generated code is borderline ridiculous. Comments are good, too many (often redundant) comments are just bloat.
34 comments for 89 lines of code. That’s a ratio of 38%. The site assumes something is 100% AI generated if the ratio goes above 20%.
Interesting that comment code was the trigger apparently. Personally, I comment a lot mainly because my brain is shit, so it’s very helpful. I haven’t run the docker compose, but I might just do that to see if AI came close. I’m sure some of the experts here would find the compose file to be wrong in some nature or another.
Hey! Thank you for testing it out, I think in my head, even the most verbose of dev wouldn’t leave >20% of comments in their codebase. The percentage works on a ratio of
(commentsCount / linesOfCode) * 100so it doesn’t just flag “a lot of comments”, it mostly checks for “too many comments”, that said, the “use common sense” at the top needs to be taken quite seriously, for example if there’s a majority of comments but none of the comments feel like written by AI, it’s clearly just the developer being verbose :)p.s. I find AI is pretty damn good at making docker compose files, it’s probably gonna work just fine <3
I think comment ratio could be a fairly misleading metric. There are programs out there that requires a lot of context upfront that distills down to just a few lines of code, and especially those that are more academically interesting.
Hey! Yeah you are right, I may remove the >20% comments to line of code resulting in the code being 100% AI driven
That said, you are obviously referencing a very niche sector, the vast majority of software engineering doesn’t require that absurd amount of comments… I can’t stress enough how verbose a ratio of 20% comments to lines of code actually is lol
I don’t claim it to be common practice, just saying that it exists. That said, it may be “niche” in the grand scheme of things, but by no means do I think it’s small and insignificant. If anything, such codebases are typically foundational libraries in the giant stack of cards most other software engineers build.
See also antirez’s article about code comments https://www.antirez.com/news/124 You’ll notice how some of the snippets would’ve easily failed your criteria.
I don’t claim it to be common practice, just saying that it exists. That said, it may be “niche” in the grand scheme of things, but by no means do I think it’s small and insignificant. If anything, such codebases are typically foundational libraries in the giant stack of cards most other software engineers build.
That’s indeed very valid! As I said, I may have been a bit too harsh on the comment rule, definitely one to review properly <3
I think in my head, even the most verbose of dev wouldn’t leave >20% of comments in their codebase.
Well, I guess that’s where it’s pretty clear that I’m no dev. LOL Give me a # key and I’ll go nuts. At my age, and with other issues, I need all the help I can get. It’s a pretty cool tool ackchyually. I tested out a few more compose files last night without commenting and they seemed to pass the test. Thanks for sharing. It was a pretty fun experience.
Aw thank you for the kindness ♥️ There’s nothing wrong in verbosity btw, I may have been a bit too harsh on my parameters, I’ll tweak it a little :)
I’m far from an expert and have never setup borg, but I see some possible problems with that config, though I don’t think I see anything that would cause it fail.
-
In the environment section of borg, borgmatic, and borgweb, the “BORG_PASSPHRASE” variable is used inconsitently, it either needs to be uncommented in borg or commented out in borgmatic and borgweb.
-
Networks: having a single “backup_net” network is less than ideal from a security standpoint. Nginx should probably be in it’s own network (an ipvlan or macvlan network) in addition to the backup_net. backup_net should be isolated to the host that are running borg, while any communication into borg should be routed through nginx on its ip.
Hey, thanks for the back-check. I’m an expert at nothing. I know just enough to get by or when something might look super squirrely. The experiment was to see OP’s tool in action. I was just curious to see what would trigger a response. I wasn’t pitting the accuracy of AI against OP’s tool. I realize. even tho I use it, that AI can be inaccurate. Sometimes I’ll pit a few of them against each other to see how each interpret the data. I even use it when creating music. I am clinically deaf, and there are some frequencies I just can’t hear that well or at all. So I use AI in the mastering process. It’s just for fun. I’m not looking to break into the biz.
-
Great that you chose weights for the final evaluation. Black, and whitelists are just too …0D/binary for any advanced use, and are unfortunately used everywhere.
Hi, yeah! For sure! Indeed the world is not black and white! But even with weight, take everything with a pinch of salt <3
