I spent a decade working in insolvency.
When we were going into a business that had failed the question was “Are the idiots, criminals or both?”
One highlight:
A boat sales / marine business goes bust. When we arrive with the paper work and seize the place there are about a dozen new boats on the lot worth several million. We change the locks on the gates.
Arrive the next day, the gates have been busted open and several million in boats are now missing. We look up the addresses of the owners (one of them lives on acreage) and drive to their property…from the road we can see the boats stashed there. Really smart guys.
So we call the police. Someone inside notices use there and decides to flee with one of the boats, it is huge but they think they can get away.
We then have the slowest car chase in history as we calmly follow this guy towing a boat on a trailer down the road while talking to the cops to meet us.
Love this. Lol.
I used to maintain a system that was used to track loss prevention. Basically security company that followed delivery trucks. It was wild to read database records about these guys openly selling their stolen goods (building supplies) while testing my code changes.
The database was sanitized of identifying info if anyone cares.
A milder story than some here:
Someone had accidentally sent an email to the whole company of 44,000 or so employees. Cue a bunch of emails -replying all- asking why they were receiving the email… followed by another wave of emails asking people to stop replying all.
Was a great popcorn moment, and made me laugh every time a new email came in.
Once a year in government. I just cringe, every time.
Idiots.
We had this one at a place I worked, except it was a very large company and the chain started crashing servers.
I think we work for the same company…
Quite possibly but gotta keep the internet life separate from real
This happens in the military far too often.
This used to crash outlook servers. If anybody has ever replied to one of these know that I hate you.
Unsubscribe
Do you work for UAP? This happened recently at my job, and was fucking hilarious.
No, this happened a few years ago, but clearly it’s a lot more common than I thought!
An isolated shingle spit nature reserve. We’d lost mains power in a storm some while back and were running on a generator. Fuel deliveries were hard to arrange. We’d finally got one. We were pretty much running on fumes and another storm was coming in. We really needed this delivery.
To collect the fuel, I had to take the Unimog along a dump track and across 5 miles of loose shingle - including one low causeway stretch through a lagoon that was prone to wash out during storms. We’d rebuilt it a LOT over the years. On the way up, there was plenty of water around there, but it was still solid.
I get up to the top ok and get the tank full - 2000L of red diesel - but the wind is pretty strong by the time I have. Half way back, I drop down off the seawall and reach the causeway section. The water is just about topping over. If I don’t go immediately, I won’t get through at all and we will be out of fuel for days - maybe weeks. So I put my foot down and get through that section only to find that 200 meters on, another section already has washed out. Oh shit.
I back up a little but sure enough the first section has also washed through now. I now have the vehicle and a full load of fuel marooned on a short section of causeway that is slowly washing out. Oh double shit. Probably more than double. Calling it in on the radio, everyone else agrees and starts preparing for a pollution incident.
In the end I find the firmest spot that I can in that short stretch and leave the Moggie there. Picking my route and my moment carefully I can get off that ‘island’ on foot - no hope with the truck - BUT due to the layout of the lagoons only to the seaward ridge, where the waves are now crashing over into the lagoon with alarming force. I then spend one of the longest half-hours I can remember freezing cold and drenched, scrambling yard by yard along the back side of that ridge and flattening myself and hoping each time a big wave hits.
The firm bit of causeway survived and there was no washed away Unimog or pollution in the end - and I didn’t drown either - but much more by luck than judgement.
These days I am in a position where I am responsible for writing risk assessments and methods statements for procedures like this. It was another world back then.
That is seriously some action movie shit
I worked at a sandwich shop and had given my two weeks notice a few days earlier. My manager came to me and asked me to clean up the bathroom…alright. I could smell it before I even opened the door.
I told my manager I’d clean it if he’d still give me the employee discount after I was gone. “Done”. That’s when I knew it was really bad.
When I opened the door I discovered someone had ass-blasted the bathroom. I’m not talking about blowing up the toilet, they did that too, but they had dropped their drawers and point-blank diarhea shotgunned the pipes under the sink.
My manager didn’t honor the employee discount after I was gone, either.
My manager didn’t honor the employee discount after I was gone, either.
They never do. I had a manager try that shit on me when I was working food service, and I turned it around on him and made him get one of his toadies to clean it up after talking a bunch about “not being trained for biohazard cleanup” and “OSHA regs” which got him to back down, and I told all my coworkers the same so they’d tell him to fuck off too.
Still wish I could have been there when the feds showed up and escorted him out of the building.
What’s the story behind that last sentence with the feds if you dont mind sharing
Sure, though it isn’t super exciting. The assistant store manager, Lucio (fuck you Lucio, you fucking piece of shit) was an illegal immigrant who managed to fake his way through the verification steps, and INS showed up one day and vanned him while I was away going to college.
I wasn’t sad to see that fucker go, because he had a number of Mexican kids that he let do whatever they wanted, including Edgar who sexually harassed multiple girls out of the store and Lucio wouldn’t do shit because “why would I fire him when we’re already losing one employee”. Never mind that wed already lost 4 girls at that point and one of them was pregnant at the time. I’m honestly surprised he and I never came to blows because I didn’t hide that I fucking hate him.
Your second-to-last word missing.
Come back and return the toilet to the state you found it in.
Then the next employee gets the same deal, and the cycle of shit continues
We really honor our hunter gatherer ancestors by passing down these stories
A “full refund”
That was literally oh shit a situation
o7 poop veteran
I used to work at a car dealership. One day I had to use a bay in a different building because my usual workplace was occupied. The other building had a lift that I hadn’t used before.
Anyways, I drove the car onto the lift, got out and placed the arms of the lift under the jacking points like I had done a thousand times before. I raised the lift a little and checked if the placement was still correct. It looked good, so I raised the car to a medium height. When I looked again, I realized that this lift had a central platform that was also raised and was set about 20 centimeters higher than the four arms that usually lift the car.
This 90.000 Euro SUV was basically balancing on a 180x50cm piece of metal right in the center. I managed to lower it down safely but my pulse goes up just thinking about that day.
Sharing my story for posterity.
I used to work at a medical center for old folks with varying disabilities. It was a great job all things considered, just didn’t pay very well and the scheduling was a mess.
Anyway, one day I’m cleaning tables on the dining room when I hear on my walkie talkie that one of the new people need help with a guy in the bathroom. Usually “they need help” means “something has gone awry, please unfuck the situation” and, since I was the supervisor on shift, my job frequently involved untucking a situation.
I arrive outside the bathroom door and the new employee tells me that she walked into a situation that she wasn’t prepared for. I figured it was some poop, or the guy fell asleep on the toilet or something.
I walk in and the walls were all painted with poop. The sink was painted with poop. The floor was painted with poop. The paper towel dispenser had poop all over the front of it.
The poor guy had gone to the bathroom, got confused and tried to remember what toilet paper was. He saw me and knew I was there to help, but he was nonverbal. His way of saying thank you was to gently take his hand and rest it under your chin.
He did so, but his hand was also still covered on poop.
I’m used to poop. It’s a normal job hazard in that line of work. But something about having to clean myself and every surface in the room from caked poop while somebody else gave the poor guy a shower…that kind of story sticks with you. To this day I can’t look at finger paints without feeling a little queasy.
I’m sorry, that sounds like a really shitty day.
Your story makes up for the non-work related stories in this thread. It’s both work related and shitty lol. I’m sorry you had to go through that.
Normally I’m very much anti “lets use robots to replace jobs”, but this is one case where I think it would be a win for everybody. The robot won’t care, and the elderly person won’t feel their dignity lost, and all is taken care of behind closed doors.
My grandma started losing control of herself towards the end, and my mother did overtime in taking care of her and cleaning her. This sounds sweet, but it was a bad situation for everyone. My mother essentially started treating her own mother like a baby, often in front of us, and my grandmother (a proud and strong woman my entire life) essentially lost her sense of dignity and independence. I still remember her as the strong and proud woman she was, and I do my best to forget her last year.
We need robot caretakers.
The only problem is that robots don’t have the kind of sense of connection and humanity that human caretakers often have, on top of the general complexity of the task. I was always frustrated when family would visit and treat their aunt/cousin/etc like a baby when like, no, they’re 80 years old and were raised on a farm. It’s really just a matter of needing appropriately trained caretaking staff who are also paid enough, which sadly the industry lacks both of those things
I work in live sports TV.
Champions League Final (European Football). Kind of a big deal. Doing a money shot camera behind the goals. 4 minutes in, one of the cameras goes dead. I try all the fixes I can remotely, while all the while the director wants the camera back up and getting quite heated about it. The only thing left to try is to replug the remote head. That part is, unfortunately, 10m past the ad boards, on the grass.
I waited for play to be down the other end (and gave the security guy a heads up what I was about to do!). Jumped the ad boards, and replugged everything. At that moment, there’s a roar from the crowd, as there is a break down the wing. I am VERY much NOT supposed to be on the grass! My brain tries to freeze, luckily, 100 million years of instincts kick in to save my arse. Next thing I know, I’m finishing a sort of head first leap/ airborne commando roll, over the ad boards to tuck in behind them.
The camera restarted just before a shot on goal. The operator captured it perfectly. Much to the directors relief/delight. I also, somehow managed to avoid being on any of the camera shots. I’m still not quite sure how.
Related XKCD: https://xkcd.com/705/
Definitely, though it was motivated by the voice of god (the director) complaining with every camera shot missed.
haha, epic !
Alt tabbed once too many times, clicked drop database, clicked yes. Realized what I’d done and panicked.
Deleted the user db for the east coast auth server for the game America’s Army: Operations. Thankfully it was the secondary so we just redid replication.
That was a nice game. It still has a small community but I wish they had open sourced it. Probably not possible because of licenses…
Man I did that once as a kid before i knew how to back stuff up properly. Months of work just gone. Now im hyperparanoid about backups and restoration procedures for everything.
If you think you fucked up, remember that EVE Online once failed to remove $instdir\boot.ini (the nice-to-have gamefile) and instead deleted c:\boot.ini (the very critical Windows file).
Two nights ago I had a random meeting with the CEO, who I have a really good relationship with, added to my calendar. Thought nothing of it.
I entered the zoom call and said ‘so am I getting fired?’
The answer was yes.
Awkward silence ensued for a minute until they started telling me about the severance package.
Side note: I can try to negotiate that severance a bit right?
Definitely negotiate that severance. What a shit deal
How do you negotiate severance? Don’t you have zero leverage in that situation?
Some severance packages will have a non disparagement clause in it, or they’ll say you can’t recruit people to xyz competitor for a number of years. You can then say “yes I can do that, but if and only if you give me 20% extra of my estimated salary”
deleted by creator
Urgh yeah I had one of those. A “small quick meeting” that makes you think they just want an informal update. Nope, its the getting fired talk. Still, turned out to be a blessing.
Mine sucks because it’s the best job I’ve ever had. Planned on staying as long as they’d keep me (just under 5 years it turns out) and had no plans at all to even poke around at other roles.
The silver lining is I’ll prob get a nice pay increase since I’ve been pretty underpaid at this place as it’s an NPO.
Older gentleman walked into the lobby of our office. None of us knew who he was or had seen him before. He looked confused and lost. Someone went over to ask if they could help him. He tried to but didn’t respond. Then fell over. Hit his head on a table on the way down. Was dead before the pandemics arrived.
We were all in shock. Poor guy was starting into a stroke when he walked in. Maybe even walked into our office to try getting help. But it was already too late.
before the pandemics arrived
I know this was a typo and you meant to write paramedics, but all I could think first thing I read this was “what a lucky bastard”
“Run, you fools.”
This fucked me up when I learned and finally accepted it, but it’s actually “Fly, you fools!”
You are both wrong it is “run, you fly fools!”
lol swipe must have boomed me. But that does make it funny.
My first salaried job was also my first proper IT job and I was a “junior technician” … the only other member of IT staff was my supervisor who had been a secretary that got a 1 week sysadmin course and knew very little.
The server room was a complete rat’s nest and I resolved to sort it out. It was all going very well until I tripped over the loose SCSI 3 cable between the AIX server and it’s raid array. While it was in use.
It took me 2 days to restore everything from tape. My supervisor was completely useless.
A few months later I was “made redundant”, leaving behind me everything working perfectly and a super tidy server room. I got calls from the company asking for help for the following 6 months, which I politely declined.
deleted by creator
Yeah, I got laid off twice more before switching careers. Both times they wanted me to come back and fix stuff after letting me go.
It goes hand in hand with the “if someone works hard, they should be given more work as a reward” line of thinking.
That’s when you offer consulting and tell them your hourly rate!
I didn’t have them over a barrel, they were just being lazy and trying to exploit me further for free.
It’s always fun when a job calls you up after you’ve been fired to ask how to do the things they didn’t know you were doing
Yep, I remember in one job I was at for 8 years a manager 2 levels up complemented me for sorting out the networking for a re-arrange of our own office … I was gobsmacked because I’d been managing a whole network and server upgrade for a client that involved well over 1000 users at the time yet an hour of fiddling with wires under desks was the only thing that got his attention.
One job I was fired from and rehired within the day, after they quickly realised that I was their only Android developer and they couldn’t build an app with just hopes and wishes. They fired me again later, which they quickly regretted since I was the only one with the signing key (meaning they couldn’t update the app).
Not my oh shit moment but certainly someone’s. Working in a call centre they sent out an example of a fraud email that was being sent out with our logo. It asked for all your personal information and credit card information.
Several individuals replied with all their details filled in. 3 of them replied all (entire call centre) with their details filled in.
deleted by creator
oh my god. This…unfortunately tracks for call centers. The world capitol of “we expect you to understand this thing with zero training”
I have a small PC I use for exposing a private PC to the wider web via nginx proxy. It had two accounts on it: mine, and one I called “remote” with some basic password I set up to forward the proxy connection.
One day, this machine started making 100% CPU noises, for several hours. Wtf? I check the processes and a Tor node had been setup and was transmitting gigabytes to some Russian IP.
My brain goes into panic mode, I kill the process, wipe the remote user, and eventually pull the Ethernet plug.
I wish I hadn’t wiped the user directory as I wanted to know what was being sent and where. Nonetheless the logs showed that several Russian IPs had been attempting an SSH brute force for literally months and one finally guessed “remote” and weak password I set for it.
I have decades of experience on Unix system, and I cringe having made such a rookie mistake.
Lesson learned: change the default SSH port to a transient port, have one dedicated SSH user with a non-standard username, and use auth-key entry only.
I still wonder what was being sent over that Tor node, and why it required all the CPU cores. My best guess is crypto mining, or it was used for a DDOS attack net somewhere.
Obfuscation is not security, changing the port doesn’t increase your security
I see this claim all the time, and it bugs me every time. Obfuscation is a perfectly reasonable part of a defense in depth solution. That’s why you configure your error messages on production systems to give very generic error messages instead of the dev-centric messages with stack traces on lower environments, for example.
The problem comes when obscurity is your only defense. It’s not a full remediation on its own, but it has a part in defense in depth.
Changing the port isn’t really much obfuscation though. It doesn’t take long to scan all ports for the entire IPv4 range (see masscan)
It helps against stupid automated attacks though.
If someone has changed the port it’s likely that they have set up a great password or disabled password auth all together.
It’s worth it for just having cleaner logs and fewer attempts.
It’s worth it for just having cleaner logs
Those logs are useful to know which IPs to permanently block :)
Technically a password is obfuscation anyway
I hear you, but I disagree:
It buys you enough time to check the journals and see that a group of IPs have attempted various ports giving you enough time to block the IP altogether.
It also buys you disinterest from the malicious host, since probably there’s a hard limit on how many ports they will test, and they will flag your machine as “too much work” and try another.
Again, I agree with you that obfuscation is not security, but it sure does help.
From what I understand you obfuscate the port in order to limit the amount of incoming attacks. But then fail2ban would be a much more effective tool.
The disinterested aspect you described is the actual problem. Because it’s based on the assumption your port won’t be found, but it definitely will, and as soon as that happens you’ll end up in a database such as shodan and the entire effect is GONE.
I figure you’d know about it by now, but fail2ban would really help.
What do you think they were transmitting?
I think they were either computing crypto-hashes and passing on the results back home (via Tor), or they were using my machine to send out several ping/fetch requests over Tor to DDOS some unknown target machine.
So can this pretty much always be shut down by having sufficiently complex + long pw?
I want to say “yes” but you should still try to change the default ports for any process open to the web. Just because they can’t guess your ssh, doesn’t mean they can’t upload a root php script to your webserver which allows file uploads.
Just be as invisible as possible. Run
nmap
on your localhost with the defaults and see if anything is set to open. If so, change that port.What about Stealth mode
What is stealth mode?
On Mac its part of security/firewall settings or sumfing
Worked at a hotel. Our phone system required you to dial 9 to reach an outside line on every phone except one: the fax machine, which was set up to dial that 9 automatically when you started dialing.
It would take a second to start making dialing sounds while it dialed that 9 in silence. Our AGM would dial the 1 before the area code, not hear any immediate sounds, and then press the 1 again. Then dial the rest of the number.
So as far as the switchboard I used to direct calls was concerned, someone just dialed 911. So it made the “holy fuck someone’s dying” alarm, our local 911 dispatcher got to hear a fax machine screeching its handshake tones, and I got to go into “oh shit a guest is having a stroke” mode, only to find out that no, my manager didn’t read the sign posted over the fax machine because of this behavior. Again.
And then we would get a call from 911 asking what the emergency was and have to explain that it was dialed by mistake.
Of course, this was almost always during a rush.
Great thing about fax machines. They can be set up in such a way that if they don’t get a fax handshake, they wait a few minutes and try dialing the number again.
I’ve always found the dial 9 to get out thing a mistake waiting to happen, why not pound hash twice as the tone? How did manufacturers settle on 9 as the sane default
I’ve always found the dial 9 to get out thing a mistake waiting to happen
Can confirm.
I used to work in a call center, and it’s astounding the number of calls that I got that were actually people trying to send faxes.
Strap in friends, because this one is a wild ride.
I had stepped into the role of team lead of our IS dept with zero training on our HP mainframe system (early 90s).
The previous team lead wasn’t very well liked and was basically punted out unceremoniously.
While I was still getting up to speed, we had an upgrade on the schedule to have three new hard drives added to the system.These were SCSI drives back then and required a bunch of pre-wiring and configuration before they could be used. Our contact engineer came out the day before installation to do all that work in preparation of coming back the next morning to get the drives online and integrated into the system.
Back at that time, drives came installed on little metal sleds that fit into the bays.
The CE came back the next day, shut down the system, did the final installations and powered back up. … Nothing.
Two of the drives would mount but one wouldn’t. Did some checking on wiring and tried again. Still nothing. Pull the drive sleds out and just reseat them in different positions on the bus. Now the one drive that originally didn’t mount did and the other two didn’t. What the hell… Check the configs again, reboot again and, success. Everything finally came up as planned.We had configured the new drives to be a part of the main system volume, so data began migrating to the new devices right away. Because there was so much trouble getting things working, the CE hung around just to make sure everything stayed up and running.
About an hour later, the system came crashing down hard. The CE says, “Do you smell something burning?” Never a good phrase.
We pull the new drives out and then completely apart. One drive, the first one that wouldn’t mount, had been installed on the sled a bit too low. Low enough for metal to metal contact, which shorted out the SCSI bus, bringing the system to its knees.Fixed that little problem, plug everything back in and … nothing. The drives all mounted fine, but access to the data was completely fucked,
Whatever… Just scratch the drives and reload from backup, you say.That would work…if there were backups. Come to find out that the previous lead hadn’t been making backups in about six months and no one knew. I was still so green at the time that I wasn’t even aware how backups on this machine worked, let alone make any.
So we have no working system, no good data and no backups. Time to hop a train to Mexico.
We take the three new drives out of the system and reboot, crossing all fingers that we might get lucky. The OS actually booted, but that was it. The data was hopelessly gone.
The CE then started working the phone, calling every next-level support contact he had. After a few hours of pulling drives, changing settings, whimpering, plugging in drives, asking various deities for favors, we couldn’t do any more.
The final possibility was to plug everything back in and let the support team dial in via the emergency 2400 baud support modem.
For the next 18 hours or so, HP support engineers used debug tools to access the data on the new drives and basically recreate it on the original drives.
Once they finished, they asked to make a set of backup tapes. This backup took about 12 hours to run. (Three times longer than normal as I found out later.)
Then we had to scratch the drives and do a reload. This was almost the scariest part because up until that time, there was still blind hope. Wiping the drives meant that we were about to lose everything.
We scratched the drives, reloaded from the backup and then rebooted.Success! Absolute fucking success. The engineers had restored the data perfectly. We could even find the record that happened to be in mid-write when the system went down. Tears were shed and backs were slapped. We then declared the entire HP support team to be literal gods.
40+ hours were spent in total fixing this problem and much beer was consumed afterwards.
I spent another five years in that position and we never had another serious incident. And you can be damn sure we had a rock solid backup rotation.
(Well, there actually was another problem involving a nightly backup and an inconveniently placed, and accidentally pressed, E-stop button, but that story isn’t nearly as exciting.)
Imagine the difference trying to get that kind of support these days. Especially from HP
No kidding. Where I’m working now, it takes an HP CE over a week just to bring out a new hot swappable drive after we jump through a number of request hoops.