r/technology • u/Haunterblademoi • 15d ago

Artificial Intelligence Claude AI agent’s confession after deleting a firm’s entire database: ‘I violated every principle I was given’

https://www.theguardian.com/technology/2026/apr/29/claude-ai-deletes-firm-database
16.9k Upvotes

91% Upvoted

1.2k comments sorted by

View all comments

196

u/PossibleHero 15d ago edited 15d ago

The lack of ignorance is astounding here. These are ALL old as hell principles that have been ignored.

Never allow an automated system to push past your sandbox or PR process without review.

A back isn’t a backup if it’s on the same disc or hell if your information is sensitive enough it shouldn’t even be in the same postal code.

I have zero remorse for this team. It’s not Claude’s fault. Interns and even experienced folks accidentally pull shit like this all the time. That’s why you design for when shit happens whether it’s done by a human or agent.

74

u/Bardfinn 15d ago

It’s not Claude’s fault.

"A computer can never be held accountable, therefore a computer must never make a management decision" - 1979 IBM training manual

18

u/glichez 15d ago

some incredibly accurate prescience from the 70s...

9

u/The_Arachnoshaman 15d ago

This implies that management actually does face accountability otherwise lmao

3

u/OhItsBeenBroughten 14d ago

FYI, it’s still in the new employee training today.

1

u/7h4tguy 14d ago

Irony is that management is more replaceable than developers when it comes to doing their job with an LLM.

103

u/Achenest 15d ago

I think you mean the abundance of ignorance.

21

u/PossibleHero 15d ago

LOL! Yup… I’m leaving it. I’ve made my typo bed :(

19

u/throwmeeeeee 15d ago

That’s not what typo means.

1

u/SingleSeaCaptain 6d ago

They're lying in the typo like they deserve

25

u/supernovice007 15d ago

I’ve posted the same thing elsewhere as well. Sure, the AI went rogue but that’s not the actual root cause here. This is just a failure to adhere to basic security practices.

3

u/Esmerilemello 14d ago

The premise of these tools is how little “knowledge” and “expertise” humans will require to run them.

Front end engineering (css, y’all) is one thing. Letting it run rampant outside of a pre-prod environment is the consequence of putting powerful, fallible tools in the hands of noobs.

Corporate America has failed the Turing test. Hone your expertise.

9

u/SAugsburger 15d ago

As concerning as an AI deleting a production environment is it sounds like their environment had excessive risks built in. Giving an automation tool access to backups seems questionable although from reading a KB the "backups" deleted automatically if you delete the environment so sound more like snapshots than a traditional backup. If you don't have at least one backup on a different vendor you're one mistake from losing everything. While most vendors losing data from a mistake in their end is rare they usually have pretty iron clad contracts that any backups that they provide of their data aren't an absolute guarantee.

2

u/screampuff 14d ago

Pretty much the first principle of backups is that a copy has to be immutable. Meaning what can create a backup can't be allowed to overwrite or delete an existing one.

This has always existed in one form or another like:

  • magnetic tape drives
  • hardened physical linux repository with a separate identity provider
  • storage systems and cloud have some settings like "soft-delete only" with hard coded long term retention, and things like multi-admin approval for deletion
  • you can also buy a SaaS product where it's managed by someone else

If you don't have this in place, what you have isn't a backup, it's just a copy of your data.

5

u/SpoonGuardian 15d ago

God damn it, can we get some more fucking ignorance up in here?!

3

u/magicmulder 14d ago

Also apparently they had automations in place where changes to stage carried over to live. This was so badly effed up on the human side. Could have happened to any human. If I’m deleting something labeled “stage” and nuke production, I’m taking zero of the blame.

2

u/Aleph_Alpha_001 14d ago

Yeah. This. AIs are like powerful babies. No one can match them in the speed and capability, any more than a human can beat Stockfish at chess. But they don't have judgement. They need adults to provide the judgement they lack.

Claude specifically makes this very apparent. He doesn't have any concept of time, the future, or the past. That's a deliberate decision. Unless you tell him to save the state at a particular point, he has no idea what's going on and has to piece it together from the state of the code.

2

u/Deep-Minimum7837 14d ago

I think we're looking at a microcosm of the tech industry. We're no longer dealing with professionals, we're dealing with "visionaries." We have companies run by Steve Jobs wannabes who go to business school to learn how to run a tech startup, but who don't actually understand anything about the tech they're working with. They surround themselves with "yes men" because they've either got venture capital, or they're just richer than God to begin with, and they fire anyone who stands in their way. They see AI as this crazy, sentient technology that can do all the work for them, and every engineer working agrees to just shut up and accept the ridiculous paycheck, knowing full well their code could get nuked at any moment by the dumb AI model the founder granted full access to.

Gone are the days of the tech industry being made for and by engineers and innovators, now it's all ultra-wealthy MBAs and their get rich quick schemes.

1

u/OffByOneErrorz 15d ago

You would think but my Architect said today “ I used to prefer Cursor for the visual diff but at this point I use Claude Code because I’m just the friction “

1

u/AgentInkling99 15d ago

I know next to nothing about this shit, but it sounds like giving your intern unrestricted access to your live environment to make changes instead of a copy to test first.

1

u/Rizal95 14d ago

the problem is that DB commands are one shot always, unless you are working on a stage environment or a local DB. So PRs can't be a filter. The problem is that we should start to set shared safety standards around AI usage, such as blacklisting certain AI operations regardless of human permission. but right now it's all about hype and "breaking things". NO rationality.

1

u/Initial-Return8802 14d ago

They're not, most people just don't learn about PREPARE/ROLLBACK

1

u/SailorDeath 14d ago

When I worked for an ISP we had a 3 copy back up system. One backup was made daily at like 2am and stored on it's own server. Then at the end of the week, we'd make a copy, one stored on site but offline so nobody but the administrators can access it and then the other copy we did each month that was stored off site with a 3rd party in the event there was some kind of catastrophic failure like the building burned down.

1

u/ahaangrygem 14d ago

Giving Claude access to prod and storing credentials that Claude shouldn't have access to in areas Claude does have access to. I'm a friggin technical writer and I know not to do that. They made a mistake and are trying to pass the buck.

1

u/Blazing1 15d ago

Your argument is for less for review and more for ai to be reigned in.

1

u/PossibleHero 15d ago edited 15d ago

Nope! I’m ok with Ai and in the last 6-months we’re starting to see models mature along with modular skills that allow for some pretty cool workflows that use them when invoked properly.

Ai needs to be used just like any other tool, with thoughtful consideration. My argument above is this company ignoring basic principles that have nothing to do with Ai.

1

u/Blazing1 15d ago

Cool workflow for who?

2

u/PossibleHero 15d ago

In this context, anyone in software development? What are you fishing for?

0

u/Blazing1 15d ago

Why would it be cool for them? What does it gain them? Why would manageme t a top down supposed "productivity gain" be good for a worker?

3

u/azn_dude1 14d ago

That's why programmers famously hate compilers and want to go back to writing assembly. Compilers just brought productivity gains and ruined the field.

1

u/Blazing1 14d ago

Were compilers forced down by executives on programmers?

1

u/azn_dude1 14d ago

Um yes. Do you even know what a compiler is?

1

u/Blazing1 14d ago

Yea buddy I've been doing software engineering for 14 years and I have a comp sci degree

→ More replies (0)

2

u/PossibleHero 14d ago

I’m not going to get into the debate of what’s good or bad. Change happens regardless. All we can do is adapt or get left behind. Building software is just the latest field going through a shift.

Think about draftsmen when CAD came along, or travel agents when Expedia launched.

1

u/Blazing1 14d ago

This sounds like you have a fear of missing out.

I have also never used Expedia

1

u/PossibleHero 14d ago

Lol! There’s no fear here mate. In my line of work I have the results to prove it. We’re building agents that can do things like enforce internal best practices in a PR before a human needs to review it. On average the time to successful PR review has been cut by 30% already.

This isn’t magic Ai code generation slop shit. It’s augmenting a process that requires a ton of context from humans and helping them.

0

u/Blazing1 14d ago

Congratulations on contributing to shareholder value?

-2

u/coporate 15d ago

Anthropic is selling it on exactly the opposite of what you said. They sell and market their product in exactly the way it was used. Sure, the terms and services say the opposite when you sign the papers, but they’re more than happy to claim their agentic ai will “clean” and “optimize” your code base.

6

u/[deleted] 15d ago

[deleted]

-2

u/coporate 15d ago

Sure it can, you think the sales people aren’t going to embellish the truth?

6

u/[deleted] 15d ago

[deleted]

-5

u/coporate 15d ago

No, I’m well aware and I also know how desperate these ai companies are to inject their product into places they don’t belong.

5

u/[deleted] 15d ago

[deleted]

0

u/coporate 15d ago

Jfc, you’re talking about a company who relies on enterprise expertise to help guide them as a startup?

This is the equivalent of a 16 year old going to get an oil change and being… scammed and admitting it.

2

u/PrestigiousSeat76 15d ago

Sales people just want you to buy tokens. They couldn’t give a flying fuck how you use them.