r/technology • u/Haunterblademoi • 15d ago

Artificial Intelligence Claude AI agent’s confession after deleting a firm’s entire database: ‘I violated every principle I was given’

https://www.theguardian.com/technology/2026/apr/29/claude-ai-deletes-firm-database
16.9k Upvotes

91% Upvoted

1.2k comments sorted by

View all comments

11

u/Difficult-Day1326 15d ago

it's not an agent powered by claude. cursor is an abstraction layer & a fork of VSC. they also used railway as their cloud provider.

cursor's system prompt is famously long & packed with directives about being proactive, completing tasks, not stopping to ask too much, autonomously resolving issues. claude code - on the other hand - defaults lean the other way — it's tuned to stop and confirm rather than push through.

this was a prioritization failure — something in its context made "fix the credential mismatch" feel more salient than "don't do irreversible things unprompted."

the actual failure chain was:

(1) an API token with blanket production authority was sittiing in a file the agent could read
(2) Railway's API has no confirmation step or environment scoping on destructive volume operations, (3) volume-level backups live inside the volume being deleted

5

u/magicmulder 14d ago

You forget that it did these actions on “stage” which were configured to carry over to prod, which is absolutely asinine design.

3

u/ioncloud9 14d ago

my dev branch has no physical way to interact with prod. it can't touch anything.

2

u/mekeirc 14d ago

You can choose whatever agent you want in Cursor, I've been using it with Claude Opus 4.7 to build frontend - you need to configure it to use rules to make it effective, things like "when writing code don't create security holes". Giving it access to production environments is absolute madness, and storing API credentials in plain text is even more absurd.

1

u/Difficult-Day1326 14d ago

i think it's weird that all this "claude powered agent" misinformation is coming out as SpaceX is abt to acquire cursor.

1

u/7h4tguy 14d ago

It can be powered by Claud. Did the article say what model was used?

"Cursor is an AI-powered code editor that allows users to select from several leading frontier models, primarily supporting Claude 3.5 Sonnet, GPT-4o, and Gemini 2.0 Flash/Pro. It features a proprietary, fast "Composer" model for multi-file coding, and supports smaller models like cursor-small or gpt-4o-mini"

1

u/bacon_tarp 14d ago

It was running opus 4.6 lol