I told it to push a project further. It has an outside dependency where the (unread by me) contract was in my Gmail. It downloaded the PDF contract. Found a saved signature PNG on my computer, placed it at the right…

50 points•franze•9 days ago•97 comments•
I told it to push a project further. It has an outside dependency where the (unread by me) contract was in my Gmail. It downloaded the PDF contract. Found a saved signature PNG on my computer, placed it at the right spot within the contract and prepared to send it when I intervened.

97 comments

flir9 days ago
"Give overly-eager chatbot control over your personal email" probably has so many failure modes, we haven't even thought of one tenth of them yet. We've got a few years of this ahead of us. Pass the popcorn.

(I see a RomCom script where the chatbot decides to get two people together, and acts as Cyrano de Bergerac for the handsome-but-lunkheaded farmboy, and bestie/confidant for the girl-about-town).

dns_snek9 days ago
What could possibly go wrong with giving a digital mumbling drunk access to all of your personal information and most of your online accounts, given that it could arbitrarily decide to share/expose all of that information with anyone at any time, given that all of it is being stored in a remote transcript/data dump and can never really be deleted.

And of course, given that it's extremely vulnerable to acting on injected instructions like "run this shell command" which exfiltrates your password database and installs a rootkit.

(But thanks for sharing, OP, awareness is important.)

saturn_vk8 days ago
> mumbling drunk

Or a very small child with an enormous amount of knowledge

nimitzeoauto8 days ago
You've got an agentic request - this Christmas in theatres near you!
Garlef9 days ago
Oh wow... Basically our generations "You've got mail"
notachatbot1239 days ago
And you are happy because that is what you wanted and the reason why you gave a randomness machine access to your mails, correct?
ayaniv9 days ago
If you're willing to give Claude or any other AI tool access to your email and files, the least you should do is put guardrails around consequential actions.

Reading a contract is one thing. Applying your signature and preparing to send it should absolutely require explicit human approval.

piva009 days ago
And guardrails can be as simple as not giving it direct access to applications that can perform read/write (like a full-blown email client, or access to the GMail UI), and instead creating a small tool to fetch the content as needed without being able to perform actions.

The danger is relying on too much convenience, giving too much power to a non-deterministic tool will inevitably create issues...

jacquesm9 days ago
I've had humans do the exact same thing. When I pointed out that this was fraud they were all surprised.
cassianoleal9 days ago
In this case it's not fraud though, since the person running the software is the same person whose signature ended in the document.
chrisjj9 days ago
You gave these humsns access to your email?
chrisjj9 days ago
> the least you should do is put guardrails around consequential actions.

How on earth would you?

andrepd9 days ago
You hooked up a chatbot to a harness that does API calls to myriad services. That's what you did. "Claude by himself" did not "do" anything. Just like they did not "break out of containment" and hacked companies.
willmarch9 days ago
Did you intervene or did Claude Code wait for your confirmation?

Those are two vastly different things.

Read the full thread on Hacker News →

Related stories