Pirate Face Rescues LLM Models from Deletion

Pirate Face Rescues LLM Models from Deletion pirateface.co

Open models as checksum-verified magnet links. Censorship-resistant, kept alive by seeders.

569 points•skepticalgenius•11 days ago•149 comments•

#llm #pirate #deletion #rescues #rescues llm #llm models #pirate face #face rescues

149 comments

Speaking to the "uncensored model" angle: there's little reason to distribute abliterated weights anyway. Instead of orthogonalising the weights that write back to the residual stream, you can just orthogonalise the activations themselves. It's equivalent.

Orthogonalising activations at runtime is computationally cheap. Just distribute the refusal vectors (few thousand floats per layer), then run against the stock weights. Antirez's DS4 already supports this: https://github.com/antirez/ds4/blob/8db1d1d155cb0400a86a86b9...

Abliterated weights are just a bad habit we've gotten into. It's also deeply suboptimal from a precision point of view to take a model that's already been QATed and distributed in pre-quantised form (DeepSeek V4, Kimi K2.5 or K3...), modify its weights, and re-quantise it. Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration -- avoidable by keeping the two separate.

derefr10 days ago

I believe abliterated models are mostly still created at this point because they're "universal": they can not only be run locally, and on cloud GPUs, but also on "managed inference" providers (i.e. services where you hand them a model URI, and they blindly fetch it, load it, and give you inference access to it through standard text/chat-completion APIs. Think HuggingFace Spaces, or Google CoLab, or CloudFlare Workers AI.)

Such managed inference providers have (for now) plausible deniability of behaving ethically (at least enough that they don't get boycotted / scare away investors) due to them being "blind" to what gets run on their systems. They're acting as the inference equivalent of data transit carriers.

But I don't think it would be possible for managed inference providers to publicly expose "runtime activation steering" in the way antirez's DS4 does, without that reading much more explicitly as them inviting unethical workloads.

(Yes, there are other things you can do with runtime steering. But almost all of those things are workload-specific, relying on you privately tuning to the needs of your own dataset. And if you can do that, you can run inference without the help of a managed inference provider. The only time a customer will come along with a pre-made runtime-steering vector file in hand, is if that vector is an alignment-orthogonalization vector.)

quotemstr10 days ago

These providers can also just ignore insinuations they're being "unethical" when people come to them with steering vectors in hand. Nobody has to listen to the scolds.

nperez10 days ago

Yeah I use a custom fork of llama.cpp that has an abliteration feature that basically does this. It's sloppily vibe coded and I don't have time to coordinate on a way to do this cleanly upstream, but it's absolutely possible and saves a lot of time and bandwidth from being wasted

doublerabbit10 days ago

> Similarly, abliterated models regain some of their refusal behaviour when they're re-quantised after abliteration

Thanks for this information, Q4 seemed fine but they reappeared again in Q5 with an vengeance, I couldn't understand why. Very Strict and I've only found one jail break that barely works around 60% of the time.

jamienk10 days ago

Can you explain this a bit to a non-expert?

I haven't wrapped my mind around this

wren699110 days ago

This is the original description of abliteration and it's quite approachable and interesting to read: Refusal in Language Models Is Mediated by a Single Direction (https://arxiv.org/abs/2406.11717). Warning: changes to your world view caused by seeing "HarmBench" used to maximise expected harm instead of minimising it may be irreversible.

There's an empirical observation that models often have a single direction in their activation space for "hmm no I shouldn't do this". It forms naturally during pre-training, and is then surfaced during post-training to make the model refuse to engage in certain behaviour.

With a little bit of linear algebra you can zap that direction from the model's activations, and it stops refusing to do things. You can also do the opposite: magnify that direction, and the model refuses to do anything at all.

mitxela10 days ago

Instead of editing the weights so they don't create the refusal signal, just let them do whatever, then delete the refusal signal itself. You don't want to edit quantised weights because it causes a loss of precision that can be pretty bad.

khalic10 days ago

I didn’t know about that method, thank you. I’ve needed a local model for security research but Qwen 27b abliterated did 30% worse than the stock weights on my internal benchmarks (I just skip the public benches now, it’s honestly useless noise on an operational level).

phoyd11 days ago

Torrents should really be the preferred method for distributing AI model weights. Why rely on a single point of failure like Hugging Face? BitTorrent was made for exactly this.

CodesInChaos10 days ago

In my experience public torrents often die as they grow older. It doesn't help that BitTorrent V1 makes long term seeding annoying, and BitTorrent V2 is almost never used.

monsieurbanana10 days ago

I never understood this, is there anything that makes it difficult for the original uploader, the one that supposedly offers the file directly, to offer a torrent instead for the same amount of time?

As far as perennity is concerned it seems strictly better.

bilegeek10 days ago

The biggest problem with BTv1 was the lack of per-file checksumming, and swarm merging (i.e. individual files have shared seeding pools across torrents). BTv2 specs the latter, but I think only BiglyBT actually implements it. Having both of those features from the get-go would've gone a LONG way to fixing the dead torrent problem.

Retr0id10 days ago

A torrent with a webseed is strictly more resilient than a direct download link alone.

zenoprax10 days ago

You only need one person/organization to commit to seeding. The majority of people do not want to seed at all without some sort of incentive.

If this site represents a coordinated datahoarding effort then there will be at least a few people who will seed indefinitely.

PunchyHamster10 days ago

Yeah but that's what provider like huggingface could just do, keep seeding the models so they are still accessible

pmdr10 days ago

HF is meant to be a single point of control. AI models and Linux distros aren't usually for normies, so distribution via torrents would make sense, especially to save the provider some bandwidth. Ubuntu has been offering torrent downloads for ages. No mention of torrents on HF. I believe most downloads will soon be account/EULA-walled.

TeMPOraL10 days ago

Yeah, I thought they were used for this already. Surprised this is news, but also relieved.

Oxodao10 days ago

IIRC Mistral used to do it, not sure if that's still the case

EDIT/ Yes they did, that no longer seems to be the case though

https://x.com/MistralAI/status/1833758285167722836

boredumb10 days ago

ages ago I tried using IPFS to more or less accomplish this, I imagined it to act more like a weights/training data network fs that everyone would be able to participate in.

mococa10 days ago

Steam & Blizzard (probably others) used to delivery games through torrent protocol in the past, before CDNs became cheaper.

When StarCraft 2 was lauched, the installer (before Battle.net installer crapware) had a complete graphical visualization of seeders & leechers.

Reference: https://warcraft.wiki.gg/wiki/Blizzard_Downloader

bayindirh10 days ago

Oh, story time:

Once I was using Blizzard's downloader to install something (StarCraft, Diablo, I don't remember), and it was kinda slow. I disabled P2P downloads and speed skyrocketed, and I said "Huh, this was unexpected".

When P2P downloads disabled you could see the list of CDNs you're downloading from and mine had a single IP on that list. It looked familiar. Then it dawned on to me. It was the Akamai server which we were hosting in our system room, at 15 minutes of driving distance. After a chuckle, I went to get a cup of tea, because that was entertaining than the game itself.

Then of course, I dived into whatever I was installing that night.

Edit: From the screenshots in the wiki, I remembered that the progress bar was red. It was possibly Diablo 3, then. However, I'm still not 100% sure about it.

skeptic_ai10 days ago

So how come was using your own ip? Become a Diablo node installer so you downloaded from there, like how can they convert your own akamai instance into anode without you knowing?

alexpotato10 days ago

I've worked at trading firms where the "reference data master" file is usually a big json or equivalent.

To get the file out to 100s or 1000s of machine they would often use private bittorent to distribute the file out.

mococa10 days ago

Same here. Inspired by both, I used to delivery videos over torrent to in door machines since at that time, there's no CDNs (or it was difficult to get one).

ehe78qhe10 days ago

I've seen this used for distributing container images in networks with awkward network topologies (e.g. a lot of bandwidth within a site or sub-site but limited bandwidth to central registries)

mitxela10 days ago

As a private network, they have the option of multicast.

ehe78qhe10 days ago

Steam has a limited form of p2p delivery, used on LANs: https://help.steampowered.com/en/faqs/view/46BD-6BA8-B012-CE...

Aurornis10 days ago

Several companies tried this for distributing software.

It was very controversial. Users were angry that software companies were using their internet bandwidth to distribute their software. Made a lot of people angry.

blharr10 days ago

>software companies were using their internet bandwidth to distribute their software

If I understand the description correctly, microsoft still does this!

https://support.microsoft.com/en-us/windows/privacy/windows-...

ivanmontillam10 days ago

If only Blizzard servers were seeders, and clients were only leechers, that'd have not been a problem.

kzrdude10 days ago

Typical example to show that people who want to be angry will be angry, even for pointless things.

JonChesterfield10 days ago

This is really important. I am already tired of hoarding rclone copies of huggingface torrents and periodically checking them for bitrot. Pirateface is not a very helpful name for it though and it doesn't seem to have scripted torrent creation either.

Is hosting the same thing at 'academictorrents' actually a viable thing? In terms of peers from either send data to one another?

edit: looks like it can treat huggingface as a backstop for torrents that are otherwise not shared which is interesting, whole load of checksum nonsense I hand rolled disappear if bittorrent handles that. Except it doesn't work?

    Magnet soon No seeders yet - a torrent mints once a seeder packages this model.

So there's some per-torrent work to be done, but I don't know what that is, and I don't see how it can be based on files I have locally _and also_ be an exact match to files on huggingface. So I'm missing something here.

edit2: Looks like an implementation error. I can create a torrent from local files and upload it, but it won't have the huggingface backstop, and I can't specify it, so that doesn't actually achieve the claimed result. Before creating community torrents in that fashion would actually be of use, the submission page needs to allow pointing at the upstream.

Also, having everyone DIY a set of files -> torrent information is insane, this should not be a SKILLS.md, it should be a bash script that makes the thing.

mmaunder10 days ago

Edit: Deleted. Been oversharing a bit re research I'm doing, and the payoff is replies from folks who haven't bothered to go and take a look themselves, which I then have to spend more cycles refuting, etc. So best to just go back to the first step and not overshare and recover the cycles I'd spend on the rest of it. Admittedly I'm tired and pissed off, but yeah. HN won't let me delete this so I guess it's just a deletion edit. Sorry.

chuckadams10 days ago

> DS 4.1 Flash, which is unguardrailed

Try asking it about Tianmen Square. I use DS myself, but let's not kid ourselves.

quaintdev10 days ago

For that specific query, just use US open weight model, oh...wait

KerrAvon10 days ago

DS 4.1 Flash absolutely has guardrails. Maybe not ones as effective as some models.

Read the full thread on Hacker News →

149 comments

Related stories