Show HN: Scrubbed – fast native web-data cleaning for LLM training | Hacker News - But Better

Show HN: Scrubbed – fast native web-data cleaning for LLM training schancel.github.io

Local tools for cleaning web text into training and evaluation corpora: encoding repair, content extraction, PII scanning, language ID, and deduplication.

1 points•eluusive•about 4 hours ago•0 comments•

#web #cleaning #training #scrubbed #native #data #llm #fast native

0 comments

No comments yet.

Related stories

The People of Utah vs. Kevin O’Learytheverge.com
The Verge · 0 points · about 12 hours ago
Inside our months-long investigation into Kevin O’Leary’s Utah data center debacletheverge.com
The Verge · 0 points · about 8 hours ago
Show HN: Capsule – Single-file web apps that save their data into SQLitewithcapsule.app
Hacker News · 377 points · 16 days ago
Show HN: Beyond "chat with your database" – agentic data explorationreportr.com
Hacker News · 1 points · 1 day ago
Show HN: Datastory, Data storytelling with AI and an open knowledge graphdatastory.tech
Hacker News · 6 points · 1 day ago
Show HN: Ssgoi – Native app-like page transitions for mobile webssgoi.dev
Hacker News · 1 points · about 13 hours ago