Building a Robust Ingestion System for Any File of Any Size | Hacker News - But Better

Building a Robust Ingestion System for Any File of Any Size mixedbread.com

A 300GB video does not fit on one machine, and a 4MB PDF can render into gigabytes of images. This is how our slicer and parser architecture processes…

1 points•sherlock_h•about 3 hours ago•0 comments•

#system #file #size #robust #ingestion #robust ingestion #ingestion system #robust ingestion system

0 comments

No comments yet.

Related stories

Show HN: Capsule – Single-file web apps that save their data into SQLitewithcapsule.app
Hacker News · 377 points · 16 days ago
File Notification Attacks: Side-Channel Leakage from the File-Notification System on Linux, Android, Windows, and macOSinoti.fyi
Lobsters · 33 points · 7 days ago
Side-Channel Leakage from the File-Notification Systeminoti.fyi
Hacker News · 1 points · 7 days ago
Show HN: System One Harness (SOH), the harness for System One modelsgithub.com
Hacker News · 1 points · 11 days ago
Applesauce: Transparent Compression for Apple File System Compression (AFSC)github.com
Hacker News · 1 points · 11 days ago
Show HN: Onesie – An expressive CLI for System One models like Jevgithub.com
Hacker News · 3 points · 3 days ago