> We built out a control plane service (CPS) responsible for automating Delta’s fleet management. Each instance of the CPS gets configured to monitor a list of Delta buckets. Its primary function includes repairing

How does the configuration spread consistently across the nodes? How does each node know if each link to the next node is still the most up-to-date, based on the configuration system. This part usually is waved away but it’s one of the trickier bits. Usually there is some paxos/raft/zookeeper system involved here.

Dammit, why can't we get more inventive with naming our projects? How we're supposed not to confuse it with https://delta.dev? There's also diffing thingies https://dandavison.github.io/delta and https://jmacd.github.io/xdelta

I'm a big fan of Chain Replication (the CRAQ paper [1] mentioned is one of my favorites)

Something I'm curious about: How does a recovering node know that it is healthy enough to serve requests? Is there some anti-entropy protocol used between the tail and the recovering node?

[1] https://www.usenix.org/legacy/event/usenix09/tech/full_paper...