Zalando’s database engineering team recently spoke at FOSDEM PGDay, an event hosted by the PostgreSQL community the day before the FOSDEM conference in Brussels. I had the opportunity to share insights on Streaming Huge Databases using Logical Decoding.

Logical decoding is a new feature of PostgreSQL (since version 9.4) that allows streaming database changes in a custom format. My talk explains what potential problems one may encounter while extracting consistent snapshot of a big database and approaches to mitigate these problems. A performance comparison of different existing Logical Decoding plugins is discussed, and a unified interface to stream pre-existing data through a plugin output function is proposed. Finally, I discussed some potential performance bottlenecks and their mitigation means.

Watch my talk below and let me know your thoughts:

Streaming huge databases using logical decoding from Alexander Shulgin