Streaming Systems by Tyler Akidau

Author:Tyler Akidau
Language: eng
Format: epub
Publisher: O'Reilly Media
Published: 2018-07-29T16:00:00+00:00

Chapter 7. The Practicalities of Persistent State

Why do people write books? When you factor out the joy of creativity, a certain fondness for grammar and punctuation, and perhaps the occasional touch of narcissism, you’re basically left with the desire to capture an otherwise ephemeral idea so that it can be revisited in the future. At a very high level, I’ve just motivated and explained persistent state in data processing pipelines.

Persistent state is, quite literally, the tables we just talked about in Chapter 6, with the additional requirement that the tables be robustly stored in a media relatively immune to loss. Stored on local disk counts, as long as you don’t ask your Site Reliability Engineers. Stored on a replicated set of disks is better. Stored on a replicated set of disks in distinct physical locations is better still. Stored in memory once definitely doesn’t count. Stored in replicated memory across multiple machines with UPS power backup and generators onsite maybe does. You get the picture.

In this chapter, our objective is to do the following:

Motivate the need for persistent state within pipelines

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.