Database Reliability Engineering: Designing and Operating Resilient Database Systems by Campbell Laine & Majors Charity

Author:Campbell, Laine & Majors, Charity [Campbell, Laine]
Language: eng
Format: azw3
Publisher: O'Reilly Media
Published: 2017-10-26T04:00:00+00:00

Pattern: high resource utilization operations

There are multiple patterns that you can utilize here, depending on the operations that are being executed.

For data modification, throttling by performing the updates in batches is a simple pattern to give engineers when performing bulk operations. For larger environments, it often makes more sense to utilize code to do lazy updating upon login of a user, or querying of a row for example.

For data removal, you can encourage SWEs to utilize soft deletes in their code. A soft delete flags a row as deleteable, which means it can be filtered out of queries in the application and removed at will. You can then throttle the deletes, removing them asynchronously. As with bulk updates, for large datasets this might prove to be impossible. If deletes are regularly performed on ranges, such as dates or ID groupings, you can utilize partitioning features to drop partitions. By dropping a table or partition, you do not create undo I/O which can reduce resource consumption.

Should you find that DDL operations such as table alters create enough I/O that latency is affected, you should consider this a red flag that capacity might be reaching its limits. Ideally, you would work with operations to add more capacity to the data stores. However, if this is not possible or is delayed, these DDL operations can be treated like blocking operations, with the appropriate pattern being applied.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.