Database Reliability Engineering: Designing and Operating Resilient Database Systems by Laine Campbell & Charity Majors
Author:Laine Campbell & Charity Majors
Language: eng
Format: azw3
Publisher: O'Reilly Media
Published: 2017-10-26T04:00:00+00:00
Pattern: high resource utilization operations
There are multiple patterns that you can utilize here, depending on the operations that are being executed.
For data modification, throttling by performing the updates in batches is a simple pattern to give engineers when performing bulk operations. For larger environments, it often makes more sense to utilize code to do lazy updating upon login of a user, or querying of a row for example.
For data removal, you can encourage SWEs to utilize soft deletes in their code. A soft delete flags a row as deleteable, which means it can be filtered out of queries in the application and removed at will. You can then throttle the deletes, removing them asynchronously. As with bulk updates, for large datasets this might prove to be impossible. If deletes are regularly performed on ranges, such as dates or ID groupings, you can utilize partitioning features to drop partitions. By dropping a table or partition, you do not create undo I/O which can reduce resource consumption.
Should you find that DDL operations such as table alters create enough I/O that latency is affected, you should consider this a red flag that capacity might be reaching its limits. Ideally, you would work with operations to add more capacity to the data stores. However, if this is not possible or is delayed, these DDL operations can be treated like blocking operations, with the appropriate pattern being applied.
Download
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.
Algorithms of the Intelligent Web by Haralambos Marmanis;Dmitry Babenko(7852)
Learning SQL by Alan Beaulieu(5412)
Weapons of Math Destruction by Cathy O'Neil(5037)
Big Data Analysis with Python by Ivan Marin(3009)
Blockchain Basics by Daniel Drescher(2891)
Hands-On Machine Learning for Algorithmic Trading by Stefan Jansen(2518)
Pandas Cookbook by Theodore Petrou(2502)
Mastering Python for Finance by Unknown(2476)
Building Statistical Models in Python by Huy Hoang Nguyen & Paul N Adams & Stuart J Miller(2475)
Azure Data and AI Architect Handbook by Olivier Mertens & Breght Van Baelen(2444)
Serverless Machine Learning with Amazon Redshift ML by Debu Panda & Phil Bates & Bhanu Pittampally & Sumeet Joshi(2379)
How The Mind Works by Steven Pinker(2213)
Data Wrangling on AWS by Navnit Shukla | Sankar M | Sam Palani(2161)
Building Machine Learning Systems with Python by Richert Willi Coelho Luis Pedro(2059)
Data Engineering with dbt by Roberto Zagni(2055)
Driving Data Quality with Data Contracts by Andrew Jones(2028)
Network Science with Python and NetworkX Quick Start Guide by Edward L. Platt(1972)
Python Natural Language Processing by Jalaj Thanaki(1892)
Machine Learning Model Serving Patterns and Best Practices by Md Johirul Islam(1824)