Hadoop For Dummies by JOnes

Author:JOnes
Language: eng
Format: mobi, epub
ISBN: 9781118652206
Published: 2014-03-19T23:40:23+00:00

hours or weeks later in subsequent service requests. This scenario would result in parts of Row 00001 being persisted to the HDFS in different HFiles. Until the HBase system performs a minor compaction, reading from Row 00001 would require three disk reads to retrieve the relevant HFile content! Minor compactions seek to minimize system overhead while keeping the number of HFiles under control. HBase designers took special care to give the HBase administrator as much tuning control as possible to make any system impact “minor.”

As its name implies, a major compaction is different from the perspective of a system impact. However, the compaction is quite important to the overall functionality of the HBase system. A major compaction seeks to combine all HFiles into one large HFile. In addition, a major compaction does the cleanup work after a user deletes a record. When a user issues a Delete call, the HBase system places a marker in the key-value pair so that it can be permanently removed during the next major compaction. Additionally, because major compactions combine all HFiles into one large HFile, the time is right for the system to review the versions of the data and compare them against the time to live (TTL) property. Values older than the TTL are purged.

Time to live refers to the variable in HBase you can set in order to define how long data with multiple versions will remain in HBase. For more information on versions in HBase see the “Understanding the HBase Data Model” section, earlier in this chapter.

For a complete list of HBase tuning parameters see http://hbase.apache.org/book/config.files.html.

You may have guessed that a major compaction significantly affects the system response time. Users who are trying to add, retrieve or manipulate data in the system during a major compaction, they may see poor system response time. In addition, the HBase cluster may have to split regions at the same time that a major compaction is taking place and balance the regions across all RegionServers. This scenario would result in a significant amount of network traffic between RegionServers. For these reasons, your HBase administrator needs to have a major compaction strategy for your deployment. We discuss a solution to the major compaction challenge at the end of this chapter, but for now we continue your tour of the basic HBase architecture.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.