architecting hbase applications by Jean-Marc Spaggiari Kevin O'Dell

Author:Jean-Marc Spaggiari, Kevin O'Dell
Language: eng
Format: epub
Publisher: O'Reilly Media, Inc.
Published: 2015-08-25T04:00:00+00:00

Warning

The reducer used to create the HFiles needs to load into memory the columns of a single row and then sort all before being able to write them all. If you have many columns in your dataset, it might not fit into memory. This should be fixed in a future release when HBASE-13897 will be implemented.

The operations on the mapper side are simple. The goal is just to split the line into different fields, assign them to an Avro object and provide this Avro object to the HBase framework to be stored into HFiles ready to be loaded.

The first thing we do is to define a set of variables that we will re-use for each and every iteration of the mapper. This is done to reduce the number of objects created.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.