分布式数据库Hypertable 0.9.7.6发布

jieforest

Over time, Hypertable will break these tables into ranges and distribute them to what are known as RangeServer processes.  These processes manage ranges of table data and run on all slave server machines in the cluster.  For example, assuming there are three slave servers, the following diagram shows what the system might look like over time.  As can be seen by the diagram, the three servers are filled to capacity.

jieforest

Adding more capacity is a simple matter of adding new commodity class servers and starting RangeServer processes on the new machines.  Hypertable will detect that there are new servers available with plenty of spare capacity and will automatically migrate ranges from the overloaded machines onto the new ones.

jieforest

This range migration process has the effect of balancing load across the entire cluster and opening up additional capacity.

GOOGLE ARCHITECTURE

Hypertable is a massively scalable database modeled after Google's Bigtable database.  Bigtable is part of a group of scalable computing technologies developed by Google which is depicted in the following diagram.

jieforest

HYPERTABLE SYSTEM OVERVIEW

The diagram below provides a high-level overview of the Hypertable system followed by a brief description of each system component.

jieforest

Hyperspace - This is Hypertable's equivalent to Google's Chubby service. Hyperspace is a highly available lock manager and provides a filesystem for storing small amounts of metadata.  Exclusive or shared locks may be obtained on any created file or directory.  High availability is achieved by running in a distributed configuration with replicas running on different physical machines.  Consistency is achieved through a distributed consensus protocol.  Google refers to Chubby as, "the root of all distributed data structures" which is a good way to think of this system.

Master - The master handles all meta operations such as creating and deleting tables. Client data does not move through the Master, so the Master can be down for short periods of time without clients being aware. The master is also responsible for detecting range server failures and re-assigning ranges if necessary. The master is also responsible for range server load balancing. Currently there is a single Master process, but high availability is achieved through hot standbys.

Range Server - Range servers are responsible for managing ranges of table data, handling all reading and writing of data.  They can manage up to potentially thousands of ranges and are agnostic to the set of ranges that they manage or the tables of which they're a part.  Ranges can move freely from one range server to another, an operation that is mostly orchestrated by the Master.

DFS Broker - Hypertable is capable of running on top of any filesystem. To achieve this, the system has abstracted the interface to the filesystem by sending all filesystem requests through a Distributed File System (DFS) broker process. The DFS broker provides a normalized filesystem interface and translates normalized filesystem requests into native filesystem requests and vice-versa.  DFS brokers have been developed for HDFS, MapR, Ceph, KFS, and local (for running on top of a local filesystem).

jieforest

DATA REPRESENTATION

Like a relational database, Hypertable represents data as tables of information. Each row in a table has cells containing related information, and each cell is identified, in part, by a row key and column name.  Support for up to 255 column names is provided when the table is created. Hypertable provides two additional features:

1. column qualifier - The column names defined in the table schema are referred to as column families.  

Applications may supply an optional column qualifier, with each distinct qualifier representing a qualified column instance belonging to the column family . The application can define an unlimited number of qualified instances of a column family.  The application supplied column name has the format family:qualifier, and column data is stored in a sparse format such that one row may have millions of qualified instances of a column family, while another row may have none or just a few instances.

2. timestamp - This is a 64-bit field associated with each cell that allows for different cell versions.  

The value represents nanoseconds since the Unix epoch and can be supplied by the application or auto-assigned by the server.  The number of versions stored can be configured in the table schema and the number of versions returned can be specified in the query predicate.  The versions are stored in reverse-chronological order, so that the newest version of the cell is returned first.

jieforest

The following diagram illustrates how data is represented in Hypertable.  The table is an example taken from a web crawler that stores information for each page that it crawls in a row of the table.



The above diagram illustrates the use of the column qualifier.  A Web search engine builds an index (much like the one in the back of a book) that points words to the Web documents that contain them.  Included in this index are not only the words included in the Web page, but also words included in the anchor text of the remote links that point to the Web page. This is how image results can appear in Web search results.  For example, given an image of a Ferrari (which contains no text), if there are enough links pointing to the image that contain the word "Ferrari" in the anchor text, then the page may get a high score for the query "Ferrari" and appear in the search results.

jieforest

The one dimension that is missing from the above diagram is the timestamp.  Imagine that each cell in the diagram above has a z-axis that contains timestamped versions of the cell.  This multi-dimensional table gets flattened out, under the hood, as sorted lists of key/value pairs as illustrated in the following diagram.

jieforest

ANATOMY OF A KEY

The following diagram illustrates the format of the key that Hypertable uses internally.

jieforest

1. control - This field is consists of bit flags that describe the format of the remaining fields.  There are certain circumstances where the timestamp or revision number may be absent, or where they are identical, in which case, they're collapsed into a single field.  This field contains that information and tells Hypertable how to properly interpret the key.

2. row key - This field contains a '\0' terminated string that represents the row key.

3. column family - This field is a single-byte field that indicates the column family code.  

4. column qualifier - This field contains a '\0' terminated string that represents the column qualifier.

5. flag - Deletes are handled through the insertion of special "delete" records (or tombstones) that indicate that some portion of a row's cells have been deleted.  These delete records are applied at query time and the deleted cells are garbage collected during major compactions.

6. timestamp - This field is an 8-byte (64-bit) field that contains the cell timestamp, represented as nanoseconds since the Unix epoch.  By default, the timestamp is stored big-endian, ones-compliment so that within a given cell, versions are stored newest to oldest.

7. revision - This field is an 8-byte (64-bit) field that contains a high resolution timestamp that currently is used internally to provide snapshot isolation for queries.