The Google File System

wangfans

Neither the client nor the chunkserver caches file data.
Client caches offer little benefit because most applications
stream through huge files or have working sets too large
to be cached. Not having them simplifies the client and
the overall system by eliminating cache coherence issues.
(Clients do cache metadata, however.) Chunkservers need
not cache file data because chunks are stored as local files
and so Linux’s buffer cache already keeps frequently accessed
data in memory.

wangfans

2.4 Single Master
Having a single master vastly simplifies our design and
enables the master to make sophisticated chunk placement
30
Legend:
Data messages
Control messages
Application
(file name, chunk index)
(chunk handle,
chunk locations)
GFS master
File namespace
/foo/bar
Instructions to chunkserver
Chunkserver state
GFS chunkserver GFS chunkserver
(chunk handle, byte range)
chunk data
chunk 2ef0
Linux file system Linux file system
GFS client
Figure 1: GFS Architecture
and replication decisions using global knowledge. However,
we must minimize its involvement in reads and writes so
that it does not become a bottleneck.

wangfans

Clients never read
and write file data through the master. Instead, a client asks
the master which chunkservers it should contact. It caches
this information for a limited time and interacts with the
chunkservers directly for many subsequent operations.

wangfans

Let us explain the interactions for a simple read with reference
to Figure 1. First, using the fixed chunks ize, the client
translates the file name and byte offset specified by the application
into a chunki ndex within the file. Then, it sends
the master a request containing the file name and chunk
index. The master replies with the corresponding chunk
handle and locations of the replicas. The client caches this
information using the file name and chunki ndex as the key.

wangfans

information using the file name and chunki ndex as the key.
The client then sends a request to one of the replicas,
most likely the closest one. The request specifies the chunk
handle and a byte range within that chunk. Further reads
of the same chunkreq uire no more client-master interaction
until the cached information expires or the file is reopened.

wangfans

In fact, the client typically asks for multiple chunks in the
same request and the master can also include the information
for chunks immediately following those requested. This
extra information sidesteps several future client-master interactions
at practically no extra cost.

wangfans

2.5 Chunk Size
Chunks ize is one of the key design parameters. We have
chosen 64 MB, which is much larger than typical file system
blocks izes. Each chunk replica is stored as a plain
Linux file on a chunkserver and is extended only as needed.
Lazy space allocation avoids wasting space due to internal
fragmentation, perhaps the greatest objection against such
a large chunksize.

wangfans

A large chunksize offers several important advantages.
First, it reduces clients’ need to interact with the master
because reads and writes on the same chunkre quire only
one initial request to the master for chunklo cation information.
The reduction is especially significant for our workloads
because applications mostly read and write large files
sequentially.

wangfans

Even for small random reads, the client can
comfortably cache all the chunklo cation information for a
multi-TB working set. Second, since on a large chunk, a
client is more likely to perform many operations on a given
chunk, it can reduce network overhead by keeping a persistent

wangfans

persistent
TCP connection to the chunkserver over an extended
period of time. Third, it reduces the size of the metadata
stored on the master. This allows us to keep the metadata
in memory, which in turn brings other advantages that we
will discuss in Section 2.6.1.