Gyanrush
Start freeOpen Gyanrush
Start freeOpen Gyanrush
  1. LPU MCA Semester 1
  2. CAP470
  3. Unit 5

CAP470 · Unit 5: Cloud access, storage and file systems

HDFS, Google File System and scalability

13 practice questions · in the ETE syllabus

Practise this topicPractise this topic

Key points

  • NameNode/master hold metadata; DataNodes/chunkservers hold data
  • HDFS blocks are 128 MB and GFS chunks 64 MB
  • Replication factor is 3 by default
  • Scale out by adding nodes

Distributed file systems split large files into big blocks and spread them over many machines.

FeatureHDFSGFS
MasterNameNode (metadata)Master (metadata)
Data serversDataNodesChunkservers
UnitBlock, 128 MB by defaultChunk, 64 MB
CopiesReplication factor 3 by defaultThree replicas by default

Block arithmetic: a 300 MB file with 128 MB blocks needs ceil(300 / 128) = 3 blocks, and a 1 GB file needs 8 blocks, which with three replicas means 24 stored blocks. A 1 GB file in 64 MB chunks makes 16 chunks. Large units reduce metadata and disk seeks for big sequential reads, and the master is not a bottleneck because data flows directly between clients and data servers.

DataNodes send heartbeats to the NameNode. If a node stops, its blocks are treated as missing and new copies are made from the surviving replicas. Scalability means handling more load. Vertical scaling makes one machine larger; horizontal scaling adds machines. Distributed file systems scale out, because one machine has limits.

Exam trap: the number of blocks is rounded up, and replicas multiply the stored blocks, not the file's block count.

Try two questions

Question 1 · Tier 4, Analyse

Why do HDFS and GFS use large units of 64 or 128 MB instead of small blocks?

  1. Large blocks make every small file load faster than in a normal file system
  2. Large blocks reduce metadata and seek overhead for big sequential reads
  3. Large blocks allow the data to be stored without replication
  4. Large blocks are required because the network cannot carry small packets
Show the answer

B. Fewer, larger blocks mean less metadata for the master and fewer disk seeks.

Why not A: These systems are tuned for big files, not many small ones.

Why not C: Replication is still used.

Why not D: Networks carry small packets without trouble.

Question 2 · Tier 3, Apply

What happens in HDFS when a DataNode stops sending heartbeats?

  1. The NameNode treats its blocks as missing and creates new copies from other replicas
  2. The whole file system stops until the node is repaired
  3. All files stored on that node are deleted permanently
  4. The client must upload the missing files again
Show the answer

A. Heartbeats let the NameNode detect failures and restore the replication factor.

Why not B: The file system continues running thanks to replicas.

Why not C: The data still exists on other nodes.

Why not D: HDFS re-replicates automatically.