Apache Hadoop community has just released Apache Hadoop 2.0.0 (alpha)
While only an alpha release (read: not ready to run in production), it is still an important step forward as it represents the very first release that delivers new and important capabilities, including:
- HDFS HA (manual failover)
- NextGen MapReduce a.k.a YARN
- HDFS Federation
- Wire-compatibility for both HDFS & YARN (via protobufs)
In addition to these new capabilities, there are several planned enhancements that are on the way from the community, includingHDFS Snapshots and auto-failover for HA NameNode, along with further improvements to the stability and performance with the next generation of MapReduce (YARN). There are definitely good times ahead.
Again, please note that the Apache Hadoop community has decided to use the alpha moniker for this release since it is a preview release that is not yet ready for production deployments for the following reasons:
- We still need to iterate over some of the APIs (especially with the switch to protobufs) before we declare them stable, i.e. something that can be supported over the long run in a compatible manner.
- Several features including HDFS HA, NextGen MapReduce et al need a lot more testing and validation before they are ready for prime time.
- While we are excited about the progress made for supporting HA for HDFS, auto-failover for HDFS NameNode and HA for NextGen MapReduce are still a work-in-progress.