Apache Hadoop 3.1.0

Apache Hadoop 3.1.0

Apache Hadoop 3.1.0 incorporates a number of significant enhancements over the previous minor release line (hadoop-3.0).

Apache Hadoop 3.1.0在之前的小版本(Hadoop -3.0)中加入了一些显著的增强。

Overview

概述

Users are encouraged to read the full set of release notes. This page provides an overview of the major changes.

鼓励用户阅读完整版的发行说明。此页面提供了主要更改的概述。

Here is a short overview of the major features and improvements.

这里是对主要特性和改进的简要概述。

Yarn Service framework provides first class support and APIs to host long running services natively in YARN.

纱线服务框架提供了一流的支持和api,使其能够在纱线中本地运行长期运行的服务。

In a nutshell, it serves as a container orchestration platform for managing containerized services on YARN. It supports both docker container and traditional process based containers in YARN.

简而言之,它作为一个容器编排平台,用于管理纱线上的集装箱化服务。它既支持docker容器,又支持传统的基于纱线的容器。

See the user documentation for more details.

有关详细信息,请参阅用户文档。

First-class GPU scheduling and isolation (For both docker/non-docker containers) on YARN.

一流的GPU调度和隔离(对于docker/非docker容器)的纱线。

See the user documentation for more details.

有关详细信息,请参阅用户文档。

First-class FPGA scheduling and isolation (For both docker/non-docker containers) on YARN.

一流的FPGA调度和隔离(对于docker/非docker容器)的纱线。

See the user documentation for more details.

有关详细信息,请参阅用户文档。

Support more expressive placement constraints in YARN. Such constraints can be crucial for the performance and resilience of applications, especially those that include long-running containers, such as services, machine-learning and streaming workloads.

在纱线中支持更有表现力的放置约束。这样的约束对于应用程序的性能和弹性至关重要,尤其是那些包括长时间运行的容器(如服务、机器学习和流媒体工作负载)的应用程序。

For example, it may be beneficial to co-locate the allocations of a job on the same rack (affinity constraints) to reduce network costs, spread allocations across machines (anti-affinity constraints) to minimize resource interference, or allow up to a specific number of allocations in a node group (cardinality constraints) to strike a balance between the two. Placement decisions also affect resilience. For example, allocations placed within the same cluster upgrade domain would go offline simultaneously.

,例如,它可能是有益的驻扎在同一地点工作的分配在同一架(关联约束),降低网络成本,传播在机器(anti-affinity约束)最小化资源分配干扰,或允许节点组中的一个特定数量的分配(基数约束)两者之间取得平衡。就业决策也会影响弹性。例如,放置在同一集群升级域中的分配将同时脱机。

See the user documentation for more details.

有关详细信息,请参阅用户文档。

Support administrators to specify absolute resources (X Memory, Y VCores, Z GPUs, etc.) to a queue instead of providing percentage based values. This provides better control for admins to configure required amount of resources for a given queue.

支持管理员为队列指定绝对资源(X内存、Y vcore、Z gpu等),而不是提供基于百分比的值。这为管理员提供了更好的控制,以便为给定的队列配置所需的资源数量。

See the user documentation for more details.

有关详细信息,请参阅用户文档。

Provided storage allows data stored outside HDFS to be mapped to and addressed from HDFS. It builds on heterogeneous storage by introducing a new storage type, PROVIDED, to the set of media in a DataNode.

提供的存储允许将存储在HDFS外部的数据映射到HDFS上。它通过引入一个新的存储类型(提供给DataNode中的媒体集)来构建异构存储。

See the user documentation for more details.

有关详细信息,请参阅用户文档。

Getting Started

开始

The Hadoop documentation includes the information you need to get started using Hadoop. Begin with the Single Node Setup which shows you how to set up a single-node Hadoop installation. Then move on to the Cluster Setup to learn how to set up a multi-node Hadoop installation.

Hadoop文档包含了使用Hadoop所需的信息。从单节点设置开始,它将向您展示如何设置单节点Hadoop安装。然后转移到集群设置,学习如何设置多节点Hadoop安装。