CDH 5 and Cloudera Manager 5 Requirements and Supported Versions

Cloudera uses the following versioning convention: major.minor.maintenance. If a cluster runs Cloudera Manager 5.14.0, the major version is 5; the minor version is 14; and the maintenance version is 0.

The Cloudera Manager minor version must always be equal to or greater than the CDH minor version. Older versions of Cloudera Manager might not support features in newer versions of CDH.

For example, Cloudera Manager 5.12.0 can manage CDH 5.12.2 because the minor versions are equal. Cloudera Manager 5.12.0 cannot manage CDH 5.14.0 because the Cloudera Manager minor version, 12, is less than the CDH minor version, 14.

Cloudera Manager and CDH come packaged with an embedded PostgreSQL database for use in a non-production environment. The embedded PostgreSQL database is not supported in production environments. For production environments, you must configure your cluster to use external databases.

In most cases (but not all), Cloudera supports versions of MariaDB, MySQL and PostgreSQL that are native to each supported Linux distribution.

After installing a database, upgrade to the latest patch and apply appropriate updates. Available updates may be specific to the operating system on which it is installed.

MySQL Support across CM/CDH 5 Releases

MySQL	Cloudera Manager/CDH 5.10 - 5.16	Cloudera Manager/CDH 5.1 - 5.9	Cloudera Manager/CDH 5.0
5.1
5.5
5.6
5.7

MariaDB Support across CM/CDH 5 Releases

MariaDB	Cloudera Manager/CDH 5.9 - 5.16	Cloudera Manager/CDH 5.5 - 5.8
5.5
10.0
10.2	Recommend 10.0 because of the known issue OPSAPS-52340.

PostgreSQL Support across CM/CDH 5 Releases

PostgreSQL	Cloudera Manager/CDH 5.16	Cloudera Manager/CDH 5.15	Cloudera Manager/CDH 5.13 - 5.14	Cloudera Manager/CDH 5.11 - 5.12	Cloudera Manager/CDH 5.5 - 5.10	Cloudera Manager/CDH 5.2 - 5.4	Cloudera Manager/CDH 5.1	Cloudera Manager/CDH 5.0
8.1
8.3
8.4
9.1
9.2
9.3
9.4
9.5
9.6
10		See Warning below.

Oracle Support across CM/CDH 5 Releases

Oracle	Cloudera Manager/CDH 5.13 - 5.16	Cloudera Manager/CDH 5.6 - 5.12	Cloudera Manager/CDH 5.0 - 5.5
11g R2
12c R1
12c R2
18	Cloudera Manager/CDH 5.16 only. Requires Java 8 JDBC driver
19	Cloudera Manager/CDH 5.16 only. Requires Java 8 JDBC driver

Only 64 bit JDKs are supported. Oracle JDK 7 is supported across all versions of Cloudera Manager 5 and CDH 5. Oracle JDK 8 is supported in Cloudera Enterprise 5.3.x and higher.

OpenJDK 8 is supported in Cloudera Enterprise 5.16.1 and higher.

Oracle JDK 9 is not supported in any Cloudera Manager or CDH version.

Unless specifically excluded, Cloudera supports later updates to a major JDK release from the release that support was introduced. For example, Oracle JDK 8u102 was released prior to CDH 5.9, but is supported in CDH 5.3 and higher because support for Oracle JDK 8 was added in CDH 5.3. Cloudera excludes or removes support for select Java updates when security is jeopardized.

Running CDH nodes within the same cluster on different JDK releases is not supported. JDK release across a cluster needs to match the patch level.

The following components are supported by the indicated versions of Transport Layer Security (TLS):

The hosts in a Cloudera Manager deployment must satisfy the following networking and security requirements:

• Networking Protocols Support
  CDH requires IPv4. IPv6 is not supported and must be disabled.

  See also Configuring Network Names.

• Multihoming Support

  – Multihoming CDH or Cloudera Manager is not supported outside specifically certified Cloudera partner appliances. Cloudera finds that current Hadoop architectures combined with modern network infrastructures and security practices remove the need for multihoming. Multihoming, however, is beneficial internally in appliance form factors to take advantage of high-bandwidth InfiniBand interconnects.

  Although some subareas of the product may work with unsupported custom multihoming configurations, there are known issues with multihoming. In addition, unknown issues may arise because multihoming is not covered by our test matrix outside the Cloudera-certified partner appliances.

• Entropy

  Data at rest encryption requires sufficient entropy to ensure randomness.

  See entropy requirements in Data at Rest Encryption Requirements.

• Cluster hosts must have a working network name resolution system and correctly formatted /etc/hosts file. All cluster hosts must have properly configured forward and reverse host resolution through DNS. The /etc/hosts files must:
  • Contain consistent information about hostnames and IP addresses across all hosts
  • Not contain uppercase hostnames
  • Not contain duplicate IP addresses

  Cluster hosts must not use aliases, either in /etc/hosts or in configuring DNS. A properly formatted /etc/hosts file should be similar to the following example:

  127.0.0.1 localhost.localdomain localhost
  192.168.1.1 cluster-01.example.com cluster-01
  192.168.1.2 cluster-02.example.com cluster-02
  192.168.1.3 cluster-03.example.com cluster-03 

• In most cases, the Cloudera Manager Server must have SSH access to the cluster hosts when you run the installation or upgrade wizard. You must log in using a root account or an account that has password-less sudo permission. For authentication during the installation and upgrade procedures, you must either enter the password or upload a public and private key pair for the root or sudo user account. If you want to use a public and private key pair, the public key must be installed on the cluster hosts before you use Cloudera Manager.

  Cloudera Manager uses SSH only during the initial install or upgrade. Once the cluster is set up, you can disable root SSH access or change the root password. Cloudera Manager does not save SSH credentials, and all credential information is discarded when the installation is complete.

• If single user mode is not enabled, the Cloudera Manager Agent runs as root so that it can make sure the required directories are created and that processes and files are owned by the appropriate user (for example, the hdfs and mapred users).
• No blocking is done by Security-Enhanced Linux (SELinux).
• No blocking by iptables or firewalls; port 7180 must be open because it is used to access Cloudera Manager after installation. Cloudera Manager communicates using specific ports, which must be open.
• For RHEL and CentOS, the /etc/sysconfig/network file on each host must contain the hostname you have just set (or verified) for that host.
• Cloudera Manager and CDH use several user accounts and groups to complete their tasks. The set of user accounts and groups varies according to the components you choose to install. Do not delete these accounts or groups and do not modify their permissions and rights. Ensure that no existing systems prevent these accounts and groups from functioning. For example, if you have scripts that delete user accounts not in a whitelist, add these accounts to the list of permitted accounts. Cloudera Manager, CDH, and managed services create and use the following accounts and groups:

This matrix contains compatibility information across versions of Impala and CDH/Cloudera Manager. For detailed information on each release, see the Impala documentation.

CDK Powered By Apache Kafka is currently distributed as a package and in a parcel that is independent of the CDH parcel. The parcel integrates with Cloudera Manager using a Custom Service Descriptor (CSD). Using the latest features of Kafka sometimes requires a newer version of Cloudera Manager and/or CDH that supports the feature. This matrix describes the compatibility between CDK Powered by Apache Kafka and Cloudera Manager and CDH.

For the latest documentation, see Kafka Documentation.

Maintaining compatibility across different Kafka clients and brokers is a common issue. This can occur as part of any of the following scenarios:

• Upgrading your Kafka cluster without upgrading your Kafka clients
• Using a third-party application that produces to or consumes from your Kafka cluster
• Having a client program communicating with two or more Kafka clusters (e.g. consuming from one cluster and producing to a different cluster)
• Using Flume or Spark as a Kafka consumer

In these cases, it is important to understand client/broker compatibility across Kafka versions. Here are general rules that apply:

1. Newer Kafka brokers can talk to older Kafka clients. The reverse is not true.
2. Changes in either the major part or the minor part of the upstream version major.minor determines whether the client and broker are compatible.

As a result, the general pattern for upgrading Kafka from version A to version B is:

1. Change Kafka server.properties to refer to version A.
2. Upgrade the brokers to version B and restart the cluster.
3. Upgrade the clients to version B.
4. After all the clients are upgraded, change the properties from the first step to version B and restart the cluster.

Flume has an embedded Kafka client which it uses to talk to Kafka clusters. Since the generally accepted practice is to have the broker running the same or newer version as the client, a CDH Flume version requires being matched to a minimum Kafka version. This matrix lists the CDK Powered By Apache Kafka (CDK) client versions used by Apache Flume in CDH, as well as the underlying Apache Kafka base version. For more information about CDK, see the Cloudera Distribution of Kafka Documentation.

Spark maintains two embedded Kafka clients and can be configured to use either one. For information on compatibility with Kafka brokers, refer to the appropriate entry in the table below.

This matrix contains compatibility information across versions of Kudu and CDH/Cloudera Manager. Starting with Apache Kudu 1.5.0 / CDH 5.13.0, Kudu is fully integrated into CDH and ships as part of the CDH parcel and packages.

Documentation for Kudu 1.5.0 / CDH 5.13.x (and higher) can be found at Apache Kudu Guide. Documentation for all lower versions can be found at Apache Kudu 1.4.0 (and lower).

Encryption comprises several components, each with its own requirements.

Data at rest encryption protection can be applied at a number of levels within Hadoop:

This section contains the various hardware and software requirements for all encryption products used for Data at Rest Encryption.

For more information on supported operating systems, see Product Compatibility Matrix for Cloudera Navigator Encryption.

For more information on the components, concepts, and architecture for encrypting data at rest, see Encrypting Data at Rest.

Cloudera Navigator encryption comprises several components. See Cloudera Navigator Data Encryption Overview for more information on the components, concepts, and architecture involved in encrypting data.

Although the version numbers differ between some Cloudera Navigator encryption components and Cloudera Enterprise, the encryption component releases are generally coordinated with Cloudera Enterprise releases. See the following table for information on which component versions correspond with a given Cloudera Enterprise release.

Sentry enables role-based, fine-grained authorization for HiveServer2 and provides classic database-style authorization for Hive, Cloudera Impala and Cloudera Search. You can use either the Sentry service (introduced in Cloudera Manager 5.1.0 and CDH 5.1.0) or the policy file approach to secure your data.

Spark is a fast, general engine for large-scale data processing. For installation and configuration instructions, see Setting Up Apache Spark Using the Command Line. To see new features introduced with each release, refer to the CDH 5 Release Notes.

Since Spark 1.6 has been integrated into the CDH package, its compatibility with Cloudera Manager and CDH depends on the CDH 5.x.x release it is shipped with.

See also Hadoop Distributions and Products Supported by OneFS on the Dell EMC Community website.

For compatibility with Cloudera Manager Replication and Snapshot features, see Product Compatibility Matrix for Backup and Disaster Recovery. For information on using Isilon with CDH, see Using CDH with Isilon Storage.

This section lists supported cloud and virtualization platforms for deploying Cloudera software. The linked reference architectures are not replacements for statements of support above, but are guides to assist with deployment and sizing options.