Cloudera AI on premises 1.5.4 SP2

Cloudera on premises 1.5.4 SP2 includes the following features for Cloudera AI.

Introducing auto scaling performance-critical components

Some performance-critical components in Cloudera AI automatically scale based on demand, enhancing the scalability and the reliability of the product. To enable utilising this feature, we have increased the recommended resource requirements for Cloudera AI by 16 cores.

You might run into some known issues while using Cloudera AI on premises.

DSE-42079: Cloudera AI Workbenches are not compatible with the Unified timezone feature

When you enable the Unified timezone feature, the Cloudera Embedded Container Service cluster timezone is synchronized with the Cloudera Manager Base time zone, and the Cloudera AI sessions fails to launch with Exit Code 34. Timestamp discrepancies with workloads are displayed.

Workaround:

If you use Cloudera AI Workbenches, disable the Unified Timezone feature by following the instructions in Cloudera Embedded Container Service unified time zone.

DSE-41757: Python workloads running multiple-line comments or strings might fail

Python workloads running multiple-line comments or strings might fail to run when using the Workbench Editor.

Workaround:

Run the code using the PBJ Workbench Editor.

DSE-42509: Creating a project does not work on NTP/Airgap proxy when using private repository with SSH key and SSH URL

When creating a project on NTP/Airgap proxy, using a private repository with an SSH key and an SSH URL, an error message is displayed that the project cannot be created.

No workaround is available.

DSE-42510: Fetching Cloudera or Huggingface AMP catalog is failing on NTP/Airgap proxy

When fetching Cloudera or Huggingface Cloudera Accelerators for Machine Learning Project (AMP) catalog on NTP/Airgap proxy, an error message is displayed Error fetching AMP catalog source URL.

No workaround is available.

This section lists issues fixed in this release for Cloudera AI on premises.

DSE-40909: Disabled 'Run experiments' function does not work as expected

When the Administrator disabled 'running experiments' in Cloudera AI Workbench > Site Administration > Settings, the user was still able to see the Experiments menu item in the global- and project-level navigation.

With the help of implementing the sufficient conditions the UI can now hide the disabled function.

DSE-38645: spark.yarn.jars are not set properly in Cloudera AI on premises Spark Pushdown mode

In case of Spark pushdown, the spark.yarn.jars parameter can indicate which jars in the driver shall be transported to YARN executors. Without this parameter set, the YARN executors use the Spark jars from the base cluster provided by Cloudera Distribution of Spark. This can lead to version mismatch between the Spark driver (used Cloudera Data Engineering-provided jars) and the executors.

Now, the spark.yarn.jars parameter is set to use jars from the Cloudera AI driver (Cloudera AI session) to keep the versions matched between the Spark driver and the executors.

DSE-39798: API v1 stop session endpoint shall perform authentication check

Previously, users could stop sessions under projects that they were not authorized to access using the session’s Universally unique identifier (UUID). This issue is now resolved.

DSE-41431: Register engine stopped status

When the reconciler is overloaded with a large number of events, the deleted status is still propagated to ensure that engines do not remain in 'Stopping' status.

DSE-41424: Better handling for data connection validation errors

In Cloudera AI on premises 1.5.4 SP1 Cloudera AI Workbench failed to start under the following conditions:

• If the HIVE_ON_TEZ service was not present or the following configurations were missing:

  • hiveserver2_load_balancer

  • hive.server2.transport.mode

  • Kerberos_princ_name

• If the IMPALA service was not present on your Base cluster or the following configurations were missing:

  • Hs2_port or hs2_http_port

  • Kerberos_princ_name

This issue has been fixed.

DSE-41218: Restrict secrets and ingress access from User Service Account role

User access to Kubernetes secrets and ingresses in their own user namespace has been removed.

DSE-34314: Cloudera Base AutoTLS property set incorrectly

The correction to Cloudera Base AutoTLS configurations resulted in the atlas.kafka.ssl.truststore.location value to be set incorrectly.

This issue has been fixed.