Cloudera Data Engineering on premises 1.5.5 CHF1

Review the features, fixes, and known issues in the Cloudera Data Engineering 1.5.5 Cumulative hotfix 1 release.

What's new

Review the new features in the Cloudera Data Engineering service of the Cloudera Data Services on premises 1.5.5 CHF1 release.

Support of Cloudera Runtime 7.3.1 version
Starting from Cloudera Data Services on premises 1.5.5 CHF1 release, Cloudera Data Engineering supports Cloudera Runtime 7.3.1 version.

Fixed issues

Review the list of issues that are resolved in the Cloudera Data Engineering service in the Cloudera Data Services 1.5.5 release.

DEX-17485: Disallow users with a role at environment level
With this fix, users who are assigned a DEAdmin or DEUser role at the environment level are filtered out and not displayed in the drop-down list for assigning a role at the Service or Virtual Cluster.
DEX-17459: When a dependent resource is denied access, the error message is wrong
Previously, when user2 tries to run a job containing private resources belonging to user1, the error message was misleading. This issue is fixed.

Known issues

The following are the known issues identified in 1.5.5 CHF1:

DEX-18727: Spark SQL functions do not work in Spark Connect sessions
Spark Connect functions require the SPARK_CONNECT_MODE_ENABLED environment variable to be set to true. Otherwise, Spark tries to use the normal Spark context functions, which do not work in Spark Connect.
To use Spark SQL functions in Spark Connect sessions, set the SPARK_CONNECT_MODE_ENABLED environment variable to true in the session by running the following command:
import os
os.environ["SPARK_CONNECT_MODE_ENABLED"]="true"