Cloudera Docs

Cloudera Data Engineering Private Cloud 1.5.4 SP1

Review the features, fixes, and known issues in the Cloudera Data Engineering 1.5.4 Service Pack 1 release.

What's new in 1.5.4 SP1

This section lists the new features added in this release for Cloudera Data Engineering Private Cloud.

Support for SAML authentication
Cloudera Data Engineering supports users to configure using SAML as Identity Provider along with LDAP. For more information about how to configure users using SAML authentication, see Configuring users for Identity Providers.
Support for NTP proxy setup
Cloudera Data Engineering requires specific proxy configurations to manage virtual cluster connections efficiently in an air-gapped setup with restricted outbound connections. This setup ensures seamless access to external resources while adhering to network security and management policies. For more information about how to setup NTP proxy, see NTP proxy setup on Cloudera Data Engineering.

Known issues in 1.5.4 SP1

This section lists the current known issues and limitations that you might run into while using the Cloudera Data Engineering service.

DEX-15429: Data connectors jobs are failing with "UNKNOWN: Channel Pipeline: [ProtocolNegotiators$ProxyProtocolNegotiationHandler#0, WriteBufferingAndExceptionHandler#0" in the proxy setup
Ozone data connector does not work when proxy is enabled.
Add the IP addresses of all the base cluster nodes to the NO_PROXY list so that request to Ozone does not use proxy.
DEX-15472: Cloudera Data Engineering service upgrade fails during backup and restore when SAML is enabled
Script-based upgrade of Cloudera Data Engineering service doesn't not work when SAML is enabled.
Make sure that the Cloudera Data Engineering service is upgraded using the dex-upgrade-utils solution before enabling the SAML (both IdP and SP initiated) in the Control Plane.
DEX-15735: The dbus-wxm-client pod appears in the dex-base namespace after editing the CPU and Memory quota the first time
After editing the Resource Pool CPU or Memory for a Cloudera Data Engineering Service for the first time, dbus-wxm-client pod gets deployed in the Cloudera Data Engineering Service namespace. The dbus-wxm-client pod keeps failing and cause alerts in the Management Console. However, this will not cause any other issues in Cloudera Data Engineering.
The dbus-wxm-client pod deployment is not needed and can be safely deleted to fix the issue. To delete it, do these steps:
  1. Identify the Cloudera Data Engineering Service namespace:
    1. In the Cloudera Data Platform (CDP) console, click the Data Engineering tile. The Cloudera Data Engineering home page displays.
    2. On the left navigation menu, click Administration.
    3. In the Services column, click for the Cloudera Data Engineering service for which you want to identify the namespace.
    4. Note the Cluster ID displayed on the page and identify the Cloudera Data Engineering service namespace. For example, if the Cluster ID is cluster-sales8098, then the Cloudera Data Engineering service namespace is dex-base-sales8098.
    5. Note this namespace.
  2. Using the Cloudera Data Engineering Serivce Namespace, run this Kubernetes command to delete the deployment:
    kubectl delete deployment dex-base-dbus-wxm-client -n [***CDE-SERVICE-NAMESPACE***]
    For example,
    kubectl delete deployment dex-base-dbus-wxm-client -n dex-base-sales8098

Fixed issues in 1.5.4 SP1

Review the list of issues that are resolved in the Cloudera Data Engineering (CDE) service in the CDP Data Services 1.5.4 SP1 release.

DEX-14484: PvC DS- Fix dex-runtime-python-builder to use python 3.8 even for Spark 3.3+
In Spark 3.3.2 virtual cluster, Python 3.8 is now used in Python virtual environment resource instead of old Python 3.6 which is not supported in Spark 3.3 or higher.