February 10, 2025
This release (1.23.1) of the Cloudera Data Engineering service on Cloudera Public Cloud introduces the following changes.
In-place upgrade enhancements
Using AWS, you can upgrade from Cloudera Data Engineering version 1.20.3 to 1.23.1. Using Azure, the minimum version of Cloudera Data Engineering for the upgrade is version 1.22.0. For more information, see Cloudera Data Engineering upgrade version compatibility and In-place upgrade with Airflow Operators and Libraries.
AWS Graviton spot instances support
With AWS Graviton, you can use spot instances as well. For more information, see AWS Graviton instances in Cloudera Data Engineering.
Data Lake 7.3.1 support
Cloudera Data Engineering version 1.23.1, besides supporting Data Lake 7.2.18, also supports Data Lake 7.3.1 with Apache Spark 3.5. For more information, see Compatibility for Cloudera Data Engineering and Runtime components.
Java upgrade to version 17
The Java version that Airflow uses is upgraded to Java 17. For more information, see Compatibility for Cloudera Data Engineering and Runtime components.
MySQL upgrade to version 8.0.39
The MySQL version that Cloudera Data Engineering 1.23.1 uses is upgraded to version 8.0.39.
Fixed issues
- DEX-15143
- Service backup failing due to Cadence size limits on the metadata
- DEX-15229
- UI glitch while accessing the Spark UI for a Cloudera Data Engineering Session
- DEX-15398
- Fix Helm upgrade failures: retrigger on context deadline exceeded and ensure correct revision on retry
- DEX-15477
- Rapid creation of successive job runs causes failures due to Jobs table locking
- DEX-15479
- Cannot upgrade Cloudera Data Engineering to 1.23 due to broken Spark Job
- DEX-15498
- Early unlock on the MutexMap causes incorrect locking behaviour
- DEX-15587
- RefreshRuns skips polling runs for Cloudera Data Engineering Job Status update for a while
- DEX-15589
- Livy marks the Batch failed when the monitoring thread is interrupted
- DEX-15713
- Jobs failing with keytab access issue
