Homepage2.10.0

Cloudera Flow Management - Kubernetes Operator

Cloudera Flow Management - Kubernetes Operator allows you to deploy and manage Apache NiFi clusters and NiFi Registry instances on Kubernetes. It simplifies data collection, transformation, and delivery across your enterprise. Leveraging containerized infrastructure, the operator streamlines the orchestration of complex data flows.

Key features and benefits

Deploying and managing Apache NiFi

  • Deploy NiFi easily and fast.
  • Manage multiple NiFi deployments.
  • Run multiple clusters with different NiFi versions (NiFi 1.x, 2.x).
  • Run NiFi in a single-node mode or as a distributed cluster.

Deploying and managing NiFi Registry

  • Deploy one or more NiFi Registries.
  • Connect a NiFi Registry instance to one or more NiFi clusters and use it for central flow versioning.
  • cfmctl command line tool for installation
  • Simplify the installation and deployment of Cloudera Flow Management - Kubernetes Operator and NiFi and NiFi Registry CRDs.
  • Quickly assess environment readiness. NiFi clusters without ZooKeeper
  • Enable NiFi deployment without ZooKeeper by handling leader election and state management directly within Kubernetes.
  • Store NiFi cluster state within Kubernetes configuration maps. Scaling NiFi clusters up and down
  • Manually scale up/down your NiFi deployments by adjusting the number of NiFi nodes in the configuration file.
  • Utilize Kubernetes Horizontal Pod Autoscaler (HPA) to monitor Central Processing Unit (CPU) utilization and trigger autoscaling based on resource consumption. Data durability
  • In-flight data remains uninterrupted even if a NiFi node goes down. Kubernetes will provision a new node and transfer the volumes from the failed node to the new one, ensuring continuous data processing. Full lifecycle management
  • Change your configuration by deploying a modified configuration file and let the operator reconcile.
  • Perform NiFi version upgrades effortlessly by specifying a new NiFi image. Support for rolling upgrades is coming soon for NiFi 2 clusters.

Components

Cloudera Flow Management - Kubernetes Operator

Cloudera Flow Management - Kubernetes Operator is an application that provides a way to run Apache NiFi and NiFi Registry clusters on Kubernetes.

The operator itself is made up of multiple components and includes various operator applications, Custom Resource Definitions (CRDs) as well as Docker (container) images.

Operator applications are purpose built Kubernetes applications that act as an extension to Kubernetes. These applications provide an easy way for you to deploy, manage, and configure NiFi and NiFi Registry.

NiFi

The core data ingestion engine that has a no-code graphical user interface and supports processors for connectivity, transformation, and content routing.

NiFi Registry

The companion to NiFi that enables DevOps style development and deployment of flow files. It also supports flow versioning, promoting flows from one environment into another, and flow deployment.

Component and feature support

Components shipped in Cloudera Flow Management - Kubernetes Operator are based on open source projects and might contain additional changes or fixes to guarantee that they work in Cloudera supported environments.

Additionally, not all NiFi features are supported. For additional information see the Release Notes as well as Component versions.

Getting started

To get started, install Cloudera Flow Management - Kubernetes Operator. Following installation, you can deploy instances of NiFi and NiFi Registry clusters.

You can learn more about the individual components as well as Kubernetes Operators by visiting the following upstream resources: