Configuring Basic Authentication for Streams Replication Manager Services that have Remote Querying enabled

If a Streams Replication Manager Service has Remote Querying enabled, it will connect to other, remote Streams Replication Manager Services to fetch the remote cluster replication metrics. If the remote Streams Replication Manager Service has Basic Authentication (BA) enabled, the Streams Replication Manager Service that has Remote Querying enabled must be provided with valid credentials that it can use to connect to the remote Streams Replication Manager Service.

Remote Querying is a feature of the Streams Replication Manager Service that enables you to configure a Streams Replication Manager Service in your deployment to connect to other, remote Streams Replication Manager Services and fetch the remote cluster replication metrics gathered by the remote Service. This allows you to monitor the replications of a whole deployment that has multiple installations of Streams Replication Manager through a single Streams Replication Manager and Streams Messaging Manager service. For more information on Remote Querying, see Remote Querying.

If the remote Streams Replication Manager Services are configured to use BA, the Streams Replication Manager Services that have Remote Querying enabled (monitoring gateways) require valid credentials that can be used to access the remote Streams Replication Manager Services. As a result, the credentials required to access the remote Streams Replication Manager Services must be provided to the monitoring gateways.

Configuration is done in Cloudera Manager by creating Streams Replication Manager Service Credentials and then configuring the monitoring gateways to use these credentials for Remote Querying.

  • Basic authentication is set up and configured for the remote Streams Replication Manager Services. For more information on how to enable Basic authentication see, Enabling Basic Authentication for the Streams Replication Manager Service.
  • Remote querying is enabled and configured. For more information see, Enabling Remote Querying
  • Ensure that you have access to the BA usernames and passwords used by the remote Streams Replication Manager Services. You will need to provide these during configuration. These usernames and passwords were configured using Basic Authentication Credentials when you enabled BA for the remote Streams Replication Manager Services.
  1. Access the Cloudera Manager instance managing the Streams Replication Manager Service that has Remote Querying enabled (monitoring gateway).
  2. Create Streams Replication Manager Credentials:
    A Streams Replication Manager Credential represents a remote Streams Replication Manager Service that has Basic Authentication enabled. The credential stores the information necessary to establish a connection with the remote Streams Replication Manager Service. Create a credential for each BA enabled remote Streams Replication Manager Service.
    1. Go to Administration > External Accounts > Streams Replication Manager Credentials.
    2. Click Add Streams Replication Manager Service Credentials.
    3. Configure the following properties:
      • Name

        Add a unique and easily identifiable name. Note down the name you configure, you will need to provide it in a later step.

      • Remote Streams Replication Manager Service target Kafka cluster alias

        This is the alias of the Kafka cluster that the remote Streams Replication Manager Service (which this credential represents) is targeting. The alias that you add here must be one of the aliases that you added to the Streams Replication Manager Service Remote Target Clusters property when you enabled Remote Querying.

      • Username and Password

        Add the username and password pair that can be used to access the remote Streams Replication Manager Service that this credential represents. Both the username and password that you add here must be valid. That is, in the Cloudera Manager instance managing the remote Streams Replication Manager Service, a Basic Authentication credential must exist with this username and password combination. Additionally, the remote Streams Replication Manager Service must be configured to use these credentials.

      To better understand how Streams Replication Manager Service Credentials should be configured, review the following two cluster example:

      In this specific example, a single Streams Replication Manager Service Credential must be created in Cloudera Manager A. This credential represents Streams Replication Manager Service B and will be used by Streams Replication Manager Service A to access Streams Replication Manager Service B. The credential would be configured as follows:
      Name: srm_b
      Remote Streams Replication Manager Service target Kafka cluster alias: kafka_b
      Username: admin
      Password: password
      
  3. Configure this Streams Replication Manager Service (monitoring gateway) to use the credentials when querying the remote Streams Replication Manager Service:
    1. Go to Clusters and select the Streams Replication Manager Service.
    2. Go to Configuration.
    3. Find and Configure the External Accounts For Remote Querying SRM Services property.
    4. Add the names of all Streams Replication Manager Service Credentials you created in Step 1. For example:
      remote_1
      remote_2
      remote_3
  4. Click Save Changes.
  5. Restart Streams Replication Manager.
The credentials required to access the remote Streams Replication Manager Services are set up and configured. When this Streams Replication Manager Service establishes a connection with the configured remote Streams Replication Manager Services, it uses the configured credentials.
  • Access the Replications page on the Streams Messaging Manager UI. Remote replications will be visible in the UI.
  • Query metrics using the Streams Replication Manager REST API. For example:
    1. Go to Streams Replication Manager > Web UI > SRM Service Swagger UI.
    2. Find and open the /v2/replications endpoint.
    3. Click Try it out then click Execute.

      The response includes all discovered replications, replicated topics, and various other metrics. This includes replications that target remote clusters.