Cloudera Docs

Enabling prefixless replication

Learn how to enable prefixless replication with the IdentityReplicationPolicy. If this replication policy is in use, remote topics on target clusters retain their original name and are not prefixed.

The default replication policy used by Streams Replication Manager is the DefaultReplicationPolicy. This replication policy renames remote (replicated) topics on target clusters and adds the source cluster alias as a prefix. For example, useast.topic1. If you do not want your topics to get renamed, you can choose to use IdentityReplicationPolicy instead, which does not rename topics during replication. This type of replication is referred to as prefixless replication.

You can configure Streams Replication Manager to use the IdentityReplicationPolicy by selecting the Enable Prefixless Replication Streams Replication Manager service property in Cloudera Manager.

  • Ensure that the Remote Topics Discovery With Internal Topic property remains selected after you enable prefixless replication. This property must be selected for prefixless replication to function. This property is selected by default and is also automatically selected when you enable prefixless replication with Enable Prefixless Replication.
  • Do not use replication.policy.class to set IdentityReplicationPolicy as the replication policy. Use Enable Prefixless Replication instead.
  • Using a mix of replications policies in a deployment with multiple instances of the Streams Replication Manager service is not supported.
  • Transitioning an already running Streams Replication Manager service from the DefaultReplicationPolicy or any other replication policy to the IdentityReplicationPolicy is not recommended.
  • The IdentityReplicationPolicy has the following limitations:
    • Replication loop detection is not supported. As a result, you must ensure that topics are not replicated in a loop between your source and target clusters.
    • The /v2/topic-metrics/{target}/{downstreamTopic}/{metric} endpoint of Streams Replication Manager Service v2 API does not work properly with prefixless replication. Use the /v2/topic-metrics/{source}/{target}/{upstreamTopic}/{metric} endpoint instead.
    • The replication metric graphs shown on the Topic Details page of the Streams Messaging Manager UI do not work with prefixless replication.
    As a result of these limitations, Cloudera recommends that you use the DefaultReplicationPolicy whenever possible.
  1. In Cloudera Manager, select the Streams Replication Manager service.
  2. Go to Configuration.
  3. Find and select the Enable Prefixless Replication property.
  4. Click Save Changes.
  5. Restart the service.
Streams Replication Manager is configured to use the IdentityReplicationPolicy. Remote topics on target clusters are no longer prefixed with the source cluster alias.