Managed Streaming for Kafka

2026/04/20 - Managed Streaming for Kafka - 4 updated api methods

Changes  Amazon MSK Replicator now supports data migration from external Apache Kafka clusters to Amazon MSK Express brokers. This release adds SaslScram authentication with TLS encryption, enhanced consumer offset synchronization, and customer log forwarding for troubleshooting.

CreateReplicator (updated) Link ¶
Changes (request)
{'KafkaClusters': {'ApacheKafkaCluster': {'ApacheKafkaClusterId': 'string',
                                          'BootstrapBrokerString': 'string'},
                   'ClientAuthentication': {'SaslScram': {'Mechanism': 'SHA256 '
                                                                       '| '
                                                                       'SHA512',
                                                          'SecretArn': 'string'}},
                   'EncryptionInTransit': {'EncryptionType': 'TLS',
                                           'RootCaCertificate': 'string'}},
 'LogDelivery': {'ReplicatorLogDelivery': {'CloudWatchLogs': {'Enabled': 'boolean',
                                                              'LogGroup': 'string'},
                                           'Firehose': {'DeliveryStream': 'string',
                                                        'Enabled': 'boolean'},
                                           'S3': {'Bucket': 'string',
                                                  'Enabled': 'boolean',
                                                  'Prefix': 'string'}}},
 'ReplicationInfoList': {'ConsumerGroupReplication': {'ConsumerGroupOffsetSyncMode': 'LEGACY '
                                                                                     '| '
                                                                                     'ENHANCED'},
                         'SourceKafkaClusterId': 'string',
                         'TargetKafkaClusterId': 'string'}}

Creates the replicator.

See also: AWS API Documentation

Request Syntax

client.create_replicator(
    Description='string',
    KafkaClusters=[
        {
            'AmazonMskCluster': {
                'MskClusterArn': 'string'
            },
            'ApacheKafkaCluster': {
                'ApacheKafkaClusterId': 'string',
                'BootstrapBrokerString': 'string'
            },
            'VpcConfig': {
                'SecurityGroupIds': [
                    'string',
                ],
                'SubnetIds': [
                    'string',
                ]
            },
            'ClientAuthentication': {
                'SaslScram': {
                    'Mechanism': 'SHA256'|'SHA512',
                    'SecretArn': 'string'
                }
            },
            'EncryptionInTransit': {
                'EncryptionType': 'TLS',
                'RootCaCertificate': 'string'
            }
        },
    ],
    ReplicationInfoList=[
        {
            'ConsumerGroupReplication': {
                'ConsumerGroupsToExclude': [
                    'string',
                ],
                'ConsumerGroupsToReplicate': [
                    'string',
                ],
                'DetectAndCopyNewConsumerGroups': True|False,
                'SynchroniseConsumerGroupOffsets': True|False,
                'ConsumerGroupOffsetSyncMode': 'LEGACY'|'ENHANCED'
            },
            'SourceKafkaClusterArn': 'string',
            'SourceKafkaClusterId': 'string',
            'TargetCompressionType': 'NONE'|'GZIP'|'SNAPPY'|'LZ4'|'ZSTD',
            'TargetKafkaClusterArn': 'string',
            'TargetKafkaClusterId': 'string',
            'TopicReplication': {
                'CopyAccessControlListsForTopics': True|False,
                'CopyTopicConfigurations': True|False,
                'DetectAndCopyNewTopics': True|False,
                'StartingPosition': {
                    'Type': 'LATEST'|'EARLIEST'
                },
                'TopicNameConfiguration': {
                    'Type': 'PREFIXED_WITH_SOURCE_CLUSTER_ALIAS'|'IDENTICAL'
                },
                'TopicsToExclude': [
                    'string',
                ],
                'TopicsToReplicate': [
                    'string',
                ]
            }
        },
    ],
    ReplicatorName='string',
    ServiceExecutionRoleArn='string',
    Tags={
        'string': 'string'
    },
    LogDelivery={
        'ReplicatorLogDelivery': {
            'CloudWatchLogs': {
                'Enabled': True|False,
                'LogGroup': 'string'
            },
            'Firehose': {
                'Enabled': True|False,
                'DeliveryStream': 'string'
            },
            'S3': {
                'Enabled': True|False,
                'Bucket': 'string',
                'Prefix': 'string'
            }
        }
    }
)
type Description:

string

param Description:

A summary description of the replicator.

type KafkaClusters:

list

param KafkaClusters:

[REQUIRED]

Kafka Clusters to use in setting up sources / targets for replication.

  • (dict) --

    Information about Kafka Cluster to be used as source / target for replication.

    • AmazonMskCluster (dict) --

      Details of an Amazon MSK Cluster.

      • MskClusterArn (string) -- [REQUIRED]

        The Amazon Resource Name (ARN) of an Amazon MSK cluster.

    • ApacheKafkaCluster (dict) --

      Details of an Apache Kafka Cluster.

      • ApacheKafkaClusterId (string) -- [REQUIRED]

        The ID of the Apache Kafka cluster.

      • BootstrapBrokerString (string) -- [REQUIRED]

        The bootstrap broker string of the Apache Kafka cluster.

    • VpcConfig (dict) --

      Details of an Amazon VPC which has network connectivity to the Apache Kafka cluster.

      • SecurityGroupIds (list) --

        The security groups to attach to the ENIs for the broker nodes.

        • (string) --

      • SubnetIds (list) -- [REQUIRED]

        The list of subnets in the client VPC to connect to.

        • (string) --

    • ClientAuthentication (dict) --

      Details of the client authentication used by the Apache Kafka cluster.

      • SaslScram (dict) -- [REQUIRED]

        Details for SASL/SCRAM client authentication.

        • Mechanism (string) -- [REQUIRED]

          The SASL/SCRAM authentication mechanism.

        • SecretArn (string) -- [REQUIRED]

          The Amazon Resource Name (ARN) of the Secrets Manager secret.

    • EncryptionInTransit (dict) --

      Details of encryption in transit to the Apache Kafka cluster.

      • EncryptionType (string) -- [REQUIRED]

        The type of encryption in transit to the Apache Kafka cluster.

      • RootCaCertificate (string) --

        The root CA certificate.

type ReplicationInfoList:

list

param ReplicationInfoList:

[REQUIRED]

A list of replication configurations, where each configuration targets a given source cluster to target cluster replication flow.

  • (dict) --

    Specifies configuration for replication between a source and target Kafka cluster.

    • ConsumerGroupReplication (dict) -- [REQUIRED]

      Configuration relating to consumer group replication.

      • ConsumerGroupsToExclude (list) --

        List of regular expression patterns indicating the consumer groups that should not be replicated.

        • (string) --

      • ConsumerGroupsToReplicate (list) -- [REQUIRED]

        List of regular expression patterns indicating the consumer groups to copy.

        • (string) --

      • DetectAndCopyNewConsumerGroups (boolean) --

        Enables synchronization of consumer groups to target cluster.

      • SynchroniseConsumerGroupOffsets (boolean) --

        Enables synchronization of consumer group offsets to target cluster. The translated offsets will be written to topic __consumer_offsets.

      • ConsumerGroupOffsetSyncMode (string) --

        The consumer group offset synchronization mode. With LEGACY, offsets are synchronized when producers write to the source cluster. With ENHANCED, consumer offsets are synchronized regardless of producer location. ENHANCED requires a corresponding replicator that replicates data from the target cluster to the source cluster.

    • SourceKafkaClusterArn (string) --

      The ARN of the source Kafka cluster.

    • SourceKafkaClusterId (string) --

      The ID of the source Kafka cluster.

    • TargetCompressionType (string) -- [REQUIRED]

      The compression type to use when producing records to target cluster.

    • TargetKafkaClusterArn (string) --

      The ARN of the target Kafka cluster.

    • TargetKafkaClusterId (string) --

      The ID of the target Kafka cluster.

    • TopicReplication (dict) -- [REQUIRED]

      Configuration relating to topic replication.

      • CopyAccessControlListsForTopics (boolean) --

        Whether to periodically configure remote topic ACLs to match their corresponding upstream topics.

      • CopyTopicConfigurations (boolean) --

        Whether to periodically configure remote topics to match their corresponding upstream topics.

      • DetectAndCopyNewTopics (boolean) --

        Whether to periodically check for new topics and partitions.

      • StartingPosition (dict) --

        Configuration for specifying the position in the topics to start replicating from.

        • Type (string) --

          The type of replication starting position.

      • TopicNameConfiguration (dict) --

        Configuration for specifying replicated topic names should be the same as their corresponding upstream topics or prefixed with source cluster alias.

        • Type (string) --

          The type of replicated topic name.

      • TopicsToExclude (list) --

        List of regular expression patterns indicating the topics that should not be replicated.

        • (string) --

      • TopicsToReplicate (list) -- [REQUIRED]

        List of regular expression patterns indicating the topics to copy.

        • (string) --

type ReplicatorName:

string

param ReplicatorName:

[REQUIRED]

The name of the replicator. Alpha-numeric characters with '-' are allowed.

type ServiceExecutionRoleArn:

string

param ServiceExecutionRoleArn:

[REQUIRED]

The ARN of the IAM role used by the replicator to access resources in the customer's account (e.g source and target clusters)

type Tags:

dict

param Tags:

List of tags to attach to created Replicator.

  • (string) --

    • (string) --

type LogDelivery:

dict

param LogDelivery:

Configuration for delivering replicator logs to customer destinations.

  • ReplicatorLogDelivery (dict) --

    Configuration for replicator log delivery.

    • CloudWatchLogs (dict) --

      Configuration for CloudWatch Logs delivery.

      • Enabled (boolean) -- [REQUIRED]

        Whether log delivery to CloudWatch Logs is enabled.

      • LogGroup (string) --

        The CloudWatch log group that is the destination for log delivery.

    • Firehose (dict) --

      Configuration for Firehose delivery.

      • Enabled (boolean) -- [REQUIRED]

        Whether log delivery to Firehose is enabled.

      • DeliveryStream (string) --

        The Firehose delivery stream that is the destination for log delivery.

    • S3 (dict) --

      Configuration for S3 delivery.

      • Enabled (boolean) -- [REQUIRED]

        Whether log delivery to S3 is enabled.

      • Bucket (string) --

        The S3 bucket that is the destination for log delivery.

      • Prefix (string) --

        The S3 prefix that is the destination for log delivery.

rtype:

dict

returns:

Response Syntax

{
    'ReplicatorArn': 'string',
    'ReplicatorName': 'string',
    'ReplicatorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED'
}

Response Structure

  • (dict) --

    HTTP Status Code 200: OK.

    • ReplicatorArn (string) --

      The Amazon Resource Name (ARN) of the replicator.

    • ReplicatorName (string) --

      Name of the replicator provided by the customer.

    • ReplicatorState (string) --

      State of the replicator.

DescribeReplicator (updated) Link ¶
Changes (response)
{'KafkaClusters': {'ApacheKafkaCluster': {'ApacheKafkaClusterId': 'string',
                                          'BootstrapBrokerString': 'string'},
                   'ClientAuthentication': {'SaslScram': {'Mechanism': 'SHA256 '
                                                                       '| '
                                                                       'SHA512',
                                                          'SecretArn': 'string'}},
                   'EncryptionInTransit': {'EncryptionType': 'TLS',
                                           'RootCaCertificate': 'string'}},
 'LogDelivery': {'ReplicatorLogDelivery': {'CloudWatchLogs': {'Enabled': 'boolean',
                                                              'LogGroup': 'string'},
                                           'Firehose': {'DeliveryStream': 'string',
                                                        'Enabled': 'boolean'},
                                           'S3': {'Bucket': 'string',
                                                  'Enabled': 'boolean',
                                                  'Prefix': 'string'}}},
 'ReplicationInfoList': {'ConsumerGroupReplication': {'ConsumerGroupOffsetSyncMode': 'LEGACY '
                                                                                     '| '
                                                                                     'ENHANCED'}}}

Describes a replicator.

See also: AWS API Documentation

Request Syntax

client.describe_replicator(
    ReplicatorArn='string'
)
type ReplicatorArn:

string

param ReplicatorArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the replicator to be described.

rtype:

dict

returns:

Response Syntax

{
    'CreationTime': datetime(2015, 1, 1),
    'CurrentVersion': 'string',
    'IsReplicatorReference': True|False,
    'KafkaClusters': [
        {
            'AmazonMskCluster': {
                'MskClusterArn': 'string'
            },
            'ApacheKafkaCluster': {
                'ApacheKafkaClusterId': 'string',
                'BootstrapBrokerString': 'string'
            },
            'KafkaClusterAlias': 'string',
            'VpcConfig': {
                'SecurityGroupIds': [
                    'string',
                ],
                'SubnetIds': [
                    'string',
                ]
            },
            'ClientAuthentication': {
                'SaslScram': {
                    'Mechanism': 'SHA256'|'SHA512',
                    'SecretArn': 'string'
                }
            },
            'EncryptionInTransit': {
                'EncryptionType': 'TLS',
                'RootCaCertificate': 'string'
            }
        },
    ],
    'ReplicationInfoList': [
        {
            'ConsumerGroupReplication': {
                'ConsumerGroupsToExclude': [
                    'string',
                ],
                'ConsumerGroupsToReplicate': [
                    'string',
                ],
                'DetectAndCopyNewConsumerGroups': True|False,
                'SynchroniseConsumerGroupOffsets': True|False,
                'ConsumerGroupOffsetSyncMode': 'LEGACY'|'ENHANCED'
            },
            'SourceKafkaClusterAlias': 'string',
            'TargetCompressionType': 'NONE'|'GZIP'|'SNAPPY'|'LZ4'|'ZSTD',
            'TargetKafkaClusterAlias': 'string',
            'TopicReplication': {
                'CopyAccessControlListsForTopics': True|False,
                'CopyTopicConfigurations': True|False,
                'DetectAndCopyNewTopics': True|False,
                'StartingPosition': {
                    'Type': 'LATEST'|'EARLIEST'
                },
                'TopicNameConfiguration': {
                    'Type': 'PREFIXED_WITH_SOURCE_CLUSTER_ALIAS'|'IDENTICAL'
                },
                'TopicsToExclude': [
                    'string',
                ],
                'TopicsToReplicate': [
                    'string',
                ]
            }
        },
    ],
    'ReplicatorArn': 'string',
    'ReplicatorDescription': 'string',
    'ReplicatorName': 'string',
    'ReplicatorResourceArn': 'string',
    'ReplicatorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED',
    'ServiceExecutionRoleArn': 'string',
    'StateInfo': {
        'Code': 'string',
        'Message': 'string'
    },
    'Tags': {
        'string': 'string'
    },
    'LogDelivery': {
        'ReplicatorLogDelivery': {
            'CloudWatchLogs': {
                'Enabled': True|False,
                'LogGroup': 'string'
            },
            'Firehose': {
                'Enabled': True|False,
                'DeliveryStream': 'string'
            },
            'S3': {
                'Enabled': True|False,
                'Bucket': 'string',
                'Prefix': 'string'
            }
        }
    }
}

Response Structure

  • (dict) --

    HTTP Status Code 200: OK.

    • CreationTime (datetime) --

      The time when the replicator was created.

    • CurrentVersion (string) --

      The current version number of the replicator.

    • IsReplicatorReference (boolean) --

      Whether this resource is a replicator reference.

    • KafkaClusters (list) --

      Kafka Clusters used in setting up sources / targets for replication.

      • (dict) --

        Information about Kafka Cluster used as source / target for replication.

        • AmazonMskCluster (dict) --

          Details of an Amazon MSK Cluster.

          • MskClusterArn (string) --

            The Amazon Resource Name (ARN) of an Amazon MSK cluster.

        • ApacheKafkaCluster (dict) --

          Details of an Apache Kafka Cluster.

          • ApacheKafkaClusterId (string) --

            The ID of the Apache Kafka cluster.

          • BootstrapBrokerString (string) --

            The bootstrap broker string of the Apache Kafka cluster.

        • KafkaClusterAlias (string) --

          The alias of the Kafka cluster. Used to prefix names of replicated topics.

        • VpcConfig (dict) --

          Details of an Amazon VPC which has network connectivity to the Apache Kafka cluster.

          • SecurityGroupIds (list) --

            The security groups to attach to the ENIs for the broker nodes.

            • (string) --

          • SubnetIds (list) --

            The list of subnets in the client VPC to connect to.

            • (string) --

        • ClientAuthentication (dict) --

          Details of the client authentication used by the Apache Kafka cluster.

          • SaslScram (dict) --

            Details for SASL/SCRAM client authentication.

            • Mechanism (string) --

              The SASL/SCRAM authentication mechanism.

            • SecretArn (string) --

              The Amazon Resource Name (ARN) of the Secrets Manager secret.

        • EncryptionInTransit (dict) --

          Details of encryption in transit to the Apache Kafka cluster.

          • EncryptionType (string) --

            The type of encryption in transit to the Apache Kafka cluster.

          • RootCaCertificate (string) --

            The root CA certificate.

    • ReplicationInfoList (list) --

      A list of replication configurations, where each configuration targets a given source cluster to target cluster replication flow.

      • (dict) --

        Specifies configuration for replication between a source and target Kafka cluster (sourceKafkaClusterAlias -> targetKafkaClusterAlias)

        • ConsumerGroupReplication (dict) --

          Configuration relating to consumer group replication.

          • ConsumerGroupsToExclude (list) --

            List of regular expression patterns indicating the consumer groups that should not be replicated.

            • (string) --

          • ConsumerGroupsToReplicate (list) --

            List of regular expression patterns indicating the consumer groups to copy.

            • (string) --

          • DetectAndCopyNewConsumerGroups (boolean) --

            Enables synchronization of consumer groups to target cluster.

          • SynchroniseConsumerGroupOffsets (boolean) --

            Enables synchronization of consumer group offsets to target cluster. The translated offsets will be written to topic __consumer_offsets.

          • ConsumerGroupOffsetSyncMode (string) --

            The consumer group offset synchronization mode. With LEGACY, offsets are synchronized when producers write to the source cluster. With ENHANCED, consumer offsets are synchronized regardless of producer location. ENHANCED requires a corresponding replicator that replicates data from the target cluster to the source cluster.

        • SourceKafkaClusterAlias (string) --

          The alias of the source Kafka cluster.

        • TargetCompressionType (string) --

          The compression type to use when producing records to target cluster.

        • TargetKafkaClusterAlias (string) --

          The alias of the target Kafka cluster.

        • TopicReplication (dict) --

          Configuration relating to topic replication.

          • CopyAccessControlListsForTopics (boolean) --

            Whether to periodically configure remote topic ACLs to match their corresponding upstream topics.

          • CopyTopicConfigurations (boolean) --

            Whether to periodically configure remote topics to match their corresponding upstream topics.

          • DetectAndCopyNewTopics (boolean) --

            Whether to periodically check for new topics and partitions.

          • StartingPosition (dict) --

            Configuration for specifying the position in the topics to start replicating from.

            • Type (string) --

              The type of replication starting position.

          • TopicNameConfiguration (dict) --

            Configuration for specifying replicated topic names should be the same as their corresponding upstream topics or prefixed with source cluster alias.

            • Type (string) --

              The type of replicated topic name.

          • TopicsToExclude (list) --

            List of regular expression patterns indicating the topics that should not be replicated.

            • (string) --

          • TopicsToReplicate (list) --

            List of regular expression patterns indicating the topics to copy.

            • (string) --

    • ReplicatorArn (string) --

      The Amazon Resource Name (ARN) of the replicator.

    • ReplicatorDescription (string) --

      The description of the replicator.

    • ReplicatorName (string) --

      The name of the replicator.

    • ReplicatorResourceArn (string) --

      The Amazon Resource Name (ARN) of the replicator resource in the region where the replicator was created.

    • ReplicatorState (string) --

      State of the replicator.

    • ServiceExecutionRoleArn (string) --

      The Amazon Resource Name (ARN) of the IAM role used by the replicator to access resources in the customer's account (e.g source and target clusters)

    • StateInfo (dict) --

      Details about the state of the replicator.

      • Code (string) -- Code that describes the current state of the replicator.

      • Message (string) -- Message that describes the state of the replicator.

    • Tags (dict) --

      List of tags attached to the Replicator.

      • (string) --

        • (string) --

    • LogDelivery (dict) --

      Configuration for log delivery.

      • ReplicatorLogDelivery (dict) --

        Configuration for replicator log delivery.

        • CloudWatchLogs (dict) --

          Configuration for CloudWatch Logs delivery.

          • Enabled (boolean) --

            Whether log delivery to CloudWatch Logs is enabled.

          • LogGroup (string) --

            The CloudWatch log group that is the destination for log delivery.

        • Firehose (dict) --

          Configuration for Firehose delivery.

          • Enabled (boolean) --

            Whether log delivery to Firehose is enabled.

          • DeliveryStream (string) --

            The Firehose delivery stream that is the destination for log delivery.

        • S3 (dict) --

          Configuration for S3 delivery.

          • Enabled (boolean) --

            Whether log delivery to S3 is enabled.

          • Bucket (string) --

            The S3 bucket that is the destination for log delivery.

          • Prefix (string) --

            The S3 prefix that is the destination for log delivery.

ListReplicators (updated) Link ¶
Changes (response)
{'Replicators': {'KafkaClustersSummary': {'ApacheKafkaCluster': {'ApacheKafkaClusterId': 'string',
                                                                 'BootstrapBrokerString': 'string'}}}}

Lists the replicators.

See also: AWS API Documentation

Request Syntax

client.list_replicators(
    MaxResults=123,
    NextToken='string',
    ReplicatorNameFilter='string'
)
type MaxResults:

integer

param MaxResults:

The maximum number of results to return in the response. If there are more results, the response includes a NextToken parameter.

type NextToken:

string

param NextToken:

If the response of ListReplicators is truncated, it returns a NextToken in the response. This NextToken should be sent in the subsequent request to ListReplicators.

type ReplicatorNameFilter:

string

param ReplicatorNameFilter:

Returns replicators starting with given name.

rtype:

dict

returns:

Response Syntax

{
    'NextToken': 'string',
    'Replicators': [
        {
            'CreationTime': datetime(2015, 1, 1),
            'CurrentVersion': 'string',
            'IsReplicatorReference': True|False,
            'KafkaClustersSummary': [
                {
                    'AmazonMskCluster': {
                        'MskClusterArn': 'string'
                    },
                    'ApacheKafkaCluster': {
                        'ApacheKafkaClusterId': 'string',
                        'BootstrapBrokerString': 'string'
                    },
                    'KafkaClusterAlias': 'string'
                },
            ],
            'ReplicationInfoSummaryList': [
                {
                    'SourceKafkaClusterAlias': 'string',
                    'TargetKafkaClusterAlias': 'string'
                },
            ],
            'ReplicatorArn': 'string',
            'ReplicatorName': 'string',
            'ReplicatorResourceArn': 'string',
            'ReplicatorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED'
        },
    ]
}

Response Structure

  • (dict) -- HTTP Status Code 200: OK.

    • NextToken (string) --

      If the response of ListReplicators is truncated, it returns a NextToken in the response. This NextToken should be sent in the subsequent request to ListReplicators.

    • Replicators (list) --

      List containing information of each of the replicators in the account.

      • (dict) --

        Information about a replicator.

        • CreationTime (datetime) --

          The time the replicator was created.

        • CurrentVersion (string) --

          The current version of the replicator.

        • IsReplicatorReference (boolean) --

          Whether this resource is a replicator reference.

        • KafkaClustersSummary (list) --

          Kafka Clusters used in setting up sources / targets for replication.

          • (dict) --

            Summarized information about Kafka Cluster used as source / target for replication.

            • AmazonMskCluster (dict) --

              Details of an Amazon MSK Cluster.

              • MskClusterArn (string) --

                The Amazon Resource Name (ARN) of an Amazon MSK cluster.

            • ApacheKafkaCluster (dict) --

              Details of an Apache Kafka Cluster.

              • ApacheKafkaClusterId (string) --

                The ID of the Apache Kafka cluster.

              • BootstrapBrokerString (string) --

                The bootstrap broker string of the Apache Kafka cluster.

            • KafkaClusterAlias (string) --

              The alias of the Kafka cluster. Used to prefix names of replicated topics.

        • ReplicationInfoSummaryList (list) --

          A list of summarized information of replications between clusters.

          • (dict) --

            Summarized information of replication between clusters.

            • SourceKafkaClusterAlias (string) --

              The alias of the source Kafka cluster.

            • TargetKafkaClusterAlias (string) --

              The alias of the target Kafka cluster.

        • ReplicatorArn (string) --

          The Amazon Resource Name (ARN) of the replicator.

        • ReplicatorName (string) --

          The name of the replicator.

        • ReplicatorResourceArn (string) --

          The Amazon Resource Name (ARN) of the replicator resource in the region where the replicator was created.

        • ReplicatorState (string) --

          State of the replicator.

UpdateReplicationInfo (updated) Link ¶
Changes (request)
{'LogDelivery': {'ReplicatorLogDelivery': {'CloudWatchLogs': {'Enabled': 'boolean',
                                                              'LogGroup': 'string'},
                                           'Firehose': {'DeliveryStream': 'string',
                                                        'Enabled': 'boolean'},
                                           'S3': {'Bucket': 'string',
                                                  'Enabled': 'boolean',
                                                  'Prefix': 'string'}}},
 'SourceKafkaClusterId': 'string',
 'TargetKafkaClusterId': 'string'}

Updates replication info of a replicator.

See also: AWS API Documentation

Request Syntax

client.update_replication_info(
    ConsumerGroupReplication={
        'ConsumerGroupsToExclude': [
            'string',
        ],
        'ConsumerGroupsToReplicate': [
            'string',
        ],
        'DetectAndCopyNewConsumerGroups': True|False,
        'SynchroniseConsumerGroupOffsets': True|False
    },
    CurrentVersion='string',
    ReplicatorArn='string',
    SourceKafkaClusterArn='string',
    SourceKafkaClusterId='string',
    TargetKafkaClusterArn='string',
    TargetKafkaClusterId='string',
    TopicReplication={
        'CopyAccessControlListsForTopics': True|False,
        'CopyTopicConfigurations': True|False,
        'DetectAndCopyNewTopics': True|False,
        'TopicsToExclude': [
            'string',
        ],
        'TopicsToReplicate': [
            'string',
        ]
    },
    LogDelivery={
        'ReplicatorLogDelivery': {
            'CloudWatchLogs': {
                'Enabled': True|False,
                'LogGroup': 'string'
            },
            'Firehose': {
                'Enabled': True|False,
                'DeliveryStream': 'string'
            },
            'S3': {
                'Enabled': True|False,
                'Bucket': 'string',
                'Prefix': 'string'
            }
        }
    }
)
type ConsumerGroupReplication:

dict

param ConsumerGroupReplication:

Updated consumer group replication information.

  • ConsumerGroupsToExclude (list) -- [REQUIRED]

    List of regular expression patterns indicating the consumer groups that should not be replicated.

    • (string) --

  • ConsumerGroupsToReplicate (list) -- [REQUIRED]

    List of regular expression patterns indicating the consumer groups to copy.

    • (string) --

  • DetectAndCopyNewConsumerGroups (boolean) -- [REQUIRED]

    Enables synchronization of consumer groups to target cluster.

  • SynchroniseConsumerGroupOffsets (boolean) -- [REQUIRED]

    Enables synchronization of consumer group offsets to target cluster. The translated offsets will be written to topic __consumer_offsets.

type CurrentVersion:

string

param CurrentVersion:

[REQUIRED]

Current replicator version.

type ReplicatorArn:

string

param ReplicatorArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the replicator to be updated.

type SourceKafkaClusterArn:

string

param SourceKafkaClusterArn:

The ARN of the source Kafka cluster.

type SourceKafkaClusterId:

string

param SourceKafkaClusterId:

The ID of the source Kafka cluster.

type TargetKafkaClusterArn:

string

param TargetKafkaClusterArn:

The ARN of the target Kafka cluster.

type TargetKafkaClusterId:

string

param TargetKafkaClusterId:

The ID of the target Kafka cluster.

type TopicReplication:

dict

param TopicReplication:

Updated topic replication information.

  • CopyAccessControlListsForTopics (boolean) -- [REQUIRED]

    Whether to periodically configure remote topic ACLs to match their corresponding upstream topics.

  • CopyTopicConfigurations (boolean) -- [REQUIRED]

    Whether to periodically configure remote topics to match their corresponding upstream topics.

  • DetectAndCopyNewTopics (boolean) -- [REQUIRED]

    Whether to periodically check for new topics and partitions.

  • TopicsToExclude (list) -- [REQUIRED]

    List of regular expression patterns indicating the topics that should not be replicated.

    • (string) --

  • TopicsToReplicate (list) -- [REQUIRED]

    List of regular expression patterns indicating the topics to copy.

    • (string) --

type LogDelivery:

dict

param LogDelivery:

Configuration for delivering replicator logs to customer destinations.

  • ReplicatorLogDelivery (dict) --

    Configuration for replicator log delivery.

    • CloudWatchLogs (dict) --

      Configuration for CloudWatch Logs delivery.

      • Enabled (boolean) -- [REQUIRED]

        Whether log delivery to CloudWatch Logs is enabled.

      • LogGroup (string) --

        The CloudWatch log group that is the destination for log delivery.

    • Firehose (dict) --

      Configuration for Firehose delivery.

      • Enabled (boolean) -- [REQUIRED]

        Whether log delivery to Firehose is enabled.

      • DeliveryStream (string) --

        The Firehose delivery stream that is the destination for log delivery.

    • S3 (dict) --

      Configuration for S3 delivery.

      • Enabled (boolean) -- [REQUIRED]

        Whether log delivery to S3 is enabled.

      • Bucket (string) --

        The S3 bucket that is the destination for log delivery.

      • Prefix (string) --

        The S3 prefix that is the destination for log delivery.

rtype:

dict

returns:

Response Syntax

{
    'ReplicatorArn': 'string',
    'ReplicatorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED'
}

Response Structure

  • (dict) -- HTTP Status Code 200: OK.

    • ReplicatorArn (string) --

      The Amazon Resource Name (ARN) of the replicator.

    • ReplicatorState (string) --

      State of the replicator.