2025/12/17 - Managed Streaming for Kafka Connect - 3 updated api methods
Changes Support dual-stack network connectivity for connectors via NetworkType field.
{'networkType': 'IPV4 | DUAL'}
Creates a connector using the specified properties.
See also: AWS API Documentation
Request Syntax
client.create_connector(
capacity={
'autoScaling': {
'maxWorkerCount': 123,
'mcuCount': 123,
'minWorkerCount': 123,
'scaleInPolicy': {
'cpuUtilizationPercentage': 123
},
'scaleOutPolicy': {
'cpuUtilizationPercentage': 123
}
},
'provisionedCapacity': {
'mcuCount': 123,
'workerCount': 123
}
},
connectorConfiguration={
'string': 'string'
},
connectorDescription='string',
connectorName='string',
kafkaCluster={
'apacheKafkaCluster': {
'bootstrapServers': 'string',
'vpc': {
'securityGroups': [
'string',
],
'subnets': [
'string',
]
}
}
},
kafkaClusterClientAuthentication={
'authenticationType': 'NONE'|'IAM'
},
kafkaClusterEncryptionInTransit={
'encryptionType': 'PLAINTEXT'|'TLS'
},
kafkaConnectVersion='string',
logDelivery={
'workerLogDelivery': {
'cloudWatchLogs': {
'enabled': True|False,
'logGroup': 'string'
},
'firehose': {
'deliveryStream': 'string',
'enabled': True|False
},
's3': {
'bucket': 'string',
'enabled': True|False,
'prefix': 'string'
}
}
},
networkType='IPV4'|'DUAL',
plugins=[
{
'customPlugin': {
'customPluginArn': 'string',
'revision': 123
}
},
],
serviceExecutionRoleArn='string',
workerConfiguration={
'revision': 123,
'workerConfigurationArn': 'string'
},
tags={
'string': 'string'
}
)
dict
[REQUIRED]
Information about the capacity allocated to the connector. Exactly one of the two properties must be specified.
autoScaling (dict) --
Information about the auto scaling parameters for the connector.
maxWorkerCount (integer) -- [REQUIRED]
The maximum number of workers allocated to the connector.
mcuCount (integer) -- [REQUIRED]
The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
minWorkerCount (integer) -- [REQUIRED]
The minimum number of workers allocated to the connector.
scaleInPolicy (dict) --
The sacle-in policy for the connector.
cpuUtilizationPercentage (integer) -- [REQUIRED]
Specifies the CPU utilization percentage threshold at which you want connector scale in to be triggered.
scaleOutPolicy (dict) --
The sacle-out policy for the connector.
cpuUtilizationPercentage (integer) -- [REQUIRED]
The CPU utilization percentage threshold at which you want connector scale out to be triggered.
provisionedCapacity (dict) --
Details about a fixed capacity allocated to a connector.
mcuCount (integer) -- [REQUIRED]
The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
workerCount (integer) -- [REQUIRED]
The number of workers that are allocated to the connector.
dict
[REQUIRED]
A map of keys to values that represent the configuration for the connector.
(string) --
(string) --
string
A summary description of the connector.
string
[REQUIRED]
The name of the connector.
dict
[REQUIRED]
Specifies which Apache Kafka cluster to connect to.
apacheKafkaCluster (dict) -- [REQUIRED]
The Apache Kafka cluster to which the connector is connected.
bootstrapServers (string) -- [REQUIRED]
The bootstrap servers of the cluster.
vpc (dict) -- [REQUIRED]
Details of an Amazon VPC which has network connectivity to the Apache Kafka cluster.
securityGroups (list) --
The security groups for the connector.
(string) --
subnets (list) -- [REQUIRED]
The subnets for the connector.
(string) --
dict
[REQUIRED]
Details of the client authentication used by the Apache Kafka cluster.
authenticationType (string) -- [REQUIRED]
The type of client authentication used to connect to the Apache Kafka cluster. Value NONE means that no client authentication is used.
dict
[REQUIRED]
Details of encryption in transit to the Apache Kafka cluster.
encryptionType (string) -- [REQUIRED]
The type of encryption in transit to the Apache Kafka cluster.
string
[REQUIRED]
The version of Kafka Connect. It has to be compatible with both the Apache Kafka cluster's version and the plugins.
dict
Details about log delivery.
workerLogDelivery (dict) -- [REQUIRED]
The workers can send worker logs to different destination types. This configuration specifies the details of these destinations.
cloudWatchLogs (dict) --
Details about delivering logs to Amazon CloudWatch Logs.
enabled (boolean) -- [REQUIRED]
Whether log delivery to Amazon CloudWatch Logs is enabled.
logGroup (string) --
The name of the CloudWatch log group that is the destination for log delivery.
firehose (dict) --
Details about delivering logs to Amazon Kinesis Data Firehose.
deliveryStream (string) --
The name of the Kinesis Data Firehose delivery stream that is the destination for log delivery.
enabled (boolean) -- [REQUIRED]
Specifies whether connector logs get delivered to Amazon Kinesis Data Firehose.
s3 (dict) --
Details about delivering logs to Amazon S3.
bucket (string) --
The name of the S3 bucket that is the destination for log delivery.
enabled (boolean) -- [REQUIRED]
Specifies whether connector logs get sent to the specified Amazon S3 destination.
prefix (string) --
The S3 prefix that is the destination for log delivery.
string
The network type of the connector. It gives connectors connectivity to either IPv4 (IPV4) or IPv4 and IPv6 (DUAL) destinations. Defaults to IPV4.
list
[REQUIRED]
Specifies which plugin to use for the connector. You must specify a single-element list containing one customPlugin object.
(dict) --
A plugin is an Amazon Web Services resource that contains the code that defines your connector logic.
customPlugin (dict) -- [REQUIRED]
Details about a custom plugin.
customPluginArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the custom plugin.
revision (integer) -- [REQUIRED]
The revision of the custom plugin.
string
[REQUIRED]
The Amazon Resource Name (ARN) of the IAM role used by the connector to access the Amazon Web Services resources that it needs. The types of resources depends on the logic of the connector. For example, a connector that has Amazon S3 as a destination must have permissions that allow it to write to the S3 destination bucket.
dict
Specifies which worker configuration to use with the connector.
revision (integer) -- [REQUIRED]
The revision of the worker configuration.
workerConfigurationArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the worker configuration.
dict
The tags you want to attach to the connector.
(string) --
(string) --
dict
Response Syntax
{
'connectorArn': 'string',
'connectorName': 'string',
'connectorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED'
}
Response Structure
(dict) --
connectorArn (string) --
The Amazon Resource Name (ARN) that Amazon assigned to the connector.
connectorName (string) --
The name of the connector.
connectorState (string) --
The state of the connector.
{'networkType': 'IPV4 | DUAL'}
Returns summary information about the connector.
See also: AWS API Documentation
Request Syntax
client.describe_connector(
connectorArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) of the connector that you want to describe.
dict
Response Syntax
{
'capacity': {
'autoScaling': {
'maxWorkerCount': 123,
'mcuCount': 123,
'minWorkerCount': 123,
'scaleInPolicy': {
'cpuUtilizationPercentage': 123
},
'scaleOutPolicy': {
'cpuUtilizationPercentage': 123
}
},
'provisionedCapacity': {
'mcuCount': 123,
'workerCount': 123
}
},
'connectorArn': 'string',
'connectorConfiguration': {
'string': 'string'
},
'connectorDescription': 'string',
'connectorName': 'string',
'connectorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED',
'creationTime': datetime(2015, 1, 1),
'currentVersion': 'string',
'kafkaCluster': {
'apacheKafkaCluster': {
'bootstrapServers': 'string',
'vpc': {
'securityGroups': [
'string',
],
'subnets': [
'string',
]
}
}
},
'kafkaClusterClientAuthentication': {
'authenticationType': 'NONE'|'IAM'
},
'kafkaClusterEncryptionInTransit': {
'encryptionType': 'PLAINTEXT'|'TLS'
},
'kafkaConnectVersion': 'string',
'logDelivery': {
'workerLogDelivery': {
'cloudWatchLogs': {
'enabled': True|False,
'logGroup': 'string'
},
'firehose': {
'deliveryStream': 'string',
'enabled': True|False
},
's3': {
'bucket': 'string',
'enabled': True|False,
'prefix': 'string'
}
}
},
'networkType': 'IPV4'|'DUAL',
'plugins': [
{
'customPlugin': {
'customPluginArn': 'string',
'revision': 123
}
},
],
'serviceExecutionRoleArn': 'string',
'workerConfiguration': {
'revision': 123,
'workerConfigurationArn': 'string'
},
'stateDescription': {
'code': 'string',
'message': 'string'
}
}
Response Structure
(dict) --
capacity (dict) --
Information about the capacity of the connector, whether it is auto scaled or provisioned.
autoScaling (dict) --
Describes the connector's auto scaling capacity.
maxWorkerCount (integer) --
The maximum number of workers allocated to the connector.
mcuCount (integer) --
The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
minWorkerCount (integer) --
The minimum number of workers allocated to the connector.
scaleInPolicy (dict) --
The sacle-in policy for the connector.
cpuUtilizationPercentage (integer) --
Specifies the CPU utilization percentage threshold at which you want connector scale in to be triggered.
scaleOutPolicy (dict) --
The sacle-out policy for the connector.>
cpuUtilizationPercentage (integer) --
The CPU utilization percentage threshold at which you want connector scale out to be triggered.
provisionedCapacity (dict) --
Describes a connector's provisioned capacity.
mcuCount (integer) --
The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
workerCount (integer) --
The number of workers that are allocated to the connector.
connectorArn (string) --
The Amazon Resource Name (ARN) of the connector.
connectorConfiguration (dict) --
A map of keys to values that represent the configuration for the connector.
(string) --
(string) --
connectorDescription (string) --
A summary description of the connector.
connectorName (string) --
The name of the connector.
connectorState (string) --
The state of the connector.
creationTime (datetime) --
The time the connector was created.
currentVersion (string) --
The current version of the connector.
kafkaCluster (dict) --
The Apache Kafka cluster that the connector is connected to.
apacheKafkaCluster (dict) --
The Apache Kafka cluster to which the connector is connected.
bootstrapServers (string) --
The bootstrap servers of the cluster.
vpc (dict) --
Details of an Amazon VPC which has network connectivity to the Apache Kafka cluster.
securityGroups (list) --
The security groups for the connector.
(string) --
subnets (list) --
The subnets for the connector.
(string) --
kafkaClusterClientAuthentication (dict) --
The type of client authentication used to connect to the Apache Kafka cluster. The value is NONE when no client authentication is used.
authenticationType (string) --
The type of client authentication used to connect to the Apache Kafka cluster. Value NONE means that no client authentication is used.
kafkaClusterEncryptionInTransit (dict) --
Details of encryption in transit to the Apache Kafka cluster.
encryptionType (string) --
The type of encryption in transit to the Apache Kafka cluster.
kafkaConnectVersion (string) --
The version of Kafka Connect. It has to be compatible with both the Apache Kafka cluster's version and the plugins.
logDelivery (dict) --
Details about delivering logs to Amazon CloudWatch Logs.
workerLogDelivery (dict) --
The workers can send worker logs to different destination types. This configuration specifies the details of these destinations.
cloudWatchLogs (dict) --
Details about delivering logs to Amazon CloudWatch Logs.
enabled (boolean) --
Whether log delivery to Amazon CloudWatch Logs is enabled.
logGroup (string) --
The name of the CloudWatch log group that is the destination for log delivery.
firehose (dict) --
Details about delivering logs to Amazon Kinesis Data Firehose.
deliveryStream (string) --
The name of the Kinesis Data Firehose delivery stream that is the destination for log delivery.
enabled (boolean) --
Specifies whether connector logs get delivered to Amazon Kinesis Data Firehose.
s3 (dict) --
Details about delivering logs to Amazon S3.
bucket (string) --
The name of the S3 bucket that is the destination for log delivery.
enabled (boolean) --
Specifies whether connector logs get sent to the specified Amazon S3 destination.
prefix (string) --
The S3 prefix that is the destination for log delivery.
networkType (string) --
The network type of the connector. It gives connectors connectivity to either IPv4 (IPV4) or IPv4 and IPv6 (DUAL) destinations. Defaults to IPV4.
plugins (list) --
Specifies which plugins were used for this connector.
(dict) --
The description of the plugin.
customPlugin (dict) --
Details about a custom plugin.
customPluginArn (string) --
The Amazon Resource Name (ARN) of the custom plugin.
revision (integer) --
The revision of the custom plugin.
serviceExecutionRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role used by the connector to access Amazon Web Services resources.
workerConfiguration (dict) --
Specifies which worker configuration was used for the connector.
revision (integer) --
The revision of the worker configuration.
workerConfigurationArn (string) --
The Amazon Resource Name (ARN) of the worker configuration.
stateDescription (dict) --
Details about the state of a connector.
code (string) --
A code that describes the state of a resource.
message (string) --
A message that describes the state of a resource.
{'connectors': {'networkType': 'IPV4 | DUAL'}}
Returns a list of all the connectors in this account and Region. The list is limited to connectors whose name starts with the specified prefix. The response also includes a description of each of the listed connectors.
See also: AWS API Documentation
Request Syntax
client.list_connectors(
connectorNamePrefix='string',
maxResults=123,
nextToken='string'
)
string
The name prefix that you want to use to search for and list connectors.
integer
The maximum number of connectors to list in one response.
string
If the response of a ListConnectors operation is truncated, it will include a NextToken. Send this NextToken in a subsequent request to continue listing from where the previous operation left off.
dict
Response Syntax
{
'connectors': [
{
'capacity': {
'autoScaling': {
'maxWorkerCount': 123,
'mcuCount': 123,
'minWorkerCount': 123,
'scaleInPolicy': {
'cpuUtilizationPercentage': 123
},
'scaleOutPolicy': {
'cpuUtilizationPercentage': 123
}
},
'provisionedCapacity': {
'mcuCount': 123,
'workerCount': 123
}
},
'connectorArn': 'string',
'connectorDescription': 'string',
'connectorName': 'string',
'connectorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED',
'creationTime': datetime(2015, 1, 1),
'currentVersion': 'string',
'kafkaCluster': {
'apacheKafkaCluster': {
'bootstrapServers': 'string',
'vpc': {
'securityGroups': [
'string',
],
'subnets': [
'string',
]
}
}
},
'kafkaClusterClientAuthentication': {
'authenticationType': 'NONE'|'IAM'
},
'kafkaClusterEncryptionInTransit': {
'encryptionType': 'PLAINTEXT'|'TLS'
},
'kafkaConnectVersion': 'string',
'logDelivery': {
'workerLogDelivery': {
'cloudWatchLogs': {
'enabled': True|False,
'logGroup': 'string'
},
'firehose': {
'deliveryStream': 'string',
'enabled': True|False
},
's3': {
'bucket': 'string',
'enabled': True|False,
'prefix': 'string'
}
}
},
'networkType': 'IPV4'|'DUAL',
'plugins': [
{
'customPlugin': {
'customPluginArn': 'string',
'revision': 123
}
},
],
'serviceExecutionRoleArn': 'string',
'workerConfiguration': {
'revision': 123,
'workerConfigurationArn': 'string'
}
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
connectors (list) --
An array of connector descriptions.
(dict) --
Summary of a connector.
capacity (dict) --
The connector's compute capacity settings.
autoScaling (dict) --
Describes the connector's auto scaling capacity.
maxWorkerCount (integer) --
The maximum number of workers allocated to the connector.
mcuCount (integer) --
The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
minWorkerCount (integer) --
The minimum number of workers allocated to the connector.
scaleInPolicy (dict) --
The sacle-in policy for the connector.
cpuUtilizationPercentage (integer) --
Specifies the CPU utilization percentage threshold at which you want connector scale in to be triggered.
scaleOutPolicy (dict) --
The sacle-out policy for the connector.>
cpuUtilizationPercentage (integer) --
The CPU utilization percentage threshold at which you want connector scale out to be triggered.
provisionedCapacity (dict) --
Describes a connector's provisioned capacity.
mcuCount (integer) --
The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
workerCount (integer) --
The number of workers that are allocated to the connector.
connectorArn (string) --
The Amazon Resource Name (ARN) of the connector.
connectorDescription (string) --
The description of the connector.
connectorName (string) --
The name of the connector.
connectorState (string) --
The state of the connector.
creationTime (datetime) --
The time that the connector was created.
currentVersion (string) --
The current version of the connector.
kafkaCluster (dict) --
The details of the Apache Kafka cluster to which the connector is connected.
apacheKafkaCluster (dict) --
The Apache Kafka cluster to which the connector is connected.
bootstrapServers (string) --
The bootstrap servers of the cluster.
vpc (dict) --
Details of an Amazon VPC which has network connectivity to the Apache Kafka cluster.
securityGroups (list) --
The security groups for the connector.
(string) --
subnets (list) --
The subnets for the connector.
(string) --
kafkaClusterClientAuthentication (dict) --
The type of client authentication used to connect to the Apache Kafka cluster. The value is NONE when no client authentication is used.
authenticationType (string) --
The type of client authentication used to connect to the Apache Kafka cluster. Value NONE means that no client authentication is used.
kafkaClusterEncryptionInTransit (dict) --
Details of encryption in transit to the Apache Kafka cluster.
encryptionType (string) --
The type of encryption in transit to the Apache Kafka cluster.
kafkaConnectVersion (string) --
The version of Kafka Connect. It has to be compatible with both the Apache Kafka cluster's version and the plugins.
logDelivery (dict) --
The settings for delivering connector logs to Amazon CloudWatch Logs.
workerLogDelivery (dict) --
The workers can send worker logs to different destination types. This configuration specifies the details of these destinations.
cloudWatchLogs (dict) --
Details about delivering logs to Amazon CloudWatch Logs.
enabled (boolean) --
Whether log delivery to Amazon CloudWatch Logs is enabled.
logGroup (string) --
The name of the CloudWatch log group that is the destination for log delivery.
firehose (dict) --
Details about delivering logs to Amazon Kinesis Data Firehose.
deliveryStream (string) --
The name of the Kinesis Data Firehose delivery stream that is the destination for log delivery.
enabled (boolean) --
Specifies whether connector logs get delivered to Amazon Kinesis Data Firehose.
s3 (dict) --
Details about delivering logs to Amazon S3.
bucket (string) --
The name of the S3 bucket that is the destination for log delivery.
enabled (boolean) --
Specifies whether connector logs get sent to the specified Amazon S3 destination.
prefix (string) --
The S3 prefix that is the destination for log delivery.
networkType (string) --
The network type of the connector. It gives connectors connectivity to either IPv4 (IPV4) or IPv4 and IPv6 (DUAL) destinations. Defaults to IPV4.
plugins (list) --
Specifies which plugins were used for this connector.
(dict) --
The description of the plugin.
customPlugin (dict) --
Details about a custom plugin.
customPluginArn (string) --
The Amazon Resource Name (ARN) of the custom plugin.
revision (integer) --
The revision of the custom plugin.
serviceExecutionRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role used by the connector to access Amazon Web Services resources.
workerConfiguration (dict) --
The worker configurations that are in use with the connector.
revision (integer) --
The revision of the worker configuration.
workerConfigurationArn (string) --
The Amazon Resource Name (ARN) of the worker configuration.
nextToken (string) --
If the response of a ListConnectors operation is truncated, it will include a NextToken. Send this NextToken in a subsequent request to continue listing from where it left off.