AWS API Changes

2024/02/26 - Managed Streaming for Kafka Connect - 4 new6 updated api methods

Changes Adds support for tagging, with new TagResource, UntagResource and ListTagsForResource APIs to manage tags and updates to existing APIs to allow tag on create. This release also adds support for the new DeleteWorkerConfiguration API.

TagResource (new)

Link ¶

Attaches tags to the specified resource.

See also: AWS API Documentation

Request Syntax

client.tag_resource(
    resourceArn='string',
    tags={
        'string': 'string'
    }
)

type resourceArn:

string

param resourceArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the resource to which you want to attach tags.

type tags:

dict

param tags:

[REQUIRED]

The tags that you want to attach to the resource.

(string) --
- (string) --

rtype:

dict

returns:

Response Syntax

{}

Response Structure

(dict) --

DeleteWorkerConfiguration (new)

Link ¶

Deletes the specified worker configuration.

See also: AWS API Documentation

Request Syntax

client.delete_worker_configuration(
    workerConfigurationArn='string'
)

type workerConfigurationArn:

string

param workerConfigurationArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the worker configuration that you want to delete.

rtype:

dict

returns:

Response Syntax

{
    'workerConfigurationArn': 'string',
    'workerConfigurationState': 'ACTIVE'|'DELETING'
}

Response Structure

(dict) --
- workerConfigurationArn (string) --
  
  The Amazon Resource Name (ARN) of the worker configuration that you requested to delete.
- workerConfigurationState (string) --
  
  The state of the worker configuration.

ListTagsForResource (new)

Link ¶

Lists all the tags attached to the specified resource.

See also: AWS API Documentation

Request Syntax

client.list_tags_for_resource(
    resourceArn='string'
)

type resourceArn:

string

param resourceArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the resource for which you want to list all attached tags.

rtype:

dict

returns:

Response Syntax

{
    'tags': {
        'string': 'string'
    }
}

Response Structure

(dict) --
- tags (dict) --
  
  Lists the tags attached to the specified resource in the corresponding request.
  - (string) --
    - (string) --

UntagResource (new)

Link ¶

Removes tags from the specified resource.

See also: AWS API Documentation

Request Syntax

client.untag_resource(
    resourceArn='string',
    tagKeys=[
        'string',
    ]
)

type resourceArn:

string

param resourceArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the resource from which you want to remove tags.

type tagKeys:

list

param tagKeys:

[REQUIRED]

The keys of the tags that you want to remove from the resource.

(string) --

rtype:

dict

returns:

Response Syntax

{}

Response Structure

(dict) --

CreateConnector (updated)

Link ¶
Changes (request)

{'tags': {'string': 'string'}}

Creates a connector using the specified properties.

See also: AWS API Documentation

Request Syntax

client.create_connector(
    capacity={
        'autoScaling': {
            'maxWorkerCount': 123,
            'mcuCount': 123,
            'minWorkerCount': 123,
            'scaleInPolicy': {
                'cpuUtilizationPercentage': 123
            },
            'scaleOutPolicy': {
                'cpuUtilizationPercentage': 123
            }
        },
        'provisionedCapacity': {
            'mcuCount': 123,
            'workerCount': 123
        }
    },
    connectorConfiguration={
        'string': 'string'
    },
    connectorDescription='string',
    connectorName='string',
    kafkaCluster={
        'apacheKafkaCluster': {
            'bootstrapServers': 'string',
            'vpc': {
                'securityGroups': [
                    'string',
                ],
                'subnets': [
                    'string',
                ]
            }
        }
    },
    kafkaClusterClientAuthentication={
        'authenticationType': 'NONE'|'IAM'
    },
    kafkaClusterEncryptionInTransit={
        'encryptionType': 'PLAINTEXT'|'TLS'
    },
    kafkaConnectVersion='string',
    logDelivery={
        'workerLogDelivery': {
            'cloudWatchLogs': {
                'enabled': True|False,
                'logGroup': 'string'
            },
            'firehose': {
                'deliveryStream': 'string',
                'enabled': True|False
            },
            's3': {
                'bucket': 'string',
                'enabled': True|False,
                'prefix': 'string'
            }
        }
    },
    plugins=[
        {
            'customPlugin': {
                'customPluginArn': 'string',
                'revision': 123
            }
        },
    ],
    serviceExecutionRoleArn='string',
    tags={
        'string': 'string'
    },
    workerConfiguration={
        'revision': 123,
        'workerConfigurationArn': 'string'
    }
)

type capacity:

dict

param capacity:

[REQUIRED]

Information about the capacity allocated to the connector. Exactly one of the two properties must be specified.

autoScaling (dict) --

Information about the auto scaling parameters for the connector.
- maxWorkerCount (integer) -- [REQUIRED]
  
  The maximum number of workers allocated to the connector.
- mcuCount (integer) -- [REQUIRED]
  
  The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
- minWorkerCount (integer) -- [REQUIRED]
  
  The minimum number of workers allocated to the connector.
- scaleInPolicy (dict) --
  
  The sacle-in policy for the connector.
  - cpuUtilizationPercentage (integer) -- [REQUIRED]
    
    Specifies the CPU utilization percentage threshold at which you want connector scale in to be triggered.
- scaleOutPolicy (dict) --
  
  The sacle-out policy for the connector.
  - cpuUtilizationPercentage (integer) -- [REQUIRED]
    
    The CPU utilization percentage threshold at which you want connector scale out to be triggered.
provisionedCapacity (dict) --

Details about a fixed capacity allocated to a connector.
- mcuCount (integer) -- [REQUIRED]
  
  The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
- workerCount (integer) -- [REQUIRED]
  
  The number of workers that are allocated to the connector.

type connectorConfiguration:

dict

param connectorConfiguration:

[REQUIRED]

A map of keys to values that represent the configuration for the connector.

(string) --
- (string) --

type connectorDescription:

string

param connectorDescription:

A summary description of the connector.

type connectorName:

string

param connectorName:

[REQUIRED]

The name of the connector.

type kafkaCluster:

dict

param kafkaCluster:

[REQUIRED]

Specifies which Apache Kafka cluster to connect to.

apacheKafkaCluster (dict) -- [REQUIRED]

The Apache Kafka cluster to which the connector is connected.
- bootstrapServers (string) -- [REQUIRED]
  
  The bootstrap servers of the cluster.
- vpc (dict) -- [REQUIRED]
  
  Details of an Amazon VPC which has network connectivity to the Apache Kafka cluster.
  - securityGroups (list) --
    
    The security groups for the connector.
    - (string) --
  - subnets (list) -- [REQUIRED]
    
    The subnets for the connector.
    - (string) --

type kafkaClusterClientAuthentication:

dict

param kafkaClusterClientAuthentication:

[REQUIRED]

Details of the client authentication used by the Apache Kafka cluster.

authenticationType (string) -- [REQUIRED]

The type of client authentication used to connect to the Apache Kafka cluster. Value NONE means that no client authentication is used.

type kafkaClusterEncryptionInTransit:

dict

param kafkaClusterEncryptionInTransit:

[REQUIRED]

Details of encryption in transit to the Apache Kafka cluster.

encryptionType (string) -- [REQUIRED]

The type of encryption in transit to the Apache Kafka cluster.

type kafkaConnectVersion:

string

param kafkaConnectVersion:

[REQUIRED]

The version of Kafka Connect. It has to be compatible with both the Apache Kafka cluster's version and the plugins.

type logDelivery:

dict

param logDelivery:

Details about log delivery.

workerLogDelivery (dict) -- [REQUIRED]

The workers can send worker logs to different destination types. This configuration specifies the details of these destinations.
- cloudWatchLogs (dict) --
  
  Details about delivering logs to Amazon CloudWatch Logs.
  - enabled (boolean) -- [REQUIRED]
    
    Whether log delivery to Amazon CloudWatch Logs is enabled.
  - logGroup (string) --
    
    The name of the CloudWatch log group that is the destination for log delivery.
- firehose (dict) --
  
  Details about delivering logs to Amazon Kinesis Data Firehose.
  - deliveryStream (string) --
    
    The name of the Kinesis Data Firehose delivery stream that is the destination for log delivery.
  - enabled (boolean) -- [REQUIRED]
    
    Specifies whether connector logs get delivered to Amazon Kinesis Data Firehose.
- s3 (dict) --
  
  Details about delivering logs to Amazon S3.
  - bucket (string) --
    
    The name of the S3 bucket that is the destination for log delivery.
  - enabled (boolean) -- [REQUIRED]
    
    Specifies whether connector logs get sent to the specified Amazon S3 destination.
  - prefix (string) --
    
    The S3 prefix that is the destination for log delivery.

type plugins:

list

param plugins:

[REQUIRED]

Specifies which plugin to use for the connector. You must specify a single-element list containing one customPlugin object.

(dict) --

A plugin is an Amazon Web Services resource that contains the code that defines your connector logic.
- customPlugin (dict) -- [REQUIRED]
  
  Details about a custom plugin.
  - customPluginArn (string) -- [REQUIRED]
    
    The Amazon Resource Name (ARN) of the custom plugin.
  - revision (integer) -- [REQUIRED]
    
    The revision of the custom plugin.

type serviceExecutionRoleArn:

string

param serviceExecutionRoleArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the IAM role used by the connector to access the Amazon Web Services resources that it needs. The types of resources depends on the logic of the connector. For example, a connector that has Amazon S3 as a destination must have permissions that allow it to write to the S3 destination bucket.

type tags:

dict

param tags:

The tags you want to attach to the connector.

(string) --
- (string) --

type workerConfiguration:

dict

param workerConfiguration:

Specifies which worker configuration to use with the connector.

revision (integer) -- [REQUIRED]

The revision of the worker configuration.
workerConfigurationArn (string) -- [REQUIRED]

The Amazon Resource Name (ARN) of the worker configuration.

rtype:

dict

returns:

Response Syntax

{
    'connectorArn': 'string',
    'connectorName': 'string',
    'connectorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED'
}

Response Structure

(dict) --
- connectorArn (string) --
  
  The Amazon Resource Name (ARN) that Amazon assigned to the connector.
- connectorName (string) --
  
  The name of the connector.
- connectorState (string) --
  
  The state of the connector.

CreateCustomPlugin (updated)

Link ¶
Changes (request)

{'tags': {'string': 'string'}}

Creates a custom plugin using the specified properties.

See also: AWS API Documentation

Request Syntax

client.create_custom_plugin(
    contentType='JAR'|'ZIP',
    description='string',
    location={
        's3Location': {
            'bucketArn': 'string',
            'fileKey': 'string',
            'objectVersion': 'string'
        }
    },
    name='string',
    tags={
        'string': 'string'
    }
)

type contentType:

string

param contentType:

[REQUIRED]

The type of the plugin file.

type description:

string

param description:

A summary description of the custom plugin.

type location:

dict

param location:

[REQUIRED]

Information about the location of a custom plugin.

s3Location (dict) -- [REQUIRED]

The S3 bucket Amazon Resource Name (ARN), file key, and object version of the plugin file stored in Amazon S3.
- bucketArn (string) -- [REQUIRED]
  
  The Amazon Resource Name (ARN) of an S3 bucket.
- fileKey (string) -- [REQUIRED]
  
  The file key for an object in an S3 bucket.
- objectVersion (string) --
  
  The version of an object in an S3 bucket.

type name:

string

param name:

[REQUIRED]

The name of the custom plugin.

type tags:

dict

param tags:

The tags you want to attach to the custom plugin.

(string) --
- (string) --

rtype:

dict

returns:

Response Syntax

{
    'customPluginArn': 'string',
    'customPluginState': 'CREATING'|'CREATE_FAILED'|'ACTIVE'|'UPDATING'|'UPDATE_FAILED'|'DELETING',
    'name': 'string',
    'revision': 123
}

Response Structure

(dict) --
- customPluginArn (string) --
  
  The Amazon Resource Name (ARN) that Amazon assigned to the custom plugin.
- customPluginState (string) --
  
  The state of the custom plugin.
- name (string) --
  
  The name of the custom plugin.
- revision (integer) --
  
  The revision of the custom plugin.

CreateWorkerConfiguration (updated)

Link ¶
Changes (request, response)
Request

{'tags': {'string': 'string'}}

Response

{'workerConfigurationState': 'ACTIVE | DELETING'}

Creates a worker configuration using the specified properties.

See also: AWS API Documentation

Request Syntax

client.create_worker_configuration(
    description='string',
    name='string',
    propertiesFileContent='string',
    tags={
        'string': 'string'
    }
)

type description:

string

param description:

A summary description of the worker configuration.

type name:

string

param name:

[REQUIRED]

The name of the worker configuration.

type propertiesFileContent:

string

param propertiesFileContent:

[REQUIRED]

Base64 encoded contents of connect-distributed.properties file.

type tags:

dict

param tags:

The tags you want to attach to the worker configuration.

(string) --
- (string) --

rtype:

dict

returns:

Response Syntax

{
    'creationTime': datetime(2015, 1, 1),
    'latestRevision': {
        'creationTime': datetime(2015, 1, 1),
        'description': 'string',
        'revision': 123
    },
    'name': 'string',
    'workerConfigurationArn': 'string',
    'workerConfigurationState': 'ACTIVE'|'DELETING'
}

Response Structure

(dict) --
- creationTime (datetime) --
  
  The time that the worker configuration was created.
- latestRevision (dict) --
  
  The latest revision of the worker configuration.
  - creationTime (datetime) --
    
    The time that a worker configuration revision was created.
  - description (string) --
    
    The description of a worker configuration revision.
  - revision (integer) --
    
    The revision of a worker configuration.
- name (string) --
  
  The name of the worker configuration.
- workerConfigurationArn (string) --
  
  The Amazon Resource Name (ARN) that Amazon assigned to the worker configuration.
- workerConfigurationState (string) --
  
  The state of the worker configuration.

DescribeWorkerConfiguration (updated)

Link ¶
Changes (response)

{'workerConfigurationState': 'ACTIVE | DELETING'}

Returns information about a worker configuration.

See also: AWS API Documentation

Request Syntax

client.describe_worker_configuration(
    workerConfigurationArn='string'
)

type workerConfigurationArn:

string

param workerConfigurationArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the worker configuration that you want to get information about.

rtype:

dict

returns:

Response Syntax

{
    'creationTime': datetime(2015, 1, 1),
    'description': 'string',
    'latestRevision': {
        'creationTime': datetime(2015, 1, 1),
        'description': 'string',
        'propertiesFileContent': 'string',
        'revision': 123
    },
    'name': 'string',
    'workerConfigurationArn': 'string',
    'workerConfigurationState': 'ACTIVE'|'DELETING'
}

Response Structure

(dict) --
- creationTime (datetime) --
  
  The time that the worker configuration was created.
- description (string) --
  
  The description of the worker configuration.
- latestRevision (dict) --
  
  The latest revision of the custom configuration.
  - creationTime (datetime) --
    
    The time that the worker configuration was created.
  - description (string) --
    
    The description of the worker configuration revision.
  - propertiesFileContent (string) --
    
    Base64 encoded contents of the connect-distributed.properties file.
  - revision (integer) --
    
    The description of a revision of the worker configuration.
- name (string) --
  
  The name of the worker configuration.
- workerConfigurationArn (string) --
  
  The Amazon Resource Name (ARN) of the custom configuration.
- workerConfigurationState (string) --
  
  The state of the worker configuration.

ListCustomPlugins (updated)

Link ¶
Changes (request)

{'namePrefix': 'string'}

Returns a list of all of the custom plugins in this account and Region.

See also: AWS API Documentation

Request Syntax

client.list_custom_plugins(
    maxResults=123,
    namePrefix='string',
    nextToken='string'
)

type maxResults:

integer

param maxResults:

The maximum number of custom plugins to list in one response.

type namePrefix:

string

param namePrefix:

Lists custom plugin names that start with the specified text string.

type nextToken:

string

param nextToken:

If the response of a ListCustomPlugins operation is truncated, it will include a NextToken. Send this NextToken in a subsequent request to continue listing from where the previous operation left off.

rtype:

dict

returns:

Response Syntax

{
    'customPlugins': [
        {
            'creationTime': datetime(2015, 1, 1),
            'customPluginArn': 'string',
            'customPluginState': 'CREATING'|'CREATE_FAILED'|'ACTIVE'|'UPDATING'|'UPDATE_FAILED'|'DELETING',
            'description': 'string',
            'latestRevision': {
                'contentType': 'JAR'|'ZIP',
                'creationTime': datetime(2015, 1, 1),
                'description': 'string',
                'fileDescription': {
                    'fileMd5': 'string',
                    'fileSize': 123
                },
                'location': {
                    's3Location': {
                        'bucketArn': 'string',
                        'fileKey': 'string',
                        'objectVersion': 'string'
                    }
                },
                'revision': 123
            },
            'name': 'string'
        },
    ],
    'nextToken': 'string'
}

Response Structure

(dict) --
- customPlugins (list) --
  
  An array of custom plugin descriptions.
  - (dict) --
    
    A summary of the custom plugin.
    - creationTime (datetime) --
      
      The time that the custom plugin was created.
    - customPluginArn (string) --
      
      The Amazon Resource Name (ARN) of the custom plugin.
    - customPluginState (string) --
      
      The state of the custom plugin.
    - description (string) --
      
      A description of the custom plugin.
    - latestRevision (dict) --
      
      The latest revision of the custom plugin.
      - contentType (string) --
        
        The format of the plugin file.
      - creationTime (datetime) --
        
        The time that the custom plugin was created.
      - description (string) --
        
        The description of the custom plugin.
      - fileDescription (dict) --
        
        Details about the custom plugin file.
        
        fileMd5 (string) --
        
        The hex-encoded MD5 checksum of the custom plugin file. You can use it to validate the file.
        
        fileSize (integer) --
        
        The size in bytes of the custom plugin file. You can use it to validate the file.
      - location (dict) --
        
        Information about the location of the custom plugin.
        
        s3Location (dict) --
        
        The S3 bucket Amazon Resource Name (ARN), file key, and object version of the plugin file stored in Amazon S3.
        
        bucketArn (string) --
        
        The Amazon Resource Name (ARN) of an S3 bucket.
        
        fileKey (string) --
        
        The file key for an object in an S3 bucket.
        
        objectVersion (string) --
        
        The version of an object in an S3 bucket.
      - revision (integer) --
        
        The revision of the custom plugin.
    - name (string) --
      
      The name of the custom plugin.
- nextToken (string) --
  
  If the response of a ListCustomPlugins operation is truncated, it will include a NextToken. Send this NextToken in a subsequent request to continue listing from where the previous operation left off.

ListWorkerConfigurations (updated)

Link ¶
Changes (request, response)
Request

{'namePrefix': 'string'}

Response

{'workerConfigurations': {'workerConfigurationState': 'ACTIVE | DELETING'}}

Returns a list of all of the worker configurations in this account and Region.

See also: AWS API Documentation

Request Syntax

client.list_worker_configurations(
    maxResults=123,
    namePrefix='string',
    nextToken='string'
)

type maxResults:

integer

param maxResults:

The maximum number of worker configurations to list in one response.

type namePrefix:

string

param namePrefix:

Lists worker configuration names that start with the specified text string.

type nextToken:

string

param nextToken:

If the response of a ListWorkerConfigurations operation is truncated, it will include a NextToken. Send this NextToken in a subsequent request to continue listing from where the previous operation left off.

rtype:

dict

returns:

Response Syntax

{
    'nextToken': 'string',
    'workerConfigurations': [
        {
            'creationTime': datetime(2015, 1, 1),
            'description': 'string',
            'latestRevision': {
                'creationTime': datetime(2015, 1, 1),
                'description': 'string',
                'revision': 123
            },
            'name': 'string',
            'workerConfigurationArn': 'string',
            'workerConfigurationState': 'ACTIVE'|'DELETING'
        },
    ]
}

Response Structure

(dict) --
- nextToken (string) --
  
  If the response of a ListWorkerConfigurations operation is truncated, it will include a NextToken. Send this NextToken in a subsequent request to continue listing from where the previous operation left off.
- workerConfigurations (list) --
  
  An array of worker configuration descriptions.
  - (dict) --
    
    The summary of a worker configuration.
    - creationTime (datetime) --
      
      The time that a worker configuration was created.
    - description (string) --
      
      The description of a worker configuration.
    - latestRevision (dict) --
      
      The latest revision of a worker configuration.
      - creationTime (datetime) --
        
        The time that a worker configuration revision was created.
      - description (string) --
        
        The description of a worker configuration revision.
      - revision (integer) --
        
        The revision of a worker configuration.
    - name (string) --
      
      The name of the worker configuration.
    - workerConfigurationArn (string) --
      
      The Amazon Resource Name (ARN) of the worker configuration.
    - workerConfigurationState (string) --
      
      The state of the worker configuration.