AWS API Changes

2026/10/28 - Managed Streaming for Kafka Connect - 2 updated api methods

Changes Amazon MSK Connect now supports updating a connector's custom plugin and worker configuration via the UpdateConnector API. MSK Connect checks connector and task health after the update and rolls back on failure. Each call returns an operation ARN to track with DescribeConnectorOperation.

DescribeConnectorOperation (updated)

Link ¶
Changes (response)

{'originWorkerSetting': {'plugins': [{'customPlugin': {'customPluginArn': 'string',
                                                       'revision': 'long'}}],
                         'useDefaultWorkerConfiguration': 'boolean',
                         'workerConfiguration': {'revision': 'long',
                                                 'workerConfigurationArn': 'string'}},
 'targetWorkerSetting': {'plugins': [{'customPlugin': {'customPluginArn': 'string',
                                                       'revision': 'long'}}],
                         'useDefaultWorkerConfiguration': 'boolean',
                         'workerConfiguration': {'revision': 'long',
                                                 'workerConfigurationArn': 'string'}}}

Returns information about the specified connector's operations.

See also: AWS API Documentation

Request Syntax

client.describe_connector_operation(
    connectorOperationArn='string'
)

type connectorOperationArn:

string

param connectorOperationArn:

[REQUIRED]

ARN of the connector operation to be described.

rtype:

dict

returns:

Response Syntax

{
    'connectorArn': 'string',
    'connectorOperationArn': 'string',
    'connectorOperationState': 'PENDING'|'UPDATE_IN_PROGRESS'|'UPDATE_COMPLETE'|'UPDATE_FAILED'|'ROLLBACK_IN_PROGRESS'|'ROLLBACK_FAILED'|'ROLLBACK_COMPLETE'|'PAUSE_COMPLETE'|'PAUSE_FAILED'|'RESUME_COMPLETE'|'RESUME_FAILED'|'PAUSE_IN_PROGRESS'|'RESUME_IN_PROGRESS'|'RESTART_IN_PROGRESS'|'RESTART_COMPLETE'|'RESTART_FAILED',
    'connectorOperationType': 'UPDATE_WORKER_SETTING'|'UPDATE_CONNECTOR_CONFIGURATION'|'ISOLATE_CONNECTOR'|'RESTORE_CONNECTOR'|'RESTART_CONNECTOR'|'PAUSE_CONNECTOR'|'RESUME_CONNECTOR',
    'operationSteps': [
        {
            'stepType': 'INITIALIZE_UPDATE'|'FINALIZE_UPDATE'|'UPDATE_WORKER_SETTING'|'UPDATE_CONNECTOR_CONFIGURATION'|'VALIDATE_UPDATE',
            'stepState': 'PENDING'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'CANCELLED'
        },
    ],
    'originWorkerSetting': {
        'capacity': {
            'autoScaling': {
                'maxWorkerCount': 123,
                'mcuCount': 123,
                'minWorkerCount': 123,
                'scaleInPolicy': {
                    'cpuUtilizationPercentage': 123
                },
                'scaleOutPolicy': {
                    'cpuUtilizationPercentage': 123
                },
                'maxAutoscalingTaskCount': 123
            },
            'provisionedCapacity': {
                'mcuCount': 123,
                'workerCount': 123
            }
        },
        'plugins': [
            {
                'customPlugin': {
                    'customPluginArn': 'string',
                    'revision': 123
                }
            },
        ],
        'workerConfiguration': {
            'revision': 123,
            'workerConfigurationArn': 'string'
        },
        'useDefaultWorkerConfiguration': True|False
    },
    'originConnectorConfiguration': {
        'string': 'string'
    },
    'targetWorkerSetting': {
        'capacity': {
            'autoScaling': {
                'maxWorkerCount': 123,
                'mcuCount': 123,
                'minWorkerCount': 123,
                'scaleInPolicy': {
                    'cpuUtilizationPercentage': 123
                },
                'scaleOutPolicy': {
                    'cpuUtilizationPercentage': 123
                },
                'maxAutoscalingTaskCount': 123
            },
            'provisionedCapacity': {
                'mcuCount': 123,
                'workerCount': 123
            }
        },
        'plugins': [
            {
                'customPlugin': {
                    'customPluginArn': 'string',
                    'revision': 123
                }
            },
        ],
        'workerConfiguration': {
            'revision': 123,
            'workerConfigurationArn': 'string'
        },
        'useDefaultWorkerConfiguration': True|False
    },
    'targetConnectorConfiguration': {
        'string': 'string'
    },
    'errorInfo': {
        'code': 'string',
        'message': 'string'
    },
    'creationTime': datetime(2015, 1, 1),
    'endTime': datetime(2015, 1, 1)
}

Response Structure

(dict) --
- connectorArn (string) --
  
  The Amazon Resource Name (ARN) of the connector.
- connectorOperationArn (string) --
  
  The Amazon Resource Name (ARN) of the connector operation.
- connectorOperationState (string) --
  
  The state of the connector operation.
- connectorOperationType (string) --
  
  The type of connector operation performed.
- operationSteps (list) --
  
  The array of operation steps taken.
  - (dict) --
    
    Details of a step that is involved in a connector's operation.
    - stepType (string) --
      
      The step type of the operation.
    - stepState (string) --
      
      The step state of the operation.
- originWorkerSetting (dict) --
  
  The origin worker setting.
  - capacity (dict) --
    
    Specifies the compute capacity settings of the connector.
    - autoScaling (dict) --
      
      Describes the connector's auto scaling capacity.
      - maxWorkerCount (integer) --
        
        The maximum number of workers allocated to the connector.
      - mcuCount (integer) --
        
        The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
      - minWorkerCount (integer) --
        
        The minimum number of workers allocated to the connector.
      - scaleInPolicy (dict) --
        
        The scale-in policy for the connector.
        
        cpuUtilizationPercentage (integer) --
        
        Specifies the CPU utilization percentage threshold at which you want connector scale in to be triggered.
      - scaleOutPolicy (dict) --
        
        The scale-out policy for the connector.
        
        cpuUtilizationPercentage (integer) --
        
        The CPU utilization percentage threshold at which you want connector scale out to be triggered.
      - maxAutoscalingTaskCount (integer) --
        
        The maximum number of tasks allocated to the connector during autoscaling operations. Must be at least equal to maxWorkerCount.
    - provisionedCapacity (dict) --
      
      Describes a connector's provisioned capacity.
      - mcuCount (integer) --
        
        The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
      - workerCount (integer) --
        
        The number of workers that are allocated to the connector.
  - plugins (list) --
    
    Specifies which plugins the connector uses.
    - (dict) --
      
      The description of the plugin.
      - customPlugin (dict) --
        
        Details about a custom plugin.
        
        customPluginArn (string) --
        
        The Amazon Resource Name (ARN) of the custom plugin.
        
        revision (integer) --
        
        The revision of the custom plugin.
  - workerConfiguration (dict) --
    
    Specifies which worker configuration the connector uses.
    - revision (integer) --
      
      The revision of the worker configuration.
    - workerConfigurationArn (string) --
      
      The Amazon Resource Name (ARN) of the worker configuration.
  - useDefaultWorkerConfiguration (boolean) --
    
    Specifies whether the connector uses the default worker configuration instead of a custom worker configuration.
- originConnectorConfiguration (dict) --
  
  The origin connector configuration.
  - (string) --
    - (string) --
- targetWorkerSetting (dict) --
  
  The target worker setting.
  - capacity (dict) --
    
    Specifies the compute capacity settings of the connector.
    - autoScaling (dict) --
      
      Describes the connector's auto scaling capacity.
      - maxWorkerCount (integer) --
        
        The maximum number of workers allocated to the connector.
      - mcuCount (integer) --
        
        The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
      - minWorkerCount (integer) --
        
        The minimum number of workers allocated to the connector.
      - scaleInPolicy (dict) --
        
        The scale-in policy for the connector.
        
        cpuUtilizationPercentage (integer) --
        
        Specifies the CPU utilization percentage threshold at which you want connector scale in to be triggered.
      - scaleOutPolicy (dict) --
        
        The scale-out policy for the connector.
        
        cpuUtilizationPercentage (integer) --
        
        The CPU utilization percentage threshold at which you want connector scale out to be triggered.
      - maxAutoscalingTaskCount (integer) --
        
        The maximum number of tasks allocated to the connector during autoscaling operations. Must be at least equal to maxWorkerCount.
    - provisionedCapacity (dict) --
      
      Describes a connector's provisioned capacity.
      - mcuCount (integer) --
        
        The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
      - workerCount (integer) --
        
        The number of workers that are allocated to the connector.
  - plugins (list) --
    
    Specifies which plugins the connector uses.
    - (dict) --
      
      The description of the plugin.
      - customPlugin (dict) --
        
        Details about a custom plugin.
        
        customPluginArn (string) --
        
        The Amazon Resource Name (ARN) of the custom plugin.
        
        revision (integer) --
        
        The revision of the custom plugin.
  - workerConfiguration (dict) --
    
    Specifies which worker configuration the connector uses.
    - revision (integer) --
      
      The revision of the worker configuration.
    - workerConfigurationArn (string) --
      
      The Amazon Resource Name (ARN) of the worker configuration.
  - useDefaultWorkerConfiguration (boolean) --
    
    Specifies whether the connector uses the default worker configuration instead of a custom worker configuration.
- targetConnectorConfiguration (dict) --
  
  The target connector configuration.
  - (string) --
    - (string) --
- errorInfo (dict) --
  
  Details about the state of a resource.
  - code (string) --
    
    A code that describes the state of a resource.
  - message (string) --
    
    A message that describes the state of a resource.
- creationTime (datetime) --
  
  The time when the operation was created.
- endTime (datetime) --
  
  The time when the operation ended.

UpdateConnector (updated)

Link ¶
Changes (request)

{'plugins': [{'customPlugin': {'customPluginArn': 'string',
                               'revision': 'long'}}],
 'useDefaultWorkerConfiguration': 'boolean',
 'workerConfiguration': {'revision': 'long',
                         'workerConfigurationArn': 'string'}}

Updates the specified connector. In the request body, you can update either the connector's worker setting or its connector configuration, but not both in the same request. To update the worker setting, specify one or more of capacity, plugins, workerConfiguration, or useDefaultWorkerConfiguration. To update the connector configuration, specify only connectorConfiguration.

See also: AWS API Documentation

Request Syntax

client.update_connector(
    capacity={
        'autoScaling': {
            'maxWorkerCount': 123,
            'mcuCount': 123,
            'minWorkerCount': 123,
            'scaleInPolicy': {
                'cpuUtilizationPercentage': 123
            },
            'scaleOutPolicy': {
                'cpuUtilizationPercentage': 123
            },
            'maxAutoscalingTaskCount': 123
        },
        'provisionedCapacity': {
            'mcuCount': 123,
            'workerCount': 123
        }
    },
    connectorConfiguration={
        'string': 'string'
    },
    skipHealthValidation=True|False,
    connectorArn='string',
    currentVersion='string',
    plugins=[
        {
            'customPlugin': {
                'customPluginArn': 'string',
                'revision': 123
            }
        },
    ],
    workerConfiguration={
        'revision': 123,
        'workerConfigurationArn': 'string'
    },
    useDefaultWorkerConfiguration=True|False
)

type capacity:

dict

param capacity:

The target capacity.

autoScaling (dict) --

The target auto scaling setting.
- maxWorkerCount (integer) -- [REQUIRED]
  
  The target maximum number of workers allocated to the connector.
- mcuCount (integer) -- [REQUIRED]
  
  The target number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
- minWorkerCount (integer) -- [REQUIRED]
  
  The target minimum number of workers allocated to the connector.
- scaleInPolicy (dict) -- [REQUIRED]
  
  The target scale-in policy for the connector.
  - cpuUtilizationPercentage (integer) -- [REQUIRED]
    
    The target CPU utilization percentage threshold at which you want connector scale in to be triggered.
- scaleOutPolicy (dict) -- [REQUIRED]
  
  The target scale-out policy for the connector.
  - cpuUtilizationPercentage (integer) -- [REQUIRED]
    
    The target CPU utilization percentage threshold at which you want connector scale out to be triggered.
- maxAutoscalingTaskCount (integer) --
  
  The maximum number of tasks allocated to the connector during autoscaling operations. Must be at least equal to maxWorkerCount.
provisionedCapacity (dict) --

The target settings for provisioned capacity.
- mcuCount (integer) -- [REQUIRED]
  
  The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.
- workerCount (integer) -- [REQUIRED]
  
  The number of workers that are allocated to the connector.

type connectorConfiguration:

dict

param connectorConfiguration:

A map of keys to values that represent the configuration for the connector.

(string) --
- (string) --

type skipHealthValidation:

boolean

param skipHealthValidation:

Specifies whether the update bypasses connector-level and task-level health checks. If true, the update skips the health checks. The update stays in effect even if the connector or its tasks are unhealthy, and Amazon MSK Connect doesn't roll it back. If false or not specified, the health checks run. If they fail, MSK Connect automatically rolls the connector back to its previous configuration, and the update doesn't take effect. To update a connector that is in the PAUSED state, you must set this parameter to true.

type connectorArn:

string

param connectorArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the connector that you want to update.

type currentVersion:

string

param currentVersion:

[REQUIRED]

The current version of the connector that you want to update.

type plugins:

list

param plugins:

Specifies which plugin to use for the connector. You must specify a single-element list containing one customPlugin object. If you don't specify this parameter, the connector keeps its current plugin.

(dict) --

A custom plugin to use when you update a connector via UpdateConnector.
- customPlugin (dict) -- [REQUIRED]
  
  The custom plugin to use for the connector.
  - customPluginArn (string) -- [REQUIRED]
    
    The Amazon Resource Name (ARN) of the custom plugin.
  - revision (integer) -- [REQUIRED]
    
    The revision of the custom plugin.

type workerConfiguration:

dict

param workerConfiguration:

The worker configuration to use with the connector. Specify the Amazon Resource Name (ARN) and the revision of the worker configuration. You can't specify both workerConfiguration and useDefaultWorkerConfiguration in the same request. If you specify neither parameter, the connector keeps its current worker configuration.

revision (integer) -- [REQUIRED]

The revision of the worker configuration.
workerConfigurationArn (string) -- [REQUIRED]

The Amazon Resource Name (ARN) of the worker configuration.

type useDefaultWorkerConfiguration:

boolean

param useDefaultWorkerConfiguration:

Specifies whether to switch the connector to the default worker configuration that Amazon MSK Connect provides. If true, the connector stops using its custom worker configuration and uses the default worker configuration instead. If false or not specified, the connector keeps its current worker configuration. You can't specify both useDefaultWorkerConfiguration and workerConfiguration in the same request.

rtype:

dict

returns:

Response Syntax

{
    'connectorArn': 'string',
    'connectorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED'|'PAUSED'|'RESTARTING'|'PAUSING'|'RESUMING',
    'connectorOperationArn': 'string'
}

Response Structure

(dict) --
- connectorArn (string) --
  
  The Amazon Resource Name (ARN) of the connector.
- connectorState (string) --
  
  The state of the connector.
- connectorOperationArn (string) --
  
  The Amazon Resource Name (ARN) of the connector operation.