Managed Streaming for Kafka Connect

2025/01/13 - Managed Streaming for Kafka Connect - 2 new1 updated api methods

Changes  Support updating connector configuration via UpdateConnector API. Release Operations API to monitor the status of the connector operation.

ListConnectorOperations (new) Link ¶

Lists information about a connector's operation(s).

See also: AWS API Documentation

Request Syntax

client.list_connector_operations(
    connectorArn='string',
    maxResults=123,
    nextToken='string'
)
type connectorArn:

string

param connectorArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the connector for which to list operations.

type maxResults:

integer

param maxResults:

Maximum number of connector operations to fetch in one get request.

type nextToken:

string

param nextToken:

If the response is truncated, it includes a NextToken. Send this NextToken in a subsequent request to continue listing from where it left off.

rtype:

dict

returns:

Response Syntax

{
    'connectorOperations': [
        {
            'connectorOperationArn': 'string',
            'connectorOperationType': 'UPDATE_WORKER_SETTING'|'UPDATE_CONNECTOR_CONFIGURATION'|'ISOLATE_CONNECTOR'|'RESTORE_CONNECTOR',
            'connectorOperationState': 'PENDING'|'UPDATE_IN_PROGRESS'|'UPDATE_COMPLETE'|'UPDATE_FAILED'|'ROLLBACK_IN_PROGRESS'|'ROLLBACK_FAILED'|'ROLLBACK_COMPLETE',
            'creationTime': datetime(2015, 1, 1),
            'endTime': datetime(2015, 1, 1)
        },
    ],
    'nextToken': 'string'
}

Response Structure

  • (dict) --

    • connectorOperations (list) --

      An array of connector operation descriptions.

      • (dict) --

        Summary of a connector operation.

        • connectorOperationArn (string) --

          The Amazon Resource Name (ARN) of the connector operation.

        • connectorOperationType (string) --

          The type of connector operation performed.

        • connectorOperationState (string) --

          The state of the connector operation.

        • creationTime (datetime) --

          The time when operation was created.

        • endTime (datetime) --

          The time when operation ended.

    • nextToken (string) --

      If the response is truncated, it includes a NextToken. Send this NextToken in a subsequent request to continue listing from where it left off.

DescribeConnectorOperation (new) Link ¶

Returns information about the specified connector's operations.

See also: AWS API Documentation

Request Syntax

client.describe_connector_operation(
    connectorOperationArn='string'
)
type connectorOperationArn:

string

param connectorOperationArn:

[REQUIRED]

ARN of the connector operation to be described.

rtype:

dict

returns:

Response Syntax

{
    'connectorArn': 'string',
    'connectorOperationArn': 'string',
    'connectorOperationState': 'PENDING'|'UPDATE_IN_PROGRESS'|'UPDATE_COMPLETE'|'UPDATE_FAILED'|'ROLLBACK_IN_PROGRESS'|'ROLLBACK_FAILED'|'ROLLBACK_COMPLETE',
    'connectorOperationType': 'UPDATE_WORKER_SETTING'|'UPDATE_CONNECTOR_CONFIGURATION'|'ISOLATE_CONNECTOR'|'RESTORE_CONNECTOR',
    'operationSteps': [
        {
            'stepType': 'INITIALIZE_UPDATE'|'FINALIZE_UPDATE'|'UPDATE_WORKER_SETTING'|'UPDATE_CONNECTOR_CONFIGURATION'|'VALIDATE_UPDATE',
            'stepState': 'PENDING'|'IN_PROGRESS'|'COMPLETED'|'FAILED'|'CANCELLED'
        },
    ],
    'originWorkerSetting': {
        'capacity': {
            'autoScaling': {
                'maxWorkerCount': 123,
                'mcuCount': 123,
                'minWorkerCount': 123,
                'scaleInPolicy': {
                    'cpuUtilizationPercentage': 123
                },
                'scaleOutPolicy': {
                    'cpuUtilizationPercentage': 123
                }
            },
            'provisionedCapacity': {
                'mcuCount': 123,
                'workerCount': 123
            }
        }
    },
    'originConnectorConfiguration': {
        'string': 'string'
    },
    'targetWorkerSetting': {
        'capacity': {
            'autoScaling': {
                'maxWorkerCount': 123,
                'mcuCount': 123,
                'minWorkerCount': 123,
                'scaleInPolicy': {
                    'cpuUtilizationPercentage': 123
                },
                'scaleOutPolicy': {
                    'cpuUtilizationPercentage': 123
                }
            },
            'provisionedCapacity': {
                'mcuCount': 123,
                'workerCount': 123
            }
        }
    },
    'targetConnectorConfiguration': {
        'string': 'string'
    },
    'errorInfo': {
        'code': 'string',
        'message': 'string'
    },
    'creationTime': datetime(2015, 1, 1),
    'endTime': datetime(2015, 1, 1)
}

Response Structure

  • (dict) --

    • connectorArn (string) --

      The Amazon Resource Name (ARN) of the connector.

    • connectorOperationArn (string) --

      The Amazon Resource Name (ARN) of the connector operation.

    • connectorOperationState (string) --

      The state of the connector operation.

    • connectorOperationType (string) --

      The type of connector operation performed.

    • operationSteps (list) --

      The array of operation steps taken.

      • (dict) --

        Details of a step that is involved in a connector's operation.

        • stepType (string) --

          The step type of the operation.

        • stepState (string) --

          The step state of the operation.

    • originWorkerSetting (dict) --

      The origin worker setting.

      • capacity (dict) --

        A description of the connector's capacity.

        • autoScaling (dict) --

          Describes the connector's auto scaling capacity.

          • maxWorkerCount (integer) --

            The maximum number of workers allocated to the connector.

          • mcuCount (integer) --

            The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.

          • minWorkerCount (integer) --

            The minimum number of workers allocated to the connector.

          • scaleInPolicy (dict) --

            The sacle-in policy for the connector.

            • cpuUtilizationPercentage (integer) --

              Specifies the CPU utilization percentage threshold at which you want connector scale in to be triggered.

          • scaleOutPolicy (dict) --

            The sacle-out policy for the connector.>

            • cpuUtilizationPercentage (integer) --

              The CPU utilization percentage threshold at which you want connector scale out to be triggered.

        • provisionedCapacity (dict) --

          Describes a connector's provisioned capacity.

          • mcuCount (integer) --

            The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.

          • workerCount (integer) --

            The number of workers that are allocated to the connector.

    • originConnectorConfiguration (dict) --

      The origin connector configuration.

      • (string) --

        • (string) --

    • targetWorkerSetting (dict) --

      The target worker setting.

      • capacity (dict) --

        A description of the connector's capacity.

        • autoScaling (dict) --

          Describes the connector's auto scaling capacity.

          • maxWorkerCount (integer) --

            The maximum number of workers allocated to the connector.

          • mcuCount (integer) --

            The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.

          • minWorkerCount (integer) --

            The minimum number of workers allocated to the connector.

          • scaleInPolicy (dict) --

            The sacle-in policy for the connector.

            • cpuUtilizationPercentage (integer) --

              Specifies the CPU utilization percentage threshold at which you want connector scale in to be triggered.

          • scaleOutPolicy (dict) --

            The sacle-out policy for the connector.>

            • cpuUtilizationPercentage (integer) --

              The CPU utilization percentage threshold at which you want connector scale out to be triggered.

        • provisionedCapacity (dict) --

          Describes a connector's provisioned capacity.

          • mcuCount (integer) --

            The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.

          • workerCount (integer) --

            The number of workers that are allocated to the connector.

    • targetConnectorConfiguration (dict) --

      The target connector configuration.

      • (string) --

        • (string) --

    • errorInfo (dict) --

      Details about the state of a resource.

      • code (string) --

        A code that describes the state of a resource.

      • message (string) --

        A message that describes the state of a resource.

    • creationTime (datetime) --

      The time when the operation was created.

    • endTime (datetime) --

      The time when the operation ended.

UpdateConnector (updated) Link ¶
Changes (request, response)
Request 
{'connectorConfiguration': {'string': 'string'}}
Response 
{'connectorOperationArn': 'string'}

Updates the specified connector.

See also: AWS API Documentation

Request Syntax

client.update_connector(
    capacity={
        'autoScaling': {
            'maxWorkerCount': 123,
            'mcuCount': 123,
            'minWorkerCount': 123,
            'scaleInPolicy': {
                'cpuUtilizationPercentage': 123
            },
            'scaleOutPolicy': {
                'cpuUtilizationPercentage': 123
            }
        },
        'provisionedCapacity': {
            'mcuCount': 123,
            'workerCount': 123
        }
    },
    connectorConfiguration={
        'string': 'string'
    },
    connectorArn='string',
    currentVersion='string'
)
type capacity:

dict

param capacity:

The target capacity.

  • autoScaling (dict) --

    The target auto scaling setting.

    • maxWorkerCount (integer) -- [REQUIRED]

      The target maximum number of workers allocated to the connector.

    • mcuCount (integer) -- [REQUIRED]

      The target number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.

    • minWorkerCount (integer) -- [REQUIRED]

      The target minimum number of workers allocated to the connector.

    • scaleInPolicy (dict) -- [REQUIRED]

      The target sacle-in policy for the connector.

      • cpuUtilizationPercentage (integer) -- [REQUIRED]

        The target CPU utilization percentage threshold at which you want connector scale in to be triggered.

    • scaleOutPolicy (dict) -- [REQUIRED]

      The target sacle-out policy for the connector.

      • cpuUtilizationPercentage (integer) -- [REQUIRED]

        The target CPU utilization percentage threshold at which you want connector scale out to be triggered.

  • provisionedCapacity (dict) --

    The target settings for provisioned capacity.

    • mcuCount (integer) -- [REQUIRED]

      The number of microcontroller units (MCUs) allocated to each connector worker. The valid values are 1,2,4,8.

    • workerCount (integer) -- [REQUIRED]

      The number of workers that are allocated to the connector.

type connectorConfiguration:

dict

param connectorConfiguration:

A map of keys to values that represent the configuration for the connector.

  • (string) --

    • (string) --

type connectorArn:

string

param connectorArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the connector that you want to update.

type currentVersion:

string

param currentVersion:

[REQUIRED]

The current version of the connector that you want to update.

rtype:

dict

returns:

Response Syntax

{
    'connectorArn': 'string',
    'connectorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED',
    'connectorOperationArn': 'string'
}

Response Structure

  • (dict) --

    • connectorArn (string) --

      The Amazon Resource Name (ARN) of the connector.

    • connectorState (string) --

      The state of the connector.

    • connectorOperationArn (string) --

      The Amazon Resource Name (ARN) of the connector operation.