AWS API Changes

2025/05/08 - AWS Glue - 3 updated api methods

Changes This new release supports customizable RefreshInterval for all Saas ZETL integrations from 15 minutes to 6 days.

CreateIntegration (updated)

Link ¶
Changes (both)

{'IntegrationConfig': {'RefreshInterval': 'string'}}

Creates a Zero-ETL integration in the caller's account between two resources with Amazon Resource Names (ARNs): the SourceArn and TargetArn.

See also: AWS API Documentation

Request Syntax

client.create_integration(
    IntegrationName='string',
    SourceArn='string',
    TargetArn='string',
    Description='string',
    DataFilter='string',
    KmsKeyId='string',
    AdditionalEncryptionContext={
        'string': 'string'
    },
    Tags=[
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    IntegrationConfig={
        'RefreshInterval': 'string'
    }
)

type IntegrationName:

string

param IntegrationName:

[REQUIRED]

A unique name for an integration in Glue.

type SourceArn:

string

param SourceArn:

[REQUIRED]

The ARN of the source resource for the integration.

type TargetArn:

string

param TargetArn:

[REQUIRED]

The ARN of the target resource for the integration.

type Description:

string

param Description:

A description of the integration.

type DataFilter:

string

param DataFilter:

Selects source tables for the integration using Maxwell filter syntax.

type KmsKeyId:

string

param KmsKeyId:

The ARN of a KMS key used for encrypting the channel.

type AdditionalEncryptionContext:

dict

param AdditionalEncryptionContext:

An optional set of non-secret key–value pairs that contains additional contextual information for encryption. This can only be provided if KMSKeyId is provided.

(string) --
- (string) --

type Tags:

list

param Tags:

Metadata assigned to the resource consisting of a list of key-value pairs.

(dict) --

The Tag object represents a label that you can assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define.

For more information about tags, and controlling access to resources in Glue, see Amazon Web Services Tags in Glue and Specifying Glue Resource ARNs in the developer guide.
- key (string) --
  
  The tag key. The key is required when you create a tag on an object. The key is case-sensitive, and must not contain the prefix aws.
- value (string) --
  
  The tag value. The value is optional when you create a tag on an object. The value is case-sensitive, and must not contain the prefix aws.

type IntegrationConfig:

dict

param IntegrationConfig:

The configuration settings.

RefreshInterval (string) --

Specifies the frequency at which CDC (Change Data Capture) pulls or incremental loads should occur. This parameter provides flexibility to align the refresh rate with your specific data update patterns, system load considerations, and performance optimization goals. Time increment can be set from 15 minutes to 8640 minutes (six days). Currently supports creation of RefreshInterval only.

rtype:

dict

returns:

Response Syntax

{
    'SourceArn': 'string',
    'TargetArn': 'string',
    'IntegrationName': 'string',
    'Description': 'string',
    'IntegrationArn': 'string',
    'KmsKeyId': 'string',
    'AdditionalEncryptionContext': {
        'string': 'string'
    },
    'Tags': [
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    'Status': 'CREATING'|'ACTIVE'|'MODIFYING'|'FAILED'|'DELETING'|'SYNCING'|'NEEDS_ATTENTION',
    'CreateTime': datetime(2015, 1, 1),
    'Errors': [
        {
            'ErrorCode': 'string',
            'ErrorMessage': 'string'
        },
    ],
    'DataFilter': 'string',
    'IntegrationConfig': {
        'RefreshInterval': 'string'
    }
}

Response Structure

(dict) --
- SourceArn (string) --
  
  The ARN of the source resource for the integration.
- TargetArn (string) --
  
  The ARN of the target resource for the integration.
- IntegrationName (string) --
  
  A unique name for an integration in Glue.
- Description (string) --
  
  A description of the integration.
- IntegrationArn (string) --
  
  The Amazon Resource Name (ARN) for the created integration.
- KmsKeyId (string) --
  
  The ARN of a KMS key used for encrypting the channel.
- AdditionalEncryptionContext (dict) --
  
  An optional set of non-secret key–value pairs that contains additional contextual information for encryption.
  - (string) --
    - (string) --
- Tags (list) --
  
  Metadata assigned to the resource consisting of a list of key-value pairs.
  - (dict) --
    
    The Tag object represents a label that you can assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define.
    
    For more information about tags, and controlling access to resources in Glue, see Amazon Web Services Tags in Glue and Specifying Glue Resource ARNs in the developer guide.
    - key (string) --
      
      The tag key. The key is required when you create a tag on an object. The key is case-sensitive, and must not contain the prefix aws.
    - value (string) --
      
      The tag value. The value is optional when you create a tag on an object. The value is case-sensitive, and must not contain the prefix aws.
- Status (string) --
  
  The status of the integration being created.
  
  The possible statuses are:
  - CREATING: The integration is being created.
  - ACTIVE: The integration creation succeeds.
  - MODIFYING: The integration is being modified.
  - FAILED: The integration creation fails.
  - DELETING: The integration is deleted.
  - SYNCING: The integration is synchronizing.
  - NEEDS_ATTENTION: The integration needs attention, such as synchronization.
- CreateTime (datetime) --
  
  The time when the integration was created, in UTC.
- Errors (list) --
  
  A list of errors associated with the integration creation.
  - (dict) --
    
    An error associated with a zero-ETL integration.
    - ErrorCode (string) --
      
      The code associated with this error.
    - ErrorMessage (string) --
      
      A message describing the error.
- DataFilter (string) --
  
  Selects source tables for the integration using Maxwell filter syntax.
- IntegrationConfig (dict) --
  
  The configuration settings.
  - RefreshInterval (string) --
    
    Specifies the frequency at which CDC (Change Data Capture) pulls or incremental loads should occur. This parameter provides flexibility to align the refresh rate with your specific data update patterns, system load considerations, and performance optimization goals. Time increment can be set from 15 minutes to 8640 minutes (six days). Currently supports creation of RefreshInterval only.

DescribeInboundIntegrations (updated)

Link ¶
Changes (response)

{'InboundIntegrations': {'IntegrationConfig': {'RefreshInterval': 'string'}}}

Returns a list of inbound integrations for the specified integration.

See also: AWS API Documentation

Request Syntax

client.describe_inbound_integrations(
    IntegrationArn='string',
    Marker='string',
    MaxRecords=123,
    TargetArn='string'
)

type IntegrationArn:

string

param IntegrationArn:

The Amazon Resource Name (ARN) of the integration.

type Marker:

string

param Marker:

A token to specify where to start paginating. This is the marker from a previously truncated response.

type MaxRecords:

integer

param MaxRecords:

The total number of items to return in the output.

type TargetArn:

string

param TargetArn:

The Amazon Resource Name (ARN) of the target resource in the integration.

rtype:

dict

returns:

Response Syntax

{
    'InboundIntegrations': [
        {
            'SourceArn': 'string',
            'TargetArn': 'string',
            'IntegrationArn': 'string',
            'Status': 'CREATING'|'ACTIVE'|'MODIFYING'|'FAILED'|'DELETING'|'SYNCING'|'NEEDS_ATTENTION',
            'CreateTime': datetime(2015, 1, 1),
            'IntegrationConfig': {
                'RefreshInterval': 'string'
            },
            'Errors': [
                {
                    'ErrorCode': 'string',
                    'ErrorMessage': 'string'
                },
            ]
        },
    ],
    'Marker': 'string'
}

Response Structure

(dict) --
- InboundIntegrations (list) --
  
  A list of inbound integrations.
  - (dict) --
    
    A structure for an integration that writes data into a resource.
    - SourceArn (string) --
      
      The ARN of the source resource for the integration.
    - TargetArn (string) --
      
      The ARN of the target resource for the integration.
    - IntegrationArn (string) --
      
      The ARN of the zero-ETL integration.
    - Status (string) --
      
      The possible statuses are:
      - CREATING: The integration is being created.
      - ACTIVE: The integration creation succeeds.
      - MODIFYING: The integration is being modified.
      - FAILED: The integration creation fails.
      - DELETING: The integration is deleted.
      - SYNCING: The integration is synchronizing.
      - NEEDS_ATTENTION: The integration needs attention, such as synchronization.
    - CreateTime (datetime) --
      
      The time that the integration was created, in UTC.
    - IntegrationConfig (dict) --
      
      Properties associated with the integration.
      - RefreshInterval (string) --
        
        Specifies the frequency at which CDC (Change Data Capture) pulls or incremental loads should occur. This parameter provides flexibility to align the refresh rate with your specific data update patterns, system load considerations, and performance optimization goals. Time increment can be set from 15 minutes to 8640 minutes (six days). Currently supports creation of RefreshInterval only.
    - Errors (list) --
      
      A list of errors associated with the integration.
      - (dict) --
        
        An error associated with a zero-ETL integration.
        
        ErrorCode (string) --
        
        The code associated with this error.
        
        ErrorMessage (string) --
        
        A message describing the error.
- Marker (string) --
  
  A value that indicates the starting point for the next set of response records in a subsequent request.

DescribeIntegrations (updated)

Link ¶
Changes (response)

{'Integrations': {'IntegrationConfig': {'RefreshInterval': 'string'}}}

The API is used to retrieve a list of integrations.

See also: AWS API Documentation

Request Syntax

client.describe_integrations(
    IntegrationIdentifier='string',
    Marker='string',
    MaxRecords=123,
    Filters=[
        {
            'Name': 'string',
            'Values': [
                'string',
            ]
        },
    ]
)

type IntegrationIdentifier:

string

param IntegrationIdentifier:

The Amazon Resource Name (ARN) for the integration.

type Marker:

string

param Marker:

A value that indicates the starting point for the next set of response records in a subsequent request.

type MaxRecords:

integer

param MaxRecords:

The total number of items to return in the output.

type Filters:

list

param Filters:

A list of key and values, to filter down the results. Supported keys are "Status", "IntegrationName", and "SourceArn". IntegrationName is limited to only one value.

(dict) --

A filter that can be used when invoking a DescribeIntegrations request.
- Name (string) --
  
  The name of the filter.
- Values (list) --
  
  A list of filter values.
  - (string) --

rtype:

dict

returns:

Response Syntax

{
    'Integrations': [
        {
            'SourceArn': 'string',
            'TargetArn': 'string',
            'Description': 'string',
            'IntegrationName': 'string',
            'IntegrationArn': 'string',
            'KmsKeyId': 'string',
            'AdditionalEncryptionContext': {
                'string': 'string'
            },
            'Tags': [
                {
                    'key': 'string',
                    'value': 'string'
                },
            ],
            'Status': 'CREATING'|'ACTIVE'|'MODIFYING'|'FAILED'|'DELETING'|'SYNCING'|'NEEDS_ATTENTION',
            'CreateTime': datetime(2015, 1, 1),
            'IntegrationConfig': {
                'RefreshInterval': 'string'
            },
            'Errors': [
                {
                    'ErrorCode': 'string',
                    'ErrorMessage': 'string'
                },
            ],
            'DataFilter': 'string'
        },
    ],
    'Marker': 'string'
}

Response Structure

(dict) --
- Integrations (list) --
  
  A list of zero-ETL integrations.
  - (dict) --
    
    Describes a zero-ETL integration.
    - SourceArn (string) --
      
      The ARN for the source of the integration.
    - TargetArn (string) --
      
      The ARN for the target of the integration.
    - Description (string) --
      
      A description for the integration.
    - IntegrationName (string) --
      
      A unique name for the integration.
    - IntegrationArn (string) --
      
      The Amazon Resource Name (ARN) for the integration.
    - KmsKeyId (string) --
      
      The ARN of a KMS key used for encrypting the channel.
    - AdditionalEncryptionContext (dict) --
      
      An optional set of non-secret key–value pairs that contains additional contextual information for encryption. This can only be provided if KMSKeyId is provided.
      - (string) --
        
        (string) --
    - Tags (list) --
      
      Metadata assigned to the resource consisting of a list of key-value pairs.
      - (dict) --
        
        The Tag object represents a label that you can assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define.
        
        For more information about tags, and controlling access to resources in Glue, see Amazon Web Services Tags in Glue and Specifying Glue Resource ARNs in the developer guide.
        
        key (string) --
        
        The tag key. The key is required when you create a tag on an object. The key is case-sensitive, and must not contain the prefix aws.
        
        value (string) --
        
        The tag value. The value is optional when you create a tag on an object. The value is case-sensitive, and must not contain the prefix aws.
    - Status (string) --
      
      The possible statuses are:
      - CREATING: The integration is being created.
      - ACTIVE: The integration creation succeeds.
      - MODIFYING: The integration is being modified.
      - FAILED: The integration creation fails.
      - DELETING: The integration is deleted.
      - SYNCING: The integration is synchronizing.
      - NEEDS_ATTENTION: The integration needs attention, such as synchronization.
    - CreateTime (datetime) --
      
      The time that the integration was created, in UTC.
    - IntegrationConfig (dict) --
      
      Properties associated with the integration.
      - RefreshInterval (string) --
        
        Specifies the frequency at which CDC (Change Data Capture) pulls or incremental loads should occur. This parameter provides flexibility to align the refresh rate with your specific data update patterns, system load considerations, and performance optimization goals. Time increment can be set from 15 minutes to 8640 minutes (six days). Currently supports creation of RefreshInterval only.
    - Errors (list) --
      
      A list of errors associated with the integration.
      - (dict) --
        
        An error associated with a zero-ETL integration.
        
        ErrorCode (string) --
        
        The code associated with this error.
        
        ErrorMessage (string) --
        
        A message describing the error.
    - DataFilter (string) --
      
      Selects source tables for the integration using Maxwell filter syntax.
- Marker (string) --
  
  A value that indicates the starting point for the next set of response records in a subsequent request.