2024/12/03 - AWS Glue - 22 new6 updated api methods
Changes This release includes(1)Zero-ETL integration to ingest data from 3P SaaS and DynamoDB to Redshift/Redlake (2)new properties on Connections to enable reuse; new connection APIs for retrieve/preview metadata (3)support of CRUD operations for Multi-catalog (4)support of automatic statistics collections
Modifies a Zero-ETL integration in the caller's account.
See also: AWS API Documentation
Request Syntax
client.modify_integration(
IntegrationIdentifier='string',
Description='string',
DataFilter='string',
IntegrationName='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) for the integration.
string
A description of the integration.
string
Selects source tables for the integration using Maxwell filter syntax.
string
A unique name for an integration in Glue.
dict
Response Syntax
{
'SourceArn': 'string',
'TargetArn': 'string',
'IntegrationName': 'string',
'Description': 'string',
'IntegrationArn': 'string',
'KmsKeyId': 'string',
'AdditionalEncryptionContext': {
'string': 'string'
},
'Tags': [
{
'key': 'string',
'value': 'string'
},
],
'Status': 'CREATING'|'ACTIVE'|'MODIFYING'|'FAILED'|'DELETING'|'SYNCING'|'NEEDS_ATTENTION',
'CreateTime': datetime(2015, 1, 1),
'Errors': [
{
'ErrorCode': 'string',
'ErrorMessage': 'string'
},
],
'DataFilter': 'string'
}
Response Structure
(dict) --
SourceArn (string) --
The ARN of the source for the integration.
TargetArn (string) --
The ARN of the target for the integration.
IntegrationName (string) --
A unique name for an integration in Glue.
Description (string) --
A description of the integration.
IntegrationArn (string) --
The Amazon Resource Name (ARN) for the integration.
KmsKeyId (string) --
The ARN of a KMS key used for encrypting the channel.
AdditionalEncryptionContext (dict) --
An optional set of non-secret key–value pairs that contains additional contextual information for encryption.
(string) --
(string) --
Tags (list) --
Metadata assigned to the resource consisting of a list of key-value pairs.
(dict) --
The Tag object represents a label that you can assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define.
For more information about tags, and controlling access to resources in Glue, see Amazon Web Services Tags in Glue and Specifying Glue Resource ARNs in the developer guide.
key (string) --
The tag key. The key is required when you create a tag on an object. The key is case-sensitive, and must not contain the prefix aws.
value (string) --
The tag value. The value is optional when you create a tag on an object. The value is case-sensitive, and must not contain the prefix aws.
Status (string) --
The status of the integration being modified.
The possible statuses are:
CREATING: The integration is being created.
ACTIVE: The integration creation succeeds.
MODIFYING: The integration is being modified.
FAILED: The integration creation fails.
DELETING: The integration is deleted.
SYNCING: The integration is synchronizing.
NEEDS_ATTENTION: The integration needs attention, such as synchronization.
CreateTime (datetime) --
The time when the integration was created, in UTC.
Errors (list) --
A list of errors associated with the integration modification.
(dict) --
An error associated with a zero-ETL integration.
ErrorCode (string) --
The code associated with this error.
ErrorMessage (string) --
A message describing the error.
DataFilter (string) --
Selects source tables for the integration using Maxwell filter syntax.
Updates an existing catalog's properties in the Glue Data Catalog.
See also: AWS API Documentation
Request Syntax
client.update_catalog(
CatalogId='string',
CatalogInput={
'Description': 'string',
'FederatedCatalog': {
'Identifier': 'string',
'ConnectionName': 'string'
},
'Parameters': {
'string': 'string'
},
'TargetRedshiftCatalog': {
'CatalogArn': 'string'
},
'CatalogProperties': {
'DataLakeAccessProperties': {
'DataLakeAccess': True|False,
'DataTransferRole': 'string',
'KmsKey': 'string',
'CatalogType': 'string'
},
'CustomProperties': {
'string': 'string'
}
},
'CreateTableDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS',
]
},
],
'CreateDatabaseDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS',
]
},
]
}
)
string
[REQUIRED]
The ID of the catalog.
dict
[REQUIRED]
A CatalogInput object specifying the new properties of an existing catalog.
Description (string) --
Description string, not more than 2048 bytes long, matching the URI address multi-line string pattern. A description of the catalog.
FederatedCatalog (dict) --
A FederatedCatalog object. A FederatedCatalog structure that references an entity outside the Glue Data Catalog, for example a Redshift database.
Identifier (string) --
A unique identifier for the federated catalog.
ConnectionName (string) --
The name of the connection to an external data source, for example a Redshift-federated catalog.
Parameters (dict) --
A map array of key-value pairs that define the parameters and properties of the catalog.
(string) --
(string) --
TargetRedshiftCatalog (dict) --
A TargetRedshiftCatalog object that describes a target catalog for resource linking.
CatalogArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the catalog resource.
CatalogProperties (dict) --
A CatalogProperties object that specifies data lake access properties and other custom properties.
DataLakeAccessProperties (dict) --
A DataLakeAccessProperties object that specifies properties to configure data lake access for your catalog resource in the Glue Data Catalog.
DataLakeAccess (boolean) --
Turns on or off data lake access for Apache Spark applications that access Amazon Redshift databases in the Data Catalog from any non-Redshift engine, such as Amazon Athena, Amazon EMR, or Glue ETL.
DataTransferRole (string) --
A role that will be assumed by Glue for transferring data into/out of the staging bucket during a query.
KmsKey (string) --
An encryption key that will be used for the staging bucket that will be created along with the catalog.
CatalogType (string) --
Specifies a federated catalog type for the native catalog resource. The currently supported type is aws:redshift.
CustomProperties (dict) --
Additional key-value properties for the catalog, such as column statistics optimizations.
(string) --
(string) --
CreateTableDefaultPermissions (list) --
An array of PrincipalPermissions objects. Creates a set of default permissions on the table(s) for principals. Used by Amazon Web Services Lake Formation. Typically should be explicitly set as an empty list.
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
CreateDatabaseDefaultPermissions (list) --
An array of PrincipalPermissions objects. Creates a set of default permissions on the database(s) for principals. Used by Amazon Web Services Lake Formation. Typically should be explicitly set as an empty list.
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
dict
Response Syntax
{}
Response Structure
(dict) --
This API can be used for setting up the ResourceProperty of the Glue connection (for the source) or Glue database ARN (for the target). These properties can include the role to access the connection or database. To set both source and target properties the same API needs to be invoked with the Glue connection ARN as ResourceArn with SourceProcessingProperties and the Glue database ARN as ResourceArn with TargetProcessingProperties respectively.
See also: AWS API Documentation
Request Syntax
client.create_integration_resource_property(
ResourceArn='string',
SourceProcessingProperties={
'RoleArn': 'string'
},
TargetProcessingProperties={
'RoleArn': 'string',
'KmsArn': 'string',
'ConnectionName': 'string',
'EventBusArn': 'string'
}
)
string
[REQUIRED]
The connection ARN of the source, or the database ARN of the target.
dict
The resource properties associated with the integration source.
RoleArn (string) --
The IAM role to access the Glue connection.
dict
The resource properties associated with the integration target.
RoleArn (string) --
The IAM role to access the Glue database.
KmsArn (string) --
The ARN of the KMS key used for encryption.
ConnectionName (string) --
The Glue network connection to configure the Glue job running in the customer VPC.
EventBusArn (string) --
The ARN of an Eventbridge event bus to receive the integration status notification.
dict
Response Syntax
{
'ResourceArn': 'string',
'SourceProcessingProperties': {
'RoleArn': 'string'
},
'TargetProcessingProperties': {
'RoleArn': 'string',
'KmsArn': 'string',
'ConnectionName': 'string',
'EventBusArn': 'string'
}
}
Response Structure
(dict) --
ResourceArn (string) --
The connection ARN of the source, or the database ARN of the target.
SourceProcessingProperties (dict) --
The resource properties associated with the integration source.
RoleArn (string) --
The IAM role to access the Glue connection.
TargetProcessingProperties (dict) --
The resource properties associated with the integration target.
RoleArn (string) --
The IAM role to access the Glue database.
KmsArn (string) --
The ARN of the KMS key used for encryption.
ConnectionName (string) --
The Glue network connection to configure the Glue job running in the customer VPC.
EventBusArn (string) --
The ARN of an Eventbridge event bus to receive the integration status notification.
This API is used to provide optional override properties for the tables that need to be replicated. These properties can include properties for filtering and partitioning for the source and target tables. To set both source and target properties the same API need to be invoked with the Glue connection ARN as ResourceArn with SourceTableConfig, and the Glue database ARN as ResourceArn with TargetTableConfig respectively.
The override will be reflected across all the integrations using same ResourceArn and source table.
See also: AWS API Documentation
Request Syntax
client.update_integration_table_properties(
ResourceArn='string',
TableName='string',
SourceTableConfig={
'Fields': [
'string',
],
'FilterPredicate': 'string',
'PrimaryKey': [
'string',
],
'RecordUpdateField': 'string'
},
TargetTableConfig={
'UnnestSpec': 'TOPLEVEL'|'FULL'|'NOUNNEST',
'PartitionSpec': [
{
'FieldName': 'string',
'FunctionSpec': 'string'
},
],
'TargetTableName': 'string'
}
)
string
[REQUIRED]
The connection ARN of the source, or the database ARN of the target.
string
[REQUIRED]
The name of the table to be replicated.
dict
A structure for the source table configuration.
Fields (list) --
A list of fields used for column-level filtering.
(string) --
FilterPredicate (string) --
A condition clause used for row-level filtering.
PrimaryKey (list) --
Unique identifier of a record.
(string) --
RecordUpdateField (string) --
Incremental pull timestamp-based field.
dict
A structure for the target table configuration.
UnnestSpec (string) --
Specifies how nested objects are flattened to top-level elements. Valid values are: "TOPLEVEL", "FULL", or "NOUNNEST".
PartitionSpec (list) --
Determines the file layout on the target.
(dict) --
A structure that describes how data is partitioned on the target.
FieldName (string) --
The field name used to partition data on the target.
FunctionSpec (string) --
Specifies a function used to partition data on the target.
TargetTableName (string) --
The optional name of a target table.
dict
Response Syntax
{}
Response Structure
(dict) --
This API is used to provide optional override properties for the the tables that need to be replicated. These properties can include properties for filtering and partitioning for the source and target tables. To set both source and target properties the same API need to be invoked with the Glue connection ARN as ResourceArn with SourceTableConfig, and the Glue database ARN as ResourceArn with TargetTableConfig respectively.
See also: AWS API Documentation
Request Syntax
client.create_integration_table_properties(
ResourceArn='string',
TableName='string',
SourceTableConfig={
'Fields': [
'string',
],
'FilterPredicate': 'string',
'PrimaryKey': [
'string',
],
'RecordUpdateField': 'string'
},
TargetTableConfig={
'UnnestSpec': 'TOPLEVEL'|'FULL'|'NOUNNEST',
'PartitionSpec': [
{
'FieldName': 'string',
'FunctionSpec': 'string'
},
],
'TargetTableName': 'string'
}
)
string
[REQUIRED]
The connection ARN of the source, or the database ARN of the target.
string
[REQUIRED]
The name of the table to be replicated.
dict
A structure for the source table configuration.
Fields (list) --
A list of fields used for column-level filtering.
(string) --
FilterPredicate (string) --
A condition clause used for row-level filtering.
PrimaryKey (list) --
Unique identifier of a record.
(string) --
RecordUpdateField (string) --
Incremental pull timestamp-based field.
dict
A structure for the target table configuration.
UnnestSpec (string) --
Specifies how nested objects are flattened to top-level elements. Valid values are: "TOPLEVEL", "FULL", or "NOUNNEST".
PartitionSpec (list) --
Determines the file layout on the target.
(dict) --
A structure that describes how data is partitioned on the target.
FieldName (string) --
The field name used to partition data on the target.
FunctionSpec (string) --
Specifies a function used to partition data on the target.
TargetTableName (string) --
The optional name of a target table.
dict
Response Syntax
{}
Response Structure
(dict) --
Returns a list of inbound integrations for the specified integration.
See also: AWS API Documentation
Request Syntax
client.describe_inbound_integrations(
IntegrationArn='string',
Marker='string',
MaxRecords=123,
TargetArn='string'
)
string
The Amazon Resource Name (ARN) of the integration.
string
A token to specify where to start paginating. This is the marker from a previously truncated response.
integer
The total number of items to return in the output.
string
The Amazon Resource Name (ARN) of the target resource in the integration.
dict
Response Syntax
{
'InboundIntegrations': [
{
'SourceArn': 'string',
'TargetArn': 'string',
'IntegrationArn': 'string',
'Status': 'CREATING'|'ACTIVE'|'MODIFYING'|'FAILED'|'DELETING'|'SYNCING'|'NEEDS_ATTENTION',
'CreateTime': datetime(2015, 1, 1),
'Errors': [
{
'ErrorCode': 'string',
'ErrorMessage': 'string'
},
]
},
],
'Marker': 'string'
}
Response Structure
(dict) --
InboundIntegrations (list) --
A list of inbound integrations.
(dict) --
A structure for an integration that writes data into a resource.
SourceArn (string) --
The ARN of the source resource for the integration.
TargetArn (string) --
The ARN of the target resource for the integration.
IntegrationArn (string) --
The ARN of the zero-ETL integration.
Status (string) --
The possible statuses are:
CREATING: The integration is being created.
ACTIVE: The integration creation succeeds.
MODIFYING: The integration is being modified.
FAILED: The integration creation fails.
DELETING: The integration is deleted.
SYNCING: The integration is synchronizing.
NEEDS_ATTENTION: The integration needs attention, such as synchronization.
CreateTime (datetime) --
The time that the integration was created, in UTC.
Errors (list) --
A list of errors associated with the integration.
(dict) --
An error associated with a zero-ETL integration.
ErrorCode (string) --
The code associated with this error.
ErrorMessage (string) --
A message describing the error.
Marker (string) --
A value that indicates the starting point for the next set of response records in a subsequent request.
The name of the Catalog to retrieve. This should be all lowercase.
See also: AWS API Documentation
Request Syntax
client.get_catalog(
CatalogId='string'
)
string
[REQUIRED]
The ID of the parent catalog in which the catalog resides. If none is provided, the Amazon Web Services Account Number is used by default.
dict
Response Syntax
{
'Catalog': {
'CatalogId': 'string',
'Name': 'string',
'ResourceArn': 'string',
'Description': 'string',
'Parameters': {
'string': 'string'
},
'CreateTime': datetime(2015, 1, 1),
'UpdateTime': datetime(2015, 1, 1),
'TargetRedshiftCatalog': {
'CatalogArn': 'string'
},
'FederatedCatalog': {
'Identifier': 'string',
'ConnectionName': 'string'
},
'CatalogProperties': {
'DataLakeAccessProperties': {
'DataLakeAccess': True|False,
'DataTransferRole': 'string',
'KmsKey': 'string',
'ManagedWorkgroupName': 'string',
'ManagedWorkgroupStatus': 'string',
'RedshiftDatabaseName': 'string',
'StatusMessage': 'string',
'CatalogType': 'string'
},
'CustomProperties': {
'string': 'string'
}
},
'CreateTableDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS',
]
},
],
'CreateDatabaseDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS',
]
},
]
}
}
Response Structure
(dict) --
Catalog (dict) --
A Catalog object. The definition of the specified catalog in the Glue Data Catalog.
CatalogId (string) --
The ID of the catalog. To grant access to the default catalog, this field should not be provided.
Name (string) --
The name of the catalog. Cannot be the same as the account ID.
ResourceArn (string) --
The Amazon Resource Name (ARN) assigned to the catalog resource.
Description (string) --
Description string, not more than 2048 bytes long, matching the URI address multi-line string pattern. A description of the catalog.
Parameters (dict) --
A map array of key-value pairs that define parameters and properties of the catalog.
(string) --
(string) --
CreateTime (datetime) --
The time at which the catalog was created.
UpdateTime (datetime) --
The time at which the catalog was last updated.
TargetRedshiftCatalog (dict) --
A TargetRedshiftCatalog object that describes a target catalog for database resource linking.
CatalogArn (string) --
The Amazon Resource Name (ARN) of the catalog resource.
FederatedCatalog (dict) --
A FederatedCatalog object that points to an entity outside the Glue Data Catalog.
Identifier (string) --
A unique identifier for the federated catalog.
ConnectionName (string) --
The name of the connection to an external data source, for example a Redshift-federated catalog.
CatalogProperties (dict) --
A CatalogProperties object that specifies data lake access properties and other custom properties.
DataLakeAccessProperties (dict) --
A DataLakeAccessProperties object with input properties to configure data lake access for your catalog resource in the Glue Data Catalog.
DataLakeAccess (boolean) --
Turns on or off data lake access for Apache Spark applications that access Amazon Redshift databases in the Data Catalog.
DataTransferRole (string) --
A role that will be assumed by Glue for transferring data into/out of the staging bucket during a query.
KmsKey (string) --
An encryption key that will be used for the staging bucket that will be created along with the catalog.
ManagedWorkgroupName (string) --
The managed Redshift Serverless compute name that is created for your catalog resource.
ManagedWorkgroupStatus (string) --
The managed Redshift Serverless compute status.
RedshiftDatabaseName (string) --
The default Redshift database resource name in the managed compute.
StatusMessage (string) --
A message that gives more detailed information about the managed workgroup status.
CatalogType (string) --
Specifies a federated catalog type for the native catalog resource. The currently supported type is aws:redshift.
CustomProperties (dict) --
Additional key-value properties for the catalog, such as column statistics optimizations.
(string) --
(string) --
CreateTableDefaultPermissions (list) --
An array of PrincipalPermissions objects. Creates a set of default permissions on the table(s) for principals. Used by Amazon Web Services Lake Formation. Not used in the normal course of Glue operations.
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
CreateDatabaseDefaultPermissions (list) --
An array of PrincipalPermissions objects. Creates a set of default permissions on the database(s) for principals. Used by Amazon Web Services Lake Formation. Not used in the normal course of Glue operations.
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
Removes the specified catalog from the Glue Data Catalog.
After completing this operation, you no longer have access to the databases, tables (and all table versions and partitions that might belong to the tables) and the user-defined functions in the deleted catalog. Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service.
To ensure the immediate deletion of all related resources before calling the DeleteCatalog operation, use DeleteTableVersion (or BatchDeleteTableVersion), DeletePartition (or BatchDeletePartition), DeleteTable (or BatchDeleteTable), DeleteUserDefinedFunction and DeleteDatabase to delete any resources that belong to the catalog.
See also: AWS API Documentation
Request Syntax
client.delete_catalog(
CatalogId='string'
)
string
[REQUIRED]
The ID of the catalog.
dict
Response Syntax
{}
Response Structure
(dict) --
This API is used to retrieve optional override properties for the tables that need to be replicated. These properties can include properties for filtering and partition for source and target tables.
See also: AWS API Documentation
Request Syntax
client.get_integration_table_properties(
ResourceArn='string',
TableName='string'
)
string
[REQUIRED]
The connection ARN of the source, or the database ARN of the target.
string
[REQUIRED]
The name of the table to be replicated.
dict
Response Syntax
{
'ResourceArn': 'string',
'TableName': 'string',
'SourceTableConfig': {
'Fields': [
'string',
],
'FilterPredicate': 'string',
'PrimaryKey': [
'string',
],
'RecordUpdateField': 'string'
},
'TargetTableConfig': {
'UnnestSpec': 'TOPLEVEL'|'FULL'|'NOUNNEST',
'PartitionSpec': [
{
'FieldName': 'string',
'FunctionSpec': 'string'
},
],
'TargetTableName': 'string'
}
}
Response Structure
(dict) --
ResourceArn (string) --
The connection ARN of the source, or the database ARN of the target.
TableName (string) --
The name of the table to be replicated.
SourceTableConfig (dict) --
A structure for the source table configuration.
Fields (list) --
A list of fields used for column-level filtering.
(string) --
FilterPredicate (string) --
A condition clause used for row-level filtering.
PrimaryKey (list) --
Unique identifier of a record.
(string) --
RecordUpdateField (string) --
Incremental pull timestamp-based field.
TargetTableConfig (dict) --
A structure for the target table configuration.
UnnestSpec (string) --
Specifies how nested objects are flattened to top-level elements. Valid values are: "TOPLEVEL", "FULL", or "NOUNNEST".
PartitionSpec (list) --
Determines the file layout on the target.
(dict) --
A structure that describes how data is partitioned on the target.
FieldName (string) --
The field name used to partition data on the target.
FunctionSpec (string) --
Specifies a function used to partition data on the target.
TargetTableName (string) --
The optional name of a target table.
Provides details regarding the entity used with the connection type, with a description of the data model for each field in the selected entity.
The response includes all the fields which make up the entity.
See also: AWS API Documentation
Request Syntax
client.describe_entity(
ConnectionName='string',
CatalogId='string',
EntityName='string',
NextToken='string',
DataStoreApiVersion='string'
)
string
[REQUIRED]
The name of the connection that contains the connection type credentials.
string
The catalog ID of the catalog that contains the connection. This can be null, By default, the Amazon Web Services Account ID is the catalog ID.
string
[REQUIRED]
The name of the entity that you want to describe from the connection type.
string
A continuation token, included if this is a continuation call.
string
The version of the API used for the data store.
dict
Response Syntax
{
'Fields': [
{
'FieldName': 'string',
'Label': 'string',
'Description': 'string',
'FieldType': 'INT'|'SMALLINT'|'BIGINT'|'FLOAT'|'LONG'|'DATE'|'BOOLEAN'|'MAP'|'ARRAY'|'STRING'|'TIMESTAMP'|'DECIMAL'|'BYTE'|'SHORT'|'DOUBLE'|'STRUCT',
'IsPrimaryKey': True|False,
'IsNullable': True|False,
'IsRetrievable': True|False,
'IsFilterable': True|False,
'IsPartitionable': True|False,
'IsCreateable': True|False,
'IsUpdateable': True|False,
'IsUpsertable': True|False,
'IsDefaultOnCreate': True|False,
'SupportedValues': [
'string',
],
'SupportedFilterOperators': [
'LESS_THAN'|'GREATER_THAN'|'BETWEEN'|'EQUAL_TO'|'NOT_EQUAL_TO'|'GREATER_THAN_OR_EQUAL_TO'|'LESS_THAN_OR_EQUAL_TO'|'CONTAINS'|'ORDER_BY',
],
'ParentField': 'string',
'NativeDataType': 'string',
'CustomProperties': {
'string': 'string'
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Fields (list) --
Describes the fields for that connector entity. This is the list of Field objects. Field is very similar to column in a database. The Field object has information about different properties associated with fields in the connector.
(dict) --
The Field object has information about the different properties associated with a field in the connector.
FieldName (string) --
A unique identifier for the field.
Label (string) --
A readable label used for the field.
Description (string) --
A description of the field.
FieldType (string) --
The type of data in the field.
IsPrimaryKey (boolean) --
Indicates whether this field can used as a primary key for the given entity.
IsNullable (boolean) --
Indicates whether this field can be nullable or not.
IsRetrievable (boolean) --
Indicates whether this field can be added in Select clause of SQL query or whether it is retrievable or not.
IsFilterable (boolean) --
Indicates whether this field can used in a filter clause ( WHERE clause) of a SQL statement when querying data.
IsPartitionable (boolean) --
Indicates whether a given field can be used in partitioning the query made to SaaS.
IsCreateable (boolean) --
Indicates whether this field can be created as part of a destination write.
IsUpdateable (boolean) --
Indicates whether this field can be updated as part of a destination write.
IsUpsertable (boolean) --
Indicates whether this field can be upserted as part of a destination write.
IsDefaultOnCreate (boolean) --
Indicates whether this field is populated automatically when the object is created, such as a created at timestamp.
SupportedValues (list) --
A list of supported values for the field.
(string) --
SupportedFilterOperators (list) --
Indicates the support filter operators for this field.
(string) --
ParentField (string) --
A parent field name for a nested field.
NativeDataType (string) --
The data type returned by the SaaS API, such as “picklist” or “textarea” from Salesforce.
CustomProperties (dict) --
Optional map of keys which may be returned.
(string) --
(string) --
NextToken (string) --
A continuation token, present if the current segment is not the last.
This API is used for fetching the ResourceProperty of the Glue connection (for the source) or Glue database ARN (for the target)
See also: AWS API Documentation
Request Syntax
client.get_integration_resource_property(
ResourceArn='string'
)
string
[REQUIRED]
The connection ARN of the source, or the database ARN of the target.
dict
Response Syntax
{
'ResourceArn': 'string',
'SourceProcessingProperties': {
'RoleArn': 'string'
},
'TargetProcessingProperties': {
'RoleArn': 'string',
'KmsArn': 'string',
'ConnectionName': 'string',
'EventBusArn': 'string'
}
}
Response Structure
(dict) --
ResourceArn (string) --
The connection ARN of the source, or the database ARN of the target.
SourceProcessingProperties (dict) --
The resource properties associated with the integration source.
RoleArn (string) --
The IAM role to access the Glue connection.
TargetProcessingProperties (dict) --
The resource properties associated with the integration target.
RoleArn (string) --
The IAM role to access the Glue database.
KmsArn (string) --
The ARN of the KMS key used for encryption.
ConnectionName (string) --
The Glue network connection to configure the Glue job running in the customer VPC.
EventBusArn (string) --
The ARN of an Eventbridge event bus to receive the integration status notification.
Deletes the specified Zero-ETL integration.
See also: AWS API Documentation
Request Syntax
client.delete_integration(
IntegrationIdentifier='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) for the integration.
dict
Response Syntax
{
'SourceArn': 'string',
'TargetArn': 'string',
'IntegrationName': 'string',
'Description': 'string',
'IntegrationArn': 'string',
'KmsKeyId': 'string',
'AdditionalEncryptionContext': {
'string': 'string'
},
'Tags': [
{
'key': 'string',
'value': 'string'
},
],
'Status': 'CREATING'|'ACTIVE'|'MODIFYING'|'FAILED'|'DELETING'|'SYNCING'|'NEEDS_ATTENTION',
'CreateTime': datetime(2015, 1, 1),
'Errors': [
{
'ErrorCode': 'string',
'ErrorMessage': 'string'
},
],
'DataFilter': 'string'
}
Response Structure
(dict) --
SourceArn (string) --
The ARN of the source for the integration.
TargetArn (string) --
The ARN of the target for the integration.
IntegrationName (string) --
A unique name for an integration in Glue.
Description (string) --
A description of the integration.
IntegrationArn (string) --
The Amazon Resource Name (ARN) for the integration.
KmsKeyId (string) --
The ARN of a KMS key used for encrypting the channel.
AdditionalEncryptionContext (dict) --
An optional set of non-secret key–value pairs that contains additional contextual information for encryption.
(string) --
(string) --
Tags (list) --
Metadata assigned to the resource consisting of a list of key-value pairs.
(dict) --
The Tag object represents a label that you can assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define.
For more information about tags, and controlling access to resources in Glue, see Amazon Web Services Tags in Glue and Specifying Glue Resource ARNs in the developer guide.
key (string) --
The tag key. The key is required when you create a tag on an object. The key is case-sensitive, and must not contain the prefix aws.
value (string) --
The tag value. The value is optional when you create a tag on an object. The value is case-sensitive, and must not contain the prefix aws.
Status (string) --
The status of the integration being deleted.
The possible statuses are:
CREATING: The integration is being created.
ACTIVE: The integration creation succeeds.
MODIFYING: The integration is being modified.
FAILED: The integration creation fails.
DELETING: The integration is deleted.
SYNCING: The integration is synchronizing.
NEEDS_ATTENTION: The integration needs attention, such as synchronization.
CreateTime (datetime) --
The time when the integration was created, in UTC.
Errors (list) --
A list of errors associated with the integration.
(dict) --
An error associated with a zero-ETL integration.
ErrorCode (string) --
The code associated with this error.
ErrorMessage (string) --
A message describing the error.
DataFilter (string) --
Selects source tables for the integration using Maxwell filter syntax.
Retrieves all catalogs defined in a catalog in the Glue Data Catalog. For a Redshift-federated catalog use case, this operation returns the list of catalogs mapped to Redshift databases in the Redshift namespace catalog.
See also: AWS API Documentation
Request Syntax
client.get_catalogs(
ParentCatalogId='string',
NextToken='string',
MaxResults=123,
Recursive=True|False
)
string
The ID of the parent catalog in which the catalog resides. If none is provided, the Amazon Web Services Account Number is used by default.
string
A continuation token, if this is a continuation call.
integer
The maximum number of catalogs to return in one response.
boolean
When specified as true, iterates through the account and returns all catalog resources (including top-level resources and child resources)
dict
Response Syntax
{
'CatalogList': [
{
'CatalogId': 'string',
'Name': 'string',
'ResourceArn': 'string',
'Description': 'string',
'Parameters': {
'string': 'string'
},
'CreateTime': datetime(2015, 1, 1),
'UpdateTime': datetime(2015, 1, 1),
'TargetRedshiftCatalog': {
'CatalogArn': 'string'
},
'FederatedCatalog': {
'Identifier': 'string',
'ConnectionName': 'string'
},
'CatalogProperties': {
'DataLakeAccessProperties': {
'DataLakeAccess': True|False,
'DataTransferRole': 'string',
'KmsKey': 'string',
'ManagedWorkgroupName': 'string',
'ManagedWorkgroupStatus': 'string',
'RedshiftDatabaseName': 'string',
'StatusMessage': 'string',
'CatalogType': 'string'
},
'CustomProperties': {
'string': 'string'
}
},
'CreateTableDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS',
]
},
],
'CreateDatabaseDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS',
]
},
]
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
CatalogList (list) --
An array of Catalog objects. A list of Catalog objects from the specified parent catalog.
(dict) --
The catalog object represents a logical grouping of databases in the Glue Data Catalog or a federated source. You can now create a Redshift-federated catalog or a catalog containing resource links to Redshift databases in another account or region.
CatalogId (string) --
The ID of the catalog. To grant access to the default catalog, this field should not be provided.
Name (string) --
The name of the catalog. Cannot be the same as the account ID.
ResourceArn (string) --
The Amazon Resource Name (ARN) assigned to the catalog resource.
Description (string) --
Description string, not more than 2048 bytes long, matching the URI address multi-line string pattern. A description of the catalog.
Parameters (dict) --
A map array of key-value pairs that define parameters and properties of the catalog.
(string) --
(string) --
CreateTime (datetime) --
The time at which the catalog was created.
UpdateTime (datetime) --
The time at which the catalog was last updated.
TargetRedshiftCatalog (dict) --
A TargetRedshiftCatalog object that describes a target catalog for database resource linking.
CatalogArn (string) --
The Amazon Resource Name (ARN) of the catalog resource.
FederatedCatalog (dict) --
A FederatedCatalog object that points to an entity outside the Glue Data Catalog.
Identifier (string) --
A unique identifier for the federated catalog.
ConnectionName (string) --
The name of the connection to an external data source, for example a Redshift-federated catalog.
CatalogProperties (dict) --
A CatalogProperties object that specifies data lake access properties and other custom properties.
DataLakeAccessProperties (dict) --
A DataLakeAccessProperties object with input properties to configure data lake access for your catalog resource in the Glue Data Catalog.
DataLakeAccess (boolean) --
Turns on or off data lake access for Apache Spark applications that access Amazon Redshift databases in the Data Catalog.
DataTransferRole (string) --
A role that will be assumed by Glue for transferring data into/out of the staging bucket during a query.
KmsKey (string) --
An encryption key that will be used for the staging bucket that will be created along with the catalog.
ManagedWorkgroupName (string) --
The managed Redshift Serverless compute name that is created for your catalog resource.
ManagedWorkgroupStatus (string) --
The managed Redshift Serverless compute status.
RedshiftDatabaseName (string) --
The default Redshift database resource name in the managed compute.
StatusMessage (string) --
A message that gives more detailed information about the managed workgroup status.
CatalogType (string) --
Specifies a federated catalog type for the native catalog resource. The currently supported type is aws:redshift.
CustomProperties (dict) --
Additional key-value properties for the catalog, such as column statistics optimizations.
(string) --
(string) --
CreateTableDefaultPermissions (list) --
An array of PrincipalPermissions objects. Creates a set of default permissions on the table(s) for principals. Used by Amazon Web Services Lake Formation. Not used in the normal course of Glue operations.
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
CreateDatabaseDefaultPermissions (list) --
An array of PrincipalPermissions objects. Creates a set of default permissions on the database(s) for principals. Used by Amazon Web Services Lake Formation. Not used in the normal course of Glue operations.
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
NextToken (string) --
A continuation token for paginating the returned list of tokens, returned if the current segment of the list is not the last.
This API is used to query preview data from a given connection type or from a native Amazon S3 based Glue Data Catalog.
Returns records as an array of JSON blobs. Each record is formatted using Jackson JsonNode based on the field type defined by the DescribeEntity API.
Spark connectors generate schemas according to the same data type mapping as in the DescribeEntity API. Spark connectors convert data to the appropriate data types matching the schema when returning rows.
See also: AWS API Documentation
Request Syntax
client.get_entity_records(
ConnectionName='string',
CatalogId='string',
EntityName='string',
NextToken='string',
DataStoreApiVersion='string',
ConnectionOptions={
'string': 'string'
},
FilterPredicate='string',
Limit=123,
OrderBy='string',
SelectedFields=[
'string',
]
)
string
The name of the connection that contains the connection type credentials.
string
The catalog ID of the catalog that contains the connection. This can be null, By default, the Amazon Web Services Account ID is the catalog ID.
string
[REQUIRED]
Name of the entity that we want to query the preview data from the given connection type.
string
A continuation token, included if this is a continuation call.
string
The API version of the SaaS connector.
dict
Connector options that are required to query the data.
(string) --
(string) --
string
A filter predicate that you can apply in the query request.
integer
[REQUIRED]
Limits the number of records fetched with the request.
string
A parameter that orders the response preview data.
list
List of fields that we want to fetch as part of preview data.
(string) --
dict
Response Syntax
{
'Records': [
{...}|[...]|123|123.4|'string'|True|None,
],
'NextToken': 'string'
}
Response Structure
(dict) --
Records (list) --
A list of the requested objects.
(:ref:`document<document>`) --
NextToken (string) --
A continuation token, present if the current segment is not the last.
This API can be used for updating the ResourceProperty of the Glue connection (for the source) or Glue database ARN (for the target). These properties can include the role to access the connection or database. Since the same resource can be used across multiple integrations, updating resource properties will impact all the integrations using it.
See also: AWS API Documentation
Request Syntax
client.update_integration_resource_property(
ResourceArn='string',
SourceProcessingProperties={
'RoleArn': 'string'
},
TargetProcessingProperties={
'RoleArn': 'string',
'KmsArn': 'string',
'ConnectionName': 'string',
'EventBusArn': 'string'
}
)
string
[REQUIRED]
The connection ARN of the source, or the database ARN of the target.
dict
The resource properties associated with the integration source.
RoleArn (string) --
The IAM role to access the Glue connection.
dict
The resource properties associated with the integration target.
RoleArn (string) --
The IAM role to access the Glue database.
KmsArn (string) --
The ARN of the KMS key used for encryption.
ConnectionName (string) --
The Glue network connection to configure the Glue job running in the customer VPC.
EventBusArn (string) --
The ARN of an Eventbridge event bus to receive the integration status notification.
dict
Response Syntax
{
'ResourceArn': 'string',
'SourceProcessingProperties': {
'RoleArn': 'string'
},
'TargetProcessingProperties': {
'RoleArn': 'string',
'KmsArn': 'string',
'ConnectionName': 'string',
'EventBusArn': 'string'
}
}
Response Structure
(dict) --
ResourceArn (string) --
The connection ARN of the source, or the database ARN of the target.
SourceProcessingProperties (dict) --
The resource properties associated with the integration source.
RoleArn (string) --
The IAM role to access the Glue connection.
TargetProcessingProperties (dict) --
The resource properties associated with the integration target.
RoleArn (string) --
The IAM role to access the Glue database.
KmsArn (string) --
The ARN of the KMS key used for encryption.
ConnectionName (string) --
The Glue network connection to configure the Glue job running in the customer VPC.
EventBusArn (string) --
The ARN of an Eventbridge event bus to receive the integration status notification.
Returns the available entities supported by the connection type.
See also: AWS API Documentation
Request Syntax
client.list_entities(
ConnectionName='string',
CatalogId='string',
ParentEntityName='string',
NextToken='string',
DataStoreApiVersion='string'
)
string
A name for the connection that has required credentials to query any connection type.
string
The catalog ID of the catalog that contains the connection. This can be null, By default, the Amazon Web Services Account ID is the catalog ID.
string
Name of the parent entity for which you want to list the children. This parameter takes a fully-qualified path of the entity in order to list the child entities.
string
A continuation token, included if this is a continuation call.
string
The API version of the SaaS connector.
dict
Response Syntax
{
'Entities': [
{
'EntityName': 'string',
'Label': 'string',
'IsParentEntity': True|False,
'Description': 'string',
'Category': 'string',
'CustomProperties': {
'string': 'string'
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
Entities (list) --
A list of Entity objects.
(dict) --
An entity supported by a given ConnectionType.
EntityName (string) --
The name of the entity.
Label (string) --
Label used for the entity.
IsParentEntity (boolean) --
A Boolean value which helps to determine whether there are sub objects that can be listed.
Description (string) --
A description of the entity.
Category (string) --
The type of entities that are present in the response. This value depends on the source connection. For example this is SObjects for Salesforce and databases or schemas or tables for sources like Amazon Redshift.
CustomProperties (dict) --
An optional map of keys which may be returned for an entity by a connector.
(string) --
(string) --
NextToken (string) --
A continuation token, present if the current segment is not the last.
Creates a Zero-ETL integration in the caller's account between two resources with Amazon Resource Names (ARNs): the SourceArn and TargetArn.
See also: AWS API Documentation
Request Syntax
client.create_integration(
IntegrationName='string',
SourceArn='string',
TargetArn='string',
Description='string',
DataFilter='string',
KmsKeyId='string',
AdditionalEncryptionContext={
'string': 'string'
},
Tags=[
{
'key': 'string',
'value': 'string'
},
]
)
string
[REQUIRED]
A unique name for an integration in Glue.
string
[REQUIRED]
The ARN of the source resource for the integration.
string
[REQUIRED]
The ARN of the target resource for the integration.
string
A description of the integration.
string
Selects source tables for the integration using Maxwell filter syntax.
string
The ARN of a KMS key used for encrypting the channel.
dict
An optional set of non-secret key–value pairs that contains additional contextual information for encryption. This can only be provided if KMSKeyId is provided.
(string) --
(string) --
list
Metadata assigned to the resource consisting of a list of key-value pairs.
(dict) --
The Tag object represents a label that you can assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define.
For more information about tags, and controlling access to resources in Glue, see Amazon Web Services Tags in Glue and Specifying Glue Resource ARNs in the developer guide.
key (string) --
The tag key. The key is required when you create a tag on an object. The key is case-sensitive, and must not contain the prefix aws.
value (string) --
The tag value. The value is optional when you create a tag on an object. The value is case-sensitive, and must not contain the prefix aws.
dict
Response Syntax
{
'SourceArn': 'string',
'TargetArn': 'string',
'IntegrationName': 'string',
'Description': 'string',
'IntegrationArn': 'string',
'KmsKeyId': 'string',
'AdditionalEncryptionContext': {
'string': 'string'
},
'Tags': [
{
'key': 'string',
'value': 'string'
},
],
'Status': 'CREATING'|'ACTIVE'|'MODIFYING'|'FAILED'|'DELETING'|'SYNCING'|'NEEDS_ATTENTION',
'CreateTime': datetime(2015, 1, 1),
'Errors': [
{
'ErrorCode': 'string',
'ErrorMessage': 'string'
},
],
'DataFilter': 'string'
}
Response Structure
(dict) --
SourceArn (string) --
The ARN of the source resource for the integration.
TargetArn (string) --
The ARN of the target resource for the integration.
IntegrationName (string) --
A unique name for an integration in Glue.
Description (string) --
A description of the integration.
IntegrationArn (string) --
The Amazon Resource Name (ARN) for the created integration.
KmsKeyId (string) --
The ARN of a KMS key used for encrypting the channel.
AdditionalEncryptionContext (dict) --
An optional set of non-secret key–value pairs that contains additional contextual information for encryption.
(string) --
(string) --
Tags (list) --
Metadata assigned to the resource consisting of a list of key-value pairs.
(dict) --
The Tag object represents a label that you can assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define.
For more information about tags, and controlling access to resources in Glue, see Amazon Web Services Tags in Glue and Specifying Glue Resource ARNs in the developer guide.
key (string) --
The tag key. The key is required when you create a tag on an object. The key is case-sensitive, and must not contain the prefix aws.
value (string) --
The tag value. The value is optional when you create a tag on an object. The value is case-sensitive, and must not contain the prefix aws.
Status (string) --
The status of the integration being created.
The possible statuses are:
CREATING: The integration is being created.
ACTIVE: The integration creation succeeds.
MODIFYING: The integration is being modified.
FAILED: The integration creation fails.
DELETING: The integration is deleted.
SYNCING: The integration is synchronizing.
NEEDS_ATTENTION: The integration needs attention, such as synchronization.
CreateTime (datetime) --
The time when the integration was created, in UTC.
Errors (list) --
A list of errors associated with the integration creation.
(dict) --
An error associated with a zero-ETL integration.
ErrorCode (string) --
The code associated with this error.
ErrorMessage (string) --
A message describing the error.
DataFilter (string) --
Selects source tables for the integration using Maxwell filter syntax.
The ListConnectionTypes API provides a discovery mechanism to learn available connection types in Glue. The response contains a list of connection types with high-level details of what is supported for each connection type. The connection types listed are the set of supported options for the ConnectionType value in the CreateConnection API.
See also: AWS API Documentation
Request Syntax
client.list_connection_types(
MaxResults=123,
NextToken='string'
)
integer
The maximum number of results to return.
string
A continuation token, if this is a continuation call.
dict
Response Syntax
{
'ConnectionTypes': [
{
'ConnectionType': 'JDBC'|'SFTP'|'MONGODB'|'KAFKA'|'NETWORK'|'MARKETPLACE'|'CUSTOM'|'SALESFORCE'|'VIEW_VALIDATION_REDSHIFT'|'VIEW_VALIDATION_ATHENA'|'GOOGLEADS'|'GOOGLESHEETS'|'GOOGLEANALYTICS4'|'SERVICENOW'|'MARKETO'|'SAPODATA'|'ZENDESK'|'JIRACLOUD'|'NETSUITEERP'|'HUBSPOT'|'FACEBOOKADS'|'INSTAGRAMADS'|'ZOHOCRM'|'SALESFORCEPARDOT'|'SALESFORCEMARKETINGCLOUD'|'SLACK'|'STRIPE'|'INTERCOM'|'SNAPCHATADS',
'Description': 'string',
'Capabilities': {
'SupportedAuthenticationTypes': [
'BASIC'|'OAUTH2'|'CUSTOM'|'IAM',
],
'SupportedDataOperations': [
'READ'|'WRITE',
],
'SupportedComputeEnvironments': [
'SPARK'|'ATHENA'|'PYTHON',
]
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
ConnectionTypes (list) --
A list of ConnectionTypeBrief objects containing brief information about the supported connection types.
(dict) --
Brief information about a supported connection type returned by the ListConnectionTypes API.
ConnectionType (string) --
The name of the connection type.
Description (string) --
A description of the connection type.
Capabilities (dict) --
The supported authentication types, data interface types (compute environments), and data operations of the connector.
SupportedAuthenticationTypes (list) --
A list of supported authentication types.
(string) --
SupportedDataOperations (list) --
A list of supported data operations.
(string) --
SupportedComputeEnvironments (list) --
A list of supported compute environments.
(string) --
NextToken (string) --
A continuation token, if the current list segment is not the last.
Creates a new catalog in the Glue Data Catalog.
See also: AWS API Documentation
Request Syntax
client.create_catalog(
Name='string',
CatalogInput={
'Description': 'string',
'FederatedCatalog': {
'Identifier': 'string',
'ConnectionName': 'string'
},
'Parameters': {
'string': 'string'
},
'TargetRedshiftCatalog': {
'CatalogArn': 'string'
},
'CatalogProperties': {
'DataLakeAccessProperties': {
'DataLakeAccess': True|False,
'DataTransferRole': 'string',
'KmsKey': 'string',
'CatalogType': 'string'
},
'CustomProperties': {
'string': 'string'
}
},
'CreateTableDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS',
]
},
],
'CreateDatabaseDefaultPermissions': [
{
'Principal': {
'DataLakePrincipalIdentifier': 'string'
},
'Permissions': [
'ALL'|'SELECT'|'ALTER'|'DROP'|'DELETE'|'INSERT'|'CREATE_DATABASE'|'CREATE_TABLE'|'DATA_LOCATION_ACCESS',
]
},
]
},
Tags={
'string': 'string'
}
)
string
[REQUIRED]
The name of the catalog to create.
dict
[REQUIRED]
A CatalogInput object that defines the metadata for the catalog.
Description (string) --
Description string, not more than 2048 bytes long, matching the URI address multi-line string pattern. A description of the catalog.
FederatedCatalog (dict) --
A FederatedCatalog object. A FederatedCatalog structure that references an entity outside the Glue Data Catalog, for example a Redshift database.
Identifier (string) --
A unique identifier for the federated catalog.
ConnectionName (string) --
The name of the connection to an external data source, for example a Redshift-federated catalog.
Parameters (dict) --
A map array of key-value pairs that define the parameters and properties of the catalog.
(string) --
(string) --
TargetRedshiftCatalog (dict) --
A TargetRedshiftCatalog object that describes a target catalog for resource linking.
CatalogArn (string) -- [REQUIRED]
The Amazon Resource Name (ARN) of the catalog resource.
CatalogProperties (dict) --
A CatalogProperties object that specifies data lake access properties and other custom properties.
DataLakeAccessProperties (dict) --
A DataLakeAccessProperties object that specifies properties to configure data lake access for your catalog resource in the Glue Data Catalog.
DataLakeAccess (boolean) --
Turns on or off data lake access for Apache Spark applications that access Amazon Redshift databases in the Data Catalog from any non-Redshift engine, such as Amazon Athena, Amazon EMR, or Glue ETL.
DataTransferRole (string) --
A role that will be assumed by Glue for transferring data into/out of the staging bucket during a query.
KmsKey (string) --
An encryption key that will be used for the staging bucket that will be created along with the catalog.
CatalogType (string) --
Specifies a federated catalog type for the native catalog resource. The currently supported type is aws:redshift.
CustomProperties (dict) --
Additional key-value properties for the catalog, such as column statistics optimizations.
(string) --
(string) --
CreateTableDefaultPermissions (list) --
An array of PrincipalPermissions objects. Creates a set of default permissions on the table(s) for principals. Used by Amazon Web Services Lake Formation. Typically should be explicitly set as an empty list.
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
CreateDatabaseDefaultPermissions (list) --
An array of PrincipalPermissions objects. Creates a set of default permissions on the database(s) for principals. Used by Amazon Web Services Lake Formation. Typically should be explicitly set as an empty list.
(dict) --
Permissions granted to a principal.
Principal (dict) --
The principal who is granted permissions.
DataLakePrincipalIdentifier (string) --
An identifier for the Lake Formation principal.
Permissions (list) --
The permissions that are granted to the principal.
(string) --
dict
A map array of key-value pairs, not more than 50 pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not more than 256 bytes long. The tags you assign to the catalog.
(string) --
(string) --
dict
Response Syntax
{}
Response Structure
(dict) --
Deletes the table properties that have been created for the tables that need to be replicated.
See also: AWS API Documentation
Request Syntax
client.delete_integration_table_properties(
ResourceArn='string',
TableName='string'
)
string
[REQUIRED]
The connection ARN of the source, or the database ARN of the target.
string
[REQUIRED]
The name of the table to be replicated.
dict
Response Syntax
{}
Response Structure
(dict) --
The DescribeConnectionType API provides full details of the supported options for a given connection type in Glue.
See also: AWS API Documentation
Request Syntax
client.describe_connection_type(
ConnectionType='string'
)
string
[REQUIRED]
The name of the connection type to be described.
dict
Response Syntax
{
'ConnectionType': 'string',
'Description': 'string',
'Capabilities': {
'SupportedAuthenticationTypes': [
'BASIC'|'OAUTH2'|'CUSTOM'|'IAM',
],
'SupportedDataOperations': [
'READ'|'WRITE',
],
'SupportedComputeEnvironments': [
'SPARK'|'ATHENA'|'PYTHON',
]
},
'ConnectionProperties': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
},
'ConnectionOptions': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
},
'AuthenticationConfiguration': {
'AuthenticationType': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
},
'SecretArn': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
},
'OAuth2Properties': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
},
'BasicAuthenticationProperties': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
},
'CustomAuthenticationProperties': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
}
},
'ComputeEnvironmentConfigurations': {
'string': {
'Name': 'string',
'Description': 'string',
'ComputeEnvironment': 'SPARK'|'ATHENA'|'PYTHON',
'SupportedAuthenticationTypes': [
'BASIC'|'OAUTH2'|'CUSTOM'|'IAM',
],
'ConnectionOptions': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
},
'ConnectionPropertyNameOverrides': {
'string': 'string'
},
'ConnectionOptionNameOverrides': {
'string': 'string'
},
'ConnectionPropertiesRequiredOverrides': [
'string',
],
'PhysicalConnectionPropertiesRequired': True|False
}
},
'PhysicalConnectionRequirements': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
},
'AthenaConnectionProperties': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
},
'PythonConnectionProperties': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
},
'SparkConnectionProperties': {
'string': {
'Name': 'string',
'Description': 'string',
'Required': True|False,
'DefaultValue': 'string',
'PropertyTypes': [
'USER_INPUT'|'SECRET'|'READ_ONLY'|'UNUSED'|'SECRET_OR_USER_INPUT',
],
'AllowedValues': [
{
'Description': 'string',
'Value': 'string'
},
],
'DataOperationScopes': [
'READ'|'WRITE',
]
}
}
}
Response Structure
(dict) --
ConnectionType (string) --
The name of the connection type.
Description (string) --
A description of the connection type.
Capabilities (dict) --
The supported authentication types, data interface types (compute environments), and data operations of the connector.
SupportedAuthenticationTypes (list) --
A list of supported authentication types.
(string) --
SupportedDataOperations (list) --
A list of supported data operations.
(string) --
SupportedComputeEnvironments (list) --
A list of supported compute environments.
(string) --
ConnectionProperties (dict) --
Connection properties which are common across compute environments.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
ConnectionOptions (dict) --
Returns properties that can be set when creating a connection in the ConnectionInput.ConnectionProperties. ConnectionOptions defines parameters that can be set in a Spark ETL script in the connection options map passed to a dataframe.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
AuthenticationConfiguration (dict) --
The type of authentication used for the connection.
AuthenticationType (dict) --
The type of authentication for a connection.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
SecretArn (dict) --
The Amazon Resource Name (ARN) for the Secrets Manager.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
OAuth2Properties (dict) --
A map of key-value pairs for the OAuth2 properties. Each value is a a Property object.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
BasicAuthenticationProperties (dict) --
A map of key-value pairs for the OAuth2 properties. Each value is a a Property object.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
CustomAuthenticationProperties (dict) --
A map of key-value pairs for the custom authentication properties. Each value is a a Property object.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
ComputeEnvironmentConfigurations (dict) --
The compute environments that are supported by the connection.
(string) --
(dict) --
An object containing configuration for a compute environment (such as Spark, Python or Athena) returned by the DescribeConnectionType API.
Name (string) --
A name for the compute environment configuration.
Description (string) --
A description of the compute environment.
ComputeEnvironment (string) --
The type of compute environment.
SupportedAuthenticationTypes (list) --
The supported authentication types for the compute environment.
(string) --
ConnectionOptions (dict) --
The parameters used as connection options for the compute environment.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
ConnectionPropertyNameOverrides (dict) --
The connection property name overrides for the compute environment.
(string) --
(string) --
ConnectionOptionNameOverrides (dict) --
The connection option name overrides for the compute environment.
(string) --
(string) --
ConnectionPropertiesRequiredOverrides (list) --
The connection properties that are required as overrides for the compute environment.
(string) --
PhysicalConnectionPropertiesRequired (boolean) --
Indicates whether PhysicalConnectionProperties are required for the compute environment.
PhysicalConnectionRequirements (dict) --
Physical requirements for a connection, such as VPC, Subnet and Security Group specifications.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
AthenaConnectionProperties (dict) --
Connection properties specific to the Athena compute environment.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
PythonConnectionProperties (dict) --
Connection properties specific to the Python compute environment.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
SparkConnectionProperties (dict) --
Connection properties specific to the Spark compute environment.
(string) --
(dict) --
An object that defines a connection type for a compute environment.
Name (string) --
The name of the property.
Description (string) --
A description of the property.
Required (boolean) --
Indicates whether the property is required.
DefaultValue (string) --
The default value for the property.
PropertyTypes (list) --
Describes the type of property.
(string) --
AllowedValues (list) --
A list of AllowedValue objects representing the values allowed for the property.
(dict) --
An object representing a value allowed for a property.
Description (string) --
A description of the allowed value.
Value (string) --
The value allowed for the property.
DataOperationScopes (list) --
Indicates which data operations are applicable to the property.
(string) --
The API is used to retrieve a list of integrations.
See also: AWS API Documentation
Request Syntax
client.describe_integrations(
IntegrationIdentifier='string',
Marker='string',
MaxRecords=123,
Filters=[
{
'Name': 'string',
'Values': [
'string',
]
},
]
)
string
The Amazon Resource Name (ARN) for the integration.
string
A value that indicates the starting point for the next set of response records in a subsequent request.
integer
The total number of items to return in the output.
list
A list of key and values, to filter down the results. Supported keys are "Status", "IntegrationName", and "SourceArn". IntegrationName is limited to only one value.
(dict) --
A filter that can be used when invoking a DescribeIntegrations request.
Name (string) --
The name of the filter.
Values (list) --
A list of filter values.
(string) --
dict
Response Syntax
{
'Integrations': [
{
'SourceArn': 'string',
'TargetArn': 'string',
'Description': 'string',
'IntegrationName': 'string',
'IntegrationArn': 'string',
'KmsKeyId': 'string',
'AdditionalEncryptionContext': {
'string': 'string'
},
'Tags': [
{
'key': 'string',
'value': 'string'
},
],
'Status': 'CREATING'|'ACTIVE'|'MODIFYING'|'FAILED'|'DELETING'|'SYNCING'|'NEEDS_ATTENTION',
'CreateTime': datetime(2015, 1, 1),
'Errors': [
{
'ErrorCode': 'string',
'ErrorMessage': 'string'
},
],
'DataFilter': 'string'
},
],
'Marker': 'string'
}
Response Structure
(dict) --
Integrations (list) --
A list of zero-ETL integrations.
(dict) --
Describes a zero-ETL integration.
SourceArn (string) --
The ARN for the source of the integration.
TargetArn (string) --
The ARN for the target of the integration.
Description (string) --
A description for the integration.
IntegrationName (string) --
A unique name for the integration.
IntegrationArn (string) --
The Amazon Resource Name (ARN) for the integration.
KmsKeyId (string) --
The ARN of a KMS key used for encrypting the channel.
AdditionalEncryptionContext (dict) --
An optional set of non-secret key–value pairs that contains additional contextual information for encryption. This can only be provided if KMSKeyId is provided.
(string) --
(string) --
Tags (list) --
Metadata assigned to the resource consisting of a list of key-value pairs.
(dict) --
The Tag object represents a label that you can assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define.
For more information about tags, and controlling access to resources in Glue, see Amazon Web Services Tags in Glue and Specifying Glue Resource ARNs in the developer guide.
key (string) --
The tag key. The key is required when you create a tag on an object. The key is case-sensitive, and must not contain the prefix aws.
value (string) --
The tag value. The value is optional when you create a tag on an object. The value is case-sensitive, and must not contain the prefix aws.
Status (string) --
The possible statuses are:
CREATING: The integration is being created.
ACTIVE: The integration creation succeeds.
MODIFYING: The integration is being modified.
FAILED: The integration creation fails.
DELETING: The integration is deleted.
SYNCING: The integration is synchronizing.
NEEDS_ATTENTION: The integration needs attention, such as synchronization.
CreateTime (datetime) --
The time that the integration was created, in UTC.
Errors (list) --
A list of errors associated with the integration.
(dict) --
An error associated with a zero-ETL integration.
ErrorCode (string) --
The code associated with this error.
ErrorMessage (string) --
A message describing the error.
DataFilter (string) --
Selects source tables for the integration using Maxwell filter syntax.
Marker (string) --
A value that indicates the starting point for the next set of response records in a subsequent request.
{'ConnectionInput': {'AuthenticationConfiguration': {'AuthenticationType': {'IAM'},
'BasicAuthenticationCredentials': {'Password': 'string',
'Username': 'string'},
'CustomAuthenticationCredentials': {'string': 'string'},
'KmsKeyArn': 'string',
'OAuth2Properties': {'OAuth2Credentials': {'AccessToken': 'string',
'JwtToken': 'string',
'RefreshToken': 'string',
'UserManagedClientApplicationClientSecret': 'string'}}},
'ConnectionType': {'FACEBOOKADS',
'GOOGLEADS',
'GOOGLEANALYTICS4',
'GOOGLESHEETS',
'HUBSPOT',
'INSTAGRAMADS',
'INTERCOM',
'JIRACLOUD',
'MARKETO',
'NETSUITEERP',
'SALESFORCEMARKETINGCLOUD',
'SALESFORCEPARDOT',
'SAPODATA',
'SERVICENOW',
'SLACK',
'SNAPCHATADS',
'STRIPE',
'ZENDESK',
'ZOHOCRM'},
'PythonProperties': {'string': 'string'},
'SparkProperties': {'string': 'string'},
'ValidateForComputeEnvironments': ['SPARK | ATHENA | '
'PYTHON']}}
Creates a connection definition in the Data Catalog.
Connections used for creating federated resources require the IAM glue:PassConnection permission.
See also: AWS API Documentation
Request Syntax
client.create_connection(
CatalogId='string',
ConnectionInput={
'Name': 'string',
'Description': 'string',
'ConnectionType': 'JDBC'|'SFTP'|'MONGODB'|'KAFKA'|'NETWORK'|'MARKETPLACE'|'CUSTOM'|'SALESFORCE'|'VIEW_VALIDATION_REDSHIFT'|'VIEW_VALIDATION_ATHENA'|'GOOGLEADS'|'GOOGLESHEETS'|'GOOGLEANALYTICS4'|'SERVICENOW'|'MARKETO'|'SAPODATA'|'ZENDESK'|'JIRACLOUD'|'NETSUITEERP'|'HUBSPOT'|'FACEBOOKADS'|'INSTAGRAMADS'|'ZOHOCRM'|'SALESFORCEPARDOT'|'SALESFORCEMARKETINGCLOUD'|'SLACK'|'STRIPE'|'INTERCOM'|'SNAPCHATADS',
'MatchCriteria': [
'string',
],
'ConnectionProperties': {
'string': 'string'
},
'SparkProperties': {
'string': 'string'
},
'AthenaProperties': {
'string': 'string'
},
'PythonProperties': {
'string': 'string'
},
'PhysicalConnectionRequirements': {
'SubnetId': 'string',
'SecurityGroupIdList': [
'string',
],
'AvailabilityZone': 'string'
},
'AuthenticationConfiguration': {
'AuthenticationType': 'BASIC'|'OAUTH2'|'CUSTOM'|'IAM',
'OAuth2Properties': {
'OAuth2GrantType': 'AUTHORIZATION_CODE'|'CLIENT_CREDENTIALS'|'JWT_BEARER',
'OAuth2ClientApplication': {
'UserManagedClientApplicationClientId': 'string',
'AWSManagedClientApplicationReference': 'string'
},
'TokenUrl': 'string',
'TokenUrlParametersMap': {
'string': 'string'
},
'AuthorizationCodeProperties': {
'AuthorizationCode': 'string',
'RedirectUri': 'string'
},
'OAuth2Credentials': {
'UserManagedClientApplicationClientSecret': 'string',
'AccessToken': 'string',
'RefreshToken': 'string',
'JwtToken': 'string'
}
},
'SecretArn': 'string',
'KmsKeyArn': 'string',
'BasicAuthenticationCredentials': {
'Username': 'string',
'Password': 'string'
},
'CustomAuthenticationCredentials': {
'string': 'string'
}
},
'ValidateCredentials': True|False,
'ValidateForComputeEnvironments': [
'SPARK'|'ATHENA'|'PYTHON',
]
},
Tags={
'string': 'string'
}
)
string
The ID of the Data Catalog in which to create the connection. If none is provided, the Amazon Web Services account ID is used by default.
dict
[REQUIRED]
A ConnectionInput object defining the connection to create.
Name (string) -- [REQUIRED]
The name of the connection.
Description (string) --
The description of the connection.
ConnectionType (string) -- [REQUIRED]
The type of the connection. Currently, these types are supported:
JDBC - Designates a connection to a database through Java Database Connectivity (JDBC). JDBC Connections use the following ConnectionParameters.
Required: All of ( HOST, PORT, JDBC_ENGINE) or JDBC_CONNECTION_URL.
Required: All of ( USERNAME, PASSWORD) or SECRET_ID.
Optional: JDBC_ENFORCE_SSL, CUSTOM_JDBC_CERT, CUSTOM_JDBC_CERT_STRING, SKIP_CUSTOM_JDBC_CERT_VALIDATION. These parameters are used to configure SSL with JDBC.
KAFKA - Designates a connection to an Apache Kafka streaming platform. KAFKA Connections use the following ConnectionParameters.
Required: KAFKA_BOOTSTRAP_SERVERS.
Optional: KAFKA_SSL_ENABLED, KAFKA_CUSTOM_CERT, KAFKA_SKIP_CUSTOM_CERT_VALIDATION. These parameters are used to configure SSL with KAFKA.
Optional: KAFKA_CLIENT_KEYSTORE, KAFKA_CLIENT_KEYSTORE_PASSWORD, KAFKA_CLIENT_KEY_PASSWORD, ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD, ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD. These parameters are used to configure TLS client configuration with SSL in KAFKA.
Optional: KAFKA_SASL_MECHANISM. Can be specified as SCRAM-SHA-512, GSSAPI, or AWS_MSK_IAM.
Optional: KAFKA_SASL_SCRAM_USERNAME, KAFKA_SASL_SCRAM_PASSWORD, ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD. These parameters are used to configure SASL/SCRAM-SHA-512 authentication with KAFKA.
Optional: KAFKA_SASL_GSSAPI_KEYTAB, KAFKA_SASL_GSSAPI_KRB5_CONF, KAFKA_SASL_GSSAPI_SERVICE, KAFKA_SASL_GSSAPI_PRINCIPAL. These parameters are used to configure SASL/GSSAPI authentication with KAFKA.
MONGODB - Designates a connection to a MongoDB document database. MONGODB Connections use the following ConnectionParameters.
Required: CONNECTION_URL.
Required: All of ( USERNAME, PASSWORD) or SECRET_ID.
VIEW_VALIDATION_REDSHIFT - Designates a connection used for view validation by Amazon Redshift.
VIEW_VALIDATION_ATHENA - Designates a connection used for view validation by Amazon Athena.
NETWORK - Designates a network connection to a data source within an Amazon Virtual Private Cloud environment (Amazon VPC). NETWORK Connections do not require ConnectionParameters. Instead, provide a PhysicalConnectionRequirements.
MARKETPLACE - Uses configuration settings contained in a connector purchased from Amazon Web Services Marketplace to read from and write to data stores that are not natively supported by Glue. MARKETPLACE Connections use the following ConnectionParameters.
Required: CONNECTOR_TYPE, CONNECTOR_URL, CONNECTOR_CLASS_NAME, CONNECTION_URL.
Required for JDBC CONNECTOR_TYPE connections: All of ( USERNAME, PASSWORD) or SECRET_ID.
CUSTOM - Uses configuration settings contained in a custom connector to read from and write to data stores that are not natively supported by Glue.
Additionally, a ConnectionType for the following SaaS connectors is supported:
FACEBOOKADS - Designates a connection to Facebook Ads.
GOOGLEADS - Designates a connection to Google Ads.
GOOGLESHEETS - Designates a connection to Google Sheets.
GOOGLEANALYTICS4 - Designates a connection to Google Analytics 4.
HUBSPOT - Designates a connection to HubSpot.
INSTAGRAMADS - Designates a connection to Instagram Ads.
INTERCOM - Designates a connection to Intercom.
JIRACLOUD - Designates a connection to Jira Cloud.
MARKETO - Designates a connection to Adobe Marketo Engage.
NETSUITEERP - Designates a connection to Oracle NetSuite.
SALESFORCE - Designates a connection to Salesforce using OAuth authentication.
SALESFORCEMARKETINGCLOUD - Designates a connection to Salesforce Marketing Cloud.
SALESFORCEPARDOT - Designates a connection to Salesforce Marketing Cloud Account Engagement (MCAE).
SAPODATA - Designates a connection to SAP OData.
SERVICENOW - Designates a connection to ServiceNow.
SLACK - Designates a connection to Slack.
SNAPCHATADS - Designates a connection to Snapchat Ads.
STRIPE - Designates a connection to Stripe.
ZENDESK - Designates a connection to Zendesk.
ZOHOCRM - Designates a connection to Zoho CRM.
For more information on the connection parameters needed for a particular connector, see the documentation for the connector in `Adding an Glue connection <https://docs.aws.amazon.com/glue/latest/dg/console-connections.html>`__in the Glue User Guide.
SFTP is not supported.
For more information about how optional ConnectionProperties are used to configure features in Glue, consult Glue connection properties.
For more information about how optional ConnectionProperties are used to configure features in Glue Studio, consult Using connectors and connections.
MatchCriteria (list) --
A list of criteria that can be used in selecting this connection.
(string) --
ConnectionProperties (dict) -- [REQUIRED]
These key-value pairs define parameters for the connection.
(string) --
(string) --
SparkProperties (dict) --
Connection properties specific to the Spark compute environment.
(string) --
(string) --
AthenaProperties (dict) --
Connection properties specific to the Athena compute environment.
(string) --
(string) --
PythonProperties (dict) --
Connection properties specific to the Python compute environment.
(string) --
(string) --
PhysicalConnectionRequirements (dict) --
The physical connection requirements, such as virtual private cloud (VPC) and SecurityGroup, that are needed to successfully make this connection.
SubnetId (string) --
The subnet ID used by the connection.
SecurityGroupIdList (list) --
The security group ID list used by the connection.
(string) --
AvailabilityZone (string) --
The connection's Availability Zone.
AuthenticationConfiguration (dict) --
The authentication properties of the connection.
AuthenticationType (string) --
A structure containing the authentication configuration in the CreateConnection request.
OAuth2Properties (dict) --
The properties for OAuth2 authentication in the CreateConnection request.
OAuth2GrantType (string) --
The OAuth2 grant type in the CreateConnection request. For example, AUTHORIZATION_CODE, JWT_BEARER, or CLIENT_CREDENTIALS.
OAuth2ClientApplication (dict) --
The client application type in the CreateConnection request. For example, AWS_MANAGED or USER_MANAGED.
UserManagedClientApplicationClientId (string) --
The client application clientID if the ClientAppType is USER_MANAGED.
AWSManagedClientApplicationReference (string) --
The reference to the SaaS-side client app that is Amazon Web Services managed.
TokenUrl (string) --
The URL of the provider's authentication server, to exchange an authorization code for an access token.
TokenUrlParametersMap (dict) --
A map of parameters that are added to the token GET request.
(string) --
(string) --
AuthorizationCodeProperties (dict) --
The set of properties required for the the OAuth2 AUTHORIZATION_CODE grant type.
AuthorizationCode (string) --
An authorization code to be used in the third leg of the AUTHORIZATION_CODE grant workflow. This is a single-use code which becomes invalid once exchanged for an access token, thus it is acceptable to have this value as a request parameter.
RedirectUri (string) --
The redirect URI where the user gets redirected to by authorization server when issuing an authorization code. The URI is subsequently used when the authorization code is exchanged for an access token.
OAuth2Credentials (dict) --
The credentials used when the authentication type is OAuth2 authentication.
UserManagedClientApplicationClientSecret (string) --
The client application client secret if the client application is user managed.
AccessToken (string) --
The access token used when the authentication type is OAuth2.
RefreshToken (string) --
The refresh token used when the authentication type is OAuth2.
JwtToken (string) --
The JSON Web Token (JWT) used when the authentication type is OAuth2.
SecretArn (string) --
The secret manager ARN to store credentials in the CreateConnection request.
KmsKeyArn (string) --
The ARN of the KMS key used to encrypt the connection. Only taken an as input in the request and stored in the Secret Manager.
BasicAuthenticationCredentials (dict) --
The credentials used when the authentication type is basic authentication.
Username (string) --
The username to connect to the data source.
Password (string) --
The password to connect to the data source.
CustomAuthenticationCredentials (dict) --
The credentials used when the authentication type is custom authentication.
(string) --
(string) --
ValidateCredentials (boolean) --
A flag to validate the credentials during create connection. Default is true.
ValidateForComputeEnvironments (list) --
The compute environments that the specified connection properties are validated against.
(string) --
dict
The tags you assign to the connection.
(string) --
(string) --
dict
Response Syntax
{
'CreateConnectionStatus': 'READY'|'IN_PROGRESS'|'FAILED'
}
Response Structure
(dict) --
CreateConnectionStatus (string) --
The status of the connection creation request. The request can take some time for certain authentication types, for example when creating an OAuth connection with token exchange over VPC.
{'ColumnStatisticsTaskSettings': {'LastExecutionAttempt': {'ColumnStatisticsTaskRunId': 'string',
'ErrorMessage': 'string',
'ExecutionTimestamp': 'timestamp',
'Status': 'FAILED | '
'STARTED'},
'ScheduleType': 'CRON | AUTO',
'SettingSource': 'CATALOG | TABLE'}}
Gets settings for a column statistics task.
See also: AWS API Documentation
Request Syntax
client.get_column_statistics_task_settings(
DatabaseName='string',
TableName='string'
)
string
[REQUIRED]
The name of the database where the table resides.
string
[REQUIRED]
The name of the table for which to retrieve column statistics.
dict
Response Syntax
{
'ColumnStatisticsTaskSettings': {
'DatabaseName': 'string',
'TableName': 'string',
'Schedule': {
'ScheduleExpression': 'string',
'State': 'SCHEDULED'|'NOT_SCHEDULED'|'TRANSITIONING'
},
'ColumnNameList': [
'string',
],
'CatalogID': 'string',
'Role': 'string',
'SampleSize': 123.0,
'SecurityConfiguration': 'string',
'ScheduleType': 'CRON'|'AUTO',
'SettingSource': 'CATALOG'|'TABLE',
'LastExecutionAttempt': {
'Status': 'FAILED'|'STARTED',
'ColumnStatisticsTaskRunId': 'string',
'ExecutionTimestamp': datetime(2015, 1, 1),
'ErrorMessage': 'string'
}
}
}
Response Structure
(dict) --
ColumnStatisticsTaskSettings (dict) --
A ColumnStatisticsTaskSettings object representing the settings for the column statistics task.
DatabaseName (string) --
The name of the database where the table resides.
TableName (string) --
The name of the table for which to generate column statistics.
Schedule (dict) --
A schedule for running the column statistics, specified in CRON syntax.
ScheduleExpression (string) --
A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).
State (string) --
The state of the schedule.
ColumnNameList (list) --
A list of column names for which to run statistics.
(string) --
CatalogID (string) --
The ID of the Data Catalog in which the database resides.
Role (string) --
The role used for running the column statistics.
SampleSize (float) --
The percentage of data to sample.
SecurityConfiguration (string) --
Name of the security configuration that is used to encrypt CloudWatch logs.
ScheduleType (string) --
The type of schedule for a column statistics task. Possible values may be CRON or AUTO.
SettingSource (string) --
The source of setting the column statistics task. Possible values may be CATALOG or TABLE.
LastExecutionAttempt (dict) --
The last ExecutionAttempt for the column statistics task run.
Status (string) --
The status of the last column statistics task run.
ColumnStatisticsTaskRunId (string) --
A task run ID for the last column statistics task run.
ExecutionTimestamp (datetime) --
A timestamp when the last column statistics task run occurred.
ErrorMessage (string) --
An error message associated with the last column statistics task run.
{'ApplyOverrideForComputeEnvironment': 'SPARK | ATHENA | PYTHON'}
Response {'Connection': {'AuthenticationConfiguration': {'AuthenticationType': {'IAM'}},
'CompatibleComputeEnvironments': ['SPARK | ATHENA | PYTHON'],
'ConnectionSchemaVersion': 'integer',
'ConnectionType': {'FACEBOOKADS',
'GOOGLEADS',
'GOOGLEANALYTICS4',
'GOOGLESHEETS',
'HUBSPOT',
'INSTAGRAMADS',
'INTERCOM',
'JIRACLOUD',
'MARKETO',
'NETSUITEERP',
'SALESFORCEMARKETINGCLOUD',
'SALESFORCEPARDOT',
'SAPODATA',
'SERVICENOW',
'SLACK',
'SNAPCHATADS',
'STRIPE',
'ZENDESK',
'ZOHOCRM'},
'PythonProperties': {'string': 'string'},
'SparkProperties': {'string': 'string'}}}
Retrieves a connection definition from the Data Catalog.
See also: AWS API Documentation
Request Syntax
client.get_connection(
CatalogId='string',
Name='string',
HidePassword=True|False,
ApplyOverrideForComputeEnvironment='SPARK'|'ATHENA'|'PYTHON'
)
string
The ID of the Data Catalog in which the connection resides. If none is provided, the Amazon Web Services account ID is used by default.
string
[REQUIRED]
The name of the connection definition to retrieve.
boolean
Allows you to retrieve the connection metadata without returning the password. For instance, the Glue console uses this flag to retrieve the connection, and does not display the password. Set this parameter when the caller might not have permission to use the KMS key to decrypt the password, but it does have permission to access the rest of the connection properties.
string
For connections that may be used in multiple services, specifies returning properties for the specified compute environment.
dict
Response Syntax
{
'Connection': {
'Name': 'string',
'Description': 'string',
'ConnectionType': 'JDBC'|'SFTP'|'MONGODB'|'KAFKA'|'NETWORK'|'MARKETPLACE'|'CUSTOM'|'SALESFORCE'|'VIEW_VALIDATION_REDSHIFT'|'VIEW_VALIDATION_ATHENA'|'GOOGLEADS'|'GOOGLESHEETS'|'GOOGLEANALYTICS4'|'SERVICENOW'|'MARKETO'|'SAPODATA'|'ZENDESK'|'JIRACLOUD'|'NETSUITEERP'|'HUBSPOT'|'FACEBOOKADS'|'INSTAGRAMADS'|'ZOHOCRM'|'SALESFORCEPARDOT'|'SALESFORCEMARKETINGCLOUD'|'SLACK'|'STRIPE'|'INTERCOM'|'SNAPCHATADS',
'MatchCriteria': [
'string',
],
'ConnectionProperties': {
'string': 'string'
},
'SparkProperties': {
'string': 'string'
},
'AthenaProperties': {
'string': 'string'
},
'PythonProperties': {
'string': 'string'
},
'PhysicalConnectionRequirements': {
'SubnetId': 'string',
'SecurityGroupIdList': [
'string',
],
'AvailabilityZone': 'string'
},
'CreationTime': datetime(2015, 1, 1),
'LastUpdatedTime': datetime(2015, 1, 1),
'LastUpdatedBy': 'string',
'Status': 'READY'|'IN_PROGRESS'|'FAILED',
'StatusReason': 'string',
'LastConnectionValidationTime': datetime(2015, 1, 1),
'AuthenticationConfiguration': {
'AuthenticationType': 'BASIC'|'OAUTH2'|'CUSTOM'|'IAM',
'SecretArn': 'string',
'OAuth2Properties': {
'OAuth2GrantType': 'AUTHORIZATION_CODE'|'CLIENT_CREDENTIALS'|'JWT_BEARER',
'OAuth2ClientApplication': {
'UserManagedClientApplicationClientId': 'string',
'AWSManagedClientApplicationReference': 'string'
},
'TokenUrl': 'string',
'TokenUrlParametersMap': {
'string': 'string'
}
}
},
'ConnectionSchemaVersion': 123,
'CompatibleComputeEnvironments': [
'SPARK'|'ATHENA'|'PYTHON',
]
}
}
Response Structure
(dict) --
Connection (dict) --
The requested connection definition.
Name (string) --
The name of the connection definition.
Description (string) --
The description of the connection.
ConnectionType (string) --
The type of the connection. Currently, SFTP is not supported.
MatchCriteria (list) --
A list of criteria that can be used in selecting this connection.
(string) --
ConnectionProperties (dict) --
These key-value pairs define parameters for the connection when using the version 1 Connection schema:
HOST - The host URI: either the fully qualified domain name (FQDN) or the IPv4 address of the database host.
PORT - The port number, between 1024 and 65535, of the port on which the database host is listening for database connections.
USER_NAME - The name under which to log in to the database. The value string for USER_NAME is " USERNAME".
PASSWORD - A password, if one is used, for the user name.
ENCRYPTED_PASSWORD - When you enable connection password protection by setting ConnectionPasswordEncryption in the Data Catalog encryption settings, this field stores the encrypted password.
JDBC_DRIVER_JAR_URI - The Amazon Simple Storage Service (Amazon S3) path of the JAR file that contains the JDBC driver to use.
JDBC_DRIVER_CLASS_NAME - The class name of the JDBC driver to use.
JDBC_ENGINE - The name of the JDBC engine to use.
JDBC_ENGINE_VERSION - The version of the JDBC engine to use.
CONFIG_FILES - (Reserved for future use.)
INSTANCE_ID - The instance ID to use.
JDBC_CONNECTION_URL - The URL for connecting to a JDBC data source.
JDBC_ENFORCE_SSL - A Boolean string (true, false) specifying whether Secure Sockets Layer (SSL) with hostname matching is enforced for the JDBC connection on the client. The default is false.
CUSTOM_JDBC_CERT - An Amazon S3 location specifying the customer's root certificate. Glue uses this root certificate to validate the customer’s certificate when connecting to the customer database. Glue only handles X.509 certificates. The certificate provided must be DER-encoded and supplied in Base64 encoding PEM format.
SKIP_CUSTOM_JDBC_CERT_VALIDATION - By default, this is false. Glue validates the Signature algorithm and Subject Public Key Algorithm for the customer certificate. The only permitted algorithms for the Signature algorithm are SHA256withRSA, SHA384withRSA or SHA512withRSA. For the Subject Public Key Algorithm, the key length must be at least 2048. You can set the value of this property to true to skip Glue’s validation of the customer certificate.
CUSTOM_JDBC_CERT_STRING - A custom JDBC certificate string which is used for domain match or distinguished name match to prevent a man-in-the-middle attack. In Oracle database, this is used as the SSL_SERVER_CERT_DN; in Microsoft SQL Server, this is used as the hostNameInCertificate.
CONNECTION_URL - The URL for connecting to a general (non-JDBC) data source.
SECRET_ID - The secret ID used for the secret manager of credentials.
CONNECTOR_URL - The connector URL for a MARKETPLACE or CUSTOM connection.
CONNECTOR_TYPE - The connector type for a MARKETPLACE or CUSTOM connection.
CONNECTOR_CLASS_NAME - The connector class name for a MARKETPLACE or CUSTOM connection.
KAFKA_BOOTSTRAP_SERVERS - A comma-separated list of host and port pairs that are the addresses of the Apache Kafka brokers in a Kafka cluster to which a Kafka client will connect to and bootstrap itself.
KAFKA_SSL_ENABLED - Whether to enable or disable SSL on an Apache Kafka connection. Default value is "true".
KAFKA_CUSTOM_CERT - The Amazon S3 URL for the private CA cert file (.pem format). The default is an empty string.
KAFKA_SKIP_CUSTOM_CERT_VALIDATION - Whether to skip the validation of the CA cert file or not. Glue validates for three algorithms: SHA256withRSA, SHA384withRSA and SHA512withRSA. Default value is "false".
KAFKA_CLIENT_KEYSTORE - The Amazon S3 location of the client keystore file for Kafka client side authentication (Optional).
KAFKA_CLIENT_KEYSTORE_PASSWORD - The password to access the provided keystore (Optional).
KAFKA_CLIENT_KEY_PASSWORD - A keystore can consist of multiple keys, so this is the password to access the client key to be used with the Kafka server side key (Optional).
ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD - The encrypted version of the Kafka client keystore password (if the user has the Glue encrypt passwords setting selected).
ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD - The encrypted version of the Kafka client key password (if the user has the Glue encrypt passwords setting selected).
KAFKA_SASL_MECHANISM - "SCRAM-SHA-512", "GSSAPI", "AWS_MSK_IAM", or "PLAIN". These are the supported SASL Mechanisms.
KAFKA_SASL_PLAIN_USERNAME - A plaintext username used to authenticate with the "PLAIN" mechanism.
KAFKA_SASL_PLAIN_PASSWORD - A plaintext password used to authenticate with the "PLAIN" mechanism.
ENCRYPTED_KAFKA_SASL_PLAIN_PASSWORD - The encrypted version of the Kafka SASL PLAIN password (if the user has the Glue encrypt passwords setting selected).
KAFKA_SASL_SCRAM_USERNAME - A plaintext username used to authenticate with the "SCRAM-SHA-512" mechanism.
KAFKA_SASL_SCRAM_PASSWORD - A plaintext password used to authenticate with the "SCRAM-SHA-512" mechanism.
ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD - The encrypted version of the Kafka SASL SCRAM password (if the user has the Glue encrypt passwords setting selected).
KAFKA_SASL_SCRAM_SECRETS_ARN - The Amazon Resource Name of a secret in Amazon Web Services Secrets Manager.
KAFKA_SASL_GSSAPI_KEYTAB - The S3 location of a Kerberos keytab file. A keytab stores long-term keys for one or more principals. For more information, see MIT Kerberos Documentation: Keytab.
KAFKA_SASL_GSSAPI_KRB5_CONF - The S3 location of a Kerberos krb5.conf file. A krb5.conf stores Kerberos configuration information, such as the location of the KDC server. For more information, see MIT Kerberos Documentation: krb5.conf.
KAFKA_SASL_GSSAPI_SERVICE - The Kerberos service name, as set with sasl.kerberos.service.name in your Kafka Configuration.
KAFKA_SASL_GSSAPI_PRINCIPAL - The name of the Kerberos princial used by Glue. For more information, see Kafka Documentation: Configuring Kafka Brokers.
ROLE_ARN - The role to be used for running queries.
REGION - The Amazon Web Services Region where queries will be run.
WORKGROUP_NAME - The name of an Amazon Redshift serverless workgroup or Amazon Athena workgroup in which queries will run.
CLUSTER_IDENTIFIER - The cluster identifier of an Amazon Redshift cluster in which queries will run.
DATABASE - The Amazon Redshift database that you are connecting to.
(string) --
(string) --
SparkProperties (dict) --
Connection properties specific to the Spark compute environment.
(string) --
(string) --
AthenaProperties (dict) --
Connection properties specific to the Athena compute environment.
(string) --
(string) --
PythonProperties (dict) --
Connection properties specific to the Python compute environment.
(string) --
(string) --
PhysicalConnectionRequirements (dict) --
The physical connection requirements, such as virtual private cloud (VPC) and SecurityGroup, that are needed to make this connection successfully.
SubnetId (string) --
The subnet ID used by the connection.
SecurityGroupIdList (list) --
The security group ID list used by the connection.
(string) --
AvailabilityZone (string) --
The connection's Availability Zone.
CreationTime (datetime) --
The timestamp of the time that this connection definition was created.
LastUpdatedTime (datetime) --
The timestamp of the last time the connection definition was updated.
LastUpdatedBy (string) --
The user, group, or role that last updated this connection definition.
Status (string) --
The status of the connection. Can be one of: READY, IN_PROGRESS, or FAILED.
StatusReason (string) --
The reason for the connection status.
LastConnectionValidationTime (datetime) --
A timestamp of the time this connection was last validated.
AuthenticationConfiguration (dict) --
The authentication properties of the connection.
AuthenticationType (string) --
A structure containing the authentication configuration.
SecretArn (string) --
The secret manager ARN to store credentials.
OAuth2Properties (dict) --
The properties for OAuth2 authentication.
OAuth2GrantType (string) --
The OAuth2 grant type. For example, AUTHORIZATION_CODE, JWT_BEARER, or CLIENT_CREDENTIALS.
OAuth2ClientApplication (dict) --
The client application type. For example, AWS_MANAGED or USER_MANAGED.
UserManagedClientApplicationClientId (string) --
The client application clientID if the ClientAppType is USER_MANAGED.
AWSManagedClientApplicationReference (string) --
The reference to the SaaS-side client app that is Amazon Web Services managed.
TokenUrl (string) --
The URL of the provider's authentication server, to exchange an authorization code for an access token.
TokenUrlParametersMap (dict) --
A map of parameters that are added to the token GET request.
(string) --
(string) --
ConnectionSchemaVersion (integer) --
The version of the connection schema for this connection. Version 2 supports properties for specific compute environments.
CompatibleComputeEnvironments (list) --
A list of compute environments compatible with the connection.
(string) --
{'Filter': {'ConnectionSchemaVersion': 'integer',
'ConnectionType': {'FACEBOOKADS',
'GOOGLEADS',
'GOOGLEANALYTICS4',
'GOOGLESHEETS',
'HUBSPOT',
'INSTAGRAMADS',
'INTERCOM',
'JIRACLOUD',
'MARKETO',
'NETSUITEERP',
'SALESFORCEMARKETINGCLOUD',
'SALESFORCEPARDOT',
'SAPODATA',
'SERVICENOW',
'SLACK',
'SNAPCHATADS',
'STRIPE',
'ZENDESK',
'ZOHOCRM'}}}
Response {'ConnectionList': {'AuthenticationConfiguration': {'AuthenticationType': {'IAM'}},
'CompatibleComputeEnvironments': ['SPARK | ATHENA | '
'PYTHON'],
'ConnectionSchemaVersion': 'integer',
'ConnectionType': {'FACEBOOKADS',
'GOOGLEADS',
'GOOGLEANALYTICS4',
'GOOGLESHEETS',
'HUBSPOT',
'INSTAGRAMADS',
'INTERCOM',
'JIRACLOUD',
'MARKETO',
'NETSUITEERP',
'SALESFORCEMARKETINGCLOUD',
'SALESFORCEPARDOT',
'SAPODATA',
'SERVICENOW',
'SLACK',
'SNAPCHATADS',
'STRIPE',
'ZENDESK',
'ZOHOCRM'},
'PythonProperties': {'string': 'string'},
'SparkProperties': {'string': 'string'}}}
Retrieves a list of connection definitions from the Data Catalog.
See also: AWS API Documentation
Request Syntax
client.get_connections(
CatalogId='string',
Filter={
'MatchCriteria': [
'string',
],
'ConnectionType': 'JDBC'|'SFTP'|'MONGODB'|'KAFKA'|'NETWORK'|'MARKETPLACE'|'CUSTOM'|'SALESFORCE'|'VIEW_VALIDATION_REDSHIFT'|'VIEW_VALIDATION_ATHENA'|'GOOGLEADS'|'GOOGLESHEETS'|'GOOGLEANALYTICS4'|'SERVICENOW'|'MARKETO'|'SAPODATA'|'ZENDESK'|'JIRACLOUD'|'NETSUITEERP'|'HUBSPOT'|'FACEBOOKADS'|'INSTAGRAMADS'|'ZOHOCRM'|'SALESFORCEPARDOT'|'SALESFORCEMARKETINGCLOUD'|'SLACK'|'STRIPE'|'INTERCOM'|'SNAPCHATADS',
'ConnectionSchemaVersion': 123
},
HidePassword=True|False,
NextToken='string',
MaxResults=123
)
string
The ID of the Data Catalog in which the connections reside. If none is provided, the Amazon Web Services account ID is used by default.
dict
A filter that controls which connections are returned.
MatchCriteria (list) --
A criteria string that must match the criteria recorded in the connection definition for that connection definition to be returned.
(string) --
ConnectionType (string) --
The type of connections to return. Currently, SFTP is not supported.
ConnectionSchemaVersion (integer) --
Denotes if the connection was created with schema version 1 or 2.
boolean
Allows you to retrieve the connection metadata without returning the password. For instance, the Glue console uses this flag to retrieve the connection, and does not display the password. Set this parameter when the caller might not have permission to use the KMS key to decrypt the password, but it does have permission to access the rest of the connection properties.
string
A continuation token, if this is a continuation call.
integer
The maximum number of connections to return in one response.
dict
Response Syntax
{
'ConnectionList': [
{
'Name': 'string',
'Description': 'string',
'ConnectionType': 'JDBC'|'SFTP'|'MONGODB'|'KAFKA'|'NETWORK'|'MARKETPLACE'|'CUSTOM'|'SALESFORCE'|'VIEW_VALIDATION_REDSHIFT'|'VIEW_VALIDATION_ATHENA'|'GOOGLEADS'|'GOOGLESHEETS'|'GOOGLEANALYTICS4'|'SERVICENOW'|'MARKETO'|'SAPODATA'|'ZENDESK'|'JIRACLOUD'|'NETSUITEERP'|'HUBSPOT'|'FACEBOOKADS'|'INSTAGRAMADS'|'ZOHOCRM'|'SALESFORCEPARDOT'|'SALESFORCEMARKETINGCLOUD'|'SLACK'|'STRIPE'|'INTERCOM'|'SNAPCHATADS',
'MatchCriteria': [
'string',
],
'ConnectionProperties': {
'string': 'string'
},
'SparkProperties': {
'string': 'string'
},
'AthenaProperties': {
'string': 'string'
},
'PythonProperties': {
'string': 'string'
},
'PhysicalConnectionRequirements': {
'SubnetId': 'string',
'SecurityGroupIdList': [
'string',
],
'AvailabilityZone': 'string'
},
'CreationTime': datetime(2015, 1, 1),
'LastUpdatedTime': datetime(2015, 1, 1),
'LastUpdatedBy': 'string',
'Status': 'READY'|'IN_PROGRESS'|'FAILED',
'StatusReason': 'string',
'LastConnectionValidationTime': datetime(2015, 1, 1),
'AuthenticationConfiguration': {
'AuthenticationType': 'BASIC'|'OAUTH2'|'CUSTOM'|'IAM',
'SecretArn': 'string',
'OAuth2Properties': {
'OAuth2GrantType': 'AUTHORIZATION_CODE'|'CLIENT_CREDENTIALS'|'JWT_BEARER',
'OAuth2ClientApplication': {
'UserManagedClientApplicationClientId': 'string',
'AWSManagedClientApplicationReference': 'string'
},
'TokenUrl': 'string',
'TokenUrlParametersMap': {
'string': 'string'
}
}
},
'ConnectionSchemaVersion': 123,
'CompatibleComputeEnvironments': [
'SPARK'|'ATHENA'|'PYTHON',
]
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
ConnectionList (list) --
A list of requested connection definitions.
(dict) --
Defines a connection to a data source.
Name (string) --
The name of the connection definition.
Description (string) --
The description of the connection.
ConnectionType (string) --
The type of the connection. Currently, SFTP is not supported.
MatchCriteria (list) --
A list of criteria that can be used in selecting this connection.
(string) --
ConnectionProperties (dict) --
These key-value pairs define parameters for the connection when using the version 1 Connection schema:
HOST - The host URI: either the fully qualified domain name (FQDN) or the IPv4 address of the database host.
PORT - The port number, between 1024 and 65535, of the port on which the database host is listening for database connections.
USER_NAME - The name under which to log in to the database. The value string for USER_NAME is " USERNAME".
PASSWORD - A password, if one is used, for the user name.
ENCRYPTED_PASSWORD - When you enable connection password protection by setting ConnectionPasswordEncryption in the Data Catalog encryption settings, this field stores the encrypted password.
JDBC_DRIVER_JAR_URI - The Amazon Simple Storage Service (Amazon S3) path of the JAR file that contains the JDBC driver to use.
JDBC_DRIVER_CLASS_NAME - The class name of the JDBC driver to use.
JDBC_ENGINE - The name of the JDBC engine to use.
JDBC_ENGINE_VERSION - The version of the JDBC engine to use.
CONFIG_FILES - (Reserved for future use.)
INSTANCE_ID - The instance ID to use.
JDBC_CONNECTION_URL - The URL for connecting to a JDBC data source.
JDBC_ENFORCE_SSL - A Boolean string (true, false) specifying whether Secure Sockets Layer (SSL) with hostname matching is enforced for the JDBC connection on the client. The default is false.
CUSTOM_JDBC_CERT - An Amazon S3 location specifying the customer's root certificate. Glue uses this root certificate to validate the customer’s certificate when connecting to the customer database. Glue only handles X.509 certificates. The certificate provided must be DER-encoded and supplied in Base64 encoding PEM format.
SKIP_CUSTOM_JDBC_CERT_VALIDATION - By default, this is false. Glue validates the Signature algorithm and Subject Public Key Algorithm for the customer certificate. The only permitted algorithms for the Signature algorithm are SHA256withRSA, SHA384withRSA or SHA512withRSA. For the Subject Public Key Algorithm, the key length must be at least 2048. You can set the value of this property to true to skip Glue’s validation of the customer certificate.
CUSTOM_JDBC_CERT_STRING - A custom JDBC certificate string which is used for domain match or distinguished name match to prevent a man-in-the-middle attack. In Oracle database, this is used as the SSL_SERVER_CERT_DN; in Microsoft SQL Server, this is used as the hostNameInCertificate.
CONNECTION_URL - The URL for connecting to a general (non-JDBC) data source.
SECRET_ID - The secret ID used for the secret manager of credentials.
CONNECTOR_URL - The connector URL for a MARKETPLACE or CUSTOM connection.
CONNECTOR_TYPE - The connector type for a MARKETPLACE or CUSTOM connection.
CONNECTOR_CLASS_NAME - The connector class name for a MARKETPLACE or CUSTOM connection.
KAFKA_BOOTSTRAP_SERVERS - A comma-separated list of host and port pairs that are the addresses of the Apache Kafka brokers in a Kafka cluster to which a Kafka client will connect to and bootstrap itself.
KAFKA_SSL_ENABLED - Whether to enable or disable SSL on an Apache Kafka connection. Default value is "true".
KAFKA_CUSTOM_CERT - The Amazon S3 URL for the private CA cert file (.pem format). The default is an empty string.
KAFKA_SKIP_CUSTOM_CERT_VALIDATION - Whether to skip the validation of the CA cert file or not. Glue validates for three algorithms: SHA256withRSA, SHA384withRSA and SHA512withRSA. Default value is "false".
KAFKA_CLIENT_KEYSTORE - The Amazon S3 location of the client keystore file for Kafka client side authentication (Optional).
KAFKA_CLIENT_KEYSTORE_PASSWORD - The password to access the provided keystore (Optional).
KAFKA_CLIENT_KEY_PASSWORD - A keystore can consist of multiple keys, so this is the password to access the client key to be used with the Kafka server side key (Optional).
ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD - The encrypted version of the Kafka client keystore password (if the user has the Glue encrypt passwords setting selected).
ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD - The encrypted version of the Kafka client key password (if the user has the Glue encrypt passwords setting selected).
KAFKA_SASL_MECHANISM - "SCRAM-SHA-512", "GSSAPI", "AWS_MSK_IAM", or "PLAIN". These are the supported SASL Mechanisms.
KAFKA_SASL_PLAIN_USERNAME - A plaintext username used to authenticate with the "PLAIN" mechanism.
KAFKA_SASL_PLAIN_PASSWORD - A plaintext password used to authenticate with the "PLAIN" mechanism.
ENCRYPTED_KAFKA_SASL_PLAIN_PASSWORD - The encrypted version of the Kafka SASL PLAIN password (if the user has the Glue encrypt passwords setting selected).
KAFKA_SASL_SCRAM_USERNAME - A plaintext username used to authenticate with the "SCRAM-SHA-512" mechanism.
KAFKA_SASL_SCRAM_PASSWORD - A plaintext password used to authenticate with the "SCRAM-SHA-512" mechanism.
ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD - The encrypted version of the Kafka SASL SCRAM password (if the user has the Glue encrypt passwords setting selected).
KAFKA_SASL_SCRAM_SECRETS_ARN - The Amazon Resource Name of a secret in Amazon Web Services Secrets Manager.
KAFKA_SASL_GSSAPI_KEYTAB - The S3 location of a Kerberos keytab file. A keytab stores long-term keys for one or more principals. For more information, see MIT Kerberos Documentation: Keytab.
KAFKA_SASL_GSSAPI_KRB5_CONF - The S3 location of a Kerberos krb5.conf file. A krb5.conf stores Kerberos configuration information, such as the location of the KDC server. For more information, see MIT Kerberos Documentation: krb5.conf.
KAFKA_SASL_GSSAPI_SERVICE - The Kerberos service name, as set with sasl.kerberos.service.name in your Kafka Configuration.
KAFKA_SASL_GSSAPI_PRINCIPAL - The name of the Kerberos princial used by Glue. For more information, see Kafka Documentation: Configuring Kafka Brokers.
ROLE_ARN - The role to be used for running queries.
REGION - The Amazon Web Services Region where queries will be run.
WORKGROUP_NAME - The name of an Amazon Redshift serverless workgroup or Amazon Athena workgroup in which queries will run.
CLUSTER_IDENTIFIER - The cluster identifier of an Amazon Redshift cluster in which queries will run.
DATABASE - The Amazon Redshift database that you are connecting to.
(string) --
(string) --
SparkProperties (dict) --
Connection properties specific to the Spark compute environment.
(string) --
(string) --
AthenaProperties (dict) --
Connection properties specific to the Athena compute environment.
(string) --
(string) --
PythonProperties (dict) --
Connection properties specific to the Python compute environment.
(string) --
(string) --
PhysicalConnectionRequirements (dict) --
The physical connection requirements, such as virtual private cloud (VPC) and SecurityGroup, that are needed to make this connection successfully.
SubnetId (string) --
The subnet ID used by the connection.
SecurityGroupIdList (list) --
The security group ID list used by the connection.
(string) --
AvailabilityZone (string) --
The connection's Availability Zone.
CreationTime (datetime) --
The timestamp of the time that this connection definition was created.
LastUpdatedTime (datetime) --
The timestamp of the last time the connection definition was updated.
LastUpdatedBy (string) --
The user, group, or role that last updated this connection definition.
Status (string) --
The status of the connection. Can be one of: READY, IN_PROGRESS, or FAILED.
StatusReason (string) --
The reason for the connection status.
LastConnectionValidationTime (datetime) --
A timestamp of the time this connection was last validated.
AuthenticationConfiguration (dict) --
The authentication properties of the connection.
AuthenticationType (string) --
A structure containing the authentication configuration.
SecretArn (string) --
The secret manager ARN to store credentials.
OAuth2Properties (dict) --
The properties for OAuth2 authentication.
OAuth2GrantType (string) --
The OAuth2 grant type. For example, AUTHORIZATION_CODE, JWT_BEARER, or CLIENT_CREDENTIALS.
OAuth2ClientApplication (dict) --
The client application type. For example, AWS_MANAGED or USER_MANAGED.
UserManagedClientApplicationClientId (string) --
The client application clientID if the ClientAppType is USER_MANAGED.
AWSManagedClientApplicationReference (string) --
The reference to the SaaS-side client app that is Amazon Web Services managed.
TokenUrl (string) --
The URL of the provider's authentication server, to exchange an authorization code for an access token.
TokenUrlParametersMap (dict) --
A map of parameters that are added to the token GET request.
(string) --
(string) --
ConnectionSchemaVersion (integer) --
The version of the connection schema for this connection. Version 2 supports properties for specific compute environments.
CompatibleComputeEnvironments (list) --
A list of compute environments compatible with the connection.
(string) --
NextToken (string) --
A continuation token, if the list of connections returned does not include the last of the filtered connections.
{'CatalogId': 'string',
'TestConnectionInput': {'AuthenticationConfiguration': {'AuthenticationType': {'IAM'},
'BasicAuthenticationCredentials': {'Password': 'string',
'Username': 'string'},
'CustomAuthenticationCredentials': {'string': 'string'},
'KmsKeyArn': 'string',
'OAuth2Properties': {'OAuth2Credentials': {'AccessToken': 'string',
'JwtToken': 'string',
'RefreshToken': 'string',
'UserManagedClientApplicationClientSecret': 'string'}}},
'ConnectionType': {'FACEBOOKADS',
'GOOGLEADS',
'GOOGLEANALYTICS4',
'GOOGLESHEETS',
'HUBSPOT',
'INSTAGRAMADS',
'INTERCOM',
'JIRACLOUD',
'MARKETO',
'NETSUITEERP',
'SALESFORCEMARKETINGCLOUD',
'SALESFORCEPARDOT',
'SAPODATA',
'SERVICENOW',
'SLACK',
'SNAPCHATADS',
'STRIPE',
'ZENDESK',
'ZOHOCRM'}}}
Tests a connection to a service to validate the service credentials that you provide.
You can either provide an existing connection name or a TestConnectionInput for testing a non-existing connection input. Providing both at the same time will cause an error.
If the action is successful, the service sends back an HTTP 200 response.
See also: AWS API Documentation
Request Syntax
client.test_connection(
ConnectionName='string',
CatalogId='string',
TestConnectionInput={
'ConnectionType': 'JDBC'|'SFTP'|'MONGODB'|'KAFKA'|'NETWORK'|'MARKETPLACE'|'CUSTOM'|'SALESFORCE'|'VIEW_VALIDATION_REDSHIFT'|'VIEW_VALIDATION_ATHENA'|'GOOGLEADS'|'GOOGLESHEETS'|'GOOGLEANALYTICS4'|'SERVICENOW'|'MARKETO'|'SAPODATA'|'ZENDESK'|'JIRACLOUD'|'NETSUITEERP'|'HUBSPOT'|'FACEBOOKADS'|'INSTAGRAMADS'|'ZOHOCRM'|'SALESFORCEPARDOT'|'SALESFORCEMARKETINGCLOUD'|'SLACK'|'STRIPE'|'INTERCOM'|'SNAPCHATADS',
'ConnectionProperties': {
'string': 'string'
},
'AuthenticationConfiguration': {
'AuthenticationType': 'BASIC'|'OAUTH2'|'CUSTOM'|'IAM',
'OAuth2Properties': {
'OAuth2GrantType': 'AUTHORIZATION_CODE'|'CLIENT_CREDENTIALS'|'JWT_BEARER',
'OAuth2ClientApplication': {
'UserManagedClientApplicationClientId': 'string',
'AWSManagedClientApplicationReference': 'string'
},
'TokenUrl': 'string',
'TokenUrlParametersMap': {
'string': 'string'
},
'AuthorizationCodeProperties': {
'AuthorizationCode': 'string',
'RedirectUri': 'string'
},
'OAuth2Credentials': {
'UserManagedClientApplicationClientSecret': 'string',
'AccessToken': 'string',
'RefreshToken': 'string',
'JwtToken': 'string'
}
},
'SecretArn': 'string',
'KmsKeyArn': 'string',
'BasicAuthenticationCredentials': {
'Username': 'string',
'Password': 'string'
},
'CustomAuthenticationCredentials': {
'string': 'string'
}
}
}
)
string
Optional. The name of the connection to test. If only name is provided, the operation will get the connection and use that for testing.
string
The catalog ID where the connection resides.
dict
A structure that is used to specify testing a connection to a service.
ConnectionType (string) -- [REQUIRED]
The type of connection to test. This operation is only available for the JDBC or SALESFORCE connection types.
ConnectionProperties (dict) -- [REQUIRED]
The key-value pairs that define parameters for the connection.
JDBC connections use the following connection properties:
Required: All of ( HOST, PORT, JDBC_ENGINE) or JDBC_CONNECTION_URL.
Required: All of ( USERNAME, PASSWORD) or SECRET_ID.
Optional: JDBC_ENFORCE_SSL, CUSTOM_JDBC_CERT, CUSTOM_JDBC_CERT_STRING, SKIP_CUSTOM_JDBC_CERT_VALIDATION. These parameters are used to configure SSL with JDBC.
SALESFORCE connections require the AuthenticationConfiguration member to be configured.
(string) --
(string) --
AuthenticationConfiguration (dict) --
A structure containing the authentication configuration in the TestConnection request. Required for a connection to Salesforce using OAuth authentication.
AuthenticationType (string) --
A structure containing the authentication configuration in the CreateConnection request.
OAuth2Properties (dict) --
The properties for OAuth2 authentication in the CreateConnection request.
OAuth2GrantType (string) --
The OAuth2 grant type in the CreateConnection request. For example, AUTHORIZATION_CODE, JWT_BEARER, or CLIENT_CREDENTIALS.
OAuth2ClientApplication (dict) --
The client application type in the CreateConnection request. For example, AWS_MANAGED or USER_MANAGED.
UserManagedClientApplicationClientId (string) --
The client application clientID if the ClientAppType is USER_MANAGED.
AWSManagedClientApplicationReference (string) --
The reference to the SaaS-side client app that is Amazon Web Services managed.
TokenUrl (string) --
The URL of the provider's authentication server, to exchange an authorization code for an access token.
TokenUrlParametersMap (dict) --
A map of parameters that are added to the token GET request.
(string) --
(string) --
AuthorizationCodeProperties (dict) --
The set of properties required for the the OAuth2 AUTHORIZATION_CODE grant type.
AuthorizationCode (string) --
An authorization code to be used in the third leg of the AUTHORIZATION_CODE grant workflow. This is a single-use code which becomes invalid once exchanged for an access token, thus it is acceptable to have this value as a request parameter.
RedirectUri (string) --
The redirect URI where the user gets redirected to by authorization server when issuing an authorization code. The URI is subsequently used when the authorization code is exchanged for an access token.
OAuth2Credentials (dict) --
The credentials used when the authentication type is OAuth2 authentication.
UserManagedClientApplicationClientSecret (string) --
The client application client secret if the client application is user managed.
AccessToken (string) --
The access token used when the authentication type is OAuth2.
RefreshToken (string) --
The refresh token used when the authentication type is OAuth2.
JwtToken (string) --
The JSON Web Token (JWT) used when the authentication type is OAuth2.
SecretArn (string) --
The secret manager ARN to store credentials in the CreateConnection request.
KmsKeyArn (string) --
The ARN of the KMS key used to encrypt the connection. Only taken an as input in the request and stored in the Secret Manager.
BasicAuthenticationCredentials (dict) --
The credentials used when the authentication type is basic authentication.
Username (string) --
The username to connect to the data source.
Password (string) --
The password to connect to the data source.
CustomAuthenticationCredentials (dict) --
The credentials used when the authentication type is custom authentication.
(string) --
(string) --
dict
Response Syntax
{}
Response Structure
(dict) --
{'ConnectionInput': {'AuthenticationConfiguration': {'AuthenticationType': {'IAM'},
'BasicAuthenticationCredentials': {'Password': 'string',
'Username': 'string'},
'CustomAuthenticationCredentials': {'string': 'string'},
'KmsKeyArn': 'string',
'OAuth2Properties': {'OAuth2Credentials': {'AccessToken': 'string',
'JwtToken': 'string',
'RefreshToken': 'string',
'UserManagedClientApplicationClientSecret': 'string'}}},
'ConnectionType': {'FACEBOOKADS',
'GOOGLEADS',
'GOOGLEANALYTICS4',
'GOOGLESHEETS',
'HUBSPOT',
'INSTAGRAMADS',
'INTERCOM',
'JIRACLOUD',
'MARKETO',
'NETSUITEERP',
'SALESFORCEMARKETINGCLOUD',
'SALESFORCEPARDOT',
'SAPODATA',
'SERVICENOW',
'SLACK',
'SNAPCHATADS',
'STRIPE',
'ZENDESK',
'ZOHOCRM'},
'PythonProperties': {'string': 'string'},
'SparkProperties': {'string': 'string'},
'ValidateForComputeEnvironments': ['SPARK | ATHENA | '
'PYTHON']}}
Updates a connection definition in the Data Catalog.
See also: AWS API Documentation
Request Syntax
client.update_connection(
CatalogId='string',
Name='string',
ConnectionInput={
'Name': 'string',
'Description': 'string',
'ConnectionType': 'JDBC'|'SFTP'|'MONGODB'|'KAFKA'|'NETWORK'|'MARKETPLACE'|'CUSTOM'|'SALESFORCE'|'VIEW_VALIDATION_REDSHIFT'|'VIEW_VALIDATION_ATHENA'|'GOOGLEADS'|'GOOGLESHEETS'|'GOOGLEANALYTICS4'|'SERVICENOW'|'MARKETO'|'SAPODATA'|'ZENDESK'|'JIRACLOUD'|'NETSUITEERP'|'HUBSPOT'|'FACEBOOKADS'|'INSTAGRAMADS'|'ZOHOCRM'|'SALESFORCEPARDOT'|'SALESFORCEMARKETINGCLOUD'|'SLACK'|'STRIPE'|'INTERCOM'|'SNAPCHATADS',
'MatchCriteria': [
'string',
],
'ConnectionProperties': {
'string': 'string'
},
'SparkProperties': {
'string': 'string'
},
'AthenaProperties': {
'string': 'string'
},
'PythonProperties': {
'string': 'string'
},
'PhysicalConnectionRequirements': {
'SubnetId': 'string',
'SecurityGroupIdList': [
'string',
],
'AvailabilityZone': 'string'
},
'AuthenticationConfiguration': {
'AuthenticationType': 'BASIC'|'OAUTH2'|'CUSTOM'|'IAM',
'OAuth2Properties': {
'OAuth2GrantType': 'AUTHORIZATION_CODE'|'CLIENT_CREDENTIALS'|'JWT_BEARER',
'OAuth2ClientApplication': {
'UserManagedClientApplicationClientId': 'string',
'AWSManagedClientApplicationReference': 'string'
},
'TokenUrl': 'string',
'TokenUrlParametersMap': {
'string': 'string'
},
'AuthorizationCodeProperties': {
'AuthorizationCode': 'string',
'RedirectUri': 'string'
},
'OAuth2Credentials': {
'UserManagedClientApplicationClientSecret': 'string',
'AccessToken': 'string',
'RefreshToken': 'string',
'JwtToken': 'string'
}
},
'SecretArn': 'string',
'KmsKeyArn': 'string',
'BasicAuthenticationCredentials': {
'Username': 'string',
'Password': 'string'
},
'CustomAuthenticationCredentials': {
'string': 'string'
}
},
'ValidateCredentials': True|False,
'ValidateForComputeEnvironments': [
'SPARK'|'ATHENA'|'PYTHON',
]
}
)
string
The ID of the Data Catalog in which the connection resides. If none is provided, the Amazon Web Services account ID is used by default.
string
[REQUIRED]
The name of the connection definition to update.
dict
[REQUIRED]
A ConnectionInput object that redefines the connection in question.
Name (string) -- [REQUIRED]
The name of the connection.
Description (string) --
The description of the connection.
ConnectionType (string) -- [REQUIRED]
The type of the connection. Currently, these types are supported:
JDBC - Designates a connection to a database through Java Database Connectivity (JDBC). JDBC Connections use the following ConnectionParameters.
Required: All of ( HOST, PORT, JDBC_ENGINE) or JDBC_CONNECTION_URL.
Required: All of ( USERNAME, PASSWORD) or SECRET_ID.
Optional: JDBC_ENFORCE_SSL, CUSTOM_JDBC_CERT, CUSTOM_JDBC_CERT_STRING, SKIP_CUSTOM_JDBC_CERT_VALIDATION. These parameters are used to configure SSL with JDBC.
KAFKA - Designates a connection to an Apache Kafka streaming platform. KAFKA Connections use the following ConnectionParameters.
Required: KAFKA_BOOTSTRAP_SERVERS.
Optional: KAFKA_SSL_ENABLED, KAFKA_CUSTOM_CERT, KAFKA_SKIP_CUSTOM_CERT_VALIDATION. These parameters are used to configure SSL with KAFKA.
Optional: KAFKA_CLIENT_KEYSTORE, KAFKA_CLIENT_KEYSTORE_PASSWORD, KAFKA_CLIENT_KEY_PASSWORD, ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD, ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD. These parameters are used to configure TLS client configuration with SSL in KAFKA.
Optional: KAFKA_SASL_MECHANISM. Can be specified as SCRAM-SHA-512, GSSAPI, or AWS_MSK_IAM.
Optional: KAFKA_SASL_SCRAM_USERNAME, KAFKA_SASL_SCRAM_PASSWORD, ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD. These parameters are used to configure SASL/SCRAM-SHA-512 authentication with KAFKA.
Optional: KAFKA_SASL_GSSAPI_KEYTAB, KAFKA_SASL_GSSAPI_KRB5_CONF, KAFKA_SASL_GSSAPI_SERVICE, KAFKA_SASL_GSSAPI_PRINCIPAL. These parameters are used to configure SASL/GSSAPI authentication with KAFKA.
MONGODB - Designates a connection to a MongoDB document database. MONGODB Connections use the following ConnectionParameters.
Required: CONNECTION_URL.
Required: All of ( USERNAME, PASSWORD) or SECRET_ID.
VIEW_VALIDATION_REDSHIFT - Designates a connection used for view validation by Amazon Redshift.
VIEW_VALIDATION_ATHENA - Designates a connection used for view validation by Amazon Athena.
NETWORK - Designates a network connection to a data source within an Amazon Virtual Private Cloud environment (Amazon VPC). NETWORK Connections do not require ConnectionParameters. Instead, provide a PhysicalConnectionRequirements.
MARKETPLACE - Uses configuration settings contained in a connector purchased from Amazon Web Services Marketplace to read from and write to data stores that are not natively supported by Glue. MARKETPLACE Connections use the following ConnectionParameters.
Required: CONNECTOR_TYPE, CONNECTOR_URL, CONNECTOR_CLASS_NAME, CONNECTION_URL.
Required for JDBC CONNECTOR_TYPE connections: All of ( USERNAME, PASSWORD) or SECRET_ID.
CUSTOM - Uses configuration settings contained in a custom connector to read from and write to data stores that are not natively supported by Glue.
Additionally, a ConnectionType for the following SaaS connectors is supported:
FACEBOOKADS - Designates a connection to Facebook Ads.
GOOGLEADS - Designates a connection to Google Ads.
GOOGLESHEETS - Designates a connection to Google Sheets.
GOOGLEANALYTICS4 - Designates a connection to Google Analytics 4.
HUBSPOT - Designates a connection to HubSpot.
INSTAGRAMADS - Designates a connection to Instagram Ads.
INTERCOM - Designates a connection to Intercom.
JIRACLOUD - Designates a connection to Jira Cloud.
MARKETO - Designates a connection to Adobe Marketo Engage.
NETSUITEERP - Designates a connection to Oracle NetSuite.
SALESFORCE - Designates a connection to Salesforce using OAuth authentication.
SALESFORCEMARKETINGCLOUD - Designates a connection to Salesforce Marketing Cloud.
SALESFORCEPARDOT - Designates a connection to Salesforce Marketing Cloud Account Engagement (MCAE).
SAPODATA - Designates a connection to SAP OData.
SERVICENOW - Designates a connection to ServiceNow.
SLACK - Designates a connection to Slack.
SNAPCHATADS - Designates a connection to Snapchat Ads.
STRIPE - Designates a connection to Stripe.
ZENDESK - Designates a connection to Zendesk.
ZOHOCRM - Designates a connection to Zoho CRM.
For more information on the connection parameters needed for a particular connector, see the documentation for the connector in `Adding an Glue connection <https://docs.aws.amazon.com/glue/latest/dg/console-connections.html>`__in the Glue User Guide.
SFTP is not supported.
For more information about how optional ConnectionProperties are used to configure features in Glue, consult Glue connection properties.
For more information about how optional ConnectionProperties are used to configure features in Glue Studio, consult Using connectors and connections.
MatchCriteria (list) --
A list of criteria that can be used in selecting this connection.
(string) --
ConnectionProperties (dict) -- [REQUIRED]
These key-value pairs define parameters for the connection.
(string) --
(string) --
SparkProperties (dict) --
Connection properties specific to the Spark compute environment.
(string) --
(string) --
AthenaProperties (dict) --
Connection properties specific to the Athena compute environment.
(string) --
(string) --
PythonProperties (dict) --
Connection properties specific to the Python compute environment.
(string) --
(string) --
PhysicalConnectionRequirements (dict) --
The physical connection requirements, such as virtual private cloud (VPC) and SecurityGroup, that are needed to successfully make this connection.
SubnetId (string) --
The subnet ID used by the connection.
SecurityGroupIdList (list) --
The security group ID list used by the connection.
(string) --
AvailabilityZone (string) --
The connection's Availability Zone.
AuthenticationConfiguration (dict) --
The authentication properties of the connection.
AuthenticationType (string) --
A structure containing the authentication configuration in the CreateConnection request.
OAuth2Properties (dict) --
The properties for OAuth2 authentication in the CreateConnection request.
OAuth2GrantType (string) --
The OAuth2 grant type in the CreateConnection request. For example, AUTHORIZATION_CODE, JWT_BEARER, or CLIENT_CREDENTIALS.
OAuth2ClientApplication (dict) --
The client application type in the CreateConnection request. For example, AWS_MANAGED or USER_MANAGED.
UserManagedClientApplicationClientId (string) --
The client application clientID if the ClientAppType is USER_MANAGED.
AWSManagedClientApplicationReference (string) --
The reference to the SaaS-side client app that is Amazon Web Services managed.
TokenUrl (string) --
The URL of the provider's authentication server, to exchange an authorization code for an access token.
TokenUrlParametersMap (dict) --
A map of parameters that are added to the token GET request.
(string) --
(string) --
AuthorizationCodeProperties (dict) --
The set of properties required for the the OAuth2 AUTHORIZATION_CODE grant type.
AuthorizationCode (string) --
An authorization code to be used in the third leg of the AUTHORIZATION_CODE grant workflow. This is a single-use code which becomes invalid once exchanged for an access token, thus it is acceptable to have this value as a request parameter.
RedirectUri (string) --
The redirect URI where the user gets redirected to by authorization server when issuing an authorization code. The URI is subsequently used when the authorization code is exchanged for an access token.
OAuth2Credentials (dict) --
The credentials used when the authentication type is OAuth2 authentication.
UserManagedClientApplicationClientSecret (string) --
The client application client secret if the client application is user managed.
AccessToken (string) --
The access token used when the authentication type is OAuth2.
RefreshToken (string) --
The refresh token used when the authentication type is OAuth2.
JwtToken (string) --
The JSON Web Token (JWT) used when the authentication type is OAuth2.
SecretArn (string) --
The secret manager ARN to store credentials in the CreateConnection request.
KmsKeyArn (string) --
The ARN of the KMS key used to encrypt the connection. Only taken an as input in the request and stored in the Secret Manager.
BasicAuthenticationCredentials (dict) --
The credentials used when the authentication type is basic authentication.
Username (string) --
The username to connect to the data source.
Password (string) --
The password to connect to the data source.
CustomAuthenticationCredentials (dict) --
The credentials used when the authentication type is custom authentication.
(string) --
(string) --
ValidateCredentials (boolean) --
A flag to validate the credentials during create connection. Default is true.
ValidateForComputeEnvironments (list) --
The compute environments that the specified connection properties are validated against.
(string) --
dict
Response Syntax
{}
Response Structure
(dict) --