2025/07/31 - AWS Glue - 5 updated api methods
Changes Added support for Route node, S3 Iceberg sources/targets, catalog Iceberg sources, DynamoDB ELT connector, AutoDataQuality evaluation, enhanced PII detection with redaction, Kinesis fan-out support, and new R-series worker types.
{'Jobs': {'CodeGenConfigurationNodes': {'AthenaConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'CatalogKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'CatalogSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'PartitionPredicate': 'string'}, 'ConnectorDataSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CustomCode': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DirectJDBCSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'DirectKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'DynamicTransform': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DynamoDBCatalogSource': {'AdditionalOptions': {'DynamodbExport': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'PitrEnabled': 'boolean'}, 'DynamoDBELTConnectorSource': {'ConnectionOptions': {'DynamodbExport': 'ddb ' '| ' 's3', 'DynamodbS3Bucket': 'string', 'DynamodbS3BucketOwner': 'string', 'DynamodbS3Prefix': 'string', 'DynamodbStsRoleArn': 'string', 'DynamodbTableArn': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'JDBCConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'JDBCConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'PIIDetection': {'DetectionParameters': 'string', 'DetectionSensitivity': 'string', 'MatchPattern': 'string', 'NumLeftCharsToExclude': 'integer', 'NumRightCharsToExclude': 'integer', 'PiiType': {'ColumnHashing', 'RowHashing', 'RowPartialMasking'}, 'RedactChar': 'string', 'RedactText': 'string'}, 'Route': {'GroupFiltersList': [{'Filters': [{'Negated': 'boolean', 'Operation': 'EQ ' '| ' 'LT ' '| ' 'GT ' '| ' 'LTE ' '| ' 'GTE ' '| ' 'REGEX ' '| ' 'ISNULL', 'Values': [{'Type': 'COLUMNEXTRACTED ' '| ' 'CONSTANT', 'Value': ['string']}]}], 'GroupName': 'string', 'LogicalOperator': 'AND ' '| ' 'OR'}], 'Inputs': ['string'], 'Name': 'string'}, 'S3CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'S3CatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3CsvSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DeltaCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3DeltaDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3DeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3ExcelSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3GlueParquetTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3HudiDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3HyperDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Format': 'json ' '| ' 'csv ' '| ' 'avro ' '| ' 'orc ' '| ' 'parquet ' '| ' 'hudi ' '| ' 'delta ' '| ' 'iceberg ' '| ' 'hyper ' '| ' 'xml', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3IcebergCatalogTarget': {'AdditionalOptions': {'string': 'string'}, 'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Database': 'string', 'Inputs': ['string'], 'Name': 'string', 'PartitionKeys': [['string']], 'SchemaChangePolicy': {'EnableUpdateCatalog': 'boolean', 'UpdateBehavior': 'UPDATE_IN_DATABASE ' '| ' 'LOG'}, 'Table': 'string'}, 'S3IcebergDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3JsonSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3ParquetSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SnowflakeSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkSQL': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}}}}
Returns a list of resource metadata for a given list of job names. After calling the ListJobs operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.
See also: AWS API Documentation
Request Syntax
client.batch_get_jobs( JobNames=[ 'string', ] )
list
[REQUIRED]
A list of job names, which might be the names returned from the ListJobs operation.
(string) --
dict
Response Syntax
# This section is too large to render. # Please see the AWS API Documentation linked below.
Response Structure
# This section is too large to render. # Please see the AWS API Documentation linked below.
{'CodeGenConfigurationNodes': {'AthenaConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'CatalogKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'CatalogSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'PartitionPredicate': 'string'}, 'ConnectorDataSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CustomCode': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DirectJDBCSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'DirectKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'DynamicTransform': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DynamoDBCatalogSource': {'AdditionalOptions': {'DynamodbExport': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'PitrEnabled': 'boolean'}, 'DynamoDBELTConnectorSource': {'ConnectionOptions': {'DynamodbExport': 'ddb ' '| ' 's3', 'DynamodbS3Bucket': 'string', 'DynamodbS3BucketOwner': 'string', 'DynamodbS3Prefix': 'string', 'DynamodbStsRoleArn': 'string', 'DynamodbTableArn': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'JDBCConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'JDBCConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'PIIDetection': {'DetectionParameters': 'string', 'DetectionSensitivity': 'string', 'MatchPattern': 'string', 'NumLeftCharsToExclude': 'integer', 'NumRightCharsToExclude': 'integer', 'PiiType': {'ColumnHashing', 'RowHashing', 'RowPartialMasking'}, 'RedactChar': 'string', 'RedactText': 'string'}, 'Route': {'GroupFiltersList': [{'Filters': [{'Negated': 'boolean', 'Operation': 'EQ ' '| ' 'LT ' '| ' 'GT ' '| ' 'LTE ' '| ' 'GTE ' '| ' 'REGEX ' '| ' 'ISNULL', 'Values': [{'Type': 'COLUMNEXTRACTED ' '| ' 'CONSTANT', 'Value': ['string']}]}], 'GroupName': 'string', 'LogicalOperator': 'AND ' '| ' 'OR'}], 'Inputs': ['string'], 'Name': 'string'}, 'S3CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'S3CatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3CsvSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DeltaCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3DeltaDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3DeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3ExcelSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3GlueParquetTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3HudiDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3HyperDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Format': 'json | csv | ' 'avro | orc | ' 'parquet | ' 'hudi | delta ' '| iceberg | ' 'hyper | xml', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3IcebergCatalogTarget': {'AdditionalOptions': {'string': 'string'}, 'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Database': 'string', 'Inputs': ['string'], 'Name': 'string', 'PartitionKeys': [['string']], 'SchemaChangePolicy': {'EnableUpdateCatalog': 'boolean', 'UpdateBehavior': 'UPDATE_IN_DATABASE ' '| ' 'LOG'}, 'Table': 'string'}, 'S3IcebergDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3JsonSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3ParquetSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SnowflakeSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkSQL': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}}}
Creates a new job definition.
See also: AWS API Documentation
Request Syntax
# This section is too large to render. # Please see the AWS API Documentation linked below.Parameters
# This section is too large to render. # Please see the AWS API Documentation linked below.
dict
Response Syntax
{ 'Name': 'string' }
Response Structure
(dict) --
Name (string) --
The unique name that was provided for this job definition.
{'Job': {'CodeGenConfigurationNodes': {'AthenaConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'CatalogKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'CatalogSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'PartitionPredicate': 'string'}, 'ConnectorDataSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CustomCode': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DirectJDBCSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'DirectKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'DynamicTransform': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DynamoDBCatalogSource': {'AdditionalOptions': {'DynamodbExport': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'PitrEnabled': 'boolean'}, 'DynamoDBELTConnectorSource': {'ConnectionOptions': {'DynamodbExport': 'ddb ' '| ' 's3', 'DynamodbS3Bucket': 'string', 'DynamodbS3BucketOwner': 'string', 'DynamodbS3Prefix': 'string', 'DynamodbStsRoleArn': 'string', 'DynamodbTableArn': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'JDBCConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'JDBCConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'PIIDetection': {'DetectionParameters': 'string', 'DetectionSensitivity': 'string', 'MatchPattern': 'string', 'NumLeftCharsToExclude': 'integer', 'NumRightCharsToExclude': 'integer', 'PiiType': {'ColumnHashing', 'RowHashing', 'RowPartialMasking'}, 'RedactChar': 'string', 'RedactText': 'string'}, 'Route': {'GroupFiltersList': [{'Filters': [{'Negated': 'boolean', 'Operation': 'EQ ' '| ' 'LT ' '| ' 'GT ' '| ' 'LTE ' '| ' 'GTE ' '| ' 'REGEX ' '| ' 'ISNULL', 'Values': [{'Type': 'COLUMNEXTRACTED ' '| ' 'CONSTANT', 'Value': ['string']}]}], 'GroupName': 'string', 'LogicalOperator': 'AND ' '| ' 'OR'}], 'Inputs': ['string'], 'Name': 'string'}, 'S3CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'S3CatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3CsvSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DeltaCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3DeltaDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3DeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3ExcelSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3GlueParquetTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3HudiDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3HyperDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Format': 'json ' '| ' 'csv ' '| ' 'avro ' '| ' 'orc ' '| ' 'parquet ' '| ' 'hudi ' '| ' 'delta ' '| ' 'iceberg ' '| ' 'hyper ' '| ' 'xml', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3IcebergCatalogTarget': {'AdditionalOptions': {'string': 'string'}, 'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Database': 'string', 'Inputs': ['string'], 'Name': 'string', 'PartitionKeys': [['string']], 'SchemaChangePolicy': {'EnableUpdateCatalog': 'boolean', 'UpdateBehavior': 'UPDATE_IN_DATABASE ' '| ' 'LOG'}, 'Table': 'string'}, 'S3IcebergDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3JsonSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3ParquetSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SnowflakeSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkSQL': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}}}}
Retrieves an existing job definition.
See also: AWS API Documentation
Request Syntax
client.get_job( JobName='string' )
string
[REQUIRED]
The name of the job definition to retrieve.
dict
Response Syntax
# This section is too large to render. # Please see the AWS API Documentation linked below.
Response Structure
# This section is too large to render. # Please see the AWS API Documentation linked below.
{'Jobs': {'CodeGenConfigurationNodes': {'AthenaConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'CatalogKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'CatalogSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'PartitionPredicate': 'string'}, 'ConnectorDataSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CustomCode': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DirectJDBCSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'DirectKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'DynamicTransform': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DynamoDBCatalogSource': {'AdditionalOptions': {'DynamodbExport': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'PitrEnabled': 'boolean'}, 'DynamoDBELTConnectorSource': {'ConnectionOptions': {'DynamodbExport': 'ddb ' '| ' 's3', 'DynamodbS3Bucket': 'string', 'DynamodbS3BucketOwner': 'string', 'DynamodbS3Prefix': 'string', 'DynamodbStsRoleArn': 'string', 'DynamodbTableArn': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'JDBCConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'JDBCConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'PIIDetection': {'DetectionParameters': 'string', 'DetectionSensitivity': 'string', 'MatchPattern': 'string', 'NumLeftCharsToExclude': 'integer', 'NumRightCharsToExclude': 'integer', 'PiiType': {'ColumnHashing', 'RowHashing', 'RowPartialMasking'}, 'RedactChar': 'string', 'RedactText': 'string'}, 'Route': {'GroupFiltersList': [{'Filters': [{'Negated': 'boolean', 'Operation': 'EQ ' '| ' 'LT ' '| ' 'GT ' '| ' 'LTE ' '| ' 'GTE ' '| ' 'REGEX ' '| ' 'ISNULL', 'Values': [{'Type': 'COLUMNEXTRACTED ' '| ' 'CONSTANT', 'Value': ['string']}]}], 'GroupName': 'string', 'LogicalOperator': 'AND ' '| ' 'OR'}], 'Inputs': ['string'], 'Name': 'string'}, 'S3CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'S3CatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3CsvSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DeltaCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3DeltaDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3DeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3ExcelSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3GlueParquetTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3HudiDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3HyperDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Format': 'json ' '| ' 'csv ' '| ' 'avro ' '| ' 'orc ' '| ' 'parquet ' '| ' 'hudi ' '| ' 'delta ' '| ' 'iceberg ' '| ' 'hyper ' '| ' 'xml', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3IcebergCatalogTarget': {'AdditionalOptions': {'string': 'string'}, 'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Database': 'string', 'Inputs': ['string'], 'Name': 'string', 'PartitionKeys': [['string']], 'SchemaChangePolicy': {'EnableUpdateCatalog': 'boolean', 'UpdateBehavior': 'UPDATE_IN_DATABASE ' '| ' 'LOG'}, 'Table': 'string'}, 'S3IcebergDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3JsonSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3ParquetSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SnowflakeSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkSQL': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}}}}
Retrieves all current job definitions.
See also: AWS API Documentation
Request Syntax
client.get_jobs( NextToken='string', MaxResults=123 )
string
A continuation token, if this is a continuation call.
integer
The maximum size of the response.
dict
Response Syntax
# This section is too large to render. # Please see the AWS API Documentation linked below.
Response Structure
# This section is too large to render. # Please see the AWS API Documentation linked below.
{'JobUpdate': {'CodeGenConfigurationNodes': {'AthenaConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'CatalogKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'CatalogSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'PartitionPredicate': 'string'}, 'ConnectorDataSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'CustomCode': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DirectJDBCSource': {'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'DirectKinesisSource': {'StreamingOptions': {'FanoutConsumerARN': 'string'}}, 'DynamicTransform': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'DynamoDBCatalogSource': {'AdditionalOptions': {'DynamodbExport': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'PitrEnabled': 'boolean'}, 'DynamoDBELTConnectorSource': {'ConnectionOptions': {'DynamodbExport': 'ddb ' '| ' 's3', 'DynamodbS3Bucket': 'string', 'DynamodbS3BucketOwner': 'string', 'DynamodbS3Prefix': 'string', 'DynamodbStsRoleArn': 'string', 'DynamodbTableArn': 'string', 'DynamodbUnnestDDBJson': 'boolean'}, 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'JDBCConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'JDBCConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'PIIDetection': {'DetectionParameters': 'string', 'DetectionSensitivity': 'string', 'MatchPattern': 'string', 'NumLeftCharsToExclude': 'integer', 'NumRightCharsToExclude': 'integer', 'PiiType': {'ColumnHashing', 'RowHashing', 'RowPartialMasking'}, 'RedactChar': 'string', 'RedactText': 'string'}, 'Route': {'GroupFiltersList': [{'Filters': [{'Negated': 'boolean', 'Operation': 'EQ ' '| ' 'LT ' '| ' 'GT ' '| ' 'LTE ' '| ' 'GTE ' '| ' 'REGEX ' '| ' 'ISNULL', 'Values': [{'Type': 'COLUMNEXTRACTED ' '| ' 'CONSTANT', 'Value': ['string']}]}], 'GroupName': 'string', 'LogicalOperator': 'AND ' '| ' 'OR'}], 'Inputs': ['string'], 'Name': 'string'}, 'S3CatalogDeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogHudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3CatalogIcebergSource': {'AdditionalIcebergOptions': {'string': 'string'}, 'Database': 'string', 'Name': 'string', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}], 'Table': 'string'}, 'S3CatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3CsvSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DeltaCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3DeltaDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3DeltaSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3DirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3ExcelSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3GlueParquetTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiCatalogTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3HudiDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}}, 'S3HudiSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3HyperDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Format': 'json ' '| ' 'csv ' '| ' 'avro ' '| ' 'orc ' '| ' 'parquet ' '| ' 'hudi ' '| ' 'delta ' '| ' 'iceberg ' '| ' 'hyper ' '| ' 'xml', 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3IcebergCatalogTarget': {'AdditionalOptions': {'string': 'string'}, 'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'Database': 'string', 'Inputs': ['string'], 'Name': 'string', 'PartitionKeys': [['string']], 'SchemaChangePolicy': {'EnableUpdateCatalog': 'boolean', 'UpdateBehavior': 'UPDATE_IN_DATABASE ' '| ' 'LOG'}, 'Table': 'string'}, 'S3IcebergDirectTarget': {'AutoDataQuality': {'EvaluationContext': 'string', 'IsEnabled': 'boolean'}, 'OutputSchemas': [{'Columns': [{'GlueStudioType': 'string', 'Name': 'string', 'Type': 'string'}]}]}, 'S3JsonSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'S3ParquetSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SnowflakeSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorSource': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkConnectorTarget': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}, 'SparkSQL': {'OutputSchemas': {'Columns': {'GlueStudioType': 'string'}}}}}}
Updates an existing job definition. The previous job definition is completely overwritten by this information.
See also: AWS API Documentation
Request Syntax
# This section is too large to render. # Please see the AWS API Documentation linked below.Parameters
# This section is too large to render. # Please see the AWS API Documentation linked below.
dict
Response Syntax
{ 'JobName': 'string' }
Response Structure
(dict) --
JobName (string) --
Returns the name of the updated job definition.