2018/12/12 - AWS Glue - 4 updated api methods
Changes API Update for Glue: this update enables encryption of password inside connection objects stored in AWS Glue Data Catalog using DataCatalogEncryptionSettings. In addition, a new "HidePassword" flag is added to GetConnection and GetConnections to return connections without passwords.
{'HidePassword': 'boolean'}
Retrieves a connection definition from the Data Catalog.
See also: AWS API Documentation
Request Syntax
client.get_connection(
CatalogId='string',
Name='string',
HidePassword=True|False
)
string
The ID of the Data Catalog in which the connection resides. If none is supplied, the AWS account ID is used by default.
string
[REQUIRED]
The name of the connection definition to retrieve.
boolean
Allow you to retrieve the connection metadata without displaying the password. For instance, the AWS Glue console uses this flag to retrieve connections, since the console does not display passwords. Set this parameter where the caller may not have permission to use the KMS key to decrypt the password, but does have permission to access the rest of the connection metadata (that is, the other connection properties).
dict
Response Syntax
{
'Connection': {
'Name': 'string',
'Description': 'string',
'ConnectionType': 'JDBC'|'SFTP',
'MatchCriteria': [
'string',
],
'ConnectionProperties': {
'string': 'string'
},
'PhysicalConnectionRequirements': {
'SubnetId': 'string',
'SecurityGroupIdList': [
'string',
],
'AvailabilityZone': 'string'
},
'CreationTime': datetime(2015, 1, 1),
'LastUpdatedTime': datetime(2015, 1, 1),
'LastUpdatedBy': 'string'
}
}
Response Structure
(dict) --
Connection (dict) --
The requested connection definition.
Name (string) --
The name of the connection definition.
Description (string) --
Description of the connection.
ConnectionType (string) --
The type of the connection. Currently, only JDBC is supported; SFTP is not supported.
MatchCriteria (list) --
A list of criteria that can be used in selecting this connection.
(string) --
ConnectionProperties (dict) --
These key-value pairs define parameters for the connection:
HOST - The host URI: either the fully qualified domain name (FQDN) or the IPv4 address of the database host.
PORT - The port number, between 1024 and 65535, of the port on which the database host is listening for database connections.
USER_NAME - The name under which to log in to the database. The value string for USER_NAME is " USERNAME ".
PASSWORD - A password, if one is used, for the user name.
ENCRYPTED_PASSWORD - When you enable connection password protection by setting ConnectionPasswordEncryption in the Data Catalog encryption settings, this field stores the key you designate to encrypt the password.
JDBC_DRIVER_JAR_URI - The S3 path of the a jar file that contains the JDBC driver to use.
JDBC_DRIVER_CLASS_NAME - The class name of the JDBC driver to use.
JDBC_ENGINE - The name of the JDBC engine to use.
JDBC_ENGINE_VERSION - The version of the JDBC engine to use.
CONFIG_FILES - (Reserved for future use).
INSTANCE_ID - The instance ID to use.
JDBC_CONNECTION_URL - The URL for the JDBC connection.
JDBC_ENFORCE_SSL - A Boolean string (true, false) specifying whether SSL with hostname matching will be enforced for the JDBC connection on the client. The default is false.
(string) --
(string) --
PhysicalConnectionRequirements (dict) --
A map of physical connection requirements, such as VPC and SecurityGroup, needed for making this connection successfully.
SubnetId (string) --
The subnet ID used by the connection.
SecurityGroupIdList (list) --
The security group ID list used by the connection.
(string) --
AvailabilityZone (string) --
The connection's availability zone. This field is redundant, since the specified subnet implies the availability zone to be used. The field must be populated now, but will be deprecated in the future.
CreationTime (datetime) --
The time this connection definition was created.
LastUpdatedTime (datetime) --
The last time this connection definition was updated.
LastUpdatedBy (string) --
The user, group or role that last updated this connection definition.
{'HidePassword': 'boolean'}
Retrieves a list of connection definitions from the Data Catalog.
See also: AWS API Documentation
Request Syntax
client.get_connections(
CatalogId='string',
Filter={
'MatchCriteria': [
'string',
],
'ConnectionType': 'JDBC'|'SFTP'
},
HidePassword=True|False,
NextToken='string',
MaxResults=123
)
string
The ID of the Data Catalog in which the connections reside. If none is supplied, the AWS account ID is used by default.
dict
A filter that controls which connections will be returned.
MatchCriteria (list) --
A criteria string that must match the criteria recorded in the connection definition for that connection definition to be returned.
(string) --
ConnectionType (string) --
The type of connections to return. Currently, only JDBC is supported; SFTP is not supported.
boolean
Allow you to retrieve the connection metadata without displaying the password. For instance, the AWS Glue console uses this flag to retrieve connections, since the console does not display passwords. Set this parameter where the caller may not have permission to use the KMS key to decrypt the password, but does have permission to access the rest of the connection metadata (that is, the other connection properties).
string
A continuation token, if this is a continuation call.
integer
The maximum number of connections to return in one response.
dict
Response Syntax
{
'ConnectionList': [
{
'Name': 'string',
'Description': 'string',
'ConnectionType': 'JDBC'|'SFTP',
'MatchCriteria': [
'string',
],
'ConnectionProperties': {
'string': 'string'
},
'PhysicalConnectionRequirements': {
'SubnetId': 'string',
'SecurityGroupIdList': [
'string',
],
'AvailabilityZone': 'string'
},
'CreationTime': datetime(2015, 1, 1),
'LastUpdatedTime': datetime(2015, 1, 1),
'LastUpdatedBy': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
ConnectionList (list) --
A list of requested connection definitions.
(dict) --
Defines a connection to a data source.
Name (string) --
The name of the connection definition.
Description (string) --
Description of the connection.
ConnectionType (string) --
The type of the connection. Currently, only JDBC is supported; SFTP is not supported.
MatchCriteria (list) --
A list of criteria that can be used in selecting this connection.
(string) --
ConnectionProperties (dict) --
These key-value pairs define parameters for the connection:
HOST - The host URI: either the fully qualified domain name (FQDN) or the IPv4 address of the database host.
PORT - The port number, between 1024 and 65535, of the port on which the database host is listening for database connections.
USER_NAME - The name under which to log in to the database. The value string for USER_NAME is " USERNAME ".
PASSWORD - A password, if one is used, for the user name.
ENCRYPTED_PASSWORD - When you enable connection password protection by setting ConnectionPasswordEncryption in the Data Catalog encryption settings, this field stores the key you designate to encrypt the password.
JDBC_DRIVER_JAR_URI - The S3 path of the a jar file that contains the JDBC driver to use.
JDBC_DRIVER_CLASS_NAME - The class name of the JDBC driver to use.
JDBC_ENGINE - The name of the JDBC engine to use.
JDBC_ENGINE_VERSION - The version of the JDBC engine to use.
CONFIG_FILES - (Reserved for future use).
INSTANCE_ID - The instance ID to use.
JDBC_CONNECTION_URL - The URL for the JDBC connection.
JDBC_ENFORCE_SSL - A Boolean string (true, false) specifying whether SSL with hostname matching will be enforced for the JDBC connection on the client. The default is false.
(string) --
(string) --
PhysicalConnectionRequirements (dict) --
A map of physical connection requirements, such as VPC and SecurityGroup, needed for making this connection successfully.
SubnetId (string) --
The subnet ID used by the connection.
SecurityGroupIdList (list) --
The security group ID list used by the connection.
(string) --
AvailabilityZone (string) --
The connection's availability zone. This field is redundant, since the specified subnet implies the availability zone to be used. The field must be populated now, but will be deprecated in the future.
CreationTime (datetime) --
The time this connection definition was created.
LastUpdatedTime (datetime) --
The last time this connection definition was updated.
LastUpdatedBy (string) --
The user, group or role that last updated this connection definition.
NextToken (string) --
A continuation token, if the list of connections returned does not include the last of the filtered connections.
{'DataCatalogEncryptionSettings': {'ConnectionPasswordEncryption': {'AwsKmsKeyId': 'string',
'ReturnConnectionPasswordEncrypted': 'boolean'}}}
Retrieves the security configuration for a specified catalog.
See also: AWS API Documentation
Request Syntax
client.get_data_catalog_encryption_settings(
CatalogId='string'
)
string
The ID of the Data Catalog for which to retrieve the security configuration. If none is supplied, the AWS account ID is used by default.
dict
Response Syntax
{
'DataCatalogEncryptionSettings': {
'EncryptionAtRest': {
'CatalogEncryptionMode': 'DISABLED'|'SSE-KMS',
'SseAwsKmsKeyId': 'string'
},
'ConnectionPasswordEncryption': {
'ReturnConnectionPasswordEncrypted': True|False,
'AwsKmsKeyId': 'string'
}
}
}
Response Structure
(dict) --
DataCatalogEncryptionSettings (dict) --
The requested security configuration.
EncryptionAtRest (dict) --
Specifies encryption-at-rest configuration for the Data Catalog.
CatalogEncryptionMode (string) --
The encryption-at-rest mode for encrypting Data Catalog data.
SseAwsKmsKeyId (string) --
The ID of the AWS KMS key to use for encryption at rest.
ConnectionPasswordEncryption (dict) --
When password protection is enabled, the Data Catalog uses a customer-provided key to encrypt the password as part of CreateConnection or UpdateConnection and store it in the ENCRYPTED_PASSWORD field in the connection properties. You can enable catalog encryption or only password encryption.
ReturnConnectionPasswordEncrypted (boolean) --
When the ReturnConnectionPasswordEncrypted flag is set to "true", passwords remain encrypted in the responses of GetConnection and GetConnections . This encryption takes effect independently from catalog encryption.
AwsKmsKeyId (string) --
A KMS key used to protect access to the JDBC source.
All users in your account should be granted the kms:encrypt permission to encrypt passwords before storing them in the Data Catalog (through the AWS Glue CreateConnection operation).
The decrypt permission should be granted only to KMS key admins and IAM roles designated for AWS Glue crawlers.
{'DataCatalogEncryptionSettings': {'ConnectionPasswordEncryption': {'AwsKmsKeyId': 'string',
'ReturnConnectionPasswordEncrypted': 'boolean'}}}
Sets the security configuration for a specified catalog. Once the configuration has been set, the specified encryption is applied to every catalog write thereafter.
See also: AWS API Documentation
Request Syntax
client.put_data_catalog_encryption_settings(
CatalogId='string',
DataCatalogEncryptionSettings={
'EncryptionAtRest': {
'CatalogEncryptionMode': 'DISABLED'|'SSE-KMS',
'SseAwsKmsKeyId': 'string'
},
'ConnectionPasswordEncryption': {
'ReturnConnectionPasswordEncrypted': True|False,
'AwsKmsKeyId': 'string'
}
}
)
string
The ID of the Data Catalog for which to set the security configuration. If none is supplied, the AWS account ID is used by default.
dict
[REQUIRED]
The security configuration to set.
EncryptionAtRest (dict) --
Specifies encryption-at-rest configuration for the Data Catalog.
CatalogEncryptionMode (string) -- [REQUIRED]
The encryption-at-rest mode for encrypting Data Catalog data.
SseAwsKmsKeyId (string) --
The ID of the AWS KMS key to use for encryption at rest.
ConnectionPasswordEncryption (dict) --
When password protection is enabled, the Data Catalog uses a customer-provided key to encrypt the password as part of CreateConnection or UpdateConnection and store it in the ENCRYPTED_PASSWORD field in the connection properties. You can enable catalog encryption or only password encryption.
ReturnConnectionPasswordEncrypted (boolean) -- [REQUIRED]
When the ReturnConnectionPasswordEncrypted flag is set to "true", passwords remain encrypted in the responses of GetConnection and GetConnections . This encryption takes effect independently from catalog encryption.
AwsKmsKeyId (string) --
A KMS key used to protect access to the JDBC source.
All users in your account should be granted the kms:encrypt permission to encrypt passwords before storing them in the Data Catalog (through the AWS Glue CreateConnection operation).
The decrypt permission should be granted only to KMS key admins and IAM roles designated for AWS Glue crawlers.
dict
Response Syntax
{}
Response Structure
(dict) --