2024/04/24 - Amazon EMR Containers - 3 new 3 updated api methods
Changes EMRonEKS Service support for SecurityConfiguration enforcement for Spark Jobs.
Lists security configurations based on a set of parameters. Security configurations in Amazon EMR on EKS are templates for different security setups. You can use security configurations to configure the Lake Formation integration setup. You can also create a security configuration to re-use a security setup each time you create a virtual cluster.
See also: AWS API Documentation
Request Syntax
client.list_security_configurations(
createdAfter=datetime(2015, 1, 1),
createdBefore=datetime(2015, 1, 1),
maxResults=123,
nextToken='string'
)
datetime
The date and time after which the security configuration was created.
datetime
The date and time before which the security configuration was created.
integer
The maximum number of security configurations the operation can list.
string
The token for the next set of security configurations to return.
dict
Response Syntax
{
'securityConfigurations': [
{
'id': 'string',
'name': 'string',
'arn': 'string',
'createdAt': datetime(2015, 1, 1),
'createdBy': 'string',
'securityConfigurationData': {
'authorizationConfiguration': {
'lakeFormationConfiguration': {
'authorizedSessionTagValue': 'string',
'secureNamespaceInfo': {
'clusterId': 'string',
'namespace': 'string'
},
'queryEngineRoleArn': 'string'
},
'encryptionConfiguration': {
'inTransitEncryptionConfiguration': {
'tlsCertificateConfiguration': {
'certificateProviderType': 'PEM',
'publicCertificateSecretArn': 'string',
'privateCertificateSecretArn': 'string'
}
}
}
}
},
'tags': {
'string': 'string'
}
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
securityConfigurations (list) --
The list of returned security configurations.
(dict) --
Inputs related to the security configuration. Security configurations in Amazon EMR on EKS are templates for different security setups. You can use security configurations to configure the Lake Formation integration setup. You can also create a security configuration to re-use a security setup each time you create a virtual cluster.
id (string) --
The ID of the security configuration.
name (string) --
The name of the security configuration.
arn (string) --
The ARN (Amazon Resource Name) of the security configuration.
createdAt (datetime) --
The date and time that the job run was created.
createdBy (string) --
The user who created the job run.
securityConfigurationData (dict) --
Security configuration inputs for the request.
authorizationConfiguration (dict) --
Authorization-related configuration input for the security configuration.
lakeFormationConfiguration (dict) --
Lake Formation related configuration inputs for the security configuration.
authorizedSessionTagValue (string) --
The session tag to authorize Amazon EMR on EKS for API calls to Lake Formation.
secureNamespaceInfo (dict) --
The namespace input of the system job.
clusterId (string) --
The ID of the Amazon EKS cluster where Amazon EMR on EKS jobs run.
namespace (string) --
The namespace of the Amazon EKS cluster where the system jobs run.
queryEngineRoleArn (string) --
The query engine IAM role ARN that is tied to the secure Spark job. The QueryEngine role assumes the JobExecutionRole to execute all the Lake Formation calls.
encryptionConfiguration (dict) --
Encryption-related configuration input for the security configuration.
inTransitEncryptionConfiguration (dict) --
In-transit encryption-related input for the security configuration.
tlsCertificateConfiguration (dict) --
TLS certificate-related configuration input for the security configuration.
certificateProviderType (string) --
The TLS certificate type. Acceptable values: PEM or Custom .
publicCertificateSecretArn (string) --
Secrets Manager ARN that contains the public TLS certificate contents, used for communication between the user job and the system job.
privateCertificateSecretArn (string) --
Secrets Manager ARN that contains the private TLS certificate contents, used for communication between the user job and the system job.
tags (dict) --
The tags to assign to the security configuration.
(string) --
(string) --
nextToken (string) --
The token for the next set of security configurations to return.
Displays detailed information about a specified security configuration. Security configurations in Amazon EMR on EKS are templates for different security setups. You can use security configurations to configure the Lake Formation integration setup. You can also create a security configuration to re-use a security setup each time you create a virtual cluster.
See also: AWS API Documentation
Request Syntax
client.describe_security_configuration(
id='string'
)
string
[REQUIRED]
The ID of the security configuration.
dict
Response Syntax
{
'securityConfiguration': {
'id': 'string',
'name': 'string',
'arn': 'string',
'createdAt': datetime(2015, 1, 1),
'createdBy': 'string',
'securityConfigurationData': {
'authorizationConfiguration': {
'lakeFormationConfiguration': {
'authorizedSessionTagValue': 'string',
'secureNamespaceInfo': {
'clusterId': 'string',
'namespace': 'string'
},
'queryEngineRoleArn': 'string'
},
'encryptionConfiguration': {
'inTransitEncryptionConfiguration': {
'tlsCertificateConfiguration': {
'certificateProviderType': 'PEM',
'publicCertificateSecretArn': 'string',
'privateCertificateSecretArn': 'string'
}
}
}
}
},
'tags': {
'string': 'string'
}
}
}
Response Structure
(dict) --
securityConfiguration (dict) --
Details of the security configuration.
id (string) --
The ID of the security configuration.
name (string) --
The name of the security configuration.
arn (string) --
The ARN (Amazon Resource Name) of the security configuration.
createdAt (datetime) --
The date and time that the job run was created.
createdBy (string) --
The user who created the job run.
securityConfigurationData (dict) --
Security configuration inputs for the request.
authorizationConfiguration (dict) --
Authorization-related configuration input for the security configuration.
lakeFormationConfiguration (dict) --
Lake Formation related configuration inputs for the security configuration.
authorizedSessionTagValue (string) --
The session tag to authorize Amazon EMR on EKS for API calls to Lake Formation.
secureNamespaceInfo (dict) --
The namespace input of the system job.
clusterId (string) --
The ID of the Amazon EKS cluster where Amazon EMR on EKS jobs run.
namespace (string) --
The namespace of the Amazon EKS cluster where the system jobs run.
queryEngineRoleArn (string) --
The query engine IAM role ARN that is tied to the secure Spark job. The QueryEngine role assumes the JobExecutionRole to execute all the Lake Formation calls.
encryptionConfiguration (dict) --
Encryption-related configuration input for the security configuration.
inTransitEncryptionConfiguration (dict) --
In-transit encryption-related input for the security configuration.
tlsCertificateConfiguration (dict) --
TLS certificate-related configuration input for the security configuration.
certificateProviderType (string) --
The TLS certificate type. Acceptable values: PEM or Custom .
publicCertificateSecretArn (string) --
Secrets Manager ARN that contains the public TLS certificate contents, used for communication between the user job and the system job.
privateCertificateSecretArn (string) --
Secrets Manager ARN that contains the private TLS certificate contents, used for communication between the user job and the system job.
tags (dict) --
The tags to assign to the security configuration.
(string) --
(string) --
Creates a security configuration. Security configurations in Amazon EMR on EKS are templates for different security setups. You can use security configurations to configure the Lake Formation integration setup. You can also create a security configuration to re-use a security setup each time you create a virtual cluster.
See also: AWS API Documentation
Request Syntax
client.create_security_configuration(
clientToken='string',
name='string',
securityConfigurationData={
'authorizationConfiguration': {
'lakeFormationConfiguration': {
'authorizedSessionTagValue': 'string',
'secureNamespaceInfo': {
'clusterId': 'string',
'namespace': 'string'
},
'queryEngineRoleArn': 'string'
},
'encryptionConfiguration': {
'inTransitEncryptionConfiguration': {
'tlsCertificateConfiguration': {
'certificateProviderType': 'PEM',
'publicCertificateSecretArn': 'string',
'privateCertificateSecretArn': 'string'
}
}
}
}
},
tags={
'string': 'string'
}
)
string
[REQUIRED]
The client idempotency token to use when creating the security configuration.
This field is autopopulated if not provided.
string
[REQUIRED]
The name of the security configuration.
dict
[REQUIRED]
Security configuration input for the request.
authorizationConfiguration (dict) --
Authorization-related configuration input for the security configuration.
lakeFormationConfiguration (dict) --
Lake Formation related configuration inputs for the security configuration.
authorizedSessionTagValue (string) --
The session tag to authorize Amazon EMR on EKS for API calls to Lake Formation.
secureNamespaceInfo (dict) --
The namespace input of the system job.
clusterId (string) --
The ID of the Amazon EKS cluster where Amazon EMR on EKS jobs run.
namespace (string) --
The namespace of the Amazon EKS cluster where the system jobs run.
queryEngineRoleArn (string) --
The query engine IAM role ARN that is tied to the secure Spark job. The QueryEngine role assumes the JobExecutionRole to execute all the Lake Formation calls.
encryptionConfiguration (dict) --
Encryption-related configuration input for the security configuration.
inTransitEncryptionConfiguration (dict) --
In-transit encryption-related input for the security configuration.
tlsCertificateConfiguration (dict) --
TLS certificate-related configuration input for the security configuration.
certificateProviderType (string) --
The TLS certificate type. Acceptable values: PEM or Custom .
publicCertificateSecretArn (string) --
Secrets Manager ARN that contains the public TLS certificate contents, used for communication between the user job and the system job.
privateCertificateSecretArn (string) --
Secrets Manager ARN that contains the private TLS certificate contents, used for communication between the user job and the system job.
dict
The tags to add to the security configuration.
(string) --
(string) --
dict
Response Syntax
{
'id': 'string',
'name': 'string',
'arn': 'string'
}
Response Structure
(dict) --
id (string) --
The ID of the security configuration.
name (string) --
The name of the security configuration.
arn (string) --
The ARN (Amazon Resource Name) of the security configuration.
{'securityConfigurationId': 'string'}
Creates a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.
See also: AWS API Documentation
Request Syntax
client.create_virtual_cluster(
name='string',
containerProvider={
'type': 'EKS',
'id': 'string',
'info': {
'eksInfo': {
'namespace': 'string'
}
}
},
clientToken='string',
tags={
'string': 'string'
},
securityConfigurationId='string'
)
string
[REQUIRED]
The specified name of the virtual cluster.
dict
[REQUIRED]
The container provider of the virtual cluster.
type (string) -- [REQUIRED]
The type of the container provider. Amazon EKS is the only supported type as of now.
id (string) -- [REQUIRED]
The ID of the container cluster.
info (dict) --
The information about the container cluster.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set: eksInfo.
eksInfo (dict) --
The information about the Amazon EKS cluster.
namespace (string) --
The namespaces of the Amazon EKS cluster.
string
[REQUIRED]
The client token of the virtual cluster.
This field is autopopulated if not provided.
dict
The tags assigned to the virtual cluster.
(string) --
(string) --
string
The ID of the security configuration.
dict
Response Syntax
{
'id': 'string',
'name': 'string',
'arn': 'string'
}
Response Structure
(dict) --
id (string) --
This output contains the virtual cluster ID.
name (string) --
This output contains the name of the virtual cluster.
arn (string) --
This output contains the ARN of virtual cluster.
{'virtualCluster': {'securityConfigurationId': 'string'}}
Displays detailed information about a specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.
See also: AWS API Documentation
Request Syntax
client.describe_virtual_cluster(
id='string'
)
string
[REQUIRED]
The ID of the virtual cluster that will be described.
dict
Response Syntax
{
'virtualCluster': {
'id': 'string',
'name': 'string',
'arn': 'string',
'state': 'RUNNING'|'TERMINATING'|'TERMINATED'|'ARRESTED',
'containerProvider': {
'type': 'EKS',
'id': 'string',
'info': {
'eksInfo': {
'namespace': 'string'
}
}
},
'createdAt': datetime(2015, 1, 1),
'tags': {
'string': 'string'
},
'securityConfigurationId': 'string'
}
}
Response Structure
(dict) --
virtualCluster (dict) --
This output displays information about the specified virtual cluster.
id (string) --
The ID of the virtual cluster.
name (string) --
The name of the virtual cluster.
arn (string) --
The ARN of the virtual cluster.
state (string) --
The state of the virtual cluster.
containerProvider (dict) --
The container provider of the virtual cluster.
type (string) --
The type of the container provider. Amazon EKS is the only supported type as of now.
id (string) --
The ID of the container cluster.
info (dict) --
The information about the container cluster.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: eksInfo. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
eksInfo (dict) --
The information about the Amazon EKS cluster.
namespace (string) --
The namespaces of the Amazon EKS cluster.
createdAt (datetime) --
The date and time when the virtual cluster is created.
tags (dict) --
The assigned tags of the virtual cluster.
(string) --
(string) --
securityConfigurationId (string) --
The ID of the security configuration.
{'virtualClusters': {'securityConfigurationId': 'string'}}
Lists information about the specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.
See also: AWS API Documentation
Request Syntax
client.list_virtual_clusters(
containerProviderId='string',
containerProviderType='EKS',
createdAfter=datetime(2015, 1, 1),
createdBefore=datetime(2015, 1, 1),
states=[
'RUNNING'|'TERMINATING'|'TERMINATED'|'ARRESTED',
],
maxResults=123,
nextToken='string',
eksAccessEntryIntegrated=True|False
)
string
The container provider ID of the virtual cluster.
string
The container provider type of the virtual cluster. Amazon EKS is the only supported type as of now.
datetime
The date and time after which the virtual clusters are created.
datetime
The date and time before which the virtual clusters are created.
list
The states of the requested virtual clusters.
(string) --
integer
The maximum number of virtual clusters that can be listed.
string
The token for the next set of virtual clusters to return.
boolean
Optional Boolean that specifies whether the operation should return the virtual clusters that have the access entry integration enabled or disabled. If not specified, the operation returns all applicable virtual clusters.
dict
Response Syntax
{
'virtualClusters': [
{
'id': 'string',
'name': 'string',
'arn': 'string',
'state': 'RUNNING'|'TERMINATING'|'TERMINATED'|'ARRESTED',
'containerProvider': {
'type': 'EKS',
'id': 'string',
'info': {
'eksInfo': {
'namespace': 'string'
}
}
},
'createdAt': datetime(2015, 1, 1),
'tags': {
'string': 'string'
},
'securityConfigurationId': 'string'
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
virtualClusters (list) --
This output lists the specified virtual clusters.
(dict) --
This entity describes a virtual cluster. A virtual cluster is a Kubernetes namespace that Amazon EMR is registered with. Amazon EMR uses virtual clusters to run jobs and host endpoints. Multiple virtual clusters can be backed by the same physical cluster. However, each virtual cluster maps to one namespace on an Amazon EKS cluster. Virtual clusters do not create any active resources that contribute to your bill or that require lifecycle management outside the service.
id (string) --
The ID of the virtual cluster.
name (string) --
The name of the virtual cluster.
arn (string) --
The ARN of the virtual cluster.
state (string) --
The state of the virtual cluster.
containerProvider (dict) --
The container provider of the virtual cluster.
type (string) --
The type of the container provider. Amazon EKS is the only supported type as of now.
id (string) --
The ID of the container cluster.
info (dict) --
The information about the container cluster.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: eksInfo. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
eksInfo (dict) --
The information about the Amazon EKS cluster.
namespace (string) --
The namespaces of the Amazon EKS cluster.
createdAt (datetime) --
The date and time when the virtual cluster is created.
tags (dict) --
The assigned tags of the virtual cluster.
(string) --
(string) --
securityConfigurationId (string) --
The ID of the security configuration.
nextToken (string) --
This output displays the token for the next set of virtual clusters.