AWS API Changes

2024/04/24 - Amazon EMR Containers - 3 new3 updated api methods

Changes EMRonEKS Service support for SecurityConfiguration enforcement for Spark Jobs.

ListSecurityConfigurations (new)

Link ¶

Lists security configurations based on a set of parameters. Security configurations in Amazon EMR on EKS are templates for different security setups. You can use security configurations to configure the Lake Formation integration setup. You can also create a security configuration to re-use a security setup each time you create a virtual cluster.

See also: AWS API Documentation

Request Syntax

client.list_security_configurations(
    createdAfter=datetime(2015, 1, 1),
    createdBefore=datetime(2015, 1, 1),
    maxResults=123,
    nextToken='string'
)

type createdAfter:

datetime

param createdAfter:

The date and time after which the security configuration was created.

type createdBefore:

datetime

param createdBefore:

The date and time before which the security configuration was created.

type maxResults:

integer

param maxResults:

The maximum number of security configurations the operation can list.

type nextToken:

string

param nextToken:

The token for the next set of security configurations to return.

rtype:

dict

returns:

Response Syntax

{
    'securityConfigurations': [
        {
            'id': 'string',
            'name': 'string',
            'arn': 'string',
            'createdAt': datetime(2015, 1, 1),
            'createdBy': 'string',
            'securityConfigurationData': {
                'authorizationConfiguration': {
                    'lakeFormationConfiguration': {
                        'authorizedSessionTagValue': 'string',
                        'secureNamespaceInfo': {
                            'clusterId': 'string',
                            'namespace': 'string'
                        },
                        'queryEngineRoleArn': 'string'
                    },
                    'encryptionConfiguration': {
                        'inTransitEncryptionConfiguration': {
                            'tlsCertificateConfiguration': {
                                'certificateProviderType': 'PEM',
                                'publicCertificateSecretArn': 'string',
                                'privateCertificateSecretArn': 'string'
                            }
                        }
                    }
                }
            },
            'tags': {
                'string': 'string'
            }
        },
    ],
    'nextToken': 'string'
}

Response Structure

(dict) --
- securityConfigurations (list) --
  
  The list of returned security configurations.
  - (dict) --
    
    Inputs related to the security configuration. Security configurations in Amazon EMR on EKS are templates for different security setups. You can use security configurations to configure the Lake Formation integration setup. You can also create a security configuration to re-use a security setup each time you create a virtual cluster.
    - id (string) --
      
      The ID of the security configuration.
    - name (string) --
      
      The name of the security configuration.
    - arn (string) --
      
      The ARN (Amazon Resource Name) of the security configuration.
    - createdAt (datetime) --
      
      The date and time that the job run was created.
    - createdBy (string) --
      
      The user who created the job run.
    - securityConfigurationData (dict) --
      
      Security configuration inputs for the request.
      - authorizationConfiguration (dict) --
        
        Authorization-related configuration input for the security configuration.
        
        lakeFormationConfiguration (dict) --
        
        Lake Formation related configuration inputs for the security configuration.
        
        authorizedSessionTagValue (string) --
        
        The session tag to authorize Amazon EMR on EKS for API calls to Lake Formation.
        
        secureNamespaceInfo (dict) --
        
        The namespace input of the system job.
        
        clusterId (string) --
        
        The ID of the Amazon EKS cluster where Amazon EMR on EKS jobs run.
        
        namespace (string) --
        
        The namespace of the Amazon EKS cluster where the system jobs run.
        
        queryEngineRoleArn (string) --
        
        The query engine IAM role ARN that is tied to the secure Spark job. The QueryEngine role assumes the JobExecutionRole to execute all the Lake Formation calls.
        
        encryptionConfiguration (dict) --
        
        Encryption-related configuration input for the security configuration.
        
        inTransitEncryptionConfiguration (dict) --
        
        In-transit encryption-related input for the security configuration.
        
        tlsCertificateConfiguration (dict) --
        
        TLS certificate-related configuration input for the security configuration.
        
        certificateProviderType (string) --
        
        The TLS certificate type. Acceptable values: PEM or Custom.
        
        publicCertificateSecretArn (string) --
        
        Secrets Manager ARN that contains the public TLS certificate contents, used for communication between the user job and the system job.
        
        privateCertificateSecretArn (string) --
        
        Secrets Manager ARN that contains the private TLS certificate contents, used for communication between the user job and the system job.
    - tags (dict) --
      
      The tags to assign to the security configuration.
      - (string) --
        
        (string) --
- nextToken (string) --
  
  The token for the next set of security configurations to return.

DescribeSecurityConfiguration (new)

Link ¶

Displays detailed information about a specified security configuration. Security configurations in Amazon EMR on EKS are templates for different security setups. You can use security configurations to configure the Lake Formation integration setup. You can also create a security configuration to re-use a security setup each time you create a virtual cluster.

See also: AWS API Documentation

Request Syntax

client.describe_security_configuration(
    id='string'
)

type id:

string

param id:

[REQUIRED]

The ID of the security configuration.

rtype:

dict

returns:

Response Syntax

{
    'securityConfiguration': {
        'id': 'string',
        'name': 'string',
        'arn': 'string',
        'createdAt': datetime(2015, 1, 1),
        'createdBy': 'string',
        'securityConfigurationData': {
            'authorizationConfiguration': {
                'lakeFormationConfiguration': {
                    'authorizedSessionTagValue': 'string',
                    'secureNamespaceInfo': {
                        'clusterId': 'string',
                        'namespace': 'string'
                    },
                    'queryEngineRoleArn': 'string'
                },
                'encryptionConfiguration': {
                    'inTransitEncryptionConfiguration': {
                        'tlsCertificateConfiguration': {
                            'certificateProviderType': 'PEM',
                            'publicCertificateSecretArn': 'string',
                            'privateCertificateSecretArn': 'string'
                        }
                    }
                }
            }
        },
        'tags': {
            'string': 'string'
        }
    }
}

Response Structure

(dict) --
- securityConfiguration (dict) --
  
  Details of the security configuration.
  - id (string) --
    
    The ID of the security configuration.
  - name (string) --
    
    The name of the security configuration.
  - arn (string) --
    
    The ARN (Amazon Resource Name) of the security configuration.
  - createdAt (datetime) --
    
    The date and time that the job run was created.
  - createdBy (string) --
    
    The user who created the job run.
  - securityConfigurationData (dict) --
    
    Security configuration inputs for the request.
    - authorizationConfiguration (dict) --
      
      Authorization-related configuration input for the security configuration.
      - lakeFormationConfiguration (dict) --
        
        Lake Formation related configuration inputs for the security configuration.
        
        authorizedSessionTagValue (string) --
        
        The session tag to authorize Amazon EMR on EKS for API calls to Lake Formation.
        
        secureNamespaceInfo (dict) --
        
        The namespace input of the system job.
        
        clusterId (string) --
        
        The ID of the Amazon EKS cluster where Amazon EMR on EKS jobs run.
        
        namespace (string) --
        
        The namespace of the Amazon EKS cluster where the system jobs run.
        
        queryEngineRoleArn (string) --
        
        The query engine IAM role ARN that is tied to the secure Spark job. The QueryEngine role assumes the JobExecutionRole to execute all the Lake Formation calls.
      - encryptionConfiguration (dict) --
        
        Encryption-related configuration input for the security configuration.
        
        inTransitEncryptionConfiguration (dict) --
        
        In-transit encryption-related input for the security configuration.
        
        tlsCertificateConfiguration (dict) --
        
        TLS certificate-related configuration input for the security configuration.
        
        certificateProviderType (string) --
        
        The TLS certificate type. Acceptable values: PEM or Custom.
        
        publicCertificateSecretArn (string) --
        
        Secrets Manager ARN that contains the public TLS certificate contents, used for communication between the user job and the system job.
        
        privateCertificateSecretArn (string) --
        
        Secrets Manager ARN that contains the private TLS certificate contents, used for communication between the user job and the system job.
  - tags (dict) --
    
    The tags to assign to the security configuration.
    - (string) --
      - (string) --

CreateSecurityConfiguration (new)

Link ¶

Creates a security configuration. Security configurations in Amazon EMR on EKS are templates for different security setups. You can use security configurations to configure the Lake Formation integration setup. You can also create a security configuration to re-use a security setup each time you create a virtual cluster.

See also: AWS API Documentation

Request Syntax

client.create_security_configuration(
    clientToken='string',
    name='string',
    securityConfigurationData={
        'authorizationConfiguration': {
            'lakeFormationConfiguration': {
                'authorizedSessionTagValue': 'string',
                'secureNamespaceInfo': {
                    'clusterId': 'string',
                    'namespace': 'string'
                },
                'queryEngineRoleArn': 'string'
            },
            'encryptionConfiguration': {
                'inTransitEncryptionConfiguration': {
                    'tlsCertificateConfiguration': {
                        'certificateProviderType': 'PEM',
                        'publicCertificateSecretArn': 'string',
                        'privateCertificateSecretArn': 'string'
                    }
                }
            }
        }
    },
    tags={
        'string': 'string'
    }
)

type clientToken:

string

param clientToken:

[REQUIRED]

The client idempotency token to use when creating the security configuration.

This field is autopopulated if not provided.

type name:

string

param name:

[REQUIRED]

The name of the security configuration.

type securityConfigurationData:

dict

param securityConfigurationData:

[REQUIRED]

Security configuration input for the request.

authorizationConfiguration (dict) --

Authorization-related configuration input for the security configuration.
- lakeFormationConfiguration (dict) --
  
  Lake Formation related configuration inputs for the security configuration.
  - authorizedSessionTagValue (string) --
    
    The session tag to authorize Amazon EMR on EKS for API calls to Lake Formation.
  - secureNamespaceInfo (dict) --
    
    The namespace input of the system job.
    - clusterId (string) --
      
      The ID of the Amazon EKS cluster where Amazon EMR on EKS jobs run.
    - namespace (string) --
      
      The namespace of the Amazon EKS cluster where the system jobs run.
  - queryEngineRoleArn (string) --
    
    The query engine IAM role ARN that is tied to the secure Spark job. The QueryEngine role assumes the JobExecutionRole to execute all the Lake Formation calls.
- encryptionConfiguration (dict) --
  
  Encryption-related configuration input for the security configuration.
  - inTransitEncryptionConfiguration (dict) --
    
    In-transit encryption-related input for the security configuration.
    - tlsCertificateConfiguration (dict) --
      
      TLS certificate-related configuration input for the security configuration.
      - certificateProviderType (string) --
        
        The TLS certificate type. Acceptable values: PEM or Custom.
      - publicCertificateSecretArn (string) --
        
        Secrets Manager ARN that contains the public TLS certificate contents, used for communication between the user job and the system job.
      - privateCertificateSecretArn (string) --
        
        Secrets Manager ARN that contains the private TLS certificate contents, used for communication between the user job and the system job.

type tags:

dict

param tags:

The tags to add to the security configuration.

(string) --
- (string) --

rtype:

dict

returns:

Response Syntax

{
    'id': 'string',
    'name': 'string',
    'arn': 'string'
}

Response Structure

(dict) --
- id (string) --
  
  The ID of the security configuration.
- name (string) --
  
  The name of the security configuration.
- arn (string) --
  
  The ARN (Amazon Resource Name) of the security configuration.

CreateVirtualCluster (updated)

Link ¶
Changes (request)

{'securityConfigurationId': 'string'}

Creates a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

See also: AWS API Documentation

Request Syntax

client.create_virtual_cluster(
    name='string',
    containerProvider={
        'type': 'EKS',
        'id': 'string',
        'info': {
            'eksInfo': {
                'namespace': 'string'
            }
        }
    },
    clientToken='string',
    tags={
        'string': 'string'
    },
    securityConfigurationId='string'
)

type name:

string

param name:

[REQUIRED]

The specified name of the virtual cluster.

type containerProvider:

dict

param containerProvider:

[REQUIRED]

The container provider of the virtual cluster.

type (string) -- [REQUIRED]

The type of the container provider. Amazon EKS is the only supported type as of now.
id (string) -- [REQUIRED]

The ID of the container cluster.
info (dict) --

The information about the container cluster.

Note

This is a Tagged Union structure. Only one of the following top level keys can be set: eksInfo.
- eksInfo (dict) --
  
  The information about the Amazon EKS cluster.
  - namespace (string) --
    
    The namespaces of the Amazon EKS cluster.

type clientToken:

string

param clientToken:

[REQUIRED]

The client token of the virtual cluster.

This field is autopopulated if not provided.

type tags:

dict

param tags:

The tags assigned to the virtual cluster.

(string) --
- (string) --

type securityConfigurationId:

string

param securityConfigurationId:

The ID of the security configuration.

rtype:

dict

returns:

Response Syntax

{
    'id': 'string',
    'name': 'string',
    'arn': 'string'
}

Response Structure

(dict) --
- id (string) --
  
  This output contains the virtual cluster ID.
- name (string) --
  
  This output contains the name of the virtual cluster.
- arn (string) --
  
  This output contains the ARN of virtual cluster.

DescribeVirtualCluster (updated)

Link ¶
Changes (response)

{'virtualCluster': {'securityConfigurationId': 'string'}}

Displays detailed information about a specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

See also: AWS API Documentation

Request Syntax

client.describe_virtual_cluster(
    id='string'
)

type id:

string

param id:

[REQUIRED]

The ID of the virtual cluster that will be described.

rtype:

dict

returns:

Response Syntax

{
    'virtualCluster': {
        'id': 'string',
        'name': 'string',
        'arn': 'string',
        'state': 'RUNNING'|'TERMINATING'|'TERMINATED'|'ARRESTED',
        'containerProvider': {
            'type': 'EKS',
            'id': 'string',
            'info': {
                'eksInfo': {
                    'namespace': 'string'
                }
            }
        },
        'createdAt': datetime(2015, 1, 1),
        'tags': {
            'string': 'string'
        },
        'securityConfigurationId': 'string'
    }
}

Response Structure

(dict) --
- virtualCluster (dict) --
  
  This output displays information about the specified virtual cluster.
  - id (string) --
    
    The ID of the virtual cluster.
  - name (string) --
    
    The name of the virtual cluster.
  - arn (string) --
    
    The ARN of the virtual cluster.
  - state (string) --
    
    The state of the virtual cluster.
  - containerProvider (dict) --
    
    The container provider of the virtual cluster.
    - type (string) --
      
      The type of the container provider. Amazon EKS is the only supported type as of now.
    - id (string) --
      
      The ID of the container cluster.
    - info (dict) --
      
      The information about the container cluster.
      Note
      
      This is a Tagged Union structure. Only one of the following top level keys will be set: eksInfo. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
      
      'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
      - eksInfo (dict) --
        
        The information about the Amazon EKS cluster.
        
        namespace (string) --
        
        The namespaces of the Amazon EKS cluster.
  - createdAt (datetime) --
    
    The date and time when the virtual cluster is created.
  - tags (dict) --
    
    The assigned tags of the virtual cluster.
    - (string) --
      - (string) --
  - securityConfigurationId (string) --
    
    The ID of the security configuration.

ListVirtualClusters (updated)

Link ¶
Changes (response)

{'virtualClusters': {'securityConfigurationId': 'string'}}

Lists information about the specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

See also: AWS API Documentation

Request Syntax

client.list_virtual_clusters(
    containerProviderId='string',
    containerProviderType='EKS',
    createdAfter=datetime(2015, 1, 1),
    createdBefore=datetime(2015, 1, 1),
    states=[
        'RUNNING'|'TERMINATING'|'TERMINATED'|'ARRESTED',
    ],
    maxResults=123,
    nextToken='string',
    eksAccessEntryIntegrated=True|False
)

type containerProviderId:

string

param containerProviderId:

The container provider ID of the virtual cluster.

type containerProviderType:

string

param containerProviderType:

The container provider type of the virtual cluster. Amazon EKS is the only supported type as of now.

type createdAfter:

datetime

param createdAfter:

The date and time after which the virtual clusters are created.

type createdBefore:

datetime

param createdBefore:

The date and time before which the virtual clusters are created.

type states:

list

param states:

The states of the requested virtual clusters.

(string) --

type maxResults:

integer

param maxResults:

The maximum number of virtual clusters that can be listed.

type nextToken:

string

param nextToken:

The token for the next set of virtual clusters to return.

type eksAccessEntryIntegrated:

boolean

param eksAccessEntryIntegrated:

Optional Boolean that specifies whether the operation should return the virtual clusters that have the access entry integration enabled or disabled. If not specified, the operation returns all applicable virtual clusters.

rtype:

dict

returns:

Response Syntax

{
    'virtualClusters': [
        {
            'id': 'string',
            'name': 'string',
            'arn': 'string',
            'state': 'RUNNING'|'TERMINATING'|'TERMINATED'|'ARRESTED',
            'containerProvider': {
                'type': 'EKS',
                'id': 'string',
                'info': {
                    'eksInfo': {
                        'namespace': 'string'
                    }
                }
            },
            'createdAt': datetime(2015, 1, 1),
            'tags': {
                'string': 'string'
            },
            'securityConfigurationId': 'string'
        },
    ],
    'nextToken': 'string'
}

Response Structure

(dict) --
- virtualClusters (list) --
  
  This output lists the specified virtual clusters.
  - (dict) --
    
    This entity describes a virtual cluster. A virtual cluster is a Kubernetes namespace that Amazon EMR is registered with. Amazon EMR uses virtual clusters to run jobs and host endpoints. Multiple virtual clusters can be backed by the same physical cluster. However, each virtual cluster maps to one namespace on an Amazon EKS cluster. Virtual clusters do not create any active resources that contribute to your bill or that require lifecycle management outside the service.
    - id (string) --
      
      The ID of the virtual cluster.
    - name (string) --
      
      The name of the virtual cluster.
    - arn (string) --
      
      The ARN of the virtual cluster.
    - state (string) --
      
      The state of the virtual cluster.
    - containerProvider (dict) --
      
      The container provider of the virtual cluster.
      - type (string) --
        
        The type of the container provider. Amazon EKS is the only supported type as of now.
      - id (string) --
        
        The ID of the container cluster.
      - info (dict) --
        
        The information about the container cluster.
        
        Note
        
        This is a Tagged Union structure. Only one of the following top level keys will be set: eksInfo. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
        
        'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
        
        eksInfo (dict) --
        
        The information about the Amazon EKS cluster.
        
        namespace (string) --
        
        The namespaces of the Amazon EKS cluster.
    - createdAt (datetime) --
      
      The date and time when the virtual cluster is created.
    - tags (dict) --
      
      The assigned tags of the virtual cluster.
      - (string) --
        
        (string) --
    - securityConfigurationId (string) --
      
      The ID of the security configuration.
- nextToken (string) --
  
  This output displays the token for the next set of virtual clusters.