EMR Serverless

2026/04/22 - EMR Serverless - 6 new3 updated api methods

Changes  This release adds support for Spark connect sessions starting with release label emr-7.13.0.

GetSessionEndpoint (new) Link ¶

Returns the session endpoint URL and a time-limited authentication token for the specified session. Use the endpoint and token to connect a client to the session. Call this operation again when the authentication token expires to obtain a new token.

See also: AWS API Documentation

Request Syntax

client.get_session_endpoint(
    applicationId='string',
    sessionId='string'
)
type applicationId:

string

param applicationId:

[REQUIRED]

The ID of the application that the session belongs to.

type sessionId:

string

param sessionId:

[REQUIRED]

The ID of the session.

rtype:

dict

returns:

Response Syntax

{
    'applicationId': 'string',
    'sessionId': 'string',
    'endpoint': 'string',
    'authToken': 'string',
    'authTokenExpiresAt': datetime(2015, 1, 1)
}

Response Structure

  • (dict) --

    • applicationId (string) --

      The output contains the ID of the application.

    • sessionId (string) --

      The output contains the ID of the session.

    • endpoint (string) --

      The endpoint URL for connecting to the session.

    • authToken (string) --

      Authentication token for accessing the session endpoint.

    • authTokenExpiresAt (datetime) --

      The expiration time of the authentication token.

TerminateSession (new) Link ¶

Terminates the specified session. After you terminate a session, it enters the TERMINATING state and then the TERMINATED state. You can still access the Spark History Server for a terminated session through the GetResourceDashboard operation.

See also: AWS API Documentation

Request Syntax

client.terminate_session(
    applicationId='string',
    sessionId='string'
)
type applicationId:

string

param applicationId:

[REQUIRED]

The ID of the application that the session belongs to.

type sessionId:

string

param sessionId:

[REQUIRED]

The ID of the session to terminate.

rtype:

dict

returns:

Response Syntax

{
    'applicationId': 'string',
    'sessionId': 'string'
}

Response Structure

  • (dict) --

    • applicationId (string) --

      The output contains the application ID on which the session was terminated.

    • sessionId (string) --

      The output contains the ID of the terminated session.

GetSession (new) Link ¶

Displays detailed information about a session.

See also: AWS API Documentation

Request Syntax

client.get_session(
    applicationId='string',
    sessionId='string'
)
type applicationId:

string

param applicationId:

[REQUIRED]

The ID of the application that the session belongs to.

type sessionId:

string

param sessionId:

[REQUIRED]

The ID of the session.

rtype:

dict

returns:

Response Syntax

{
    'session': {
        'applicationId': 'string',
        'sessionId': 'string',
        'arn': 'string',
        'name': 'string',
        'state': 'SUBMITTED'|'STARTING'|'STARTED'|'IDLE'|'BUSY'|'FAILED'|'TERMINATING'|'TERMINATED',
        'stateDetails': 'string',
        'releaseLabel': 'string',
        'executionRoleArn': 'string',
        'createdBy': 'string',
        'createdAt': datetime(2015, 1, 1),
        'updatedAt': datetime(2015, 1, 1),
        'startedAt': datetime(2015, 1, 1),
        'endedAt': datetime(2015, 1, 1),
        'idleSince': datetime(2015, 1, 1),
        'configurationOverrides': {
            'runtimeConfiguration': [
                {
                    'classification': 'string',
                    'properties': {
                        'string': 'string'
                    },
                    'configurations': {'... recursive ...'}
                },
            ]
        },
        'networkConfiguration': {
            'subnetIds': [
                'string',
            ],
            'securityGroupIds': [
                'string',
            ]
        },
        'idleTimeoutMinutes': 123,
        'tags': {
            'string': 'string'
        },
        'totalResourceUtilization': {
            'vCPUHour': 123.0,
            'memoryGBHour': 123.0,
            'storageGBHour': 123.0
        },
        'billedResourceUtilization': {
            'vCPUHour': 123.0,
            'memoryGBHour': 123.0,
            'storageGBHour': 123.0
        },
        'totalExecutionDurationSeconds': 123
    }
}

Response Structure

  • (dict) --

    • session (dict) --

      The output displays information about the session.

      • applicationId (string) --

        The ID of the application that the session belongs to.

      • sessionId (string) --

        The ID of the session.

      • arn (string) --

        The Amazon Resource Name (ARN) of the session.

      • name (string) --

        The optional name of the session.

      • state (string) --

        The state of the session.

      • stateDetails (string) --

        Additional details about the current state of the session.

      • releaseLabel (string) --

        The Amazon EMR release label associated with the session.

      • executionRoleArn (string) --

        The Amazon Resource Name (ARN) of the execution role for the session.

      • createdBy (string) --

        The IAM principal that created the session.

      • createdAt (datetime) --

        The date and time that the session was created.

      • updatedAt (datetime) --

        The date and time that the session was last updated.

      • startedAt (datetime) --

        The date and time that the session moved to a running state.

      • endedAt (datetime) --

        The date and time that the session was terminated or failed.

      • idleSince (datetime) --

        The date and time that the session became idle.

      • configurationOverrides (dict) --

        The configuration overrides for the session, including runtime configuration properties.

        • runtimeConfiguration (list) --

          The runtime configuration for the session. Contains Spark configuration properties specified at session creation time.

          • (dict) --

            A configuration specification to be used when provisioning an application. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file.

            • classification (string) --

              The classification within a configuration.

            • properties (dict) --

              A set of properties specified within a configuration classification.

              • (string) --

                • (string) --

            • configurations (list) --

              A list of additional configurations to apply within a configuration object.

      • networkConfiguration (dict) --

        The network configuration for customer VPC connectivity for the session.

        • subnetIds (list) --

          The array of subnet Ids for customer VPC connectivity.

          • (string) --

        • securityGroupIds (list) --

          The array of security group Ids for customer VPC connectivity.

          • (string) --

      • idleTimeoutMinutes (integer) --

        The idle timeout in minutes for the session. After the session remains idle for this duration, it is automatically terminated.

      • tags (dict) --

        The tags assigned to the session.

        • (string) --

          • (string) --

      • totalResourceUtilization (dict) --

        The aggregate vCPU, memory, and storage resources used from the time the session starts to execute, until the time the session terminates, rounded up to the nearest second.

        • vCPUHour (float) --

          The aggregated vCPU used per hour from the time job start executing till the time job is terminated.

        • memoryGBHour (float) --

          The aggregated memory used per hour from the time job start executing till the time job is terminated.

        • storageGBHour (float) --

          The aggregated storage used per hour from the time job start executing till the time job is terminated.

      • billedResourceUtilization (dict) --

        The aggregate vCPU, memory, and storage that Amazon Web Services has billed for the session. The billed resources include a 1-minute minimum usage for workers, plus additional storage over 20 GB per worker. Note that billed resources do not include usage for idle pre-initialized workers.

        • vCPUHour (float) --

          The aggregated vCPU used per hour from the time the job starts executing until the job is terminated.

        • memoryGBHour (float) --

          The aggregated memory used per hour from the time the job starts executing until the job is terminated.

        • storageGBHour (float) --

          The aggregated storage used per hour from the time the job starts executing until the job is terminated.

      • totalExecutionDurationSeconds (integer) --

        The total execution duration of the session in seconds.

GetResourceDashboard (new) Link ¶

Returns a URL that you can use to access the application UIs for a specified resource, such as a session.

For resources in a running state, the application UI is a live user interface such as the Spark web UI. For terminated resources, the application UI is a persistent application user interface such as the Spark History Server.

See also: AWS API Documentation

Request Syntax

client.get_resource_dashboard(
    applicationId='string',
    resourceId='string',
    resourceType='SESSION'
)
type applicationId:

string

param applicationId:

[REQUIRED]

The ID of the application that the resource belongs to.

type resourceId:

string

param resourceId:

[REQUIRED]

The ID of the resource.

type resourceType:

string

param resourceType:

[REQUIRED]

The type of resource to access the dashboard for. Currently, only Session is supported.

rtype:

dict

returns:

Response Syntax

{
    'url': 'string'
}

Response Structure

  • (dict) --

    • url (string) --

      A URL to the resource dashboard. For an active resource, this URL opens the live application UI. For a terminated resource, this URL opens the persistent application UI. This value is not included in the response if the URL is not available.

StartSession (new) Link ¶

Creates and starts a new session on the specified application. The application must be in the STARTED state or have AutoStart enabled, and have interactive sessions enabled. This operation is supported for EMR release 7.13.0 and later.

See also: AWS API Documentation

Request Syntax

client.start_session(
    applicationId='string',
    clientToken='string',
    executionRoleArn='string',
    configurationOverrides={
        'runtimeConfiguration': [
            {
                'classification': 'string',
                'properties': {
                    'string': 'string'
                },
                'configurations': {'... recursive ...'}
            },
        ]
    },
    tags={
        'string': 'string'
    },
    idleTimeoutMinutes=123,
    name='string'
)
type applicationId:

string

param applicationId:

[REQUIRED]

The ID of the application on which to start the session.

type clientToken:

string

param clientToken:

[REQUIRED]

A unique, case-sensitive identifier that you provide to ensure the idempotency of the request. If you retry a request that completed successfully using the same client token, the server returns the successful response without performing the operation again.

This field is autopopulated if not provided.

type executionRoleArn:

string

param executionRoleArn:

[REQUIRED]

The execution role ARN for the session. Amazon EMR Serverless uses this role to access Amazon Web Services resources on your behalf during session execution.

type configurationOverrides:

dict

param configurationOverrides:

The configuration overrides for the session. Only runtime configuration overrides are supported.

  • runtimeConfiguration (list) --

    The runtime configuration for the session. Contains Spark configuration properties specified at session creation time.

    • (dict) --

      A configuration specification to be used when provisioning an application. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file.

      • classification (string) -- [REQUIRED]

        The classification within a configuration.

      • properties (dict) --

        A set of properties specified within a configuration classification.

        • (string) --

          • (string) --

      • configurations (list) --

        A list of additional configurations to apply within a configuration object.

type tags:

dict

param tags:

The tags to assign to the session.

  • (string) --

    • (string) --

type idleTimeoutMinutes:

integer

param idleTimeoutMinutes:

The idle timeout in minutes for the session. After the session remains idle for this duration, Amazon EMR Serverless automatically terminates it.

type name:

string

param name:

The optional name for the session.

rtype:

dict

returns:

Response Syntax

{
    'applicationId': 'string',
    'sessionId': 'string',
    'arn': 'string'
}

Response Structure

  • (dict) --

    • applicationId (string) --

      The output contains the application ID on which the session was started.

    • sessionId (string) --

      The output contains the ID of the session.

    • arn (string) --

      The output contains the ARN of the session.

ListSessions (new) Link ¶

Lists sessions for the specified application. You can filter sessions by state and creation time.

See also: AWS API Documentation

Request Syntax

client.list_sessions(
    applicationId='string',
    nextToken='string',
    maxResults=123,
    states=[
        'SUBMITTED'|'STARTING'|'STARTED'|'IDLE'|'BUSY'|'FAILED'|'TERMINATING'|'TERMINATED',
    ],
    createdAtAfter=datetime(2015, 1, 1),
    createdAtBefore=datetime(2015, 1, 1)
)
type applicationId:

string

param applicationId:

[REQUIRED]

The ID of the application to list sessions for.

type nextToken:

string

param nextToken:

The token for the next set of session results.

type maxResults:

integer

param maxResults:

The maximum number of sessions to return in each page of results.

type states:

list

param states:

An optional filter for session states. Note that if this filter contains multiple states, the resulting list will be grouped by the state.

  • (string) --

type createdAtAfter:

datetime

param createdAtAfter:

The lower bound of the option to filter by creation date and time.

type createdAtBefore:

datetime

param createdAtBefore:

The upper bound of the option to filter by creation date and time.

rtype:

dict

returns:

Response Syntax

{
    'sessions': [
        {
            'applicationId': 'string',
            'sessionId': 'string',
            'arn': 'string',
            'name': 'string',
            'state': 'SUBMITTED'|'STARTING'|'STARTED'|'IDLE'|'BUSY'|'FAILED'|'TERMINATING'|'TERMINATED',
            'stateDetails': 'string',
            'releaseLabel': 'string',
            'executionRoleArn': 'string',
            'createdBy': 'string',
            'createdAt': datetime(2015, 1, 1),
            'updatedAt': datetime(2015, 1, 1)
        },
    ],
    'nextToken': 'string'
}

Response Structure

  • (dict) --

    • sessions (list) --

      The output lists information about the specified sessions.

      • (dict) --

        The summary of attributes associated with a session.

        • applicationId (string) --

          The ID of the application that the session belongs to.

        • sessionId (string) --

          The ID of the session.

        • arn (string) --

          The Amazon Resource Name (ARN) of the session.

        • name (string) --

          The optional name of the session.

        • state (string) --

          The state of the session.

        • stateDetails (string) --

          Additional details about the current state of the session.

        • releaseLabel (string) --

          The Amazon EMR release label associated with the session.

        • executionRoleArn (string) --

          The Amazon Resource Name (ARN) of the execution role for the session.

        • createdBy (string) --

          The IAM principal that created the session.

        • createdAt (datetime) --

          The date and time that the session was created.

        • updatedAt (datetime) --

          The date and time that the session was last updated.

    • nextToken (string) --

      The output displays the token for the next set of session results. This is required for pagination and is available as a response of the previous request.

CreateApplication (updated) Link ¶
Changes (request)
{'interactiveConfiguration': {'sessionEnabled': 'boolean'}}

Creates an application.

See also: AWS API Documentation

Request Syntax

client.create_application(
    name='string',
    releaseLabel='string',
    type='string',
    clientToken='string',
    initialCapacity={
        'string': {
            'workerCount': 123,
            'workerConfiguration': {
                'cpu': 'string',
                'memory': 'string',
                'disk': 'string',
                'diskType': 'string'
            }
        }
    },
    maximumCapacity={
        'cpu': 'string',
        'memory': 'string',
        'disk': 'string'
    },
    tags={
        'string': 'string'
    },
    autoStartConfiguration={
        'enabled': True|False
    },
    autoStopConfiguration={
        'enabled': True|False,
        'idleTimeoutMinutes': 123
    },
    networkConfiguration={
        'subnetIds': [
            'string',
        ],
        'securityGroupIds': [
            'string',
        ]
    },
    architecture='ARM64'|'X86_64',
    imageConfiguration={
        'imageUri': 'string'
    },
    workerTypeSpecifications={
        'string': {
            'imageConfiguration': {
                'imageUri': 'string'
            }
        }
    },
    runtimeConfiguration=[
        {
            'classification': 'string',
            'properties': {
                'string': 'string'
            },
            'configurations': {'... recursive ...'}
        },
    ],
    monitoringConfiguration={
        's3MonitoringConfiguration': {
            'logUri': 'string',
            'encryptionKeyArn': 'string'
        },
        'managedPersistenceMonitoringConfiguration': {
            'enabled': True|False,
            'encryptionKeyArn': 'string'
        },
        'cloudWatchLoggingConfiguration': {
            'enabled': True|False,
            'logGroupName': 'string',
            'logStreamNamePrefix': 'string',
            'encryptionKeyArn': 'string',
            'logTypes': {
                'string': [
                    'string',
                ]
            }
        },
        'prometheusMonitoringConfiguration': {
            'remoteWriteUrl': 'string'
        }
    },
    diskEncryptionConfiguration={
        'encryptionContext': {
            'string': 'string'
        },
        'encryptionKeyArn': 'string'
    },
    interactiveConfiguration={
        'studioEnabled': True|False,
        'livyEndpointEnabled': True|False,
        'sessionEnabled': True|False
    },
    schedulerConfiguration={
        'queueTimeoutMinutes': 123,
        'maxConcurrentRuns': 123
    },
    identityCenterConfiguration={
        'identityCenterInstanceArn': 'string',
        'userBackgroundSessionsEnabled': True|False
    },
    jobLevelCostAllocationConfiguration={
        'enabled': True|False
    }
)
type name:

string

param name:

The name of the application.

type releaseLabel:

string

param releaseLabel:

[REQUIRED]

The Amazon EMR release associated with the application.

type type:

string

param type:

[REQUIRED]

The type of application you want to start, such as Spark or Hive.

type clientToken:

string

param clientToken:

[REQUIRED]

The client idempotency token of the application to create. Its value must be unique for each request.

This field is autopopulated if not provided.

type initialCapacity:

dict

param initialCapacity:

The capacity to initialize when the application is created.

  • (string) --

    Worker type for an analytics framework.

    • (dict) --

      The initial capacity configuration per worker.

      • workerCount (integer) -- [REQUIRED]

        The number of workers in the initial capacity configuration.

      • workerConfiguration (dict) --

        The resource configuration of the initial capacity configuration.

        • cpu (string) -- [REQUIRED]

          The CPU requirements for every worker instance of the worker type.

        • memory (string) -- [REQUIRED]

          The memory requirements for every worker instance of the worker type.

        • disk (string) --

          The disk requirements for every worker instance of the worker type.

        • diskType (string) --

          The disk type for every worker instance of the work type. Shuffle optimized disks have higher performance characteristics and are better for shuffle heavy workloads. Default is STANDARD.

type maximumCapacity:

dict

param maximumCapacity:

The maximum capacity to allocate when the application is created. This is cumulative across all workers at any given point in time, not just when an application is created. No new resources will be created once any one of the defined limits is hit.

  • cpu (string) -- [REQUIRED]

    The maximum allowed CPU for an application.

  • memory (string) -- [REQUIRED]

    The maximum allowed resources for an application.

  • disk (string) --

    The maximum allowed disk for an application.

type tags:

dict

param tags:

The tags assigned to the application.

  • (string) --

    • (string) --

type autoStartConfiguration:

dict

param autoStartConfiguration:

The configuration for an application to automatically start on job submission.

  • enabled (boolean) --

    Enables the application to automatically start on job submission. Defaults to true.

type autoStopConfiguration:

dict

param autoStopConfiguration:

The configuration for an application to automatically stop after a certain amount of time being idle.

  • enabled (boolean) --

    Enables the application to automatically stop after a certain amount of time being idle. Defaults to true.

  • idleTimeoutMinutes (integer) --

    The amount of idle time in minutes after which your application will automatically stop. Defaults to 15 minutes.

type networkConfiguration:

dict

param networkConfiguration:

The network configuration for customer VPC connectivity.

  • subnetIds (list) --

    The array of subnet Ids for customer VPC connectivity.

    • (string) --

  • securityGroupIds (list) --

    The array of security group Ids for customer VPC connectivity.

    • (string) --

type architecture:

string

param architecture:

The CPU architecture of an application.

type imageConfiguration:

dict

param imageConfiguration:

The image configuration for all worker types. You can either set this parameter or imageConfiguration for each worker type in workerTypeSpecifications.

  • imageUri (string) --

    The URI of an image in the Amazon ECR registry. This field is required when you create a new application. If you leave this field blank in an update, Amazon EMR will remove the image configuration.

type workerTypeSpecifications:

dict

param workerTypeSpecifications:

The key-value pairs that specify worker type to WorkerTypeSpecificationInput. This parameter must contain all valid worker types for a Spark or Hive application. Valid worker types include Driver and Executor for Spark applications and HiveDriver and TezTask for Hive applications. You can either set image details in this parameter for each worker type, or in imageConfiguration for all worker types.

  • (string) --

    Worker type for an analytics framework.

    • (dict) --

      The specifications for a worker type.

      • imageConfiguration (dict) --

        The image configuration for a worker type.

        • imageUri (string) --

          The URI of an image in the Amazon ECR registry. This field is required when you create a new application. If you leave this field blank in an update, Amazon EMR will remove the image configuration.

type runtimeConfiguration:

list

param runtimeConfiguration:

The Configuration specifications to use when creating an application. Each configuration consists of a classification and properties. This configuration is applied to all the job runs submitted under the application.

  • (dict) --

    A configuration specification to be used when provisioning an application. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file.

    • classification (string) -- [REQUIRED]

      The classification within a configuration.

    • properties (dict) --

      A set of properties specified within a configuration classification.

      • (string) --

        • (string) --

    • configurations (list) --

      A list of additional configurations to apply within a configuration object.

type monitoringConfiguration:

dict

param monitoringConfiguration:

The configuration setting for monitoring.

  • s3MonitoringConfiguration (dict) --

    The Amazon S3 configuration for monitoring log publishing.

    • logUri (string) --

      The Amazon S3 destination URI for log publishing.

    • encryptionKeyArn (string) --

      The KMS key ARN to encrypt the logs published to the given Amazon S3 destination.

  • managedPersistenceMonitoringConfiguration (dict) --

    The managed log persistence configuration for a job run.

    • enabled (boolean) --

      Enables managed logging and defaults to true. If set to false, managed logging will be turned off.

    • encryptionKeyArn (string) --

      The KMS key ARN to encrypt the logs stored in managed log persistence.

  • cloudWatchLoggingConfiguration (dict) --

    The Amazon CloudWatch configuration for monitoring logs. You can configure your jobs to send log information to CloudWatch.

    • enabled (boolean) -- [REQUIRED]

      Enables CloudWatch logging.

    • logGroupName (string) --

      The name of the log group in Amazon CloudWatch Logs where you want to publish your logs.

    • logStreamNamePrefix (string) --

      Prefix for the CloudWatch log stream name.

    • encryptionKeyArn (string) --

      The Key Management Service (KMS) key ARN to encrypt the logs that you store in CloudWatch Logs.

    • logTypes (dict) --

      The types of logs that you want to publish to CloudWatch. If you don't specify any log types, driver STDOUT and STDERR logs will be published to CloudWatch Logs by default. For more information including the supported worker types for Hive and Spark, see Logging for EMR Serverless with CloudWatch.

      • Key Valid Values: SPARK_DRIVER, SPARK_EXECUTOR, HIVE_DRIVER, TEZ_TASK

      • Array Members Valid Values: STDOUT, STDERR, HIVE_LOG, TEZ_AM, SYSTEM_LOGS

      • (string) --

        Worker type for an analytics framework.

        • (list) --

          • (string) --

            Log type for a Spark/Hive job-run.

  • prometheusMonitoringConfiguration (dict) --

    The monitoring configuration object you can configure to send metrics to Amazon Managed Service for Prometheus for a job run.

    • remoteWriteUrl (string) --

      The remote write URL in the Amazon Managed Service for Prometheus workspace to send metrics to.

type diskEncryptionConfiguration:

dict

param diskEncryptionConfiguration:

The configuration object that allows encrypting local disks.

  • encryptionContext (dict) --

    Specifies the optional encryption context that will be used when encrypting the data. An encryption context is a collection of non-secret key-value pairs that represent additional authenticated data.

    • (string) --

      • (string) --

  • encryptionKeyArn (string) --

    The KMS key ARN to encrypt local disks.

type interactiveConfiguration:

dict

param interactiveConfiguration:

The interactive configuration object that enables the interactive use cases to use when running an application.

  • studioEnabled (boolean) --

    Enables you to connect an application to Amazon EMR Studio to run interactive workloads in a notebook.

  • livyEndpointEnabled (boolean) --

    Enables an Apache Livy endpoint that you can connect to and run interactive jobs.

  • sessionEnabled (boolean) --

    Enables interactive sessions on the application. When set to true, you can start interactive sessions using the StartSession operation.

type schedulerConfiguration:

dict

param schedulerConfiguration:

The scheduler configuration for batch and streaming jobs running on this application. Supported with release labels emr-7.0.0 and above.

  • queueTimeoutMinutes (integer) --

    The maximum duration in minutes for the job in QUEUED state. If scheduler configuration is enabled on your application, the default value is 360 minutes (6 hours). The valid range is from 15 to 720.

  • maxConcurrentRuns (integer) --

    The maximum concurrent job runs on this application. If scheduler configuration is enabled on your application, the default value is 15. The valid range is 1 to 1000.

type identityCenterConfiguration:

dict

param identityCenterConfiguration:

The IAM Identity Center Configuration accepts the Identity Center instance parameter required to enable trusted identity propagation. This configuration allows identity propagation between integrated services and the Identity Center instance.

  • identityCenterInstanceArn (string) --

    The ARN of the IAM Identity Center instance.

  • userBackgroundSessionsEnabled (boolean) --

    Enables user background sessions for this application so Livy sessions can continue running after users log out of their interactive notebook or their Identity Center sessions expire.

type jobLevelCostAllocationConfiguration:

dict

param jobLevelCostAllocationConfiguration:

The configuration object that enables job level cost allocation.

  • enabled (boolean) --

    Enables job level cost allocation for the application.

rtype:

dict

returns:

Response Syntax

{
    'applicationId': 'string',
    'name': 'string',
    'arn': 'string'
}

Response Structure

  • (dict) --

    • applicationId (string) --

      The output contains the application ID.

    • name (string) --

      The output contains the name of the application.

    • arn (string) --

      The output contains the ARN of the application.

GetApplication (updated) Link ¶
Changes (response)
{'application': {'interactiveConfiguration': {'sessionEnabled': 'boolean'}}}

Displays detailed information about a specified application.

See also: AWS API Documentation

Request Syntax

client.get_application(
    applicationId='string'
)
type applicationId:

string

param applicationId:

[REQUIRED]

The ID of the application that will be described.

rtype:

dict

returns:

Response Syntax

{
    'application': {
        'applicationId': 'string',
        'name': 'string',
        'arn': 'string',
        'releaseLabel': 'string',
        'type': 'string',
        'state': 'CREATING'|'CREATED'|'STARTING'|'STARTED'|'STOPPING'|'STOPPED'|'TERMINATED',
        'stateDetails': 'string',
        'initialCapacity': {
            'string': {
                'workerCount': 123,
                'workerConfiguration': {
                    'cpu': 'string',
                    'memory': 'string',
                    'disk': 'string',
                    'diskType': 'string'
                }
            }
        },
        'maximumCapacity': {
            'cpu': 'string',
            'memory': 'string',
            'disk': 'string'
        },
        'createdAt': datetime(2015, 1, 1),
        'updatedAt': datetime(2015, 1, 1),
        'tags': {
            'string': 'string'
        },
        'autoStartConfiguration': {
            'enabled': True|False
        },
        'autoStopConfiguration': {
            'enabled': True|False,
            'idleTimeoutMinutes': 123
        },
        'networkConfiguration': {
            'subnetIds': [
                'string',
            ],
            'securityGroupIds': [
                'string',
            ]
        },
        'architecture': 'ARM64'|'X86_64',
        'imageConfiguration': {
            'imageUri': 'string',
            'resolvedImageDigest': 'string'
        },
        'workerTypeSpecifications': {
            'string': {
                'imageConfiguration': {
                    'imageUri': 'string',
                    'resolvedImageDigest': 'string'
                }
            }
        },
        'runtimeConfiguration': [
            {
                'classification': 'string',
                'properties': {
                    'string': 'string'
                },
                'configurations': {'... recursive ...'}
            },
        ],
        'monitoringConfiguration': {
            's3MonitoringConfiguration': {
                'logUri': 'string',
                'encryptionKeyArn': 'string'
            },
            'managedPersistenceMonitoringConfiguration': {
                'enabled': True|False,
                'encryptionKeyArn': 'string'
            },
            'cloudWatchLoggingConfiguration': {
                'enabled': True|False,
                'logGroupName': 'string',
                'logStreamNamePrefix': 'string',
                'encryptionKeyArn': 'string',
                'logTypes': {
                    'string': [
                        'string',
                    ]
                }
            },
            'prometheusMonitoringConfiguration': {
                'remoteWriteUrl': 'string'
            }
        },
        'diskEncryptionConfiguration': {
            'encryptionContext': {
                'string': 'string'
            },
            'encryptionKeyArn': 'string'
        },
        'interactiveConfiguration': {
            'studioEnabled': True|False,
            'livyEndpointEnabled': True|False,
            'sessionEnabled': True|False
        },
        'schedulerConfiguration': {
            'queueTimeoutMinutes': 123,
            'maxConcurrentRuns': 123
        },
        'identityCenterConfiguration': {
            'identityCenterInstanceArn': 'string',
            'identityCenterApplicationArn': 'string',
            'userBackgroundSessionsEnabled': True|False
        },
        'jobLevelCostAllocationConfiguration': {
            'enabled': True|False
        }
    }
}

Response Structure

  • (dict) --

    • application (dict) --

      The output displays information about the specified application.

      • applicationId (string) --

        The ID of the application.

      • name (string) --

        The name of the application.

      • arn (string) --

        The ARN of the application.

      • releaseLabel (string) --

        The Amazon EMR release associated with the application.

      • type (string) --

        The type of application, such as Spark or Hive.

      • state (string) --

        The state of the application.

      • stateDetails (string) --

        The state details of the application.

      • initialCapacity (dict) --

        The initial capacity of the application.

        • (string) --

          Worker type for an analytics framework.

          • (dict) --

            The initial capacity configuration per worker.

            • workerCount (integer) --

              The number of workers in the initial capacity configuration.

            • workerConfiguration (dict) --

              The resource configuration of the initial capacity configuration.

              • cpu (string) --

                The CPU requirements for every worker instance of the worker type.

              • memory (string) --

                The memory requirements for every worker instance of the worker type.

              • disk (string) --

                The disk requirements for every worker instance of the worker type.

              • diskType (string) --

                The disk type for every worker instance of the work type. Shuffle optimized disks have higher performance characteristics and are better for shuffle heavy workloads. Default is STANDARD.

      • maximumCapacity (dict) --

        The maximum capacity of the application. This is cumulative across all workers at any given point in time during the lifespan of the application is created. No new resources will be created once any one of the defined limits is hit.

        • cpu (string) --

          The maximum allowed CPU for an application.

        • memory (string) --

          The maximum allowed resources for an application.

        • disk (string) --

          The maximum allowed disk for an application.

      • createdAt (datetime) --

        The date and time when the application run was created.

      • updatedAt (datetime) --

        The date and time when the application run was last updated.

      • tags (dict) --

        The tags assigned to the application.

        • (string) --

          • (string) --

      • autoStartConfiguration (dict) --

        The configuration for an application to automatically start on job submission.

        • enabled (boolean) --

          Enables the application to automatically start on job submission. Defaults to true.

      • autoStopConfiguration (dict) --

        The configuration for an application to automatically stop after a certain amount of time being idle.

        • enabled (boolean) --

          Enables the application to automatically stop after a certain amount of time being idle. Defaults to true.

        • idleTimeoutMinutes (integer) --

          The amount of idle time in minutes after which your application will automatically stop. Defaults to 15 minutes.

      • networkConfiguration (dict) --

        The network configuration for customer VPC connectivity for the application.

        • subnetIds (list) --

          The array of subnet Ids for customer VPC connectivity.

          • (string) --

        • securityGroupIds (list) --

          The array of security group Ids for customer VPC connectivity.

          • (string) --

      • architecture (string) --

        The CPU architecture of an application.

      • imageConfiguration (dict) --

        The image configuration applied to all worker types.

        • imageUri (string) --

          The image URI.

        • resolvedImageDigest (string) --

          The SHA256 digest of the image URI. This indicates which specific image the application is configured for. The image digest doesn't exist until an application has started.

      • workerTypeSpecifications (dict) --

        The specification applied to each worker type.

        • (string) --

          Worker type for an analytics framework.

          • (dict) --

            The specifications for a worker type.

            • imageConfiguration (dict) --

              The image configuration for a worker type.

              • imageUri (string) --

                The image URI.

              • resolvedImageDigest (string) --

                The SHA256 digest of the image URI. This indicates which specific image the application is configured for. The image digest doesn't exist until an application has started.

      • runtimeConfiguration (list) --

        The Configuration specifications of an application. Each configuration consists of a classification and properties. You use this parameter when creating or updating an application. To see the runtimeConfiguration object of an application, run the GetApplication API operation.

        • (dict) --

          A configuration specification to be used when provisioning an application. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file.

          • classification (string) --

            The classification within a configuration.

          • properties (dict) --

            A set of properties specified within a configuration classification.

            • (string) --

              • (string) --

          • configurations (list) --

            A list of additional configurations to apply within a configuration object.

      • monitoringConfiguration (dict) --

        The configuration setting for monitoring.

        • s3MonitoringConfiguration (dict) --

          The Amazon S3 configuration for monitoring log publishing.

          • logUri (string) --

            The Amazon S3 destination URI for log publishing.

          • encryptionKeyArn (string) --

            The KMS key ARN to encrypt the logs published to the given Amazon S3 destination.

        • managedPersistenceMonitoringConfiguration (dict) --

          The managed log persistence configuration for a job run.

          • enabled (boolean) --

            Enables managed logging and defaults to true. If set to false, managed logging will be turned off.

          • encryptionKeyArn (string) --

            The KMS key ARN to encrypt the logs stored in managed log persistence.

        • cloudWatchLoggingConfiguration (dict) --

          The Amazon CloudWatch configuration for monitoring logs. You can configure your jobs to send log information to CloudWatch.

          • enabled (boolean) --

            Enables CloudWatch logging.

          • logGroupName (string) --

            The name of the log group in Amazon CloudWatch Logs where you want to publish your logs.

          • logStreamNamePrefix (string) --

            Prefix for the CloudWatch log stream name.

          • encryptionKeyArn (string) --

            The Key Management Service (KMS) key ARN to encrypt the logs that you store in CloudWatch Logs.

          • logTypes (dict) --

            The types of logs that you want to publish to CloudWatch. If you don't specify any log types, driver STDOUT and STDERR logs will be published to CloudWatch Logs by default. For more information including the supported worker types for Hive and Spark, see Logging for EMR Serverless with CloudWatch.

            • Key Valid Values: SPARK_DRIVER, SPARK_EXECUTOR, HIVE_DRIVER, TEZ_TASK

            • Array Members Valid Values: STDOUT, STDERR, HIVE_LOG, TEZ_AM, SYSTEM_LOGS

            • (string) --

              Worker type for an analytics framework.

              • (list) --

                • (string) --

                  Log type for a Spark/Hive job-run.

        • prometheusMonitoringConfiguration (dict) --

          The monitoring configuration object you can configure to send metrics to Amazon Managed Service for Prometheus for a job run.

          • remoteWriteUrl (string) --

            The remote write URL in the Amazon Managed Service for Prometheus workspace to send metrics to.

      • diskEncryptionConfiguration (dict) --

        The configuration object that allows encrypting local disks.

        • encryptionContext (dict) --

          Specifies the optional encryption context that will be used when encrypting the data. An encryption context is a collection of non-secret key-value pairs that represent additional authenticated data.

          • (string) --

            • (string) --

        • encryptionKeyArn (string) --

          The KMS key ARN to encrypt local disks.

      • interactiveConfiguration (dict) --

        The interactive configuration object that enables the interactive use cases for an application.

        • studioEnabled (boolean) --

          Enables you to connect an application to Amazon EMR Studio to run interactive workloads in a notebook.

        • livyEndpointEnabled (boolean) --

          Enables an Apache Livy endpoint that you can connect to and run interactive jobs.

        • sessionEnabled (boolean) --

          Enables interactive sessions on the application. When set to true, you can start interactive sessions using the StartSession operation.

      • schedulerConfiguration (dict) --

        The scheduler configuration for batch and streaming jobs running on this application. Supported with release labels emr-7.0.0 and above.

        • queueTimeoutMinutes (integer) --

          The maximum duration in minutes for the job in QUEUED state. If scheduler configuration is enabled on your application, the default value is 360 minutes (6 hours). The valid range is from 15 to 720.

        • maxConcurrentRuns (integer) --

          The maximum concurrent job runs on this application. If scheduler configuration is enabled on your application, the default value is 15. The valid range is 1 to 1000.

      • identityCenterConfiguration (dict) --

        The IAM Identity Center configuration applied to enable trusted identity propagation.

        • identityCenterInstanceArn (string) --

          The ARN of the IAM Identity Center instance.

        • identityCenterApplicationArn (string) --

          The ARN of the EMR Serverless created IAM Identity Center Application that provides trusted-identity propagation.

        • userBackgroundSessionsEnabled (boolean) --

          Enables user background sessions for this application so Livy sessions can continue running after users log out of their interactive notebook or their Identity Center sessions expire.

      • jobLevelCostAllocationConfiguration (dict) --

        The configuration object that enables job level cost allocation.

        • enabled (boolean) --

          Enables job level cost allocation for the application.

UpdateApplication (updated) Link ¶
Changes (request, response)
Request
{'interactiveConfiguration': {'sessionEnabled': 'boolean'}}
Response
{'application': {'interactiveConfiguration': {'sessionEnabled': 'boolean'}}}

Updates a specified application. An application has to be in a stopped or created state in order to be updated.

See also: AWS API Documentation

Request Syntax

client.update_application(
    applicationId='string',
    clientToken='string',
    initialCapacity={
        'string': {
            'workerCount': 123,
            'workerConfiguration': {
                'cpu': 'string',
                'memory': 'string',
                'disk': 'string',
                'diskType': 'string'
            }
        }
    },
    maximumCapacity={
        'cpu': 'string',
        'memory': 'string',
        'disk': 'string'
    },
    autoStartConfiguration={
        'enabled': True|False
    },
    autoStopConfiguration={
        'enabled': True|False,
        'idleTimeoutMinutes': 123
    },
    networkConfiguration={
        'subnetIds': [
            'string',
        ],
        'securityGroupIds': [
            'string',
        ]
    },
    architecture='ARM64'|'X86_64',
    imageConfiguration={
        'imageUri': 'string'
    },
    workerTypeSpecifications={
        'string': {
            'imageConfiguration': {
                'imageUri': 'string'
            }
        }
    },
    interactiveConfiguration={
        'studioEnabled': True|False,
        'livyEndpointEnabled': True|False,
        'sessionEnabled': True|False
    },
    releaseLabel='string',
    runtimeConfiguration=[
        {
            'classification': 'string',
            'properties': {
                'string': 'string'
            },
            'configurations': {'... recursive ...'}
        },
    ],
    monitoringConfiguration={
        's3MonitoringConfiguration': {
            'logUri': 'string',
            'encryptionKeyArn': 'string'
        },
        'managedPersistenceMonitoringConfiguration': {
            'enabled': True|False,
            'encryptionKeyArn': 'string'
        },
        'cloudWatchLoggingConfiguration': {
            'enabled': True|False,
            'logGroupName': 'string',
            'logStreamNamePrefix': 'string',
            'encryptionKeyArn': 'string',
            'logTypes': {
                'string': [
                    'string',
                ]
            }
        },
        'prometheusMonitoringConfiguration': {
            'remoteWriteUrl': 'string'
        }
    },
    diskEncryptionConfiguration={
        'encryptionContext': {
            'string': 'string'
        },
        'encryptionKeyArn': 'string'
    },
    schedulerConfiguration={
        'queueTimeoutMinutes': 123,
        'maxConcurrentRuns': 123
    },
    identityCenterConfiguration={
        'identityCenterInstanceArn': 'string',
        'userBackgroundSessionsEnabled': True|False
    },
    jobLevelCostAllocationConfiguration={
        'enabled': True|False
    }
)
type applicationId:

string

param applicationId:

[REQUIRED]

The ID of the application to update.

type clientToken:

string

param clientToken:

[REQUIRED]

The client idempotency token of the application to update. Its value must be unique for each request.

This field is autopopulated if not provided.

type initialCapacity:

dict

param initialCapacity:

The capacity to initialize when the application is updated.

  • (string) --

    Worker type for an analytics framework.

    • (dict) --

      The initial capacity configuration per worker.

      • workerCount (integer) -- [REQUIRED]

        The number of workers in the initial capacity configuration.

      • workerConfiguration (dict) --

        The resource configuration of the initial capacity configuration.

        • cpu (string) -- [REQUIRED]

          The CPU requirements for every worker instance of the worker type.

        • memory (string) -- [REQUIRED]

          The memory requirements for every worker instance of the worker type.

        • disk (string) --

          The disk requirements for every worker instance of the worker type.

        • diskType (string) --

          The disk type for every worker instance of the work type. Shuffle optimized disks have higher performance characteristics and are better for shuffle heavy workloads. Default is STANDARD.

type maximumCapacity:

dict

param maximumCapacity:

The maximum capacity to allocate when the application is updated. This is cumulative across all workers at any given point in time during the lifespan of the application. No new resources will be created once any one of the defined limits is hit.

  • cpu (string) -- [REQUIRED]

    The maximum allowed CPU for an application.

  • memory (string) -- [REQUIRED]

    The maximum allowed resources for an application.

  • disk (string) --

    The maximum allowed disk for an application.

type autoStartConfiguration:

dict

param autoStartConfiguration:

The configuration for an application to automatically start on job submission.

  • enabled (boolean) --

    Enables the application to automatically start on job submission. Defaults to true.

type autoStopConfiguration:

dict

param autoStopConfiguration:

The configuration for an application to automatically stop after a certain amount of time being idle.

  • enabled (boolean) --

    Enables the application to automatically stop after a certain amount of time being idle. Defaults to true.

  • idleTimeoutMinutes (integer) --

    The amount of idle time in minutes after which your application will automatically stop. Defaults to 15 minutes.

type networkConfiguration:

dict

param networkConfiguration:

The network configuration for customer VPC connectivity.

  • subnetIds (list) --

    The array of subnet Ids for customer VPC connectivity.

    • (string) --

  • securityGroupIds (list) --

    The array of security group Ids for customer VPC connectivity.

    • (string) --

type architecture:

string

param architecture:

The CPU architecture of an application.

type imageConfiguration:

dict

param imageConfiguration:

The image configuration to be used for all worker types. You can either set this parameter or imageConfiguration for each worker type in WorkerTypeSpecificationInput.

  • imageUri (string) --

    The URI of an image in the Amazon ECR registry. This field is required when you create a new application. If you leave this field blank in an update, Amazon EMR will remove the image configuration.

type workerTypeSpecifications:

dict

param workerTypeSpecifications:

The key-value pairs that specify worker type to WorkerTypeSpecificationInput. This parameter must contain all valid worker types for a Spark or Hive application. Valid worker types include Driver and Executor for Spark applications and HiveDriver and TezTask for Hive applications. You can either set image details in this parameter for each worker type, or in imageConfiguration for all worker types.

  • (string) --

    Worker type for an analytics framework.

    • (dict) --

      The specifications for a worker type.

      • imageConfiguration (dict) --

        The image configuration for a worker type.

        • imageUri (string) --

          The URI of an image in the Amazon ECR registry. This field is required when you create a new application. If you leave this field blank in an update, Amazon EMR will remove the image configuration.

type interactiveConfiguration:

dict

param interactiveConfiguration:

The interactive configuration object that contains new interactive use cases when the application is updated.

  • studioEnabled (boolean) --

    Enables you to connect an application to Amazon EMR Studio to run interactive workloads in a notebook.

  • livyEndpointEnabled (boolean) --

    Enables an Apache Livy endpoint that you can connect to and run interactive jobs.

  • sessionEnabled (boolean) --

    Enables interactive sessions on the application. When set to true, you can start interactive sessions using the StartSession operation.

type releaseLabel:

string

param releaseLabel:

The Amazon EMR release label for the application. You can change the release label to use a different release of Amazon EMR.

type runtimeConfiguration:

list

param runtimeConfiguration:

The Configuration specifications to use when updating an application. Each configuration consists of a classification and properties. This configuration is applied across all the job runs submitted under the application.

  • (dict) --

    A configuration specification to be used when provisioning an application. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file.

    • classification (string) -- [REQUIRED]

      The classification within a configuration.

    • properties (dict) --

      A set of properties specified within a configuration classification.

      • (string) --

        • (string) --

    • configurations (list) --

      A list of additional configurations to apply within a configuration object.

type monitoringConfiguration:

dict

param monitoringConfiguration:

The configuration setting for monitoring.

  • s3MonitoringConfiguration (dict) --

    The Amazon S3 configuration for monitoring log publishing.

    • logUri (string) --

      The Amazon S3 destination URI for log publishing.

    • encryptionKeyArn (string) --

      The KMS key ARN to encrypt the logs published to the given Amazon S3 destination.

  • managedPersistenceMonitoringConfiguration (dict) --

    The managed log persistence configuration for a job run.

    • enabled (boolean) --

      Enables managed logging and defaults to true. If set to false, managed logging will be turned off.

    • encryptionKeyArn (string) --

      The KMS key ARN to encrypt the logs stored in managed log persistence.

  • cloudWatchLoggingConfiguration (dict) --

    The Amazon CloudWatch configuration for monitoring logs. You can configure your jobs to send log information to CloudWatch.

    • enabled (boolean) -- [REQUIRED]

      Enables CloudWatch logging.

    • logGroupName (string) --

      The name of the log group in Amazon CloudWatch Logs where you want to publish your logs.

    • logStreamNamePrefix (string) --

      Prefix for the CloudWatch log stream name.

    • encryptionKeyArn (string) --

      The Key Management Service (KMS) key ARN to encrypt the logs that you store in CloudWatch Logs.

    • logTypes (dict) --

      The types of logs that you want to publish to CloudWatch. If you don't specify any log types, driver STDOUT and STDERR logs will be published to CloudWatch Logs by default. For more information including the supported worker types for Hive and Spark, see Logging for EMR Serverless with CloudWatch.

      • Key Valid Values: SPARK_DRIVER, SPARK_EXECUTOR, HIVE_DRIVER, TEZ_TASK

      • Array Members Valid Values: STDOUT, STDERR, HIVE_LOG, TEZ_AM, SYSTEM_LOGS

      • (string) --

        Worker type for an analytics framework.

        • (list) --

          • (string) --

            Log type for a Spark/Hive job-run.

  • prometheusMonitoringConfiguration (dict) --

    The monitoring configuration object you can configure to send metrics to Amazon Managed Service for Prometheus for a job run.

    • remoteWriteUrl (string) --

      The remote write URL in the Amazon Managed Service for Prometheus workspace to send metrics to.

type diskEncryptionConfiguration:

dict

param diskEncryptionConfiguration:

The configuration object that allows encrypting local disks.

  • encryptionContext (dict) --

    Specifies the optional encryption context that will be used when encrypting the data. An encryption context is a collection of non-secret key-value pairs that represent additional authenticated data.

    • (string) --

      • (string) --

  • encryptionKeyArn (string) --

    The KMS key ARN to encrypt local disks.

type schedulerConfiguration:

dict

param schedulerConfiguration:

The scheduler configuration for batch and streaming jobs running on this application. Supported with release labels emr-7.0.0 and above.

  • queueTimeoutMinutes (integer) --

    The maximum duration in minutes for the job in QUEUED state. If scheduler configuration is enabled on your application, the default value is 360 minutes (6 hours). The valid range is from 15 to 720.

  • maxConcurrentRuns (integer) --

    The maximum concurrent job runs on this application. If scheduler configuration is enabled on your application, the default value is 15. The valid range is 1 to 1000.

type identityCenterConfiguration:

dict

param identityCenterConfiguration:

Specifies the IAM Identity Center configuration used to enable or disable trusted identity propagation. When provided, this configuration determines how the application interacts with IAM Identity Center for user authentication and access control.

  • identityCenterInstanceArn (string) --

    The ARN of the IAM Identity Center instance.

  • userBackgroundSessionsEnabled (boolean) --

    Enables user background sessions for this application so Livy sessions can continue running after users log out of their interactive notebook or their Identity Center sessions expire.

type jobLevelCostAllocationConfiguration:

dict

param jobLevelCostAllocationConfiguration:

The configuration object that enables job level cost allocation.

  • enabled (boolean) --

    Enables job level cost allocation for the application.

rtype:

dict

returns:

Response Syntax

{
    'application': {
        'applicationId': 'string',
        'name': 'string',
        'arn': 'string',
        'releaseLabel': 'string',
        'type': 'string',
        'state': 'CREATING'|'CREATED'|'STARTING'|'STARTED'|'STOPPING'|'STOPPED'|'TERMINATED',
        'stateDetails': 'string',
        'initialCapacity': {
            'string': {
                'workerCount': 123,
                'workerConfiguration': {
                    'cpu': 'string',
                    'memory': 'string',
                    'disk': 'string',
                    'diskType': 'string'
                }
            }
        },
        'maximumCapacity': {
            'cpu': 'string',
            'memory': 'string',
            'disk': 'string'
        },
        'createdAt': datetime(2015, 1, 1),
        'updatedAt': datetime(2015, 1, 1),
        'tags': {
            'string': 'string'
        },
        'autoStartConfiguration': {
            'enabled': True|False
        },
        'autoStopConfiguration': {
            'enabled': True|False,
            'idleTimeoutMinutes': 123
        },
        'networkConfiguration': {
            'subnetIds': [
                'string',
            ],
            'securityGroupIds': [
                'string',
            ]
        },
        'architecture': 'ARM64'|'X86_64',
        'imageConfiguration': {
            'imageUri': 'string',
            'resolvedImageDigest': 'string'
        },
        'workerTypeSpecifications': {
            'string': {
                'imageConfiguration': {
                    'imageUri': 'string',
                    'resolvedImageDigest': 'string'
                }
            }
        },
        'runtimeConfiguration': [
            {
                'classification': 'string',
                'properties': {
                    'string': 'string'
                },
                'configurations': {'... recursive ...'}
            },
        ],
        'monitoringConfiguration': {
            's3MonitoringConfiguration': {
                'logUri': 'string',
                'encryptionKeyArn': 'string'
            },
            'managedPersistenceMonitoringConfiguration': {
                'enabled': True|False,
                'encryptionKeyArn': 'string'
            },
            'cloudWatchLoggingConfiguration': {
                'enabled': True|False,
                'logGroupName': 'string',
                'logStreamNamePrefix': 'string',
                'encryptionKeyArn': 'string',
                'logTypes': {
                    'string': [
                        'string',
                    ]
                }
            },
            'prometheusMonitoringConfiguration': {
                'remoteWriteUrl': 'string'
            }
        },
        'diskEncryptionConfiguration': {
            'encryptionContext': {
                'string': 'string'
            },
            'encryptionKeyArn': 'string'
        },
        'interactiveConfiguration': {
            'studioEnabled': True|False,
            'livyEndpointEnabled': True|False,
            'sessionEnabled': True|False
        },
        'schedulerConfiguration': {
            'queueTimeoutMinutes': 123,
            'maxConcurrentRuns': 123
        },
        'identityCenterConfiguration': {
            'identityCenterInstanceArn': 'string',
            'identityCenterApplicationArn': 'string',
            'userBackgroundSessionsEnabled': True|False
        },
        'jobLevelCostAllocationConfiguration': {
            'enabled': True|False
        }
    }
}

Response Structure

  • (dict) --

    • application (dict) --

      Information about the updated application.

      • applicationId (string) --

        The ID of the application.

      • name (string) --

        The name of the application.

      • arn (string) --

        The ARN of the application.

      • releaseLabel (string) --

        The Amazon EMR release associated with the application.

      • type (string) --

        The type of application, such as Spark or Hive.

      • state (string) --

        The state of the application.

      • stateDetails (string) --

        The state details of the application.

      • initialCapacity (dict) --

        The initial capacity of the application.

        • (string) --

          Worker type for an analytics framework.

          • (dict) --

            The initial capacity configuration per worker.

            • workerCount (integer) --

              The number of workers in the initial capacity configuration.

            • workerConfiguration (dict) --

              The resource configuration of the initial capacity configuration.

              • cpu (string) --

                The CPU requirements for every worker instance of the worker type.

              • memory (string) --

                The memory requirements for every worker instance of the worker type.

              • disk (string) --

                The disk requirements for every worker instance of the worker type.

              • diskType (string) --

                The disk type for every worker instance of the work type. Shuffle optimized disks have higher performance characteristics and are better for shuffle heavy workloads. Default is STANDARD.

      • maximumCapacity (dict) --

        The maximum capacity of the application. This is cumulative across all workers at any given point in time during the lifespan of the application is created. No new resources will be created once any one of the defined limits is hit.

        • cpu (string) --

          The maximum allowed CPU for an application.

        • memory (string) --

          The maximum allowed resources for an application.

        • disk (string) --

          The maximum allowed disk for an application.

      • createdAt (datetime) --

        The date and time when the application run was created.

      • updatedAt (datetime) --

        The date and time when the application run was last updated.

      • tags (dict) --

        The tags assigned to the application.

        • (string) --

          • (string) --

      • autoStartConfiguration (dict) --

        The configuration for an application to automatically start on job submission.

        • enabled (boolean) --

          Enables the application to automatically start on job submission. Defaults to true.

      • autoStopConfiguration (dict) --

        The configuration for an application to automatically stop after a certain amount of time being idle.

        • enabled (boolean) --

          Enables the application to automatically stop after a certain amount of time being idle. Defaults to true.

        • idleTimeoutMinutes (integer) --

          The amount of idle time in minutes after which your application will automatically stop. Defaults to 15 minutes.

      • networkConfiguration (dict) --

        The network configuration for customer VPC connectivity for the application.

        • subnetIds (list) --

          The array of subnet Ids for customer VPC connectivity.

          • (string) --

        • securityGroupIds (list) --

          The array of security group Ids for customer VPC connectivity.

          • (string) --

      • architecture (string) --

        The CPU architecture of an application.

      • imageConfiguration (dict) --

        The image configuration applied to all worker types.

        • imageUri (string) --

          The image URI.

        • resolvedImageDigest (string) --

          The SHA256 digest of the image URI. This indicates which specific image the application is configured for. The image digest doesn't exist until an application has started.

      • workerTypeSpecifications (dict) --

        The specification applied to each worker type.

        • (string) --

          Worker type for an analytics framework.

          • (dict) --

            The specifications for a worker type.

            • imageConfiguration (dict) --

              The image configuration for a worker type.

              • imageUri (string) --

                The image URI.

              • resolvedImageDigest (string) --

                The SHA256 digest of the image URI. This indicates which specific image the application is configured for. The image digest doesn't exist until an application has started.

      • runtimeConfiguration (list) --

        The Configuration specifications of an application. Each configuration consists of a classification and properties. You use this parameter when creating or updating an application. To see the runtimeConfiguration object of an application, run the GetApplication API operation.

        • (dict) --

          A configuration specification to be used when provisioning an application. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file.

          • classification (string) --

            The classification within a configuration.

          • properties (dict) --

            A set of properties specified within a configuration classification.

            • (string) --

              • (string) --

          • configurations (list) --

            A list of additional configurations to apply within a configuration object.

      • monitoringConfiguration (dict) --

        The configuration setting for monitoring.

        • s3MonitoringConfiguration (dict) --

          The Amazon S3 configuration for monitoring log publishing.

          • logUri (string) --

            The Amazon S3 destination URI for log publishing.

          • encryptionKeyArn (string) --

            The KMS key ARN to encrypt the logs published to the given Amazon S3 destination.

        • managedPersistenceMonitoringConfiguration (dict) --

          The managed log persistence configuration for a job run.

          • enabled (boolean) --

            Enables managed logging and defaults to true. If set to false, managed logging will be turned off.

          • encryptionKeyArn (string) --

            The KMS key ARN to encrypt the logs stored in managed log persistence.

        • cloudWatchLoggingConfiguration (dict) --

          The Amazon CloudWatch configuration for monitoring logs. You can configure your jobs to send log information to CloudWatch.

          • enabled (boolean) --

            Enables CloudWatch logging.

          • logGroupName (string) --

            The name of the log group in Amazon CloudWatch Logs where you want to publish your logs.

          • logStreamNamePrefix (string) --

            Prefix for the CloudWatch log stream name.

          • encryptionKeyArn (string) --

            The Key Management Service (KMS) key ARN to encrypt the logs that you store in CloudWatch Logs.

          • logTypes (dict) --

            The types of logs that you want to publish to CloudWatch. If you don't specify any log types, driver STDOUT and STDERR logs will be published to CloudWatch Logs by default. For more information including the supported worker types for Hive and Spark, see Logging for EMR Serverless with CloudWatch.

            • Key Valid Values: SPARK_DRIVER, SPARK_EXECUTOR, HIVE_DRIVER, TEZ_TASK

            • Array Members Valid Values: STDOUT, STDERR, HIVE_LOG, TEZ_AM, SYSTEM_LOGS

            • (string) --

              Worker type for an analytics framework.

              • (list) --

                • (string) --

                  Log type for a Spark/Hive job-run.

        • prometheusMonitoringConfiguration (dict) --

          The monitoring configuration object you can configure to send metrics to Amazon Managed Service for Prometheus for a job run.

          • remoteWriteUrl (string) --

            The remote write URL in the Amazon Managed Service for Prometheus workspace to send metrics to.

      • diskEncryptionConfiguration (dict) --

        The configuration object that allows encrypting local disks.

        • encryptionContext (dict) --

          Specifies the optional encryption context that will be used when encrypting the data. An encryption context is a collection of non-secret key-value pairs that represent additional authenticated data.

          • (string) --

            • (string) --

        • encryptionKeyArn (string) --

          The KMS key ARN to encrypt local disks.

      • interactiveConfiguration (dict) --

        The interactive configuration object that enables the interactive use cases for an application.

        • studioEnabled (boolean) --

          Enables you to connect an application to Amazon EMR Studio to run interactive workloads in a notebook.

        • livyEndpointEnabled (boolean) --

          Enables an Apache Livy endpoint that you can connect to and run interactive jobs.

        • sessionEnabled (boolean) --

          Enables interactive sessions on the application. When set to true, you can start interactive sessions using the StartSession operation.

      • schedulerConfiguration (dict) --

        The scheduler configuration for batch and streaming jobs running on this application. Supported with release labels emr-7.0.0 and above.

        • queueTimeoutMinutes (integer) --

          The maximum duration in minutes for the job in QUEUED state. If scheduler configuration is enabled on your application, the default value is 360 minutes (6 hours). The valid range is from 15 to 720.

        • maxConcurrentRuns (integer) --

          The maximum concurrent job runs on this application. If scheduler configuration is enabled on your application, the default value is 15. The valid range is 1 to 1000.

      • identityCenterConfiguration (dict) --

        The IAM Identity Center configuration applied to enable trusted identity propagation.

        • identityCenterInstanceArn (string) --

          The ARN of the IAM Identity Center instance.

        • identityCenterApplicationArn (string) --

          The ARN of the EMR Serverless created IAM Identity Center Application that provides trusted-identity propagation.

        • userBackgroundSessionsEnabled (boolean) --

          Enables user background sessions for this application so Livy sessions can continue running after users log out of their interactive notebook or their Identity Center sessions expire.

      • jobLevelCostAllocationConfiguration (dict) --

        The configuration object that enables job level cost allocation.

        • enabled (boolean) --

          Enables job level cost allocation for the application.