AWS Glue

2018/03/20 - AWS Glue - 2 updated api methods

Changes  Update glue client to latest version

GetDevEndpoint (updated) Link ¶
Changes (response)
{'DevEndpoint': {'PrivateAddress': 'string'}}

Retrieves information about a specified DevEndpoint.

See also: AWS API Documentation

Request Syntax

client.get_dev_endpoint(
    EndpointName='string'
)
type EndpointName

string

param EndpointName

[REQUIRED]

Name of the DevEndpoint for which to retrieve information.

rtype

dict

returns

Response Syntax

{
    'DevEndpoint': {
        'EndpointName': 'string',
        'RoleArn': 'string',
        'SecurityGroupIds': [
            'string',
        ],
        'SubnetId': 'string',
        'YarnEndpointAddress': 'string',
        'PrivateAddress': 'string',
        'ZeppelinRemoteSparkInterpreterPort': 123,
        'PublicAddress': 'string',
        'Status': 'string',
        'NumberOfNodes': 123,
        'AvailabilityZone': 'string',
        'VpcId': 'string',
        'ExtraPythonLibsS3Path': 'string',
        'ExtraJarsS3Path': 'string',
        'FailureReason': 'string',
        'LastUpdateStatus': 'string',
        'CreatedTimestamp': datetime(2015, 1, 1),
        'LastModifiedTimestamp': datetime(2015, 1, 1),
        'PublicKey': 'string'
    }
}

Response Structure

  • (dict) --

    • DevEndpoint (dict) --

      A DevEndpoint definition.

      • EndpointName (string) --

        The name of the DevEndpoint.

      • RoleArn (string) --

        The AWS ARN of the IAM role used in this DevEndpoint.

      • SecurityGroupIds (list) --

        A list of security group identifiers used in this DevEndpoint.

        • (string) --

      • SubnetId (string) --

        The subnet ID for this DevEndpoint.

      • YarnEndpointAddress (string) --

        The YARN endpoint address used by this DevEndpoint.

      • PrivateAddress (string) --

        The private address used by this DevEndpoint.

      • ZeppelinRemoteSparkInterpreterPort (integer) --

        The Apache Zeppelin port for the remote Apache Spark interpreter.

      • PublicAddress (string) --

        The public VPC address used by this DevEndpoint.

      • Status (string) --

        The current status of this DevEndpoint.

      • NumberOfNodes (integer) --

        The number of AWS Glue Data Processing Units (DPUs) allocated to this DevEndpoint.

      • AvailabilityZone (string) --

        The AWS availability zone where this DevEndpoint is located.

      • VpcId (string) --

        The ID of the virtual private cloud (VPC) used by this DevEndpoint.

      • ExtraPythonLibsS3Path (string) --

        Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

        Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.

      • ExtraJarsS3Path (string) --

        Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.

        Please note that only pure Java/Scala libraries can currently be used on a DevEndpoint.

      • FailureReason (string) --

        The reason for a current failure in this DevEndpoint.

      • LastUpdateStatus (string) --

        The status of the last update.

      • CreatedTimestamp (datetime) --

        The point in time at which this DevEndpoint was created.

      • LastModifiedTimestamp (datetime) --

        The point in time at which this DevEndpoint was last modified.

      • PublicKey (string) --

        The public key to be used by this DevEndpoint for authentication.

GetDevEndpoints (updated) Link ¶
Changes (response)
{'DevEndpoints': {'PrivateAddress': 'string'}}

Retrieves all the DevEndpoints in this AWS account.

See also: AWS API Documentation

Request Syntax

client.get_dev_endpoints(
    MaxResults=123,
    NextToken='string'
)
type MaxResults

integer

param MaxResults

The maximum size of information to return.

type NextToken

string

param NextToken

A continuation token, if this is a continuation call.

rtype

dict

returns

Response Syntax

{
    'DevEndpoints': [
        {
            'EndpointName': 'string',
            'RoleArn': 'string',
            'SecurityGroupIds': [
                'string',
            ],
            'SubnetId': 'string',
            'YarnEndpointAddress': 'string',
            'PrivateAddress': 'string',
            'ZeppelinRemoteSparkInterpreterPort': 123,
            'PublicAddress': 'string',
            'Status': 'string',
            'NumberOfNodes': 123,
            'AvailabilityZone': 'string',
            'VpcId': 'string',
            'ExtraPythonLibsS3Path': 'string',
            'ExtraJarsS3Path': 'string',
            'FailureReason': 'string',
            'LastUpdateStatus': 'string',
            'CreatedTimestamp': datetime(2015, 1, 1),
            'LastModifiedTimestamp': datetime(2015, 1, 1),
            'PublicKey': 'string'
        },
    ],
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • DevEndpoints (list) --

      A list of DevEndpoint definitions.

      • (dict) --

        A development endpoint where a developer can remotely debug ETL scripts.

        • EndpointName (string) --

          The name of the DevEndpoint.

        • RoleArn (string) --

          The AWS ARN of the IAM role used in this DevEndpoint.

        • SecurityGroupIds (list) --

          A list of security group identifiers used in this DevEndpoint.

          • (string) --

        • SubnetId (string) --

          The subnet ID for this DevEndpoint.

        • YarnEndpointAddress (string) --

          The YARN endpoint address used by this DevEndpoint.

        • PrivateAddress (string) --

          The private address used by this DevEndpoint.

        • ZeppelinRemoteSparkInterpreterPort (integer) --

          The Apache Zeppelin port for the remote Apache Spark interpreter.

        • PublicAddress (string) --

          The public VPC address used by this DevEndpoint.

        • Status (string) --

          The current status of this DevEndpoint.

        • NumberOfNodes (integer) --

          The number of AWS Glue Data Processing Units (DPUs) allocated to this DevEndpoint.

        • AvailabilityZone (string) --

          The AWS availability zone where this DevEndpoint is located.

        • VpcId (string) --

          The ID of the virtual private cloud (VPC) used by this DevEndpoint.

        • ExtraPythonLibsS3Path (string) --

          Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

          Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.

        • ExtraJarsS3Path (string) --

          Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.

          Please note that only pure Java/Scala libraries can currently be used on a DevEndpoint.

        • FailureReason (string) --

          The reason for a current failure in this DevEndpoint.

        • LastUpdateStatus (string) --

          The status of the last update.

        • CreatedTimestamp (datetime) --

          The point in time at which this DevEndpoint was created.

        • LastModifiedTimestamp (datetime) --

          The point in time at which this DevEndpoint was last modified.

        • PublicKey (string) --

          The public key to be used by this DevEndpoint for authentication.

    • NextToken (string) --

      A continuation token, if not all DevEndpoint definitions have yet been returned.