AWS Lake Formation

Allows a user or application in a secure environment to access data in a specific Amazon S3 location registered with Lake Formation by providing temporary scoped credentials that are limited to the requested data location and the caller's authorized access level.

The API operation returns an error in the following scenarios:

• The data location is not registered with Lake Formation.

• No Glue table is associated with the data location.

• The caller doesn't have required permissions on the associated table. The caller must have SELECT or SUPER permissions on the associated table, and credential vending for full table access must be enabled in the data lake settings. For more information, see Application integration for full table access.

• The data location is in a different Amazon Web Services Region. Lake Formation doesn't support cross-Region access when vending credentials for a data location. Lake Formation only supports Amazon S3 paths registered within the same Region as the API call.

See also: AWS API Documentation

Request Syntax

client.get_temporary_data_location_credentials(
    DurationSeconds=123,
    AuditContext={
        'AdditionalAuditContext': 'string'
    },
    DataLocations=[
        'string',
    ],
    CredentialsScope='READ'|'READWRITE'
)

type DurationSeconds:

integer

param DurationSeconds:

The time period, between 900 and 43,200 seconds, for the timeout of the temporary credentials.

type AuditContext:

dict

param AuditContext:

A structure used to include auditing information on the privileged API.

• AdditionalAuditContext (string) --

  The filter engine can populate the 'AdditionalAuditContext' information with the request ID for you to track. This information will be displayed in CloudTrail log in your account.

type DataLocations:

list

param DataLocations:

The Amazon S3 data location that you want to access.

• (string) --

type CredentialsScope:

string

param CredentialsScope:

The credential scope is determined by the caller's Lake Formation permission on the associated table. Credential scope can be either:

• READ - Provides read-only access to the data location.

• READ_WRITE - Provides both read and write access to the data location.

rtype:

dict

returns:

Response Syntax

{
    'Credentials': {
        'AccessKeyId': 'string',
        'SecretAccessKey': 'string',
        'SessionToken': 'string',
        'Expiration': datetime(2015, 1, 1)
    },
    'AccessibleDataLocations': [
        'string',
    ],
    'CredentialsScope': 'READ'|'READWRITE'
}

Response Structure

• (dict) --

  • Credentials (dict) --

    A temporary set of credentials for an Lake Formation user. These credentials are scoped down to only access the raw data sources that the user has access to.

    The temporary security credentials consist of an access key and a session token. The access key consists of an access key ID and a secret key. When the credentials are created, they are associated with an IAM access control policy that limits what the user can do when using the credentials.

    • AccessKeyId (string) --

      The access key ID for the temporary credentials.

    • SecretAccessKey (string) --

      The secret key for the temporary credentials.

    • SessionToken (string) --

      The session token for the temporary credentials.

    • Expiration (datetime) --

      The date and time when the temporary credentials expire.

  • AccessibleDataLocations (list) --

    Refers to the Amazon S3 locations that can be accessed through the GetTemporaryCredentialsForLocation API operation.

    • (string) --

  • CredentialsScope (string) --

    The credential scope is determined by the caller's Lake Formation permission on the associated table. Credential scope can be either:

    • READ - Provides read-only access to the data location.

    • READ_WRITE - Provides both read and write access to the data location.

{'ResourceInfo': {'ExpectedResourceOwnerAccount': 'string',
                  'VerificationStatus': 'VERIFIED | VERIFICATION_FAILED | '
                                        'NOT_VERIFIED'}}

Retrieves the current data access role for the given resource registered in Lake Formation.

See also: AWS API Documentation

Request Syntax

client.describe_resource(
    ResourceArn='string'
)

type ResourceArn:

string

param ResourceArn:

[REQUIRED]

The resource ARN.

rtype:

dict

returns:

Response Syntax

{
    'ResourceInfo': {
        'ResourceArn': 'string',
        'RoleArn': 'string',
        'LastModified': datetime(2015, 1, 1),
        'WithFederation': True|False,
        'HybridAccessEnabled': True|False,
        'WithPrivilegedAccess': True|False,
        'VerificationStatus': 'VERIFIED'|'VERIFICATION_FAILED'|'NOT_VERIFIED',
        'ExpectedResourceOwnerAccount': 'string'
    }
}

Response Structure

• (dict) --

  • ResourceInfo (dict) --

    A structure containing information about an Lake Formation resource.

    • ResourceArn (string) --

      The Amazon Resource Name (ARN) of the resource.

    • RoleArn (string) --

      The IAM role that registered a resource.

    • LastModified (datetime) --

      The date and time the resource was last modified.

    • WithFederation (boolean) --

      Whether or not the resource is a federated resource.

    • HybridAccessEnabled (boolean) --

      Indicates whether the data access of tables pointing to the location can be managed by both Lake Formation permissions as well as Amazon S3 bucket policies.

    • WithPrivilegedAccess (boolean) --

      Grants the calling principal the permissions to perform all supported Lake Formation operations on the registered data location.

    • VerificationStatus (string) --

      Indicates whether the registered role has sufficient permissions to access registered Amazon S3 location. Verification Status can be one of the following:

      • VERIFIED - Registered role has sufficient permissions to access registered Amazon S3 location.

      • NOT_VERIFIED - Registered role does not have sufficient permissions to access registered Amazon S3 location.

      • VERIFICATION_FAILED - Unable to verify if the registered role can access the registered Amazon S3 location.

    • ExpectedResourceOwnerAccount (string) --

      The Amazon Web Services account that owns the Glue tables associated with specific Amazon S3 locations.

{'ResourceInfoList': {'ExpectedResourceOwnerAccount': 'string',
                      'VerificationStatus': 'VERIFIED | VERIFICATION_FAILED | '
                                            'NOT_VERIFIED'}}

Lists the resources registered to be managed by the Data Catalog.

See also: AWS API Documentation

Request Syntax

client.list_resources(
    FilterConditionList=[
        {
            'Field': 'RESOURCE_ARN'|'ROLE_ARN'|'LAST_MODIFIED',
            'ComparisonOperator': 'EQ'|'NE'|'LE'|'LT'|'GE'|'GT'|'CONTAINS'|'NOT_CONTAINS'|'BEGINS_WITH'|'IN'|'BETWEEN',
            'StringValueList': [
                'string',
            ]
        },
    ],
    MaxResults=123,
    NextToken='string'
)

type FilterConditionList:

list

param FilterConditionList:

Any applicable row-level and/or column-level filtering conditions for the resources.

• (dict) --

  This structure describes the filtering of columns in a table based on a filter condition.

  • Field (string) --

    The field to filter in the filter condition.

  • ComparisonOperator (string) --

    The comparison operator used in the filter condition.

  • StringValueList (list) --

    A string with values used in evaluating the filter condition.

    • (string) --

type MaxResults:

integer

param MaxResults:

The maximum number of resource results.

type NextToken:

string

param NextToken:

A continuation token, if this is not the first call to retrieve these resources.

rtype:

dict

returns:

Response Syntax

{
    'ResourceInfoList': [
        {
            'ResourceArn': 'string',
            'RoleArn': 'string',
            'LastModified': datetime(2015, 1, 1),
            'WithFederation': True|False,
            'HybridAccessEnabled': True|False,
            'WithPrivilegedAccess': True|False,
            'VerificationStatus': 'VERIFIED'|'VERIFICATION_FAILED'|'NOT_VERIFIED',
            'ExpectedResourceOwnerAccount': 'string'
        },
    ],
    'NextToken': 'string'
}

Response Structure

• (dict) --

  • ResourceInfoList (list) --

    A summary of the data lake resources.

    • (dict) --

      A structure containing information about an Lake Formation resource.

      • ResourceArn (string) --

        The Amazon Resource Name (ARN) of the resource.

      • RoleArn (string) --

        The IAM role that registered a resource.

      • LastModified (datetime) --

        The date and time the resource was last modified.

      • WithFederation (boolean) --

        Whether or not the resource is a federated resource.

      • HybridAccessEnabled (boolean) --

        Indicates whether the data access of tables pointing to the location can be managed by both Lake Formation permissions as well as Amazon S3 bucket policies.

      • WithPrivilegedAccess (boolean) --

        Grants the calling principal the permissions to perform all supported Lake Formation operations on the registered data location.

      • VerificationStatus (string) --

        Indicates whether the registered role has sufficient permissions to access registered Amazon S3 location. Verification Status can be one of the following:

        • VERIFIED - Registered role has sufficient permissions to access registered Amazon S3 location.

        • NOT_VERIFIED - Registered role does not have sufficient permissions to access registered Amazon S3 location.

        • VERIFICATION_FAILED - Unable to verify if the registered role can access the registered Amazon S3 location.

      • ExpectedResourceOwnerAccount (string) --

        The Amazon Web Services account that owns the Glue tables associated with specific Amazon S3 locations.

  • NextToken (string) --

    A continuation token, if this is not the first call to retrieve these resources.

{'ExpectedResourceOwnerAccount': 'string'}

Registers the resource as managed by the Data Catalog.

To add or update data, Lake Formation needs read/write access to the chosen data location. Choose a role that you know has permission to do this, or choose the AWSServiceRoleForLakeFormationDataAccess service-linked role. When you register the first Amazon S3 path, the service-linked role and a new inline policy are created on your behalf. Lake Formation adds the first path to the inline policy and attaches it to the service-linked role. When you register subsequent paths, Lake Formation adds the path to the existing policy.

The following request registers a new location and gives Lake Formation permission to use the service-linked role to access that location.

ResourceArn = arn:aws:s3:::my-bucket/ UseServiceLinkedRole = true

If UseServiceLinkedRole is not set to true, you must provide or set the RoleArn:

arn:aws:iam::12345:role/my-data-access-role

See also: AWS API Documentation

Request Syntax

client.register_resource(
    ResourceArn='string',
    UseServiceLinkedRole=True|False,
    RoleArn='string',
    WithFederation=True|False,
    HybridAccessEnabled=True|False,
    WithPrivilegedAccess=True|False,
    ExpectedResourceOwnerAccount='string'
)

type ResourceArn:

string

param ResourceArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the resource that you want to register.

type UseServiceLinkedRole:

boolean

param UseServiceLinkedRole:

Designates an Identity and Access Management (IAM) service-linked role by registering this role with the Data Catalog. A service-linked role is a unique type of IAM role that is linked directly to Lake Formation.

For more information, see Using Service-Linked Roles for Lake Formation.

type RoleArn:

string

param RoleArn:

The identifier for the role that registers the resource.

type WithFederation:

boolean

param WithFederation:

Whether or not the resource is a federated resource.

type HybridAccessEnabled:

boolean

param HybridAccessEnabled:

Specifies whether the data access of tables pointing to the location can be managed by both Lake Formation permissions as well as Amazon S3 bucket policies.

type WithPrivilegedAccess:

boolean

param WithPrivilegedAccess:

Grants the calling principal the permissions to perform all supported Lake Formation operations on the registered data location.

type ExpectedResourceOwnerAccount:

string

param ExpectedResourceOwnerAccount:

The Amazon Web Services account that owns the Glue tables associated with specific Amazon S3 locations.

rtype:

dict

returns:

Response Syntax

{}

Response Structure

• (dict) --

{'ExpectedResourceOwnerAccount': 'string'}

Updates the data access role used for vending access to the given (registered) resource in Lake Formation.

See also: AWS API Documentation

Request Syntax

client.update_resource(
    RoleArn='string',
    ResourceArn='string',
    WithFederation=True|False,
    HybridAccessEnabled=True|False,
    ExpectedResourceOwnerAccount='string'
)

type RoleArn:

string

param RoleArn:

[REQUIRED]

The new role to use for the given resource registered in Lake Formation.

type ResourceArn:

string

param ResourceArn:

[REQUIRED]

The resource ARN.

type WithFederation:

boolean

param WithFederation:

Whether or not the resource is a federated resource.

type HybridAccessEnabled:

boolean

param HybridAccessEnabled:

Specifies whether the data access of tables pointing to the location can be managed by both Lake Formation permissions as well as Amazon S3 bucket policies.

type ExpectedResourceOwnerAccount:

string

param ExpectedResourceOwnerAccount:

The Amazon Web Services account that owns the Glue tables associated with specific Amazon S3 locations.

rtype:

dict

returns:

Response Syntax

{}

Response Structure

• (dict) --