Amazon DataZone

2024/06/27 - Amazon DataZone - 3 new 1 updated api methods

Changes  This release supports the data lineage feature of business data catalog in Amazon DataZone.

ListLineageNodeHistory (new) Link ¶

Lists the history of the specified data lineage node.

See also: AWS API Documentation

Request Syntax

client.list_lineage_node_history(
    direction='UPSTREAM'|'DOWNSTREAM',
    domainIdentifier='string',
    eventTimestampGTE=datetime(2015, 1, 1),
    eventTimestampLTE=datetime(2015, 1, 1),
    identifier='string',
    maxResults=123,
    nextToken='string',
    sortOrder='ASCENDING'|'DESCENDING'
)
type direction

string

param direction

The direction of the data lineage node refers to the lineage node having neighbors in that direction. For example, if direction is UPSTREAM , the ListLineageNodeHistory API responds with historical versions with upstream neighbors only.

type domainIdentifier

string

param domainIdentifier

[REQUIRED]

The ID of the domain where you want to list the history of the specified data lineage node.

type eventTimestampGTE

datetime

param eventTimestampGTE

Specifies whether the action is to return data lineage node history from the time after the event timestamp.

type eventTimestampLTE

datetime

param eventTimestampLTE

Specifies whether the action is to return data lineage node history from the time prior of the event timestamp.

type identifier

string

param identifier

[REQUIRED]

The ID of the data lineage node whose history you want to list.

type maxResults

integer

param maxResults

The maximum number of history items to return in a single call to ListLineageNodeHistory. When the number of memberships to be listed is greater than the value of MaxResults, the response contains a NextToken value that you can use in a subsequent call to ListLineageNodeHistory to list the next set of items.

type nextToken

string

param nextToken

When the number of history items is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of items, the response includes a pagination token named NextToken. You can specify this NextToken value in a subsequent call to ListLineageNodeHistory to list the next set of items.

type sortOrder

string

param sortOrder

The order by which you want data lineage node history to be sorted.

rtype

dict

returns

Response Syntax

{
    'nextToken': 'string',
    'nodes': [
        {
            'createdAt': datetime(2015, 1, 1),
            'createdBy': 'string',
            'description': 'string',
            'domainId': 'string',
            'eventTimestamp': datetime(2015, 1, 1),
            'id': 'string',
            'name': 'string',
            'sourceIdentifier': 'string',
            'typeName': 'string',
            'typeRevision': 'string',
            'updatedAt': datetime(2015, 1, 1),
            'updatedBy': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    • nextToken (string) --

      When the number of history items is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of items, the response includes a pagination token named NextToken. You can specify this NextToken value in a subsequent call to ListLineageNodeHistory to list the next set of items.

    • nodes (list) --

      The nodes returned by the ListLineageNodeHistory action.

      • (dict) --

        The summary of the data lineage node.

        • createdAt (datetime) --

          The timestamp at which the data lineage node was created.

        • createdBy (string) --

          The user who created the data lineage node.

        • description (string) --

          The description of the data lineage node.

        • domainId (string) --

          The ID of the domain of the data lineage node.

        • eventTimestamp (datetime) --

          The event timestamp of the data lineage node.

        • id (string) --

          The ID of the data lineage node.

        • name (string) --

          The name of the data lineage node.

        • sourceIdentifier (string) --

          The alternate ID of the data lineage node.

        • typeName (string) --

          The name of the type of the data lineage node.

        • typeRevision (string) --

          The type of the revision of the data lineage node.

        • updatedAt (datetime) --

          The timestamp at which the data lineage node was updated.

        • updatedBy (string) --

          The user who updated the data lineage node.

GetLineageNode (new) Link ¶

Gets the data lineage node.

See also: AWS API Documentation

Request Syntax

client.get_lineage_node(
    domainIdentifier='string',
    eventTimestamp=datetime(2015, 1, 1),
    identifier='string'
)
type domainIdentifier

string

param domainIdentifier

[REQUIRED]

The ID of the domain in which you want to get the data lineage node.

type eventTimestamp

datetime

param eventTimestamp

The event time stamp for which you want to get the data lineage node.

type identifier

string

param identifier

[REQUIRED]

The ID of the data lineage node that you want to get.

Both, a lineage node identifier generated by Amazon DataZone and a sourceIdentifier of the lineage node are supported. If sourceIdentifier is greater than 1800 characters, you can use lineage node identifier generated by Amazon DataZone to get the node details.

rtype

dict

returns

Response Syntax

{
    'createdAt': datetime(2015, 1, 1),
    'createdBy': 'string',
    'description': 'string',
    'domainId': 'string',
    'downstreamNodes': [
        {
            'eventTimestamp': datetime(2015, 1, 1),
            'id': 'string'
        },
    ],
    'eventTimestamp': datetime(2015, 1, 1),
    'formsOutput': [
        {
            'content': 'string',
            'formName': 'string',
            'typeName': 'string',
            'typeRevision': 'string'
        },
    ],
    'id': 'string',
    'name': 'string',
    'sourceIdentifier': 'string',
    'typeName': 'string',
    'typeRevision': 'string',
    'updatedAt': datetime(2015, 1, 1),
    'updatedBy': 'string',
    'upstreamNodes': [
        {
            'eventTimestamp': datetime(2015, 1, 1),
            'id': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    • createdAt (datetime) --

      The timestamp at which the data lineage node was created.

    • createdBy (string) --

      The user who created the data lineage node.

    • description (string) --

      The description of the data lineage node.

    • domainId (string) --

      The ID of the domain where you're getting the data lineage node.

    • downstreamNodes (list) --

      The downsteam nodes of the specified data lineage node.

      • (dict) --

        The reference details for the data lineage node.

        • eventTimestamp (datetime) --

          The event timestamp of the data lineage node.

        • id (string) --

          The ID of the data lineage node.

    • eventTimestamp (datetime) --

      The timestamp of the event described in the data lineage node.

    • formsOutput (list) --

      The metadata of the specified data lineage node.

      • (dict) --

        The details of a metadata form.

        • content (string) --

          The content of the metadata form.

        • formName (string) --

          The name of the metadata form.

        • typeName (string) --

          The name of the metadata form type.

        • typeRevision (string) --

          The revision of the metadata form type.

    • id (string) --

      The ID of the data lineage node.

    • name (string) --

      The name of the data lineage node.

    • sourceIdentifier (string) --

      The source identifier of the data lineage node.

    • typeName (string) --

      The name of the type of the specified data lineage node.

    • typeRevision (string) --

      The revision type of the specified data lineage node.

    • updatedAt (datetime) --

      The timestamp at which the data lineage node was updated.

    • updatedBy (string) --

      The user who updated the data lineage node.

    • upstreamNodes (list) --

      The upstream nodes of the specified data lineage node.

      • (dict) --

        The reference details for the data lineage node.

        • eventTimestamp (datetime) --

          The event timestamp of the data lineage node.

        • id (string) --

          The ID of the data lineage node.

PostLineageEvent (new) Link ¶

Posts a data lineage event.

See also: AWS API Documentation

Request Syntax

client.post_lineage_event(
    clientToken='string',
    domainIdentifier='string',
    event=b'bytes'|file
)
type clientToken

string

param clientToken

A unique, case-sensitive identifier that is provided to ensure the idempotency of the request.

This field is autopopulated if not provided.

type domainIdentifier

string

param domainIdentifier

[REQUIRED]

The ID of the domain where you want to post a data lineage event.

type event

bytes or seekable file-like object

param event

[REQUIRED]

The data lineage event that you want to post. Only open-lineage run event are supported as events.

rtype

dict

returns

Response Syntax

{}

Response Structure

  • (dict) --

SearchTypes (updated) Link ¶
Changes (request, response)
Request
{'searchScope': {'LINEAGE_NODE_TYPE'}}
Response
{'items': {'lineageNodeTypeItem': {'createdAt': 'timestamp',
                                   'createdBy': 'string',
                                   'description': 'string',
                                   'domainId': 'string',
                                   'formsOutput': {'string': {'required': 'boolean',
                                                              'typeName': 'string',
                                                              'typeRevision': 'string'}},
                                   'name': 'string',
                                   'revision': 'string',
                                   'updatedAt': 'timestamp',
                                   'updatedBy': 'string'}}}

Searches for types in Amazon DataZone.

See also: AWS API Documentation

Request Syntax

client.search_types(
    domainIdentifier='string',
    filters={
        'and': [
            {'... recursive ...'},
        ],
        'filter': {
            'attribute': 'string',
            'value': 'string'
        },
        'or': [
            {'... recursive ...'},
        ]
    },
    managed=True|False,
    maxResults=123,
    nextToken='string',
    searchIn=[
        {
            'attribute': 'string'
        },
    ],
    searchScope='ASSET_TYPE'|'FORM_TYPE'|'LINEAGE_NODE_TYPE',
    searchText='string',
    sort={
        'attribute': 'string',
        'order': 'ASCENDING'|'DESCENDING'
    }
)
type domainIdentifier

string

param domainIdentifier

[REQUIRED]

The identifier of the Amazon DataZone domain in which to invoke the SearchTypes action.

type filters

dict

param filters

The filters for the SearchTypes action.

Note

This is a Tagged Union structure. Only one of the following top level keys can be set: and, filter, or.

  • and (list) --

    The 'and' search filter clause in Amazon DataZone.

    • (dict) --

      A search filter clause in Amazon DataZone.

      Note

      This is a Tagged Union structure. Only one of the following top level keys can be set: and, filter, or.

  • filter (dict) --

    A search filter in Amazon DataZone.

    • attribute (string) -- [REQUIRED]

      A search filter attribute in Amazon DataZone.

    • value (string) -- [REQUIRED]

      A search filter value in Amazon DataZone.

  • or (list) --

    The 'or' search filter clause in Amazon DataZone.

    • (dict) --

      A search filter clause in Amazon DataZone.

      Note

      This is a Tagged Union structure. Only one of the following top level keys can be set: and, filter, or.

type managed

boolean

param managed

[REQUIRED]

Specifies whether the search is managed.

type maxResults

integer

param maxResults

The maximum number of results to return in a single call to SearchTypes . When the number of results to be listed is greater than the value of MaxResults , the response contains a NextToken value that you can use in a subsequent call to SearchTypes to list the next set of results.

type nextToken

string

param nextToken

When the number of results is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of results, the response includes a pagination token named NextToken . You can specify this NextToken value in a subsequent call to SearchTypes to list the next set of results.

type searchIn

list

param searchIn

The details of the search.

  • (dict) --

    The details of the search.

    • attribute (string) -- [REQUIRED]

      The search attribute.

type searchScope

string

param searchScope

[REQUIRED]

Specifies the scope of the search for types.

type searchText

string

param searchText

Specifies the text for which to search.

type sort

dict

param sort

The specifies the way to sort the SearchTypes results.

  • attribute (string) -- [REQUIRED]

    The attribute detail of the way to sort search results.

  • order (string) --

    The order detail of the wya to sort search results.

rtype

dict

returns

Response Syntax

{
    'items': [
        {
            'assetTypeItem': {
                'createdAt': datetime(2015, 1, 1),
                'createdBy': 'string',
                'description': 'string',
                'domainId': 'string',
                'formsOutput': {
                    'string': {
                        'required': True|False,
                        'typeName': 'string',
                        'typeRevision': 'string'
                    }
                },
                'name': 'string',
                'originDomainId': 'string',
                'originProjectId': 'string',
                'owningProjectId': 'string',
                'revision': 'string',
                'updatedAt': datetime(2015, 1, 1),
                'updatedBy': 'string'
            },
            'formTypeItem': {
                'createdAt': datetime(2015, 1, 1),
                'createdBy': 'string',
                'description': 'string',
                'domainId': 'string',
                'imports': [
                    {
                        'name': 'string',
                        'revision': 'string'
                    },
                ],
                'model': {
                    'smithy': 'string'
                },
                'name': 'string',
                'originDomainId': 'string',
                'originProjectId': 'string',
                'owningProjectId': 'string',
                'revision': 'string',
                'status': 'ENABLED'|'DISABLED'
            },
            'lineageNodeTypeItem': {
                'createdAt': datetime(2015, 1, 1),
                'createdBy': 'string',
                'description': 'string',
                'domainId': 'string',
                'formsOutput': {
                    'string': {
                        'required': True|False,
                        'typeName': 'string',
                        'typeRevision': 'string'
                    }
                },
                'name': 'string',
                'revision': 'string',
                'updatedAt': datetime(2015, 1, 1),
                'updatedBy': 'string'
            }
        },
    ],
    'nextToken': 'string',
    'totalMatchCount': 123
}

Response Structure

  • (dict) --

    • items (list) --

      The results of the SearchTypes action.

      • (dict) --

        The details of the results of the SearchTypes action.

        Note

        This is a Tagged Union structure. Only one of the following top level keys will be set: assetTypeItem, formTypeItem, lineageNodeTypeItem. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:

        'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
        • assetTypeItem (dict) --

          The asset type included in the results of the SearchTypes action.

          • createdAt (datetime) --

            The timestamp of when the asset type was created.

          • createdBy (string) --

            The Amazon DataZone user who created the asset type.

          • description (string) --

            The description of the asset type.

          • domainId (string) --

            The identifier of the Amazon DataZone domain where the asset type exists.

          • formsOutput (dict) --

            The forms included in the details of the asset type.

            • (string) --

              • (dict) --

                The details of the form entry.

                • required (boolean) --

                  Specifies whether a form entry is required.

                • typeName (string) --

                  The name of the type of the form entry.

                • typeRevision (string) --

                  The type revision of the form entry.

          • name (string) --

            The name of the asset type.

          • originDomainId (string) --

            The identifier of the Amazon DataZone domain where the asset type was originally created.

          • originProjectId (string) --

            The identifier of the Amazon DataZone project where the asset type exists.

          • owningProjectId (string) --

            The identifier of the Amazon DataZone project that owns the asset type.

          • revision (string) --

            The revision of the asset type.

          • updatedAt (datetime) --

            The timestamp of when the asset type was updated.

          • updatedBy (string) --

            The Amazon DataZone user who updated the asset type.

        • formTypeItem (dict) --

          The form type included in the results of the SearchTypes action.

          • createdAt (datetime) --

            The timestamp of when the metadata form type was created.

          • createdBy (string) --

            The Amazon DataZone user who created teh metadata form type.

          • description (string) --

            The description of the metadata form type.

          • domainId (string) --

            The identifier of the Amazon DataZone domain in which the form type exists.

          • imports (list) --

            The imports specified in the form type.

            • (dict) --

              The details of the import of the metadata form type.

              • name (string) --

                The name of the import.

              • revision (string) --

                The revision of the import.

          • model (dict) --

            The model of the form type.

            Note

            This is a Tagged Union structure. Only one of the following top level keys will be set: smithy. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:

            'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
            • smithy (string) --

              Indicates the smithy model of the API.

          • name (string) --

            The name of the form type.

          • originDomainId (string) --

            The identifier of the Amazon DataZone domain in which the form type was originally created.

          • originProjectId (string) --

            The identifier of the project in which the form type was originally created.

          • owningProjectId (string) --

            The identifier of the project that owns the form type.

          • revision (string) --

            The revision of the form type.

          • status (string) --

            The status of the form type.

        • lineageNodeTypeItem (dict) --

          The details of a data lineage node type.

          • createdAt (datetime) --

            The timestamp at which the data lineage node type was created.

          • createdBy (string) --

            The user who created the data lineage node type.

          • description (string) --

            The description of the data lineage node type.

          • domainId (string) --

            The ID of the domain where the data lineage node type lives.

          • formsOutput (dict) --

            The forms output of the data lineage node type.

            • (string) --

              • (dict) --

                The details of the form entry.

                • required (boolean) --

                  Specifies whether a form entry is required.

                • typeName (string) --

                  The name of the type of the form entry.

                • typeRevision (string) --

                  The type revision of the form entry.

          • name (string) --

            The name of the data lineage node type.

          • revision (string) --

            The revision of the data lineage node type.

          • updatedAt (datetime) --

            The timestamp at which the data lineage node type was updated.

          • updatedBy (string) --

            The user who updated the data lineage node type.

    • nextToken (string) --

      When the number of results is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of results, the response includes a pagination token named NextToken . You can specify this NextToken value in a subsequent call to SearchTypes to list the next set of results.

    • totalMatchCount (integer) --

      Total number of search results.