AWS API Changes

2024/06/27 - Amazon DataZone - 3 new1 updated api methods

Changes This release supports the data lineage feature of business data catalog in Amazon DataZone.

ListLineageNodeHistory (new)

Link ¶

Lists the history of the specified data lineage node.

Request Syntax

client.list_lineage_node_history(
    direction='UPSTREAM'|'DOWNSTREAM',
    domainIdentifier='string',
    eventTimestampGTE=datetime(2015, 1, 1),
    eventTimestampLTE=datetime(2015, 1, 1),
    identifier='string',
    maxResults=123,
    nextToken='string',
    sortOrder='ASCENDING'|'DESCENDING'
)

type direction:

string

param direction:

The direction of the data lineage node refers to the lineage node having neighbors in that direction. For example, if direction is UPSTREAM, the ListLineageNodeHistory API responds with historical versions with upstream neighbors only.

type domainIdentifier:

string

param domainIdentifier:

[REQUIRED]

The ID of the domain where you want to list the history of the specified data lineage node.

type eventTimestampGTE:

datetime

param eventTimestampGTE:

Specifies whether the action is to return data lineage node history from the time after the event timestamp.

type eventTimestampLTE:

datetime

param eventTimestampLTE:

Specifies whether the action is to return data lineage node history from the time prior of the event timestamp.

type identifier:

string

param identifier:

[REQUIRED]

The ID of the data lineage node whose history you want to list.

type maxResults:

integer

param maxResults:

The maximum number of history items to return in a single call to ListLineageNodeHistory. When the number of memberships to be listed is greater than the value of MaxResults, the response contains a NextToken value that you can use in a subsequent call to ListLineageNodeHistory to list the next set of items.

type nextToken:

string

param nextToken:

When the number of history items is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of items, the response includes a pagination token named NextToken. You can specify this NextToken value in a subsequent call to ListLineageNodeHistory to list the next set of items.

type sortOrder:

string

param sortOrder:

The order by which you want data lineage node history to be sorted.

rtype:

dict

returns:

Response Syntax

{
    'nextToken': 'string',
    'nodes': [
        {
            'createdAt': datetime(2015, 1, 1),
            'createdBy': 'string',
            'description': 'string',
            'domainId': 'string',
            'eventTimestamp': datetime(2015, 1, 1),
            'id': 'string',
            'name': 'string',
            'sourceIdentifier': 'string',
            'typeName': 'string',
            'typeRevision': 'string',
            'updatedAt': datetime(2015, 1, 1),
            'updatedBy': 'string'
        },
    ]
}

Response Structure

(dict) --
- nextToken (string) --
  
  When the number of history items is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of items, the response includes a pagination token named NextToken. You can specify this NextToken value in a subsequent call to ListLineageNodeHistory to list the next set of items.
- nodes (list) --
  
  The nodes returned by the ListLineageNodeHistory action.
  - (dict) --
    
    The summary of the data lineage node.
    - createdAt (datetime) --
      
      The timestamp at which the data lineage node was created.
    - createdBy (string) --
      
      The user who created the data lineage node.
    - description (string) --
      
      The description of the data lineage node.
    - domainId (string) --
      
      The ID of the domain of the data lineage node.
    - eventTimestamp (datetime) --
      
      The event timestamp of the data lineage node.
    - id (string) --
      
      The ID of the data lineage node.
    - name (string) --
      
      The name of the data lineage node.
    - sourceIdentifier (string) --
      
      The alternate ID of the data lineage node.
    - typeName (string) --
      
      The name of the type of the data lineage node.
    - typeRevision (string) --
      
      The type of the revision of the data lineage node.
    - updatedAt (datetime) --
      
      The timestamp at which the data lineage node was updated.
    - updatedBy (string) --
      
      The user who updated the data lineage node.

PostLineageEvent (new)

Link ¶

Posts a data lineage event.

See also: AWS API Documentation

Request Syntax

client.post_lineage_event(
    clientToken='string',
    domainIdentifier='string',
    event=b'bytes'|file
)

type clientToken:

string

param clientToken:

A unique, case-sensitive identifier that is provided to ensure the idempotency of the request.

This field is autopopulated if not provided.

type domainIdentifier:

string

param domainIdentifier:

[REQUIRED]

The ID of the domain where you want to post a data lineage event.

type event:

bytes or seekable file-like object

param event:

[REQUIRED]

The data lineage event that you want to post. Only open-lineage run event are supported as events.

rtype:

dict

returns:

Response Syntax

{}

Response Structure

(dict) --

GetLineageNode (new)

Link ¶

Gets the data lineage node.

See also: AWS API Documentation

Request Syntax

client.get_lineage_node(
    domainIdentifier='string',
    eventTimestamp=datetime(2015, 1, 1),
    identifier='string'
)

type domainIdentifier:

string

param domainIdentifier:

[REQUIRED]

The ID of the domain in which you want to get the data lineage node.

type eventTimestamp:

datetime

param eventTimestamp:

The event time stamp for which you want to get the data lineage node.

type identifier:

string

param identifier:

[REQUIRED]

The ID of the data lineage node that you want to get.

Both, a lineage node identifier generated by Amazon DataZone and a sourceIdentifier of the lineage node are supported. If sourceIdentifier is greater than 1800 characters, you can use lineage node identifier generated by Amazon DataZone to get the node details.

rtype:

dict

returns:

Response Syntax

{
    'createdAt': datetime(2015, 1, 1),
    'createdBy': 'string',
    'description': 'string',
    'domainId': 'string',
    'downstreamNodes': [
        {
            'eventTimestamp': datetime(2015, 1, 1),
            'id': 'string'
        },
    ],
    'eventTimestamp': datetime(2015, 1, 1),
    'formsOutput': [
        {
            'content': 'string',
            'formName': 'string',
            'typeName': 'string',
            'typeRevision': 'string'
        },
    ],
    'id': 'string',
    'name': 'string',
    'sourceIdentifier': 'string',
    'typeName': 'string',
    'typeRevision': 'string',
    'updatedAt': datetime(2015, 1, 1),
    'updatedBy': 'string',
    'upstreamNodes': [
        {
            'eventTimestamp': datetime(2015, 1, 1),
            'id': 'string'
        },
    ]
}

Response Structure

(dict) --
- createdAt (datetime) --
  
  The timestamp at which the data lineage node was created.
- createdBy (string) --
  
  The user who created the data lineage node.
- description (string) --
  
  The description of the data lineage node.
- domainId (string) --
  
  The ID of the domain where you're getting the data lineage node.
- downstreamNodes (list) --
  
  The downsteam nodes of the specified data lineage node.
  - (dict) --
    
    The reference details for the data lineage node.
    - eventTimestamp (datetime) --
      
      The event timestamp of the data lineage node.
    - id (string) --
      
      The ID of the data lineage node.
- eventTimestamp (datetime) --
  
  The timestamp of the event described in the data lineage node.
- formsOutput (list) --
  
  The metadata of the specified data lineage node.
  - (dict) --
    
    The details of a metadata form.
    - content (string) --
      
      The content of the metadata form.
    - formName (string) --
      
      The name of the metadata form.
    - typeName (string) --
      
      The name of the metadata form type.
    - typeRevision (string) --
      
      The revision of the metadata form type.
- id (string) --
  
  The ID of the data lineage node.
- name (string) --
  
  The name of the data lineage node.
- sourceIdentifier (string) --
  
  The source identifier of the data lineage node.
- typeName (string) --
  
  The name of the type of the specified data lineage node.
- typeRevision (string) --
  
  The revision type of the specified data lineage node.
- updatedAt (datetime) --
  
  The timestamp at which the data lineage node was updated.
- updatedBy (string) --
  
  The user who updated the data lineage node.
- upstreamNodes (list) --
  
  The upstream nodes of the specified data lineage node.
  - (dict) --
    
    The reference details for the data lineage node.
    - eventTimestamp (datetime) --
      
      The event timestamp of the data lineage node.
    - id (string) --
      
      The ID of the data lineage node.

SearchTypes (updated)

Link ¶
Changes (request, response)
Request

{'searchScope': {'LINEAGE_NODE_TYPE'}}

Response

{'items': {'lineageNodeTypeItem': {'createdAt': 'timestamp',
                                   'createdBy': 'string',
                                   'description': 'string',
                                   'domainId': 'string',
                                   'formsOutput': {'string': {'required': 'boolean',
                                                              'typeName': 'string',
                                                              'typeRevision': 'string'}},
                                   'name': 'string',
                                   'revision': 'string',
                                   'updatedAt': 'timestamp',
                                   'updatedBy': 'string'}}}

Searches for types in Amazon DataZone.

See also: AWS API Documentation

Request Syntax

client.search_types(
    domainIdentifier='string',
    filters={
        'and': [
            {'... recursive ...'},
        ],
        'filter': {
            'attribute': 'string',
            'value': 'string'
        },
        'or': [
            {'... recursive ...'},
        ]
    },
    managed=True|False,
    maxResults=123,
    nextToken='string',
    searchIn=[
        {
            'attribute': 'string'
        },
    ],
    searchScope='ASSET_TYPE'|'FORM_TYPE'|'LINEAGE_NODE_TYPE',
    searchText='string',
    sort={
        'attribute': 'string',
        'order': 'ASCENDING'|'DESCENDING'
    }
)

type domainIdentifier:

string

param domainIdentifier:

[REQUIRED]

The identifier of the Amazon DataZone domain in which to invoke the SearchTypes action.

type filters:

dict

param filters:

The filters for the SearchTypes action.

and (list) --

The 'and' search filter clause in Amazon DataZone.
- (dict) --
  
  A search filter clause in Amazon DataZone.
  
  Note
  
  This is a Tagged Union structure. Only one of the following top level keys can be set: and, filter, or.
filter (dict) --

A search filter in Amazon DataZone.
- attribute (string) -- [REQUIRED]
  
  A search filter attribute in Amazon DataZone.
- value (string) -- [REQUIRED]
  
  A search filter value in Amazon DataZone.
or (list) --

The 'or' search filter clause in Amazon DataZone.
- (dict) --
  
  A search filter clause in Amazon DataZone.
  
  Note
  
  This is a Tagged Union structure. Only one of the following top level keys can be set: and, filter, or.

type managed:

boolean

param managed:

[REQUIRED]

Specifies whether the search is managed.

type maxResults:

integer

param maxResults:

The maximum number of results to return in a single call to SearchTypes. When the number of results to be listed is greater than the value of MaxResults, the response contains a NextToken value that you can use in a subsequent call to SearchTypes to list the next set of results.

type nextToken:

string

param nextToken:

When the number of results is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of results, the response includes a pagination token named NextToken. You can specify this NextToken value in a subsequent call to SearchTypes to list the next set of results.

type searchIn:

list

param searchIn:

The details of the search.

(dict) --

The details of the search.
- attribute (string) -- [REQUIRED]
  
  The search attribute.

type searchScope:

string

param searchScope:

[REQUIRED]

Specifies the scope of the search for types.

type searchText:

string

param searchText:

Specifies the text for which to search.

type sort:

dict

param sort:

The specifies the way to sort the SearchTypes results.

attribute (string) -- [REQUIRED]

The attribute detail of the way to sort search results.
order (string) --

The order detail of the wya to sort search results.

rtype:

dict

returns:

Response Syntax

{
    'items': [
        {
            'assetTypeItem': {
                'createdAt': datetime(2015, 1, 1),
                'createdBy': 'string',
                'description': 'string',
                'domainId': 'string',
                'formsOutput': {
                    'string': {
                        'required': True|False,
                        'typeName': 'string',
                        'typeRevision': 'string'
                    }
                },
                'name': 'string',
                'originDomainId': 'string',
                'originProjectId': 'string',
                'owningProjectId': 'string',
                'revision': 'string',
                'updatedAt': datetime(2015, 1, 1),
                'updatedBy': 'string'
            },
            'formTypeItem': {
                'createdAt': datetime(2015, 1, 1),
                'createdBy': 'string',
                'description': 'string',
                'domainId': 'string',
                'imports': [
                    {
                        'name': 'string',
                        'revision': 'string'
                    },
                ],
                'model': {
                    'smithy': 'string'
                },
                'name': 'string',
                'originDomainId': 'string',
                'originProjectId': 'string',
                'owningProjectId': 'string',
                'revision': 'string',
                'status': 'ENABLED'|'DISABLED'
            },
            'lineageNodeTypeItem': {
                'createdAt': datetime(2015, 1, 1),
                'createdBy': 'string',
                'description': 'string',
                'domainId': 'string',
                'formsOutput': {
                    'string': {
                        'required': True|False,
                        'typeName': 'string',
                        'typeRevision': 'string'
                    }
                },
                'name': 'string',
                'revision': 'string',
                'updatedAt': datetime(2015, 1, 1),
                'updatedBy': 'string'
            }
        },
    ],
    'nextToken': 'string',
    'totalMatchCount': 123
}

Response Structure

(dict) --
- items (list) --
  
  The results of the SearchTypes action.
  - (dict) --
    
    The details of the results of the SearchTypes action.
    Note
    
    This is a Tagged Union structure. Only one of the following top level keys will be set: assetTypeItem, formTypeItem, lineageNodeTypeItem. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
```
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
```
    - assetTypeItem (dict) --
      
      The asset type included in the results of the SearchTypes action.
      - createdAt (datetime) --
        
        The timestamp of when the asset type was created.
      - createdBy (string) --
        
        The Amazon DataZone user who created the asset type.
      - description (string) --
        
        The description of the asset type.
      - domainId (string) --
        
        The identifier of the Amazon DataZone domain where the asset type exists.
      - formsOutput (dict) --
        
        The forms included in the details of the asset type.
        
        (string) --
        
        (dict) --
        
        The details of the form entry.
        
        required (boolean) --
        
        Specifies whether a form entry is required.
        
        typeName (string) --
        
        The name of the type of the form entry.
        
        typeRevision (string) --
        
        The type revision of the form entry.
      - name (string) --
        
        The name of the asset type.
      - originDomainId (string) --
        
        The identifier of the Amazon DataZone domain where the asset type was originally created.
      - originProjectId (string) --
        
        The identifier of the Amazon DataZone project where the asset type exists.
      - owningProjectId (string) --
        
        The identifier of the Amazon DataZone project that owns the asset type.
      - revision (string) --
        
        The revision of the asset type.
      - updatedAt (datetime) --
        
        The timestamp of when the asset type was updated.
      - updatedBy (string) --
        
        The Amazon DataZone user who updated the asset type.
    - formTypeItem (dict) --
      
      The form type included in the results of the SearchTypes action.
      - createdAt (datetime) --
        
        The timestamp of when the metadata form type was created.
      - createdBy (string) --
        
        The Amazon DataZone user who created teh metadata form type.
      - description (string) --
        
        The description of the metadata form type.
      - domainId (string) --
        
        The identifier of the Amazon DataZone domain in which the form type exists.
      - imports (list) --
        
        The imports specified in the form type.
        
        (dict) --
        
        The details of the import of the metadata form type.
        
        name (string) --
        
        The name of the import.
        
        revision (string) --
        
        The revision of the import.
      - model (dict) --
        
        The model of the form type.
        
        Note
        
        This is a Tagged Union structure. Only one of the following top level keys will be set: smithy. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
        
        'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
        
        smithy (string) --
        
        Indicates the smithy model of the API.
      - name (string) --
        
        The name of the form type.
      - originDomainId (string) --
        
        The identifier of the Amazon DataZone domain in which the form type was originally created.
      - originProjectId (string) --
        
        The identifier of the project in which the form type was originally created.
      - owningProjectId (string) --
        
        The identifier of the project that owns the form type.
      - revision (string) --
        
        The revision of the form type.
      - status (string) --
        
        The status of the form type.
    - lineageNodeTypeItem (dict) --
      
      The details of a data lineage node type.
      - createdAt (datetime) --
        
        The timestamp at which the data lineage node type was created.
      - createdBy (string) --
        
        The user who created the data lineage node type.
      - description (string) --
        
        The description of the data lineage node type.
      - domainId (string) --
        
        The ID of the domain where the data lineage node type lives.
      - formsOutput (dict) --
        
        The forms output of the data lineage node type.
        
        (string) --
        
        (dict) --
        
        The details of the form entry.
        
        required (boolean) --
        
        Specifies whether a form entry is required.
        
        typeName (string) --
        
        The name of the type of the form entry.
        
        typeRevision (string) --
        
        The type revision of the form entry.
      - name (string) --
        
        The name of the data lineage node type.
      - revision (string) --
        
        The revision of the data lineage node type.
      - updatedAt (datetime) --
        
        The timestamp at which the data lineage node type was updated.
      - updatedBy (string) --
        
        The user who updated the data lineage node type.
- nextToken (string) --
  
  When the number of results is greater than the default value for the MaxResults parameter, or if you explicitly specify a value for MaxResults that is less than the number of results, the response includes a pagination token named NextToken. You can specify this NextToken value in a subsequent call to SearchTypes to list the next set of results.
- totalMatchCount (integer) --
  
  Total number of search results.