AWS Data Pipeline

2015/02/23 - AWS Data Pipeline - 2 new 2 updated api methods

AddTags (new) Link ¶

Add or modify tags in an existing pipeline.

Request Syntax

client.add_tags(
    pipelineId='string',
    tags=[
        {
            'key': 'string',
            'value': 'string'
        },
    ]
)
type pipelineId

string

param pipelineId

[REQUIRED]

The identifier of the pipeline to which you want to add the tags.

type tags

list

param tags

[REQUIRED]

The tags as key/value pairs to add to the pipeline.

  • (dict) --

    Tags are key/value pairs defined by a user and associated with a pipeline to control access. AWS Data Pipeline allows you to associate ten tags per pipeline. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .

    • key (string) -- [REQUIRED]

      The key name of a tag defined by a user. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .

    • value (string) -- [REQUIRED]

      The optional value portion of a tag defined by a user. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .

rtype

dict

returns

Response Syntax

{}

Response Structure

  • (dict) --

    The response from the AddTags action.

RemoveTags (new) Link ¶

Remove existing tags from a pipeline.

Request Syntax

client.remove_tags(
    pipelineId='string',
    tagKeys=[
        'string',
    ]
)
type pipelineId

string

param pipelineId

[REQUIRED]

The pipeline from which you want to remove tags.

type tagKeys

list

param tagKeys

[REQUIRED]

The keys of the tags you wish to remove.

  • (string) --

rtype

dict

returns

Response Syntax

{}

Response Structure

  • (dict) --

    The result of the RemoveTags action.

CreatePipeline (updated) Link ¶
Changes (request)
{'tags': [{'key': 'string', 'value': 'string'}]}

Creates a new empty pipeline. When this action succeeds, you can then use the PutPipelineDefinition action to populate the pipeline.

Request Syntax

client.create_pipeline(
    name='string',
    uniqueId='string',
    description='string',
    tags=[
        {
            'key': 'string',
            'value': 'string'
        },
    ]
)
type name

string

param name

[REQUIRED]

The name of the new pipeline. You can use the same name for multiple pipelines associated with your AWS account, because AWS Data Pipeline assigns each new pipeline a unique pipeline identifier.

type uniqueId

string

param uniqueId

[REQUIRED]

A unique identifier that you specify. This identifier is not the same as the pipeline identifier assigned by AWS Data Pipeline. You are responsible for defining the format and ensuring the uniqueness of this identifier. You use this parameter to ensure idempotency during repeated calls to CreatePipeline. For example, if the first call to CreatePipeline does not return a clear success, you can pass in the same unique identifier and pipeline name combination on a subsequent call to CreatePipeline. CreatePipeline ensures that if a pipeline already exists with the same name and unique identifier, a new pipeline will not be created. Instead, you'll receive the pipeline identifier from the previous attempt. The uniqueness of the name and unique identifier combination is scoped to the AWS account or IAM user credentials.

type description

string

param description

The description of the new pipeline.

type tags

list

param tags

A list of tags to associate with a pipeline at creation time. Tags let you control access to pipelines. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .

  • (dict) --

    Tags are key/value pairs defined by a user and associated with a pipeline to control access. AWS Data Pipeline allows you to associate ten tags per pipeline. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .

    • key (string) -- [REQUIRED]

      The key name of a tag defined by a user. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .

    • value (string) -- [REQUIRED]

      The optional value portion of a tag defined by a user. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .

rtype

dict

returns

Response Syntax

{
    'pipelineId': 'string'
}

Response Structure

  • (dict) --

    Contains the output from the CreatePipeline action.

    • pipelineId (string) --

      The ID that AWS Data Pipeline assigns the newly created pipeline. The ID is a string of the form: df-06372391ZG65EXAMPLE.

DescribePipelines (updated) Link ¶
Changes (response)
{'pipelineDescriptionList': {'tags': [{'key': 'string', 'value': 'string'}]}}

Retrieve metadata about one or more pipelines. The information retrieved includes the name of the pipeline, the pipeline identifier, its current state, and the user account that owns the pipeline. Using account credentials, you can retrieve metadata about pipelines that you or your IAM users have created. If you are using an IAM user account, you can retrieve metadata about only those pipelines you have read permission for.

To retrieve the full pipeline definition instead of metadata about the pipeline, call the GetPipelineDefinition action.

Request Syntax

client.describe_pipelines(
    pipelineIds=[
        'string',
    ]
)
type pipelineIds

list

param pipelineIds

[REQUIRED]

Identifiers of the pipelines to describe. You can pass as many as 25 identifiers in a single call to DescribePipelines. You can obtain pipeline identifiers by calling ListPipelines.

  • (string) --

rtype

dict

returns

Response Syntax

{
    'pipelineDescriptionList': [
        {
            'pipelineId': 'string',
            'name': 'string',
            'fields': [
                {
                    'key': 'string',
                    'stringValue': 'string',
                    'refValue': 'string'
                },
            ],
            'description': 'string',
            'tags': [
                {
                    'key': 'string',
                    'value': 'string'
                },
            ]
        },
    ]
}

Response Structure

  • (dict) --

    Contains the output from the DescribePipelines action.

    • pipelineDescriptionList (list) --

      An array of descriptions returned for the specified pipelines.

      • (dict) --

        Contains pipeline metadata.

        • pipelineId (string) --

          The pipeline identifier that was assigned by AWS Data Pipeline. This is a string of the form df-297EG78HU43EEXAMPLE .

        • name (string) --

          Name of the pipeline.

        • fields (list) --

          A list of read-only fields that contain metadata about the pipeline: @userId, @accountId, and @pipelineState.

          • (dict) --

            A key-value pair that describes a property of a pipeline object. The value is specified as either a string value ( StringValue ) or a reference to another object ( RefValue ) but not as both.

            • key (string) --

              The field identifier.

            • stringValue (string) --

              The field value, expressed as a String.

            • refValue (string) --

              The field value, expressed as the identifier of another object.

        • description (string) --

          Description of the pipeline.

        • tags (list) --

          A list of tags to associated with a pipeline. Tags let you control access to pipelines. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .

          • (dict) --

            Tags are key/value pairs defined by a user and associated with a pipeline to control access. AWS Data Pipeline allows you to associate ten tags per pipeline. For more information, see Controlling User Access to Pipelines in the AWS Data Pipeline Developer Guide .