2024/06/12 - Amazon OpenSearch Ingestion - 5 updated api methods
Changes SDK changes for self-managed vpc endpoint to OpenSearch ingestion pipelines.
{'VpcOptions': {'VpcEndpointManagement': 'CUSTOMER | SERVICE'}}
Response {'Pipeline': {'VpcEndpointService': 'string',
'VpcEndpoints': {'VpcOptions': {'VpcEndpointManagement': 'CUSTOMER '
'| '
'SERVICE'}}}}
Creates an OpenSearch Ingestion pipeline. For more information, see Creating Amazon OpenSearch Ingestion pipelines.
See also: AWS API Documentation
Request Syntax
client.create_pipeline(
PipelineName='string',
MinUnits=123,
MaxUnits=123,
PipelineConfigurationBody='string',
LogPublishingOptions={
'IsLoggingEnabled': True|False,
'CloudWatchLogDestination': {
'LogGroup': 'string'
}
},
VpcOptions={
'SubnetIds': [
'string',
],
'SecurityGroupIds': [
'string',
],
'VpcAttachmentOptions': {
'AttachToVpc': True|False,
'CidrBlock': 'string'
},
'VpcEndpointManagement': 'CUSTOMER'|'SERVICE'
},
BufferOptions={
'PersistentBufferEnabled': True|False
},
EncryptionAtRestOptions={
'KmsKeyArn': 'string'
},
Tags=[
{
'Key': 'string',
'Value': 'string'
},
]
)
string
[REQUIRED]
The name of the OpenSearch Ingestion pipeline to create. Pipeline names are unique across the pipelines owned by an account within an Amazon Web Services Region.
integer
[REQUIRED]
The minimum pipeline capacity, in Ingestion Compute Units (ICUs).
integer
[REQUIRED]
The maximum pipeline capacity, in Ingestion Compute Units (ICUs).
string
[REQUIRED]
The pipeline configuration in YAML format. The command accepts the pipeline configuration as a string or within a .yaml file. If you provide the configuration as a string, each new line must be escaped with \n.
dict
Key-value pairs to configure log publishing.
IsLoggingEnabled (boolean) --
Whether logs should be published.
CloudWatchLogDestination (dict) --
The destination for OpenSearch Ingestion logs sent to Amazon CloudWatch Logs. This parameter is required if IsLoggingEnabled is set to true.
LogGroup (string) -- [REQUIRED]
The name of the CloudWatch Logs group to send pipeline logs to. You can specify an existing log group or create a new one. For example, /aws/vendedlogs/OpenSearchService/pipelines.
dict
Container for the values required to configure VPC access for the pipeline. If you don't specify these values, OpenSearch Ingestion creates the pipeline with a public endpoint.
SubnetIds (list) -- [REQUIRED]
A list of subnet IDs associated with the VPC endpoint.
(string) --
SecurityGroupIds (list) --
A list of security groups associated with the VPC endpoint.
(string) --
VpcAttachmentOptions (dict) --
Options for attaching a VPC to a pipeline.
AttachToVpc (boolean) -- [REQUIRED]
Whether a VPC is attached to the pipeline.
CidrBlock (string) --
The CIDR block to be reserved for OpenSearch Ingestion to create elastic network interfaces (ENIs).
VpcEndpointManagement (string) --
Defines whether you or Amazon OpenSearch Ingestion service create and manage the VPC endpoint configured for the pipeline.
dict
Key-value pairs to configure persistent buffering for the pipeline.
PersistentBufferEnabled (boolean) -- [REQUIRED]
Whether persistent buffering should be enabled.
dict
Key-value pairs to configure encryption for data that is written to a persistent buffer.
KmsKeyArn (string) -- [REQUIRED]
The ARN of the KMS key used to encrypt buffer data. By default, data is encrypted using an Amazon Web Services owned key.
list
List of tags to add to the pipeline upon creation.
(dict) --
A tag (key-value pair) for an OpenSearch Ingestion pipeline.
Key (string) -- [REQUIRED]
The tag key. Tag keys must be unique for the pipeline to which they are attached.
Value (string) -- [REQUIRED]
The value assigned to the corresponding tag key. Tag values can be null and don't have to be unique in a tag set. For example, you can have a key value pair in a tag set of project : Trinity and cost-center : Trinity
dict
Response Syntax
{
'Pipeline': {
'PipelineName': 'string',
'PipelineArn': 'string',
'MinUnits': 123,
'MaxUnits': 123,
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'CREATE_FAILED'|'UPDATE_FAILED'|'STARTING'|'START_FAILED'|'STOPPING'|'STOPPED',
'StatusReason': {
'Description': 'string'
},
'PipelineConfigurationBody': 'string',
'CreatedAt': datetime(2015, 1, 1),
'LastUpdatedAt': datetime(2015, 1, 1),
'IngestEndpointUrls': [
'string',
],
'LogPublishingOptions': {
'IsLoggingEnabled': True|False,
'CloudWatchLogDestination': {
'LogGroup': 'string'
}
},
'VpcEndpoints': [
{
'VpcEndpointId': 'string',
'VpcId': 'string',
'VpcOptions': {
'SubnetIds': [
'string',
],
'SecurityGroupIds': [
'string',
],
'VpcAttachmentOptions': {
'AttachToVpc': True|False,
'CidrBlock': 'string'
},
'VpcEndpointManagement': 'CUSTOMER'|'SERVICE'
}
},
],
'BufferOptions': {
'PersistentBufferEnabled': True|False
},
'EncryptionAtRestOptions': {
'KmsKeyArn': 'string'
},
'VpcEndpointService': 'string',
'ServiceVpcEndpoints': [
{
'ServiceName': 'OPENSEARCH_SERVERLESS',
'VpcEndpointId': 'string'
},
],
'Destinations': [
{
'ServiceName': 'string',
'Endpoint': 'string'
},
],
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
}
Response Structure
(dict) --
Pipeline (dict) --
Container for information about the created pipeline.
PipelineName (string) --
The name of the pipeline.
PipelineArn (string) --
The Amazon Resource Name (ARN) of the pipeline.
MinUnits (integer) --
The minimum pipeline capacity, in Ingestion Compute Units (ICUs).
MaxUnits (integer) --
The maximum pipeline capacity, in Ingestion Compute Units (ICUs).
Status (string) --
The current status of the pipeline.
StatusReason (dict) --
The reason for the current status of the pipeline.
Description (string) --
A description of why a pipeline has a certain status.
PipelineConfigurationBody (string) --
The Data Prepper pipeline configuration in YAML format.
CreatedAt (datetime) --
The date and time when the pipeline was created.
LastUpdatedAt (datetime) --
The date and time when the pipeline was last updated.
IngestEndpointUrls (list) --
The ingestion endpoints for the pipeline, which you can send data to.
(string) --
LogPublishingOptions (dict) --
Key-value pairs that represent log publishing settings.
IsLoggingEnabled (boolean) --
Whether logs should be published.
CloudWatchLogDestination (dict) --
The destination for OpenSearch Ingestion logs sent to Amazon CloudWatch Logs. This parameter is required if IsLoggingEnabled is set to true.
LogGroup (string) --
The name of the CloudWatch Logs group to send pipeline logs to. You can specify an existing log group or create a new one. For example, /aws/vendedlogs/OpenSearchService/pipelines.
VpcEndpoints (list) --
The VPC interface endpoints that have access to the pipeline.
(dict) --
An OpenSearch Ingestion-managed VPC endpoint that will access one or more pipelines.
VpcEndpointId (string) --
The unique identifier of the endpoint.
VpcId (string) --
The ID for your VPC. Amazon Web Services PrivateLink generates this value when you create a VPC.
VpcOptions (dict) --
Information about the VPC, including associated subnets and security groups.
SubnetIds (list) --
A list of subnet IDs associated with the VPC endpoint.
(string) --
SecurityGroupIds (list) --
A list of security groups associated with the VPC endpoint.
(string) --
VpcAttachmentOptions (dict) --
Options for attaching a VPC to a pipeline.
AttachToVpc (boolean) --
Whether a VPC is attached to the pipeline.
CidrBlock (string) --
The CIDR block to be reserved for OpenSearch Ingestion to create elastic network interfaces (ENIs).
VpcEndpointManagement (string) --
Defines whether you or Amazon OpenSearch Ingestion service create and manage the VPC endpoint configured for the pipeline.
BufferOptions (dict) --
Options that specify the configuration of a persistent buffer. To configure how OpenSearch Ingestion encrypts this data, set the EncryptionAtRestOptions. For more information, see Persistent buffering.
PersistentBufferEnabled (boolean) --
Whether persistent buffering should be enabled.
EncryptionAtRestOptions (dict) --
Options to control how OpenSearch encrypts buffer data.
KmsKeyArn (string) --
The ARN of the KMS key used to encrypt buffer data. By default, data is encrypted using an Amazon Web Services owned key.
VpcEndpointService (string) --
The VPC endpoint service name for the pipeline.
ServiceVpcEndpoints (list) --
A list of VPC endpoints that OpenSearch Ingestion has created to other Amazon Web Services services.
(dict) --
A container for information about VPC endpoints that were created to other services
ServiceName (string) --
The name of the service for which a VPC endpoint was created.
VpcEndpointId (string) --
The unique identifier of the VPC endpoint that was created.
Destinations (list) --
Destinations to which the pipeline writes data.
(dict) --
An object representing the destination of a pipeline.
ServiceName (string) --
The name of the service receiving data from the pipeline.
Endpoint (string) --
The endpoint receiving data from the pipeline.
Tags (list) --
A list of tags associated with the given pipeline.
(dict) --
A tag (key-value pair) for an OpenSearch Ingestion pipeline.
Key (string) --
The tag key. Tag keys must be unique for the pipeline to which they are attached.
Value (string) --
The value assigned to the corresponding tag key. Tag values can be null and don't have to be unique in a tag set. For example, you can have a key value pair in a tag set of project : Trinity and cost-center : Trinity
{'Pipeline': {'VpcEndpointService': 'string',
'VpcEndpoints': {'VpcOptions': {'VpcEndpointManagement': 'CUSTOMER '
'| '
'SERVICE'}}}}
Retrieves information about an OpenSearch Ingestion pipeline.
See also: AWS API Documentation
Request Syntax
client.get_pipeline(
PipelineName='string'
)
string
[REQUIRED]
The name of the pipeline.
dict
Response Syntax
{
'Pipeline': {
'PipelineName': 'string',
'PipelineArn': 'string',
'MinUnits': 123,
'MaxUnits': 123,
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'CREATE_FAILED'|'UPDATE_FAILED'|'STARTING'|'START_FAILED'|'STOPPING'|'STOPPED',
'StatusReason': {
'Description': 'string'
},
'PipelineConfigurationBody': 'string',
'CreatedAt': datetime(2015, 1, 1),
'LastUpdatedAt': datetime(2015, 1, 1),
'IngestEndpointUrls': [
'string',
],
'LogPublishingOptions': {
'IsLoggingEnabled': True|False,
'CloudWatchLogDestination': {
'LogGroup': 'string'
}
},
'VpcEndpoints': [
{
'VpcEndpointId': 'string',
'VpcId': 'string',
'VpcOptions': {
'SubnetIds': [
'string',
],
'SecurityGroupIds': [
'string',
],
'VpcAttachmentOptions': {
'AttachToVpc': True|False,
'CidrBlock': 'string'
},
'VpcEndpointManagement': 'CUSTOMER'|'SERVICE'
}
},
],
'BufferOptions': {
'PersistentBufferEnabled': True|False
},
'EncryptionAtRestOptions': {
'KmsKeyArn': 'string'
},
'VpcEndpointService': 'string',
'ServiceVpcEndpoints': [
{
'ServiceName': 'OPENSEARCH_SERVERLESS',
'VpcEndpointId': 'string'
},
],
'Destinations': [
{
'ServiceName': 'string',
'Endpoint': 'string'
},
],
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
}
Response Structure
(dict) --
Pipeline (dict) --
Detailed information about the requested pipeline.
PipelineName (string) --
The name of the pipeline.
PipelineArn (string) --
The Amazon Resource Name (ARN) of the pipeline.
MinUnits (integer) --
The minimum pipeline capacity, in Ingestion Compute Units (ICUs).
MaxUnits (integer) --
The maximum pipeline capacity, in Ingestion Compute Units (ICUs).
Status (string) --
The current status of the pipeline.
StatusReason (dict) --
The reason for the current status of the pipeline.
Description (string) --
A description of why a pipeline has a certain status.
PipelineConfigurationBody (string) --
The Data Prepper pipeline configuration in YAML format.
CreatedAt (datetime) --
The date and time when the pipeline was created.
LastUpdatedAt (datetime) --
The date and time when the pipeline was last updated.
IngestEndpointUrls (list) --
The ingestion endpoints for the pipeline, which you can send data to.
(string) --
LogPublishingOptions (dict) --
Key-value pairs that represent log publishing settings.
IsLoggingEnabled (boolean) --
Whether logs should be published.
CloudWatchLogDestination (dict) --
The destination for OpenSearch Ingestion logs sent to Amazon CloudWatch Logs. This parameter is required if IsLoggingEnabled is set to true.
LogGroup (string) --
The name of the CloudWatch Logs group to send pipeline logs to. You can specify an existing log group or create a new one. For example, /aws/vendedlogs/OpenSearchService/pipelines.
VpcEndpoints (list) --
The VPC interface endpoints that have access to the pipeline.
(dict) --
An OpenSearch Ingestion-managed VPC endpoint that will access one or more pipelines.
VpcEndpointId (string) --
The unique identifier of the endpoint.
VpcId (string) --
The ID for your VPC. Amazon Web Services PrivateLink generates this value when you create a VPC.
VpcOptions (dict) --
Information about the VPC, including associated subnets and security groups.
SubnetIds (list) --
A list of subnet IDs associated with the VPC endpoint.
(string) --
SecurityGroupIds (list) --
A list of security groups associated with the VPC endpoint.
(string) --
VpcAttachmentOptions (dict) --
Options for attaching a VPC to a pipeline.
AttachToVpc (boolean) --
Whether a VPC is attached to the pipeline.
CidrBlock (string) --
The CIDR block to be reserved for OpenSearch Ingestion to create elastic network interfaces (ENIs).
VpcEndpointManagement (string) --
Defines whether you or Amazon OpenSearch Ingestion service create and manage the VPC endpoint configured for the pipeline.
BufferOptions (dict) --
Options that specify the configuration of a persistent buffer. To configure how OpenSearch Ingestion encrypts this data, set the EncryptionAtRestOptions. For more information, see Persistent buffering.
PersistentBufferEnabled (boolean) --
Whether persistent buffering should be enabled.
EncryptionAtRestOptions (dict) --
Options to control how OpenSearch encrypts buffer data.
KmsKeyArn (string) --
The ARN of the KMS key used to encrypt buffer data. By default, data is encrypted using an Amazon Web Services owned key.
VpcEndpointService (string) --
The VPC endpoint service name for the pipeline.
ServiceVpcEndpoints (list) --
A list of VPC endpoints that OpenSearch Ingestion has created to other Amazon Web Services services.
(dict) --
A container for information about VPC endpoints that were created to other services
ServiceName (string) --
The name of the service for which a VPC endpoint was created.
VpcEndpointId (string) --
The unique identifier of the VPC endpoint that was created.
Destinations (list) --
Destinations to which the pipeline writes data.
(dict) --
An object representing the destination of a pipeline.
ServiceName (string) --
The name of the service receiving data from the pipeline.
Endpoint (string) --
The endpoint receiving data from the pipeline.
Tags (list) --
A list of tags associated with the given pipeline.
(dict) --
A tag (key-value pair) for an OpenSearch Ingestion pipeline.
Key (string) --
The tag key. Tag keys must be unique for the pipeline to which they are attached.
Value (string) --
The value assigned to the corresponding tag key. Tag values can be null and don't have to be unique in a tag set. For example, you can have a key value pair in a tag set of project : Trinity and cost-center : Trinity
{'Pipeline': {'VpcEndpointService': 'string',
'VpcEndpoints': {'VpcOptions': {'VpcEndpointManagement': 'CUSTOMER '
'| '
'SERVICE'}}}}
Starts an OpenSearch Ingestion pipeline. For more information, see Starting an OpenSearch Ingestion pipeline.
See also: AWS API Documentation
Request Syntax
client.start_pipeline(
PipelineName='string'
)
string
[REQUIRED]
The name of the pipeline to start.
dict
Response Syntax
{
'Pipeline': {
'PipelineName': 'string',
'PipelineArn': 'string',
'MinUnits': 123,
'MaxUnits': 123,
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'CREATE_FAILED'|'UPDATE_FAILED'|'STARTING'|'START_FAILED'|'STOPPING'|'STOPPED',
'StatusReason': {
'Description': 'string'
},
'PipelineConfigurationBody': 'string',
'CreatedAt': datetime(2015, 1, 1),
'LastUpdatedAt': datetime(2015, 1, 1),
'IngestEndpointUrls': [
'string',
],
'LogPublishingOptions': {
'IsLoggingEnabled': True|False,
'CloudWatchLogDestination': {
'LogGroup': 'string'
}
},
'VpcEndpoints': [
{
'VpcEndpointId': 'string',
'VpcId': 'string',
'VpcOptions': {
'SubnetIds': [
'string',
],
'SecurityGroupIds': [
'string',
],
'VpcAttachmentOptions': {
'AttachToVpc': True|False,
'CidrBlock': 'string'
},
'VpcEndpointManagement': 'CUSTOMER'|'SERVICE'
}
},
],
'BufferOptions': {
'PersistentBufferEnabled': True|False
},
'EncryptionAtRestOptions': {
'KmsKeyArn': 'string'
},
'VpcEndpointService': 'string',
'ServiceVpcEndpoints': [
{
'ServiceName': 'OPENSEARCH_SERVERLESS',
'VpcEndpointId': 'string'
},
],
'Destinations': [
{
'ServiceName': 'string',
'Endpoint': 'string'
},
],
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
}
Response Structure
(dict) --
Pipeline (dict) --
Information about an existing OpenSearch Ingestion pipeline.
PipelineName (string) --
The name of the pipeline.
PipelineArn (string) --
The Amazon Resource Name (ARN) of the pipeline.
MinUnits (integer) --
The minimum pipeline capacity, in Ingestion Compute Units (ICUs).
MaxUnits (integer) --
The maximum pipeline capacity, in Ingestion Compute Units (ICUs).
Status (string) --
The current status of the pipeline.
StatusReason (dict) --
The reason for the current status of the pipeline.
Description (string) --
A description of why a pipeline has a certain status.
PipelineConfigurationBody (string) --
The Data Prepper pipeline configuration in YAML format.
CreatedAt (datetime) --
The date and time when the pipeline was created.
LastUpdatedAt (datetime) --
The date and time when the pipeline was last updated.
IngestEndpointUrls (list) --
The ingestion endpoints for the pipeline, which you can send data to.
(string) --
LogPublishingOptions (dict) --
Key-value pairs that represent log publishing settings.
IsLoggingEnabled (boolean) --
Whether logs should be published.
CloudWatchLogDestination (dict) --
The destination for OpenSearch Ingestion logs sent to Amazon CloudWatch Logs. This parameter is required if IsLoggingEnabled is set to true.
LogGroup (string) --
The name of the CloudWatch Logs group to send pipeline logs to. You can specify an existing log group or create a new one. For example, /aws/vendedlogs/OpenSearchService/pipelines.
VpcEndpoints (list) --
The VPC interface endpoints that have access to the pipeline.
(dict) --
An OpenSearch Ingestion-managed VPC endpoint that will access one or more pipelines.
VpcEndpointId (string) --
The unique identifier of the endpoint.
VpcId (string) --
The ID for your VPC. Amazon Web Services PrivateLink generates this value when you create a VPC.
VpcOptions (dict) --
Information about the VPC, including associated subnets and security groups.
SubnetIds (list) --
A list of subnet IDs associated with the VPC endpoint.
(string) --
SecurityGroupIds (list) --
A list of security groups associated with the VPC endpoint.
(string) --
VpcAttachmentOptions (dict) --
Options for attaching a VPC to a pipeline.
AttachToVpc (boolean) --
Whether a VPC is attached to the pipeline.
CidrBlock (string) --
The CIDR block to be reserved for OpenSearch Ingestion to create elastic network interfaces (ENIs).
VpcEndpointManagement (string) --
Defines whether you or Amazon OpenSearch Ingestion service create and manage the VPC endpoint configured for the pipeline.
BufferOptions (dict) --
Options that specify the configuration of a persistent buffer. To configure how OpenSearch Ingestion encrypts this data, set the EncryptionAtRestOptions. For more information, see Persistent buffering.
PersistentBufferEnabled (boolean) --
Whether persistent buffering should be enabled.
EncryptionAtRestOptions (dict) --
Options to control how OpenSearch encrypts buffer data.
KmsKeyArn (string) --
The ARN of the KMS key used to encrypt buffer data. By default, data is encrypted using an Amazon Web Services owned key.
VpcEndpointService (string) --
The VPC endpoint service name for the pipeline.
ServiceVpcEndpoints (list) --
A list of VPC endpoints that OpenSearch Ingestion has created to other Amazon Web Services services.
(dict) --
A container for information about VPC endpoints that were created to other services
ServiceName (string) --
The name of the service for which a VPC endpoint was created.
VpcEndpointId (string) --
The unique identifier of the VPC endpoint that was created.
Destinations (list) --
Destinations to which the pipeline writes data.
(dict) --
An object representing the destination of a pipeline.
ServiceName (string) --
The name of the service receiving data from the pipeline.
Endpoint (string) --
The endpoint receiving data from the pipeline.
Tags (list) --
A list of tags associated with the given pipeline.
(dict) --
A tag (key-value pair) for an OpenSearch Ingestion pipeline.
Key (string) --
The tag key. Tag keys must be unique for the pipeline to which they are attached.
Value (string) --
The value assigned to the corresponding tag key. Tag values can be null and don't have to be unique in a tag set. For example, you can have a key value pair in a tag set of project : Trinity and cost-center : Trinity
{'Pipeline': {'VpcEndpointService': 'string',
'VpcEndpoints': {'VpcOptions': {'VpcEndpointManagement': 'CUSTOMER '
'| '
'SERVICE'}}}}
Stops an OpenSearch Ingestion pipeline. For more information, see Stopping an OpenSearch Ingestion pipeline.
See also: AWS API Documentation
Request Syntax
client.stop_pipeline(
PipelineName='string'
)
string
[REQUIRED]
The name of the pipeline to stop.
dict
Response Syntax
{
'Pipeline': {
'PipelineName': 'string',
'PipelineArn': 'string',
'MinUnits': 123,
'MaxUnits': 123,
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'CREATE_FAILED'|'UPDATE_FAILED'|'STARTING'|'START_FAILED'|'STOPPING'|'STOPPED',
'StatusReason': {
'Description': 'string'
},
'PipelineConfigurationBody': 'string',
'CreatedAt': datetime(2015, 1, 1),
'LastUpdatedAt': datetime(2015, 1, 1),
'IngestEndpointUrls': [
'string',
],
'LogPublishingOptions': {
'IsLoggingEnabled': True|False,
'CloudWatchLogDestination': {
'LogGroup': 'string'
}
},
'VpcEndpoints': [
{
'VpcEndpointId': 'string',
'VpcId': 'string',
'VpcOptions': {
'SubnetIds': [
'string',
],
'SecurityGroupIds': [
'string',
],
'VpcAttachmentOptions': {
'AttachToVpc': True|False,
'CidrBlock': 'string'
},
'VpcEndpointManagement': 'CUSTOMER'|'SERVICE'
}
},
],
'BufferOptions': {
'PersistentBufferEnabled': True|False
},
'EncryptionAtRestOptions': {
'KmsKeyArn': 'string'
},
'VpcEndpointService': 'string',
'ServiceVpcEndpoints': [
{
'ServiceName': 'OPENSEARCH_SERVERLESS',
'VpcEndpointId': 'string'
},
],
'Destinations': [
{
'ServiceName': 'string',
'Endpoint': 'string'
},
],
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
}
Response Structure
(dict) --
Pipeline (dict) --
Information about an existing OpenSearch Ingestion pipeline.
PipelineName (string) --
The name of the pipeline.
PipelineArn (string) --
The Amazon Resource Name (ARN) of the pipeline.
MinUnits (integer) --
The minimum pipeline capacity, in Ingestion Compute Units (ICUs).
MaxUnits (integer) --
The maximum pipeline capacity, in Ingestion Compute Units (ICUs).
Status (string) --
The current status of the pipeline.
StatusReason (dict) --
The reason for the current status of the pipeline.
Description (string) --
A description of why a pipeline has a certain status.
PipelineConfigurationBody (string) --
The Data Prepper pipeline configuration in YAML format.
CreatedAt (datetime) --
The date and time when the pipeline was created.
LastUpdatedAt (datetime) --
The date and time when the pipeline was last updated.
IngestEndpointUrls (list) --
The ingestion endpoints for the pipeline, which you can send data to.
(string) --
LogPublishingOptions (dict) --
Key-value pairs that represent log publishing settings.
IsLoggingEnabled (boolean) --
Whether logs should be published.
CloudWatchLogDestination (dict) --
The destination for OpenSearch Ingestion logs sent to Amazon CloudWatch Logs. This parameter is required if IsLoggingEnabled is set to true.
LogGroup (string) --
The name of the CloudWatch Logs group to send pipeline logs to. You can specify an existing log group or create a new one. For example, /aws/vendedlogs/OpenSearchService/pipelines.
VpcEndpoints (list) --
The VPC interface endpoints that have access to the pipeline.
(dict) --
An OpenSearch Ingestion-managed VPC endpoint that will access one or more pipelines.
VpcEndpointId (string) --
The unique identifier of the endpoint.
VpcId (string) --
The ID for your VPC. Amazon Web Services PrivateLink generates this value when you create a VPC.
VpcOptions (dict) --
Information about the VPC, including associated subnets and security groups.
SubnetIds (list) --
A list of subnet IDs associated with the VPC endpoint.
(string) --
SecurityGroupIds (list) --
A list of security groups associated with the VPC endpoint.
(string) --
VpcAttachmentOptions (dict) --
Options for attaching a VPC to a pipeline.
AttachToVpc (boolean) --
Whether a VPC is attached to the pipeline.
CidrBlock (string) --
The CIDR block to be reserved for OpenSearch Ingestion to create elastic network interfaces (ENIs).
VpcEndpointManagement (string) --
Defines whether you or Amazon OpenSearch Ingestion service create and manage the VPC endpoint configured for the pipeline.
BufferOptions (dict) --
Options that specify the configuration of a persistent buffer. To configure how OpenSearch Ingestion encrypts this data, set the EncryptionAtRestOptions. For more information, see Persistent buffering.
PersistentBufferEnabled (boolean) --
Whether persistent buffering should be enabled.
EncryptionAtRestOptions (dict) --
Options to control how OpenSearch encrypts buffer data.
KmsKeyArn (string) --
The ARN of the KMS key used to encrypt buffer data. By default, data is encrypted using an Amazon Web Services owned key.
VpcEndpointService (string) --
The VPC endpoint service name for the pipeline.
ServiceVpcEndpoints (list) --
A list of VPC endpoints that OpenSearch Ingestion has created to other Amazon Web Services services.
(dict) --
A container for information about VPC endpoints that were created to other services
ServiceName (string) --
The name of the service for which a VPC endpoint was created.
VpcEndpointId (string) --
The unique identifier of the VPC endpoint that was created.
Destinations (list) --
Destinations to which the pipeline writes data.
(dict) --
An object representing the destination of a pipeline.
ServiceName (string) --
The name of the service receiving data from the pipeline.
Endpoint (string) --
The endpoint receiving data from the pipeline.
Tags (list) --
A list of tags associated with the given pipeline.
(dict) --
A tag (key-value pair) for an OpenSearch Ingestion pipeline.
Key (string) --
The tag key. Tag keys must be unique for the pipeline to which they are attached.
Value (string) --
The value assigned to the corresponding tag key. Tag values can be null and don't have to be unique in a tag set. For example, you can have a key value pair in a tag set of project : Trinity and cost-center : Trinity
{'Pipeline': {'VpcEndpointService': 'string',
'VpcEndpoints': {'VpcOptions': {'VpcEndpointManagement': 'CUSTOMER '
'| '
'SERVICE'}}}}
Updates an OpenSearch Ingestion pipeline. For more information, see Updating Amazon OpenSearch Ingestion pipelines.
See also: AWS API Documentation
Request Syntax
client.update_pipeline(
PipelineName='string',
MinUnits=123,
MaxUnits=123,
PipelineConfigurationBody='string',
LogPublishingOptions={
'IsLoggingEnabled': True|False,
'CloudWatchLogDestination': {
'LogGroup': 'string'
}
},
BufferOptions={
'PersistentBufferEnabled': True|False
},
EncryptionAtRestOptions={
'KmsKeyArn': 'string'
}
)
string
[REQUIRED]
The name of the pipeline to update.
integer
The minimum pipeline capacity, in Ingestion Compute Units (ICUs).
integer
The maximum pipeline capacity, in Ingestion Compute Units (ICUs)
string
The pipeline configuration in YAML format. The command accepts the pipeline configuration as a string or within a .yaml file. If you provide the configuration as a string, each new line must be escaped with \n.
dict
Key-value pairs to configure log publishing.
IsLoggingEnabled (boolean) --
Whether logs should be published.
CloudWatchLogDestination (dict) --
The destination for OpenSearch Ingestion logs sent to Amazon CloudWatch Logs. This parameter is required if IsLoggingEnabled is set to true.
LogGroup (string) -- [REQUIRED]
The name of the CloudWatch Logs group to send pipeline logs to. You can specify an existing log group or create a new one. For example, /aws/vendedlogs/OpenSearchService/pipelines.
dict
Key-value pairs to configure persistent buffering for the pipeline.
PersistentBufferEnabled (boolean) -- [REQUIRED]
Whether persistent buffering should be enabled.
dict
Key-value pairs to configure encryption for data that is written to a persistent buffer.
KmsKeyArn (string) -- [REQUIRED]
The ARN of the KMS key used to encrypt buffer data. By default, data is encrypted using an Amazon Web Services owned key.
dict
Response Syntax
{
'Pipeline': {
'PipelineName': 'string',
'PipelineArn': 'string',
'MinUnits': 123,
'MaxUnits': 123,
'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'CREATE_FAILED'|'UPDATE_FAILED'|'STARTING'|'START_FAILED'|'STOPPING'|'STOPPED',
'StatusReason': {
'Description': 'string'
},
'PipelineConfigurationBody': 'string',
'CreatedAt': datetime(2015, 1, 1),
'LastUpdatedAt': datetime(2015, 1, 1),
'IngestEndpointUrls': [
'string',
],
'LogPublishingOptions': {
'IsLoggingEnabled': True|False,
'CloudWatchLogDestination': {
'LogGroup': 'string'
}
},
'VpcEndpoints': [
{
'VpcEndpointId': 'string',
'VpcId': 'string',
'VpcOptions': {
'SubnetIds': [
'string',
],
'SecurityGroupIds': [
'string',
],
'VpcAttachmentOptions': {
'AttachToVpc': True|False,
'CidrBlock': 'string'
},
'VpcEndpointManagement': 'CUSTOMER'|'SERVICE'
}
},
],
'BufferOptions': {
'PersistentBufferEnabled': True|False
},
'EncryptionAtRestOptions': {
'KmsKeyArn': 'string'
},
'VpcEndpointService': 'string',
'ServiceVpcEndpoints': [
{
'ServiceName': 'OPENSEARCH_SERVERLESS',
'VpcEndpointId': 'string'
},
],
'Destinations': [
{
'ServiceName': 'string',
'Endpoint': 'string'
},
],
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
}
}
Response Structure
(dict) --
Pipeline (dict) --
Container for information about the updated pipeline.
PipelineName (string) --
The name of the pipeline.
PipelineArn (string) --
The Amazon Resource Name (ARN) of the pipeline.
MinUnits (integer) --
The minimum pipeline capacity, in Ingestion Compute Units (ICUs).
MaxUnits (integer) --
The maximum pipeline capacity, in Ingestion Compute Units (ICUs).
Status (string) --
The current status of the pipeline.
StatusReason (dict) --
The reason for the current status of the pipeline.
Description (string) --
A description of why a pipeline has a certain status.
PipelineConfigurationBody (string) --
The Data Prepper pipeline configuration in YAML format.
CreatedAt (datetime) --
The date and time when the pipeline was created.
LastUpdatedAt (datetime) --
The date and time when the pipeline was last updated.
IngestEndpointUrls (list) --
The ingestion endpoints for the pipeline, which you can send data to.
(string) --
LogPublishingOptions (dict) --
Key-value pairs that represent log publishing settings.
IsLoggingEnabled (boolean) --
Whether logs should be published.
CloudWatchLogDestination (dict) --
The destination for OpenSearch Ingestion logs sent to Amazon CloudWatch Logs. This parameter is required if IsLoggingEnabled is set to true.
LogGroup (string) --
The name of the CloudWatch Logs group to send pipeline logs to. You can specify an existing log group or create a new one. For example, /aws/vendedlogs/OpenSearchService/pipelines.
VpcEndpoints (list) --
The VPC interface endpoints that have access to the pipeline.
(dict) --
An OpenSearch Ingestion-managed VPC endpoint that will access one or more pipelines.
VpcEndpointId (string) --
The unique identifier of the endpoint.
VpcId (string) --
The ID for your VPC. Amazon Web Services PrivateLink generates this value when you create a VPC.
VpcOptions (dict) --
Information about the VPC, including associated subnets and security groups.
SubnetIds (list) --
A list of subnet IDs associated with the VPC endpoint.
(string) --
SecurityGroupIds (list) --
A list of security groups associated with the VPC endpoint.
(string) --
VpcAttachmentOptions (dict) --
Options for attaching a VPC to a pipeline.
AttachToVpc (boolean) --
Whether a VPC is attached to the pipeline.
CidrBlock (string) --
The CIDR block to be reserved for OpenSearch Ingestion to create elastic network interfaces (ENIs).
VpcEndpointManagement (string) --
Defines whether you or Amazon OpenSearch Ingestion service create and manage the VPC endpoint configured for the pipeline.
BufferOptions (dict) --
Options that specify the configuration of a persistent buffer. To configure how OpenSearch Ingestion encrypts this data, set the EncryptionAtRestOptions. For more information, see Persistent buffering.
PersistentBufferEnabled (boolean) --
Whether persistent buffering should be enabled.
EncryptionAtRestOptions (dict) --
Options to control how OpenSearch encrypts buffer data.
KmsKeyArn (string) --
The ARN of the KMS key used to encrypt buffer data. By default, data is encrypted using an Amazon Web Services owned key.
VpcEndpointService (string) --
The VPC endpoint service name for the pipeline.
ServiceVpcEndpoints (list) --
A list of VPC endpoints that OpenSearch Ingestion has created to other Amazon Web Services services.
(dict) --
A container for information about VPC endpoints that were created to other services
ServiceName (string) --
The name of the service for which a VPC endpoint was created.
VpcEndpointId (string) --
The unique identifier of the VPC endpoint that was created.
Destinations (list) --
Destinations to which the pipeline writes data.
(dict) --
An object representing the destination of a pipeline.
ServiceName (string) --
The name of the service receiving data from the pipeline.
Endpoint (string) --
The endpoint receiving data from the pipeline.
Tags (list) --
A list of tags associated with the given pipeline.
(dict) --
A tag (key-value pair) for an OpenSearch Ingestion pipeline.
Key (string) --
The tag key. Tag keys must be unique for the pipeline to which they are attached.
Value (string) --
The value assigned to the corresponding tag key. Tag values can be null and don't have to be unique in a tag set. For example, you can have a key value pair in a tag set of project : Trinity and cost-center : Trinity