AWS API Changes

2022/03/14 - Amazon Timestream Query - 2 updated api methods

Changes Amazon Timestream Scheduled Queries now support Timestamp datatype in a multi-measure record.

CreateScheduledQuery (updated)

Link ¶
Changes (request)

{'TargetConfiguration': {'TimestreamConfiguration': {'MixedMeasureMappings': {'MultiMeasureAttributeMappings': {'MeasureValueType': {'TIMESTAMP'}}},
                                                     'MultiMeasureMappings': {'MultiMeasureAttributeMappings': {'MeasureValueType': {'TIMESTAMP'}}}}}}

Create a scheduled query that will be run on your behalf at the configured schedule. Timestream assumes the execution role provided as part of the ScheduledQueryExecutionRoleArn parameter to run the query. You can use the NotificationConfiguration parameter to configure notification for your scheduled query operations.

See also: AWS API Documentation

Request Syntax

client.create_scheduled_query(
    Name='string',
    QueryString='string',
    ScheduleConfiguration={
        'ScheduleExpression': 'string'
    },
    NotificationConfiguration={
        'SnsConfiguration': {
            'TopicArn': 'string'
        }
    },
    TargetConfiguration={
        'TimestreamConfiguration': {
            'DatabaseName': 'string',
            'TableName': 'string',
            'TimeColumn': 'string',
            'DimensionMappings': [
                {
                    'Name': 'string',
                    'DimensionValueType': 'VARCHAR'
                },
            ],
            'MultiMeasureMappings': {
                'TargetMultiMeasureName': 'string',
                'MultiMeasureAttributeMappings': [
                    {
                        'SourceColumn': 'string',
                        'TargetMultiMeasureAttributeName': 'string',
                        'MeasureValueType': 'BIGINT'|'BOOLEAN'|'DOUBLE'|'VARCHAR'|'TIMESTAMP'
                    },
                ]
            },
            'MixedMeasureMappings': [
                {
                    'MeasureName': 'string',
                    'SourceColumn': 'string',
                    'TargetMeasureName': 'string',
                    'MeasureValueType': 'BIGINT'|'BOOLEAN'|'DOUBLE'|'VARCHAR'|'MULTI',
                    'MultiMeasureAttributeMappings': [
                        {
                            'SourceColumn': 'string',
                            'TargetMultiMeasureAttributeName': 'string',
                            'MeasureValueType': 'BIGINT'|'BOOLEAN'|'DOUBLE'|'VARCHAR'|'TIMESTAMP'
                        },
                    ]
                },
            ],
            'MeasureNameColumn': 'string'
        }
    },
    ClientToken='string',
    ScheduledQueryExecutionRoleArn='string',
    Tags=[
        {
            'Key': 'string',
            'Value': 'string'
        },
    ],
    KmsKeyId='string',
    ErrorReportConfiguration={
        'S3Configuration': {
            'BucketName': 'string',
            'ObjectKeyPrefix': 'string',
            'EncryptionOption': 'SSE_S3'|'SSE_KMS'
        }
    }
)

type Name:

string

param Name:

[REQUIRED]

Name of the scheduled query.

type QueryString:

string

param QueryString:

[REQUIRED]

The query string to run. Parameter names can be specified in the query string @ character followed by an identifier. The named Parameter @scheduled_runtime is reserved and can be used in the query to get the time at which the query is scheduled to run.

The timestamp calculated according to the ScheduleConfiguration parameter, will be the value of @scheduled_runtime paramater for each query run. For example, consider an instance of a scheduled query executing on 2021-12-01 00:00:00. For this instance, the @scheduled_runtime parameter is initialized to the timestamp 2021-12-01 00:00:00 when invoking the query.

type ScheduleConfiguration:

dict

param ScheduleConfiguration:

[REQUIRED]

The schedule configuration for the query.

ScheduleExpression (string) -- [REQUIRED]

An expression that denotes when to trigger the scheduled query run. This can be a cron expression or a rate expression.

type NotificationConfiguration:

dict

param NotificationConfiguration:

[REQUIRED]

Notification configuration for the scheduled query. A notification is sent by Timestream when a query run finishes, when the state is updated or when you delete it.

SnsConfiguration (dict) -- [REQUIRED]

Details on SNS configuration.
- TopicArn (string) -- [REQUIRED]
  
  SNS topic ARN that the scheduled query status notifications will be sent to.

type TargetConfiguration:

dict

param TargetConfiguration:

Configuration used for writing the result of a query.

TimestreamConfiguration (dict) -- [REQUIRED]

Configuration needed to write data into the Timestream database and table.
- DatabaseName (string) -- [REQUIRED]
  
  Name of Timestream database to which the query result will be written.
- TableName (string) -- [REQUIRED]
  
  Name of Timestream table that the query result will be written to. The table should be within the same database that is provided in Timestream configuration.
- TimeColumn (string) -- [REQUIRED]
  
  Column from query result that should be used as the time column in destination table. Column type for this should be TIMESTAMP.
- DimensionMappings (list) -- [REQUIRED]
  
  This is to allow mapping column(s) from the query result to the dimension in the destination table.
  - (dict) --
    
    This type is used to map column(s) from the query result to a dimension in the destination table.
    - Name (string) -- [REQUIRED]
      
      Column name from query result.
    - DimensionValueType (string) -- [REQUIRED]
      
      Type for the dimension.
- MultiMeasureMappings (dict) --
  
  Multi-measure mappings.
  - TargetMultiMeasureName (string) --
    
    The name of the target multi-measure name in the derived table. This input is required when measureNameColumn is not provided. If MeasureNameColumn is provided, then value from that column will be used as multi-measure name.
  - MultiMeasureAttributeMappings (list) -- [REQUIRED]
    
    Required. Attribute mappings to be used for mapping query results to ingest data for multi-measure attributes.
    - (dict) --
      
      Attribute mapping for MULTI value measures.
      - SourceColumn (string) -- [REQUIRED]
        
        Source column from where the attribute value is to be read.
      - TargetMultiMeasureAttributeName (string) --
        
        Custom name to be used for attribute name in derived table. If not provided, source column name would be used.
      - MeasureValueType (string) -- [REQUIRED]
        
        Type of the attribute to be read from the source column.
- MixedMeasureMappings (list) --
  
  Specifies how to map measures to multi-measure records.
  - (dict) --
    
    MixedMeasureMappings are mappings that can be used to ingest data into a mixture of narrow and multi measures in the derived table.
    - MeasureName (string) --
      
      Refers to the value of measure_name in a result row. This field is required if MeasureNameColumn is provided.
    - SourceColumn (string) --
      
      This field refers to the source column from which measure-value is to be read for result materialization.
    - TargetMeasureName (string) --
      
      Target measure name to be used. If not provided, the target measure name by default would be measure-name if provided, or sourceColumn otherwise.
    - MeasureValueType (string) -- [REQUIRED]
      
      Type of the value that is to be read from sourceColumn. If the mapping is for MULTI, use MeasureValueType.MULTI.
    - MultiMeasureAttributeMappings (list) --
      
      Required when measureValueType is MULTI. Attribute mappings for MULTI value measures.
      - (dict) --
        
        Attribute mapping for MULTI value measures.
        
        SourceColumn (string) -- [REQUIRED]
        
        Source column from where the attribute value is to be read.
        
        TargetMultiMeasureAttributeName (string) --
        
        Custom name to be used for attribute name in derived table. If not provided, source column name would be used.
        
        MeasureValueType (string) -- [REQUIRED]
        
        Type of the attribute to be read from the source column.
- MeasureNameColumn (string) --
  
  Name of the measure column.

type ClientToken:

string

param ClientToken:

Using a ClientToken makes the call to CreateScheduledQuery idempotent, in other words, making the same request repeatedly will produce the same result. Making multiple identical CreateScheduledQuery requests has the same effect as making a single request.

If CreateScheduledQuery is called without a ClientToken, the Query SDK generates a ClientToken on your behalf.
After 8 hours, any request with the same ClientToken is treated as a new request.

This field is autopopulated if not provided.

type ScheduledQueryExecutionRoleArn:

string

param ScheduledQueryExecutionRoleArn:

[REQUIRED]

The ARN for the IAM role that Timestream will assume when running the scheduled query.

type Tags:

list

param Tags:

A list of key-value pairs to label the scheduled query.

(dict) --

A tag is a label that you assign to a Timestream database and/or table. Each tag consists of a key and an optional value, both of which you define. Tags enable you to categorize databases and/or tables, for example, by purpose, owner, or environment.
- Key (string) -- [REQUIRED]
  
  The key of the tag. Tag keys are case sensitive.
- Value (string) -- [REQUIRED]
  
  The value of the tag. Tag values are case sensitive and can be null.

type KmsKeyId:

string

param KmsKeyId:

The Amazon KMS key used to encrypt the scheduled query resource, at-rest. If the Amazon KMS key is not specified, the scheduled query resource will be encrypted with a Timestream owned Amazon KMS key. To specify a KMS key, use the key ID, key ARN, alias name, or alias ARN. When using an alias name, prefix the name with alias/

If ErrorReportConfiguration uses SSE_KMS as encryption type, the same KmsKeyId is used to encrypt the error report at rest.

type ErrorReportConfiguration:

dict

param ErrorReportConfiguration:

[REQUIRED]

Configuration for error reporting. Error reports will be generated when a problem is encountered when writing the query results.

S3Configuration (dict) -- [REQUIRED]

The S3 configuration for the error reports.
- BucketName (string) -- [REQUIRED]
  
  Name of the S3 bucket under which error reports will be created.
- ObjectKeyPrefix (string) --
  
  Prefix for the error report key. Timestream by default adds the following prefix to the error report path.
- EncryptionOption (string) --
  
  Encryption at rest options for the error reports. If no encryption option is specified, Timestream will choose SSE_S3 as default.

rtype:

dict

returns:

Response Syntax

{
    'Arn': 'string'
}

Response Structure

(dict) --
- Arn (string) --
  
  ARN for the created scheduled query.

DescribeScheduledQuery (updated)

Link ¶
Changes (response)

{'ScheduledQuery': {'TargetConfiguration': {'TimestreamConfiguration': {'MixedMeasureMappings': {'MultiMeasureAttributeMappings': {'MeasureValueType': {'TIMESTAMP'}}},
                                                                        'MultiMeasureMappings': {'MultiMeasureAttributeMappings': {'MeasureValueType': {'TIMESTAMP'}}}}}}}

Provides detailed information about a scheduled query.

See also: AWS API Documentation

Request Syntax

client.describe_scheduled_query(
    ScheduledQueryArn='string'
)

type ScheduledQueryArn:

string

param ScheduledQueryArn:

[REQUIRED]

The ARN of the scheduled query.

rtype:

dict

returns:

Response Syntax

{
    'ScheduledQuery': {
        'Arn': 'string',
        'Name': 'string',
        'QueryString': 'string',
        'CreationTime': datetime(2015, 1, 1),
        'State': 'ENABLED'|'DISABLED',
        'PreviousInvocationTime': datetime(2015, 1, 1),
        'NextInvocationTime': datetime(2015, 1, 1),
        'ScheduleConfiguration': {
            'ScheduleExpression': 'string'
        },
        'NotificationConfiguration': {
            'SnsConfiguration': {
                'TopicArn': 'string'
            }
        },
        'TargetConfiguration': {
            'TimestreamConfiguration': {
                'DatabaseName': 'string',
                'TableName': 'string',
                'TimeColumn': 'string',
                'DimensionMappings': [
                    {
                        'Name': 'string',
                        'DimensionValueType': 'VARCHAR'
                    },
                ],
                'MultiMeasureMappings': {
                    'TargetMultiMeasureName': 'string',
                    'MultiMeasureAttributeMappings': [
                        {
                            'SourceColumn': 'string',
                            'TargetMultiMeasureAttributeName': 'string',
                            'MeasureValueType': 'BIGINT'|'BOOLEAN'|'DOUBLE'|'VARCHAR'|'TIMESTAMP'
                        },
                    ]
                },
                'MixedMeasureMappings': [
                    {
                        'MeasureName': 'string',
                        'SourceColumn': 'string',
                        'TargetMeasureName': 'string',
                        'MeasureValueType': 'BIGINT'|'BOOLEAN'|'DOUBLE'|'VARCHAR'|'MULTI',
                        'MultiMeasureAttributeMappings': [
                            {
                                'SourceColumn': 'string',
                                'TargetMultiMeasureAttributeName': 'string',
                                'MeasureValueType': 'BIGINT'|'BOOLEAN'|'DOUBLE'|'VARCHAR'|'TIMESTAMP'
                            },
                        ]
                    },
                ],
                'MeasureNameColumn': 'string'
            }
        },
        'ScheduledQueryExecutionRoleArn': 'string',
        'KmsKeyId': 'string',
        'ErrorReportConfiguration': {
            'S3Configuration': {
                'BucketName': 'string',
                'ObjectKeyPrefix': 'string',
                'EncryptionOption': 'SSE_S3'|'SSE_KMS'
            }
        },
        'LastRunSummary': {
            'InvocationTime': datetime(2015, 1, 1),
            'TriggerTime': datetime(2015, 1, 1),
            'RunStatus': 'AUTO_TRIGGER_SUCCESS'|'AUTO_TRIGGER_FAILURE'|'MANUAL_TRIGGER_SUCCESS'|'MANUAL_TRIGGER_FAILURE',
            'ExecutionStats': {
                'ExecutionTimeInMillis': 123,
                'DataWrites': 123,
                'BytesMetered': 123,
                'RecordsIngested': 123,
                'QueryResultRows': 123
            },
            'ErrorReportLocation': {
                'S3ReportLocation': {
                    'BucketName': 'string',
                    'ObjectKey': 'string'
                }
            },
            'FailureReason': 'string'
        },
        'RecentlyFailedRuns': [
            {
                'InvocationTime': datetime(2015, 1, 1),
                'TriggerTime': datetime(2015, 1, 1),
                'RunStatus': 'AUTO_TRIGGER_SUCCESS'|'AUTO_TRIGGER_FAILURE'|'MANUAL_TRIGGER_SUCCESS'|'MANUAL_TRIGGER_FAILURE',
                'ExecutionStats': {
                    'ExecutionTimeInMillis': 123,
                    'DataWrites': 123,
                    'BytesMetered': 123,
                    'RecordsIngested': 123,
                    'QueryResultRows': 123
                },
                'ErrorReportLocation': {
                    'S3ReportLocation': {
                        'BucketName': 'string',
                        'ObjectKey': 'string'
                    }
                },
                'FailureReason': 'string'
            },
        ]
    }
}

Response Structure

(dict) --
- ScheduledQuery (dict) --
  
  The scheduled query.
  - Arn (string) --
    
    Scheduled query ARN.
  - Name (string) --
    
    Name of the scheduled query.
  - QueryString (string) --
    
    The query to be run.
  - CreationTime (datetime) --
    
    Creation time of the scheduled query.
  - State (string) --
    
    State of the scheduled query.
  - PreviousInvocationTime (datetime) --
    
    Last time the query was run.
  - NextInvocationTime (datetime) --
    
    The next time the scheduled query is scheduled to run.
  - ScheduleConfiguration (dict) --
    
    Schedule configuration.
    - ScheduleExpression (string) --
      
      An expression that denotes when to trigger the scheduled query run. This can be a cron expression or a rate expression.
  - NotificationConfiguration (dict) --
    
    Notification configuration.
    - SnsConfiguration (dict) --
      
      Details on SNS configuration.
      - TopicArn (string) --
        
        SNS topic ARN that the scheduled query status notifications will be sent to.
  - TargetConfiguration (dict) --
    
    Scheduled query target store configuration.
    - TimestreamConfiguration (dict) --
      
      Configuration needed to write data into the Timestream database and table.
      - DatabaseName (string) --
        
        Name of Timestream database to which the query result will be written.
      - TableName (string) --
        
        Name of Timestream table that the query result will be written to. The table should be within the same database that is provided in Timestream configuration.
      - TimeColumn (string) --
        
        Column from query result that should be used as the time column in destination table. Column type for this should be TIMESTAMP.
      - DimensionMappings (list) --
        
        This is to allow mapping column(s) from the query result to the dimension in the destination table.
        
        (dict) --
        
        This type is used to map column(s) from the query result to a dimension in the destination table.
        
        Name (string) --
        
        Column name from query result.
        
        DimensionValueType (string) --
        
        Type for the dimension.
      - MultiMeasureMappings (dict) --
        
        Multi-measure mappings.
        
        TargetMultiMeasureName (string) --
        
        The name of the target multi-measure name in the derived table. This input is required when measureNameColumn is not provided. If MeasureNameColumn is provided, then value from that column will be used as multi-measure name.
        
        MultiMeasureAttributeMappings (list) --
        
        Required. Attribute mappings to be used for mapping query results to ingest data for multi-measure attributes.
        
        (dict) --
        
        Attribute mapping for MULTI value measures.
        
        SourceColumn (string) --
        
        Source column from where the attribute value is to be read.
        
        TargetMultiMeasureAttributeName (string) --
        
        Custom name to be used for attribute name in derived table. If not provided, source column name would be used.
        
        MeasureValueType (string) --
        
        Type of the attribute to be read from the source column.
      - MixedMeasureMappings (list) --
        
        Specifies how to map measures to multi-measure records.
        
        (dict) --
        
        MixedMeasureMappings are mappings that can be used to ingest data into a mixture of narrow and multi measures in the derived table.
        
        MeasureName (string) --
        
        Refers to the value of measure_name in a result row. This field is required if MeasureNameColumn is provided.
        
        SourceColumn (string) --
        
        This field refers to the source column from which measure-value is to be read for result materialization.
        
        TargetMeasureName (string) --
        
        Target measure name to be used. If not provided, the target measure name by default would be measure-name if provided, or sourceColumn otherwise.
        
        MeasureValueType (string) --
        
        Type of the value that is to be read from sourceColumn. If the mapping is for MULTI, use MeasureValueType.MULTI.
        
        MultiMeasureAttributeMappings (list) --
        
        Required when measureValueType is MULTI. Attribute mappings for MULTI value measures.
        
        (dict) --
        
        Attribute mapping for MULTI value measures.
        
        SourceColumn (string) --
        
        Source column from where the attribute value is to be read.
        
        TargetMultiMeasureAttributeName (string) --
        
        Custom name to be used for attribute name in derived table. If not provided, source column name would be used.
        
        MeasureValueType (string) --
        
        Type of the attribute to be read from the source column.
      - MeasureNameColumn (string) --
        
        Name of the measure column.
  - ScheduledQueryExecutionRoleArn (string) --
    
    IAM role that Timestream uses to run the schedule query.
  - KmsKeyId (string) --
    
    A customer provided KMS key used to encrypt the scheduled query resource.
  - ErrorReportConfiguration (dict) --
    
    Error-reporting configuration for the scheduled query.
    - S3Configuration (dict) --
      
      The S3 configuration for the error reports.
      - BucketName (string) --
        
        Name of the S3 bucket under which error reports will be created.
      - ObjectKeyPrefix (string) --
        
        Prefix for the error report key. Timestream by default adds the following prefix to the error report path.
      - EncryptionOption (string) --
        
        Encryption at rest options for the error reports. If no encryption option is specified, Timestream will choose SSE_S3 as default.
  - LastRunSummary (dict) --
    
    Runtime summary for the last scheduled query run.
    - InvocationTime (datetime) --
      
      InvocationTime for this run. This is the time at which the query is scheduled to run. Parameter @scheduled_runtime can be used in the query to get the value.
    - TriggerTime (datetime) --
      
      The actual time when the query was run.
    - RunStatus (string) --
      
      The status of a scheduled query run.
    - ExecutionStats (dict) --
      
      Runtime statistics for a scheduled run.
      - ExecutionTimeInMillis (integer) --
        
        Total time, measured in milliseconds, that was needed for the scheduled query run to complete.
      - DataWrites (integer) --
        
        Data writes metered for records ingested in a single scheduled query run.
      - BytesMetered (integer) --
        
        Bytes metered for a single scheduled query run.
      - RecordsIngested (integer) --
        
        The number of records ingested for a single scheduled query run.
      - QueryResultRows (integer) --
        
        Number of rows present in the output from running a query before ingestion to destination data source.
    - ErrorReportLocation (dict) --
      
      S3 location for error report.
      - S3ReportLocation (dict) --
        
        The S3 location where error reports are written.
        
        BucketName (string) --
        
        S3 bucket name.
        
        ObjectKey (string) --
        
        S3 key.
    - FailureReason (string) --
      
      Error message for the scheduled query in case of failure. You might have to look at the error report to get more detailed error reasons.
  - RecentlyFailedRuns (list) --
    
    Runtime summary for the last five failed scheduled query runs.
    - (dict) --
      
      Run summary for the scheduled query
      - InvocationTime (datetime) --
        
        InvocationTime for this run. This is the time at which the query is scheduled to run. Parameter @scheduled_runtime can be used in the query to get the value.
      - TriggerTime (datetime) --
        
        The actual time when the query was run.
      - RunStatus (string) --
        
        The status of a scheduled query run.
      - ExecutionStats (dict) --
        
        Runtime statistics for a scheduled run.
        
        ExecutionTimeInMillis (integer) --
        
        Total time, measured in milliseconds, that was needed for the scheduled query run to complete.
        
        DataWrites (integer) --
        
        Data writes metered for records ingested in a single scheduled query run.
        
        BytesMetered (integer) --
        
        Bytes metered for a single scheduled query run.
        
        RecordsIngested (integer) --
        
        The number of records ingested for a single scheduled query run.
        
        QueryResultRows (integer) --
        
        Number of rows present in the output from running a query before ingestion to destination data source.
      - ErrorReportLocation (dict) --
        
        S3 location for error report.
        
        S3ReportLocation (dict) --
        
        The S3 location where error reports are written.
        
        BucketName (string) --
        
        S3 bucket name.
        
        ObjectKey (string) --
        
        S3 key.
      - FailureReason (string) --
        
        Error message for the scheduled query in case of failure. You might have to look at the error report to get more detailed error reasons.