2024/12/02 - Amazon CloudWatch Logs - 4 new5 updated api methods
Changes Adds PutIntegration, GetIntegration, ListIntegrations and DeleteIntegration APIs. Adds QueryLanguage support to StartQuery, GetQueryResults, DescribeQueries, DescribeQueryDefinitions, and PutQueryDefinition APIs.
Creates an integration between CloudWatch Logs and another service in this account. Currently, only integrations with OpenSearch Service are supported, and currently you can have only one integration in your account.
Integrating with OpenSearch Service makes it possible for you to create curated vended logs dashboards, powered by OpenSearch Service analytics. For more information, see Vended log dashboards powered by Amazon OpenSearch Service.
You can use this operation only to create a new integration. You can't modify an existing integration.
See also: AWS API Documentation
Request Syntax
client.put_integration(
integrationName='string',
resourceConfig={
'openSearchResourceConfig': {
'kmsKeyArn': 'string',
'dataSourceRoleArn': 'string',
'dashboardViewerPrincipals': [
'string',
],
'applicationArn': 'string',
'retentionDays': 123
}
},
integrationType='OPENSEARCH'
)
string
[REQUIRED]
A name for the integration.
dict
[REQUIRED]
A structure that contains configuration information for the integration that you are creating.
openSearchResourceConfig (dict) --
This structure contains configuration details about an integration between CloudWatch Logs and OpenSearch Service.
kmsKeyArn (string) --
To have the vended dashboard data encrypted with KMS instead of the CloudWatch Logs default encryption method, specify the ARN of the KMS key that you want to use.
dataSourceRoleArn (string) -- [REQUIRED]
Specify the ARN of an IAM role that CloudWatch Logs will use to create the integration. This role must have the permissions necessary to access the OpenSearch Service collection to be able to create the dashboards. For more information about the permissions needed, see Create an IAM role to access the OpenSearch Service collection in the CloudWatch Logs User Guide.
dashboardViewerPrincipals (list) -- [REQUIRED]
Specify the ARNs of IAM roles and IAM users who you want to grant permission to for viewing the dashboards.
(string) --
applicationArn (string) --
If you want to use an existing OpenSearch Service application for your integration with OpenSearch Service, specify it here. If you omit this, a new application will be created.
retentionDays (integer) -- [REQUIRED]
Specify how many days that you want the data derived by OpenSearch Service to be retained in the index that the dashboard refers to. This also sets the maximum time period that you can choose when viewing data in the dashboard. Choosing a longer time frame will incur additional costs.
string
[REQUIRED]
The type of integration. Currently, the only supported type is OPENSEARCH.
dict
Response Syntax
{
'integrationName': 'string',
'integrationStatus': 'PROVISIONING'|'ACTIVE'|'FAILED'
}
Response Structure
(dict) --
integrationName (string) --
The name of the integration that you just created.
integrationStatus (string) --
The status of the integration that you just created.
After you create an integration, it takes a few minutes to complete. During this time, you'll see the status as PROVISIONING.
Deletes the integration between CloudWatch Logs and OpenSearch Service. If your integration has active vended logs dashboards, you must specify true for the force parameter, otherwise the operation will fail. If you delete the integration by setting force to true, all your vended logs dashboards powered by OpenSearch Service will be deleted and the data that was on them will no longer be accessible.
See also: AWS API Documentation
Request Syntax
client.delete_integration(
integrationName='string',
force=True|False
)
string
[REQUIRED]
The name of the integration to delete. To find the name of your integration, use ListIntegrations.
boolean
Specify true to force the deletion of the integration even if vended logs dashboards currently exist.
The default is false.
dict
Response Syntax
{}
Response Structure
(dict) --
Returns a list of integrations between CloudWatch Logs and other services in this account. Currently, only one integration can be created in an account, and this integration must be with OpenSearch Service.
See also: AWS API Documentation
Request Syntax
client.list_integrations(
integrationNamePrefix='string',
integrationType='OPENSEARCH',
integrationStatus='PROVISIONING'|'ACTIVE'|'FAILED'
)
string
To limit the results to integrations that start with a certain name prefix, specify that name prefix here.
string
To limit the results to integrations of a certain type, specify that type here.
string
To limit the results to integrations with a certain status, specify that status here.
dict
Response Syntax
{
'integrationSummaries': [
{
'integrationName': 'string',
'integrationType': 'OPENSEARCH',
'integrationStatus': 'PROVISIONING'|'ACTIVE'|'FAILED'
},
]
}
Response Structure
(dict) --
integrationSummaries (list) --
An array, where each object in the array contains information about one CloudWatch Logs integration in this account.
(dict) --
This structure contains information about one CloudWatch Logs integration. This structure is returned by a ListIntegrations operation.
integrationName (string) --
The name of this integration.
integrationType (string) --
The type of integration. Integrations with OpenSearch Service have the type OPENSEARCH.
integrationStatus (string) --
The current status of this integration.
Returns information about one integration between CloudWatch Logs and OpenSearch Service.
See also: AWS API Documentation
Request Syntax
client.get_integration(
integrationName='string'
)
string
[REQUIRED]
The name of the integration that you want to find information about. To find the name of your integration, use ListIntegrations
dict
Response Syntax
{
'integrationName': 'string',
'integrationType': 'OPENSEARCH',
'integrationStatus': 'PROVISIONING'|'ACTIVE'|'FAILED',
'integrationDetails': {
'openSearchIntegrationDetails': {
'dataSource': {
'dataSourceName': 'string',
'status': {
'status': 'ACTIVE'|'NOT_FOUND'|'ERROR',
'statusMessage': 'string'
}
},
'application': {
'applicationEndpoint': 'string',
'applicationArn': 'string',
'applicationId': 'string',
'status': {
'status': 'ACTIVE'|'NOT_FOUND'|'ERROR',
'statusMessage': 'string'
}
},
'collection': {
'collectionEndpoint': 'string',
'collectionArn': 'string',
'status': {
'status': 'ACTIVE'|'NOT_FOUND'|'ERROR',
'statusMessage': 'string'
}
},
'workspace': {
'workspaceId': 'string',
'status': {
'status': 'ACTIVE'|'NOT_FOUND'|'ERROR',
'statusMessage': 'string'
}
},
'encryptionPolicy': {
'policyName': 'string',
'status': {
'status': 'ACTIVE'|'NOT_FOUND'|'ERROR',
'statusMessage': 'string'
}
},
'networkPolicy': {
'policyName': 'string',
'status': {
'status': 'ACTIVE'|'NOT_FOUND'|'ERROR',
'statusMessage': 'string'
}
},
'accessPolicy': {
'policyName': 'string',
'status': {
'status': 'ACTIVE'|'NOT_FOUND'|'ERROR',
'statusMessage': 'string'
}
},
'lifecyclePolicy': {
'policyName': 'string',
'status': {
'status': 'ACTIVE'|'NOT_FOUND'|'ERROR',
'statusMessage': 'string'
}
}
}
}
}
Response Structure
(dict) --
integrationName (string) --
The name of the integration.
integrationType (string) --
The type of integration. Integrations with OpenSearch Service have the type OPENSEARCH.
integrationStatus (string) --
The current status of this integration.
integrationDetails (dict) --
A structure that contains information about the integration configuration. For an integration with OpenSearch Service, this includes information about OpenSearch Service resources such as the collection, the workspace, and policies.
openSearchIntegrationDetails (dict) --
This structure contains complete information about one integration between CloudWatch Logs and OpenSearch Service.
dataSource (dict) --
This structure contains information about the OpenSearch Service data source used for this integration. This data source was created as part of the integration setup. An OpenSearch Service data source defines the source and destination for OpenSearch Service queries. It includes the role required to execute queries and write to collections.
For more information about OpenSearch Service data sources , see Creating OpenSearch Service data source integrations with Amazon S3.
dataSourceName (string) --
The name of the OpenSearch Service data source.
status (dict) --
This structure contains information about the status of this OpenSearch Service resource.
status (string) --
The current status of this resource.
statusMessage (string) --
A message with additional information about the status of this resource.
application (dict) --
This structure contains information about the OpenSearch Service application used for this integration. An OpenSearch Service application is the web application that was created by the integration with CloudWatch Logs. It hosts the vended logs dashboards.
applicationEndpoint (string) --
The endpoint of the application.
applicationArn (string) --
The Amazon Resource Name (ARN) of the application.
applicationId (string) --
The ID of the application.
status (dict) --
This structure contains information about the status of this OpenSearch Service resource.
status (string) --
The current status of this resource.
statusMessage (string) --
A message with additional information about the status of this resource.
collection (dict) --
This structure contains information about the OpenSearch Service collection used for this integration. This collection was created as part of the integration setup. An OpenSearch Service collection is a logical grouping of one or more indexes that represent an analytics workload. For more information, see Creating and managing OpenSearch Service Serverless collections.
collectionEndpoint (string) --
The endpoint of the collection.
collectionArn (string) --
The ARN of the collection.
status (dict) --
This structure contains information about the status of this OpenSearch Service resource.
status (string) --
The current status of this resource.
statusMessage (string) --
A message with additional information about the status of this resource.
workspace (dict) --
This structure contains information about the OpenSearch Service workspace used for this integration. An OpenSearch Service workspace is the collection of dashboards along with other OpenSearch Service tools. This workspace was created automatically as part of the integration setup. For more information, see Centralized OpenSearch user interface (Dashboards) with OpenSearch Service.
workspaceId (string) --
The ID of this workspace.
status (dict) --
This structure contains information about the status of an OpenSearch Service resource.
status (string) --
The current status of this resource.
statusMessage (string) --
A message with additional information about the status of this resource.
encryptionPolicy (dict) --
This structure contains information about the OpenSearch Service encryption policy used for this integration. The encryption policy was created automatically when you created the integration. For more information, see Encryption policies in the OpenSearch Service Developer Guide.
policyName (string) --
The name of the encryption policy.
status (dict) --
This structure contains information about the status of this OpenSearch Service resource.
status (string) --
The current status of this resource.
statusMessage (string) --
A message with additional information about the status of this resource.
networkPolicy (dict) --
This structure contains information about the OpenSearch Service network policy used for this integration. The network policy assigns network access settings to collections. For more information, see Network policies in the OpenSearch Service Developer Guide.
policyName (string) --
The name of the network policy.
status (dict) --
This structure contains information about the status of this OpenSearch Service resource.
status (string) --
The current status of this resource.
statusMessage (string) --
A message with additional information about the status of this resource.
accessPolicy (dict) --
This structure contains information about the OpenSearch Service data access policy used for this integration. The access policy defines the access controls for the collection. This data access policy was automatically created as part of the integration setup. For more information about OpenSearch Service data access policies, see Data access control for Amazon OpenSearch Serverless in the OpenSearch Service Developer Guide.
policyName (string) --
The name of the data access policy.
status (dict) --
This structure contains information about the status of this OpenSearch Service resource.
status (string) --
The current status of this resource.
statusMessage (string) --
A message with additional information about the status of this resource.
lifecyclePolicy (dict) --
This structure contains information about the OpenSearch Service data lifecycle policy used for this integration. The lifecycle policy determines the lifespan of the data in the collection. It was automatically created as part of the integration setup.
For more information, see Using data lifecycle policies with OpenSearch Service Serverless in the OpenSearch Service Developer Guide.
policyName (string) --
The name of the lifecycle policy.
status (dict) --
This structure contains information about the status of this OpenSearch Service resource.
status (string) --
The current status of this resource.
statusMessage (string) --
A message with additional information about the status of this resource.
{'queryLanguage': 'CWLI | SQL | PPL'}
Response {'queries': {'queryLanguage': 'CWLI | SQL | PPL'}}
Returns a list of CloudWatch Logs Insights queries that are scheduled, running, or have been run recently in this account. You can request all queries or limit it to queries of a specific log group or queries with a certain status.
See also: AWS API Documentation
Request Syntax
client.describe_queries(
logGroupName='string',
status='Scheduled'|'Running'|'Complete'|'Failed'|'Cancelled'|'Timeout'|'Unknown',
maxResults=123,
nextToken='string',
queryLanguage='CWLI'|'SQL'|'PPL'
)
string
Limits the returned queries to only those for the specified log group.
string
Limits the returned queries to only those that have the specified status. Valid values are Cancelled, Complete, Failed, Running, and Scheduled.
integer
Limits the number of returned queries to the specified number.
string
The token for the next set of items to return. The token expires after 24 hours.
string
Limits the returned queries to only the queries that use the specified query language.
dict
Response Syntax
{
'queries': [
{
'queryLanguage': 'CWLI'|'SQL'|'PPL',
'queryId': 'string',
'queryString': 'string',
'status': 'Scheduled'|'Running'|'Complete'|'Failed'|'Cancelled'|'Timeout'|'Unknown',
'createTime': 123,
'logGroupName': 'string'
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
queries (list) --
The list of queries that match the request.
(dict) --
Information about one CloudWatch Logs Insights query that matches the request in a DescribeQueries operation.
queryLanguage (string) --
The query language used for this query. For more information about the query languages that CloudWatch Logs supports, see Supported query languages.
queryId (string) --
The unique ID number of this query.
queryString (string) --
The query string used in this query.
status (string) --
The status of this query. Possible values are Cancelled, Complete, Failed, Running, Scheduled, and Unknown.
createTime (integer) --
The date and time that this query was created.
logGroupName (string) --
The name of the log group scanned by this query.
nextToken (string) --
The token for the next set of items to return. The token expires after 24 hours.
{'queryLanguage': 'CWLI | SQL | PPL'}
Response {'queryDefinitions': {'queryLanguage': 'CWLI | SQL | PPL'}}
This operation returns a paginated list of your saved CloudWatch Logs Insights query definitions. You can retrieve query definitions from the current account or from a source account that is linked to the current account.
You can use the queryDefinitionNamePrefix parameter to limit the results to only the query definitions that have names that start with a certain string.
See also: AWS API Documentation
Request Syntax
client.describe_query_definitions(
queryLanguage='CWLI'|'SQL'|'PPL',
queryDefinitionNamePrefix='string',
maxResults=123,
nextToken='string'
)
string
The query language used for this query. For more information about the query languages that CloudWatch Logs supports, see Supported query languages.
string
Use this parameter to filter your results to only the query definitions that have names that start with the prefix you specify.
integer
Limits the number of returned query definitions to the specified number.
string
The token for the next set of items to return. The token expires after 24 hours.
dict
Response Syntax
{
'queryDefinitions': [
{
'queryLanguage': 'CWLI'|'SQL'|'PPL',
'queryDefinitionId': 'string',
'name': 'string',
'queryString': 'string',
'lastModified': 123,
'logGroupNames': [
'string',
]
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
queryDefinitions (list) --
The list of query definitions that match your request.
(dict) --
This structure contains details about a saved CloudWatch Logs Insights query definition.
queryLanguage (string) --
The query language used for this query. For more information about the query languages that CloudWatch Logs supports, see Supported query languages.
queryDefinitionId (string) --
The unique ID of the query definition.
name (string) --
The name of the query definition.
queryString (string) --
The query string to use for this definition. For more information, see CloudWatch Logs Insights Query Syntax.
lastModified (integer) --
The date that the query definition was most recently modified.
logGroupNames (list) --
If this query definition contains a list of log groups that it is limited to, that list appears here.
(string) --
nextToken (string) --
The token for the next set of items to return. The token expires after 24 hours.
{'queryLanguage': 'CWLI | SQL | PPL'}
Returns the results from the specified query.
Only the fields requested in the query are returned, along with a @ptr field, which is the identifier for the log record. You can use the value of @ptr in a GetLogRecord operation to get the full log record.
GetQueryResults does not start running a query. To run a query, use StartQuery. For more information about how long results of previous queries are available, see CloudWatch Logs quotas.
If the value of the Status field in the output is Running, this operation returns only partial results. If you see a value of Scheduled or Running for the status, you can retry the operation later to see the final results.
If you are using CloudWatch cross-account observability, you can use this operation in a monitoring account to start queries in linked source accounts. For more information, see CloudWatch cross-account observability.
See also: AWS API Documentation
Request Syntax
client.get_query_results(
queryId='string'
)
string
[REQUIRED]
The ID number of the query.
dict
Response Syntax
{
'queryLanguage': 'CWLI'|'SQL'|'PPL',
'results': [
[
{
'field': 'string',
'value': 'string'
},
],
],
'statistics': {
'recordsMatched': 123.0,
'recordsScanned': 123.0,
'estimatedRecordsSkipped': 123.0,
'bytesScanned': 123.0,
'estimatedBytesSkipped': 123.0,
'logGroupsScanned': 123.0
},
'status': 'Scheduled'|'Running'|'Complete'|'Failed'|'Cancelled'|'Timeout'|'Unknown',
'encryptionKey': 'string'
}
Response Structure
(dict) --
queryLanguage (string) --
The query language used for this query. For more information about the query languages that CloudWatch Logs supports, see Supported query languages.
results (list) --
The log events that matched the query criteria during the most recent time it ran.
The results value is an array of arrays. Each log event is one object in the top-level array. Each of these log event objects is an array of field/ value pairs.
(list) --
(dict) --
Contains one field from one log event returned by a CloudWatch Logs Insights query, along with the value of that field.
For more information about the fields that are generated by CloudWatch logs, see Supported Logs and Discovered Fields.
field (string) --
The log event field.
value (string) --
The value of this field.
statistics (dict) --
Includes the number of log events scanned by the query, the number of log events that matched the query criteria, and the total number of bytes in the scanned log events. These values reflect the full raw results of the query.
recordsMatched (float) --
The number of log events that matched the query string.
recordsScanned (float) --
The total number of log events scanned during the query.
estimatedRecordsSkipped (float) --
An estimate of the number of log events that were skipped when processing this query, because the query contained an indexed field. Skipping these entries lowers query costs and improves the query performance time. For more information about field indexes, see PutIndexPolicy.
bytesScanned (float) --
The total number of bytes in the log events scanned during the query.
estimatedBytesSkipped (float) --
An estimate of the number of bytes in the log events that were skipped when processing this query, because the query contained an indexed field. Skipping these entries lowers query costs and improves the query performance time. For more information about field indexes, see PutIndexPolicy.
logGroupsScanned (float) --
The number of log groups that were scanned by this query.
status (string) --
The status of the most recent running of the query. Possible values are Cancelled, Complete, Failed, Running, Scheduled, Timeout, and Unknown.
Queries time out after 60 minutes of runtime. To avoid having your queries time out, reduce the time range being searched or partition your query into a number of queries.
encryptionKey (string) --
If you associated an KMS key with the CloudWatch Logs Insights query results in this account, this field displays the ARN of the key that's used to encrypt the query results when StartQuery stores them.
{'queryLanguage': 'CWLI | SQL | PPL'}
Creates or updates a query definition for CloudWatch Logs Insights. For more information, see Analyzing Log Data with CloudWatch Logs Insights.
To update a query definition, specify its queryDefinitionId in your request. The values of name, queryString, and logGroupNames are changed to the values that you specify in your update operation. No current values are retained from the current query definition. For example, imagine updating a current query definition that includes log groups. If you don't specify the logGroupNames parameter in your update operation, the query definition changes to contain no log groups.
You must have the logs:PutQueryDefinition permission to be able to perform this operation.
See also: AWS API Documentation
Request Syntax
client.put_query_definition(
queryLanguage='CWLI'|'SQL'|'PPL',
name='string',
queryDefinitionId='string',
logGroupNames=[
'string',
],
queryString='string',
clientToken='string'
)
string
Specify the query language to use for this query. The options are Logs Insights QL, OpenSearch PPL, and OpenSearch SQL. For more information about the query languages that CloudWatch Logs supports, see Supported query languages.
string
[REQUIRED]
A name for the query definition. If you are saving numerous query definitions, we recommend that you name them. This way, you can find the ones you want by using the first part of the name as a filter in the queryDefinitionNamePrefix parameter of DescribeQueryDefinitions.
string
If you are updating a query definition, use this parameter to specify the ID of the query definition that you want to update. You can use DescribeQueryDefinitions to retrieve the IDs of your saved query definitions.
If you are creating a query definition, do not specify this parameter. CloudWatch generates a unique ID for the new query definition and include it in the response to this operation.
list
Use this parameter to include specific log groups as part of your query definition. If your query uses the OpenSearch Service query language, you specify the log group names inside the querystring instead of here.
If you are updating an existing query definition for the Logs Insights QL or OpenSearch Service PPL and you omit this parameter, then the updated definition will contain no log groups.
(string) --
string
[REQUIRED]
The query string to use for this definition. For more information, see CloudWatch Logs Insights Query Syntax.
string
Used as an idempotency token, to avoid returning an exception if the service receives the same request twice because of a network error.
This field is autopopulated if not provided.
dict
Response Syntax
{
'queryDefinitionId': 'string'
}
Response Structure
(dict) --
queryDefinitionId (string) --
The ID of the query definition.
{'queryLanguage': 'CWLI | SQL | PPL'}
Starts a query of one or more log groups using CloudWatch Logs Insights. You specify the log groups and time range to query and the query string to use.
For more information, see CloudWatch Logs Insights Query Syntax.
After you run a query using StartQuery, the query results are stored by CloudWatch Logs. You can use GetQueryResults to retrieve the results of a query, using the queryId that StartQuery returns.
If you have associated a KMS key with the query results in this account, then StartQuery uses that key to encrypt the results when it stores them. If no key is associated with query results, the query results are encrypted with the default CloudWatch Logs encryption method.
Queries time out after 60 minutes of runtime. If your queries are timing out, reduce the time range being searched or partition your query into a number of queries.
If you are using CloudWatch cross-account observability, you can use this operation in a monitoring account to start a query in a linked source account. For more information, see CloudWatch cross-account observability. For a cross-account StartQuery operation, the query definition must be defined in the monitoring account.
You can have up to 30 concurrent CloudWatch Logs insights queries, including queries that have been added to dashboards.
See also: AWS API Documentation
Request Syntax
client.start_query(
queryLanguage='CWLI'|'SQL'|'PPL',
logGroupName='string',
logGroupNames=[
'string',
],
logGroupIdentifiers=[
'string',
],
startTime=123,
endTime=123,
queryString='string',
limit=123
)
string
Specify the query language to use for this query. The options are Logs Insights QL, OpenSearch PPL, and OpenSearch SQL. For more information about the query languages that CloudWatch Logs supports, see Supported query languages.
string
The log group on which to perform the query.
list
The list of log groups to be queried. You can include up to 50 log groups.
(string) --
list
The list of log groups to query. You can include up to 50 log groups.
You can specify them by the log group name or ARN. If a log group that you're querying is in a source account and you're using a monitoring account, you must specify the ARN of the log group here. The query definition must also be defined in the monitoring account.
If you specify an ARN, use the format arn:aws:logs:region:account-id:log-group:log_group_name Don't include an * at the end.
A StartQuery operation must include exactly one of the following parameters: logGroupName, logGroupNames, or logGroupIdentifiers. The exception is queries using the OpenSearch Service SQL query language, where you specify the log group names inside the querystring instead of here.
(string) --
integer
[REQUIRED]
The beginning of the time range to query. The range is inclusive, so the specified start time is included in the query. Specified as epoch time, the number of seconds since January 1, 1970, 00:00:00 UTC.
integer
[REQUIRED]
The end of the time range to query. The range is inclusive, so the specified end time is included in the query. Specified as epoch time, the number of seconds since January 1, 1970, 00:00:00 UTC.
string
[REQUIRED]
The query string to use. For more information, see CloudWatch Logs Insights Query Syntax.
integer
The maximum number of log events to return in the query. If the query string uses the fields command, only the specified fields and their values are returned. The default is 10,000.
dict
Response Syntax
{
'queryId': 'string'
}
Response Structure
(dict) --
queryId (string) --
The unique ID of the query.