2023/11/27 - Amazon CloudWatch Logs - 7 new 2 updated api methods
Changes Added APIs to Create, Update, Get, List and Delete LogAnomalyDetectors and List and Update Anomalies in Detector. Added LogGroupClass attribute for LogGroups to classify loggroup as Standard loggroup with all capabilities or InfrequentAccess loggroup with limited capabilities.
Creates an anomaly detector that regularly scans one or more log groups and look for patterns and anomalies in the logs.
An anomaly detector can help surface issues by automatically discovering anomalies in your log event traffic. An anomaly detector uses machine learning algorithms to scan log events and find patterns . A pattern is a shared text structure that recurs among your log fields. Patterns provide a useful tool for analyzing large sets of logs because a large number of log events can often be compressed into a few patterns.
The anomaly detector uses pattern recognition to find anomalies , which are unusual log events. It uses the evaluationFrequency to compare current log events and patterns with trained baselines.
Fields within a pattern are called tokens . Fields that vary within a pattern, such as a request ID or timestamp, are referred to as dynamic tokens and represented by <*> .
The following is an example of a pattern:
[INFO] Request time: <*> ms
This pattern represents log events like [INFO] Request time: 327 ms and other similar log events that differ only by the number, in this csse 327. When the pattern is displayed, the different numbers are replaced by <*>
Note
Any parts of log events that are masked as sensitive data are not scanned for anomalies. For more information about masking sensitive data, see Help protect sensitive log data with masking.
See also: AWS API Documentation
Request Syntax
client.create_log_anomaly_detector(
logGroupArnList=[
'string',
],
detectorName='string',
evaluationFrequency='ONE_MIN'|'FIVE_MIN'|'TEN_MIN'|'FIFTEEN_MIN'|'THIRTY_MIN'|'ONE_HOUR',
filterPattern='string',
kmsKeyId='string',
anomalyVisibilityTime=123,
tags={
'string': 'string'
}
)
list
[REQUIRED]
An array containing the ARNs of the log groups that this anomaly detector will watch. You must specify at least one ARN.
(string) --
string
A name for this anomaly detector.
string
Specifies how often the anomaly detector is to run and look for anomalies. Set this value according to the frequency that the log group receives new logs. For example, if the log group receives new log events every 10 minutes, then 15 minutes might be a good setting for evaluationFrequency .
string
You can use this parameter to limit the anomaly detection model to examine only log events that match the pattern you specify here. For more information, see Filter and Pattern Syntax.
string
Optionally assigns a KMS key to secure this anomaly detector and its findings. If a key is assigned, the anomalies found and the model used by this detector are encrypted at rest with the key. If a key is assigned to an anomaly detector, a user must have permissions for both this key and for the anomaly detector to retrieve information about the anomalies that it finds.
For more information about using a KMS key and to see the required IAM policy, see Use a KMS key with an anomaly detector.
integer
The number of days to have visibility on an anomaly. After this time period has elapsed for an anomaly, it will be automatically baselined and the anomaly detector will treat new occurrences of a similar anomaly as normal. Therefore, if you do not correct the cause of an anomaly during the time period specified in anomalyVisibilityTime , it will be considered normal going forward and will not be detected as an anomaly.
dict
An optional list of key-value pairs to associate with the resource.
For more information about tagging, see Tagging Amazon Web Services resources
(string) --
(string) --
dict
Response Syntax
{
'anomalyDetectorArn': 'string'
}
Response Structure
(dict) --
anomalyDetectorArn (string) --
The ARN of the log anomaly detector that you just created.
Returns a list of anomalies that log anomaly detectors have found. For details about the structure format of each anomaly object that is returned, see the example in this section.
See also: AWS API Documentation
Request Syntax
client.list_anomalies(
anomalyDetectorArn='string',
suppressionState='SUPPRESSED'|'UNSUPPRESSED',
limit=123,
nextToken='string'
)
string
Use this to optionally limit the results to only the anomalies found by a certain anomaly detector.
string
You can specify this parameter if you want to the operation to return only anomalies that are currently either suppressed or unsuppressed.
integer
The maximum number of items to return. If you don't specify a value, the default maximum value of 50 items is used.
string
The token for the next set of items to return. The token expires after 24 hours.
dict
Response Syntax
{
'anomalies': [
{
'anomalyId': 'string',
'patternId': 'string',
'anomalyDetectorArn': 'string',
'patternString': 'string',
'patternRegex': 'string',
'priority': 'string',
'firstSeen': 123,
'lastSeen': 123,
'description': 'string',
'active': True|False,
'state': 'Active'|'Suppressed'|'Baseline',
'histogram': {
'string': 123
},
'logSamples': [
'string',
],
'patternTokens': [
{
'dynamicTokenPosition': 123,
'isDynamic': True|False,
'tokenString': 'string',
'enumerations': {
'string': 123
}
},
],
'logGroupArnList': [
'string',
],
'suppressed': True|False,
'suppressedDate': 123,
'suppressedUntil': 123,
'isPatternLevelSuppression': True|False
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
anomalies (list) --
An array of structures, where each structure contains information about one anomaly that a log anomaly detector has found.
(dict) --
This structure represents one anomaly that has been found by a logs anomaly detector.
For more information about patterns and anomalies, see CreateLogAnomalyDetector.
anomalyId (string) --
The unique ID that CloudWatch Logs assigned to this anomaly.
patternId (string) --
The ID of the pattern used to help identify this anomaly.
anomalyDetectorArn (string) --
The ARN of the anomaly detector that identified this anomaly.
patternString (string) --
The pattern used to help identify this anomaly, in string format.
patternRegex (string) --
The pattern used to help identify this anomaly, in regular expression format.
priority (string) --
The priority level of this anomaly, as determined by CloudWatch Logs. Priority is computed based on log severity labels such as FATAL and ERROR and the amount of deviation from the baseline. Possible values are HIGH , MEDIUM , and LOW .
firstSeen (integer) --
The date and time when the anomaly detector first saw this anomaly. It is specified as epoch time, which is the number of seconds since January 1, 1970, 00:00:00 UTC .
lastSeen (integer) --
The date and time when the anomaly detector most recently saw this anomaly. It is specified as epoch time, which is the number of seconds since January 1, 1970, 00:00:00 UTC .
description (string) --
A human-readable description of the anomaly. This description is generated by CloudWatch Logs.
active (boolean) --
Specifies whether this anomaly is still ongoing.
state (string) --
Indicates the current state of this anomaly. If it is still being treated as an anomaly, the value is Active . If you have suppressed this anomaly by using the UpdateAnomaly operation, the value is Suppressed . If this behavior is now considered to be normal, the value is Baseline .
histogram (dict) --
A map showing times when the anomaly detector ran, and the number of occurrences of this anomaly that were detected at each of those runs. The times are specified in epoch time, which is the number of seconds since January 1, 1970, 00:00:00 UTC .
(string) --
(integer) --
logSamples (list) --
An array of sample log event messages that are considered to be part of this anomaly.
(string) --
patternTokens (list) --
An array of structures where each structure contains information about one token that makes up the pattern.
(dict) --
A tructures that contains information about one pattern token related to an anomaly.
For more information about patterns and tokens, see CreateLogAnomalyDetector.
dynamicTokenPosition (integer) --
For a dynamic token, this indicates where in the pattern that this token appears, related to other dynamic tokens. The dynamic token that appears first has a value of 1 , the one that appears second is 2 , and so on.
isDynamic (boolean) --
Specifies whether this is a dynamic token.
tokenString (string) --
The string represented by this token. If this is a dynamic token, the value will be <*>
enumerations (dict) --
Contains the values found for a dynamic token, and the number of times each value was found.
(string) --
(integer) --
logGroupArnList (list) --
An array of ARNS of the log groups that contained log events considered to be part of this anomaly.
(string) --
suppressed (boolean) --
Indicates whether this anomaly is currently suppressed. To suppress an anomaly, use UpdateAnomaly.
suppressedDate (integer) --
If the anomaly is suppressed, this indicates when it was suppressed.
suppressedUntil (integer) --
If the anomaly is suppressed, this indicates when the suppression will end. If this value is 0 , the anomaly was suppressed with no expiration, with the INFINITE value.
isPatternLevelSuppression (boolean) --
If this anomaly is suppressed, this field is true if the suppression is because the pattern is suppressed. If false , then only this particular anomaly is suppressed.
nextToken (string) --
The token for the next set of items to return. The token expires after 24 hours.
Deletes the specified CloudWatch Logs anomaly detector.
See also: AWS API Documentation
Request Syntax
client.delete_log_anomaly_detector(
anomalyDetectorArn='string'
)
string
[REQUIRED]
The ARN of the anomaly detector to delete. You can find the ARNs of log anomaly detectors in your account by using the ListLogAnomalyDetectors operation.
None
Retrieves information about the log anomaly detector that you specify.
See also: AWS API Documentation
Request Syntax
client.get_log_anomaly_detector(
anomalyDetectorArn='string'
)
string
[REQUIRED]
The ARN of the anomaly detector to retrieve information about. You can find the ARNs of log anomaly detectors in your account by using the ListLogAnomalyDetectors operation.
dict
Response Syntax
{
'detectorName': 'string',
'logGroupArnList': [
'string',
],
'evaluationFrequency': 'ONE_MIN'|'FIVE_MIN'|'TEN_MIN'|'FIFTEEN_MIN'|'THIRTY_MIN'|'ONE_HOUR',
'filterPattern': 'string',
'anomalyDetectorStatus': 'INITIALIZING'|'TRAINING'|'ANALYZING'|'FAILED'|'DELETED'|'PAUSED',
'kmsKeyId': 'string',
'creationTimeStamp': 123,
'lastModifiedTimeStamp': 123,
'anomalyVisibilityTime': 123
}
Response Structure
(dict) --
detectorName (string) --
The name of the log anomaly detector
logGroupArnList (list) --
An array of structures, where each structure contains the ARN of a log group associated with this anomaly detector.
(string) --
evaluationFrequency (string) --
Specifies how often the anomaly detector runs and look for anomalies. Set this value according to the frequency that the log group receives new logs. For example, if the log group receives new log events every 10 minutes, then setting evaluationFrequency to FIFTEEN_MIN might be appropriate.
filterPattern (string) --
A symbolic description of how CloudWatch Logs should interpret the data in each log event. For example, a log event can contain timestamps, IP addresses, strings, and so on. You use the filter pattern to specify what to look for in the log event message.
anomalyDetectorStatus (string) --
Specifies whether the anomaly detector is currently active. To change its status, use the enabled parameter in the UpdateLogAnomalyDetector operation.
kmsKeyId (string) --
The ID of the KMS key assigned to this anomaly detector, if any.
creationTimeStamp (integer) --
The date and time when this anomaly detector was created.
lastModifiedTimeStamp (integer) --
The date and time when this anomaly detector was most recently modified.
anomalyVisibilityTime (integer) --
The number of days used as the life cycle of anomalies. After this time, anomalies are automatically baselined and the anomaly detector model will treat new occurrences of similar event as normal.
Retrieves a list of the log anomaly detectors in the account.
See also: AWS API Documentation
Request Syntax
client.list_log_anomaly_detectors(
filterLogGroupArn='string',
limit=123,
nextToken='string'
)
string
Use this to optionally filter the results to only include anomaly detectors that are associated with the specified log group.
integer
The maximum number of items to return. If you don't specify a value, the default maximum value of 50 items is used.
string
The token for the next set of items to return. The token expires after 24 hours.
dict
Response Syntax
{
'anomalyDetectors': [
{
'anomalyDetectorArn': 'string',
'detectorName': 'string',
'logGroupArnList': [
'string',
],
'evaluationFrequency': 'ONE_MIN'|'FIVE_MIN'|'TEN_MIN'|'FIFTEEN_MIN'|'THIRTY_MIN'|'ONE_HOUR',
'filterPattern': 'string',
'anomalyDetectorStatus': 'INITIALIZING'|'TRAINING'|'ANALYZING'|'FAILED'|'DELETED'|'PAUSED',
'kmsKeyId': 'string',
'creationTimeStamp': 123,
'lastModifiedTimeStamp': 123,
'anomalyVisibilityTime': 123
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
anomalyDetectors (list) --
An array of structures, where each structure in the array contains information about one anomaly detector.
(dict) --
Contains information about one anomaly detector in the account.
anomalyDetectorArn (string) --
The ARN of the anomaly detector.
detectorName (string) --
The name of the anomaly detector.
logGroupArnList (list) --
A list of the ARNs of the log groups that this anomaly detector watches.
(string) --
evaluationFrequency (string) --
Specifies how often the anomaly detector runs and look for anomalies.
filterPattern (string) --
A symbolic description of how CloudWatch Logs should interpret the data in each log event. For example, a log event can contain timestamps, IP addresses, strings, and so on. You use the filter pattern to specify what to look for in the log event message.
anomalyDetectorStatus (string) --
Specifies the current status of the anomaly detector. To pause an anomaly detector, use the enabled parameter in the UpdateLogAnomalyDetector operation.
kmsKeyId (string) --
The ID of the KMS key assigned to this anomaly detector, if any.
creationTimeStamp (integer) --
The date and time when this anomaly detector was created.
lastModifiedTimeStamp (integer) --
The date and time when this anomaly detector was most recently modified.
anomalyVisibilityTime (integer) --
The number of days used as the life cycle of anomalies. After this time, anomalies are automatically baselined and the anomaly detector model will treat new occurrences of similar event as normal.
nextToken (string) --
The token for the next set of items to return. The token expires after 24 hours.
Updates an existing log anomaly detector.
See also: AWS API Documentation
Request Syntax
client.update_log_anomaly_detector(
anomalyDetectorArn='string',
evaluationFrequency='ONE_MIN'|'FIVE_MIN'|'TEN_MIN'|'FIFTEEN_MIN'|'THIRTY_MIN'|'ONE_HOUR',
filterPattern='string',
anomalyVisibilityTime=123,
enabled=True|False
)
string
[REQUIRED]
The ARN of the anomaly detector that you want to update.
string
Specifies how often the anomaly detector runs and look for anomalies. Set this value according to the frequency that the log group receives new logs. For example, if the log group receives new log events every 10 minutes, then setting evaluationFrequency to FIFTEEN_MIN might be appropriate.
string
A symbolic description of how CloudWatch Logs should interpret the data in each log event. For example, a log event can contain timestamps, IP addresses, strings, and so on. You use the filter pattern to specify what to look for in the log event message.
integer
The number of days to use as the life cycle of anomalies. After this time, anomalies are automatically baselined and the anomaly detector model will treat new occurrences of similar event as normal. Therefore, if you do not correct the cause of an anomaly during this time, it will be considered normal going forward and will not be detected.
boolean
[REQUIRED]
Use this parameter to pause or restart the anomaly detector.
None
Use this operation to suppress anomaly detection for a specified anomaly or pattern. If you suppress an anomaly, CloudWatch Logs won’t report new occurrences of that anomaly and won't update that anomaly with new data. If you suppress a pattern, CloudWatch Logs won’t report any anomalies related to that pattern.
You must specify either anomalyId or patternId , but you can't specify both parameters in the same operation.
If you have previously used this operation to suppress detection of a pattern or anomaly, you can use it again to cause CloudWatch Logs to end the suppression. To do this, use this operation and specify the anomaly or pattern to stop suppressing, and omit the suppressionType and suppressionPeriod parameters.
See also: AWS API Documentation
Request Syntax
client.update_anomaly(
anomalyId='string',
patternId='string',
anomalyDetectorArn='string',
suppressionType='LIMITED'|'INFINITE',
suppressionPeriod={
'value': 123,
'suppressionUnit': 'SECONDS'|'MINUTES'|'HOURS'
}
)
string
If you are suppressing or unsuppressing an anomaly, specify its unique ID here. You can find anomaly IDs by using the ListAnomalies operation.
string
If you are suppressing or unsuppressing an pattern, specify its unique ID here. You can find pattern IDs by using the ListAnomalies operation.
string
[REQUIRED]
The ARN of the anomaly detector that this operation is to act on.
string
Use this to specify whether the suppression to be temporary or infinite. If you specify LIMITED , you must also specify a suppressionPeriod . If you specify INFINITE , any value for suppressionPeriod is ignored.
dict
If you are temporarily suppressing an anomaly or pattern, use this structure to specify how long the suppression is to last.
value (integer) --
Specifies the number of seconds, minutes or hours to suppress this anomaly. There is no maximum.
suppressionUnit (string) --
Specifies whether the value of value is in seconds, minutes, or hours.
None
{'logGroupClass': 'STANDARD | INFREQUENT_ACCESS'}
Creates a log group with the specified name. You can create up to 1,000,000 log groups per Region per account.
You must use the following guidelines when naming a log group:
Log group names must be unique within a Region for an Amazon Web Services account.
Log group names can be between 1 and 512 characters long.
Log group names consist of the following characters: a-z, A-Z, 0-9, '_' (underscore), '-' (hyphen), '/' (forward slash), '.' (period), and '#' (number sign)
When you create a log group, by default the log events in the log group do not expire. To set a retention policy so that events expire and are deleted after a specified time, use PutRetentionPolicy.
If you associate an KMS key with the log group, ingested data is encrypted using the KMS key. This association is stored as long as the data encrypted with the KMS key is still within CloudWatch Logs. This enables CloudWatch Logs to decrypt this data whenever it is requested.
If you attempt to associate a KMS key with the log group but the KMS key does not exist or the KMS key is disabled, you receive an InvalidParameterException error.
Warning
CloudWatch Logs supports only symmetric KMS keys. Do not associate an asymmetric KMS key with your log group. For more information, see Using Symmetric and Asymmetric Keys.
See also: AWS API Documentation
Request Syntax
client.create_log_group(
logGroupName='string',
kmsKeyId='string',
tags={
'string': 'string'
},
logGroupClass='STANDARD'|'INFREQUENT_ACCESS'
)
string
[REQUIRED]
A name for the log group.
string
The Amazon Resource Name (ARN) of the KMS key to use when encrypting log data. For more information, see Amazon Resource Names.
dict
The key-value pairs to use for the tags.
You can grant users access to certain log groups while preventing them from accessing other log groups. To do so, tag your groups and use IAM policies that refer to those tags. To assign tags when you create a log group, you must have either the logs:TagResource or logs:TagLogGroup permission. For more information about tagging, see Tagging Amazon Web Services resources. For more information about using tags to control access, see Controlling access to Amazon Web Services resources using tags.
(string) --
(string) --
string
Use this parameter to specify the log group class for this log group. There are two classes:
The Standard log class supports all CloudWatch Logs features.
The Infrequent Access log class supports a subset of CloudWatch Logs features and incurs lower costs.
If you omit this parameter, the default of STANDARD is used.
For details about the features supported by each class, see Log classes
None
{'logGroupClass': 'STANDARD | INFREQUENT_ACCESS'}
Response {'logGroups': {'logGroupClass': 'STANDARD | INFREQUENT_ACCESS'}}
Lists the specified log groups. You can list all your log groups or filter the results by prefix. The results are ASCII-sorted by log group name.
CloudWatch Logs doesn’t support IAM policies that control access to the DescribeLogGroups action by using the aws:ResourceTag/key-name condition key. Other CloudWatch Logs actions do support the use of the aws:ResourceTag/key-name condition key to control access. For more information about using tags to control access, see Controlling access to Amazon Web Services resources using tags.
If you are using CloudWatch cross-account observability, you can use this operation in a monitoring account and view data from the linked source accounts. For more information, see CloudWatch cross-account observability.
See also: AWS API Documentation
Request Syntax
client.describe_log_groups(
accountIdentifiers=[
'string',
],
logGroupNamePrefix='string',
logGroupNamePattern='string',
nextToken='string',
limit=123,
includeLinkedAccounts=True|False,
logGroupClass='STANDARD'|'INFREQUENT_ACCESS'
)
list
When includeLinkedAccounts is set to True , use this parameter to specify the list of accounts to search. You can specify as many as 20 account IDs in the array.
(string) --
string
The prefix to match.
Note
logGroupNamePrefix and logGroupNamePattern are mutually exclusive. Only one of these parameters can be passed.
string
If you specify a string for this parameter, the operation returns only log groups that have names that match the string based on a case-sensitive substring search. For example, if you specify Foo , log groups named FooBar , aws/Foo , and GroupFoo would match, but foo , F/o/o and Froo would not match.
If you specify logGroupNamePattern in your request, then only arn , creationTime , and logGroupName are included in the response.
Note
logGroupNamePattern and logGroupNamePrefix are mutually exclusive. Only one of these parameters can be passed.
string
The token for the next set of items to return. (You received this token from a previous call.)
integer
The maximum number of items returned. If you don't specify a value, the default is up to 50 items.
boolean
If you are using a monitoring account, set this to True to have the operation return log groups in the accounts listed in accountIdentifiers .
If this parameter is set to true and accountIdentifiers contains a null value, the operation returns all log groups in the monitoring account and all log groups in all source accounts that are linked to the monitoring account.
string
Specifies the log group class for this log group. There are two classes:
The Standard log class supports all CloudWatch Logs features.
The Infrequent Access log class supports a subset of CloudWatch Logs features and incurs lower costs.
For details about the features supported by each class, see Log classes
dict
Response Syntax
{
'logGroups': [
{
'logGroupName': 'string',
'creationTime': 123,
'retentionInDays': 123,
'metricFilterCount': 123,
'arn': 'string',
'storedBytes': 123,
'kmsKeyId': 'string',
'dataProtectionStatus': 'ACTIVATED'|'DELETED'|'ARCHIVED'|'DISABLED',
'inheritedProperties': [
'ACCOUNT_DATA_PROTECTION',
],
'logGroupClass': 'STANDARD'|'INFREQUENT_ACCESS'
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
logGroups (list) --
The log groups.
If the retentionInDays value is not included for a log group, then that log group's events do not expire.
(dict) --
Represents a log group.
logGroupName (string) --
The name of the log group.
creationTime (integer) --
The creation time of the log group, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.
retentionInDays (integer) --
The number of days to retain the log events in the specified log group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, and 3653.
To set a log group so that its log events do not expire, use DeleteRetentionPolicy.
metricFilterCount (integer) --
The number of metric filters.
arn (string) --
The Amazon Resource Name (ARN) of the log group.
storedBytes (integer) --
The number of bytes stored.
kmsKeyId (string) --
The Amazon Resource Name (ARN) of the KMS key to use when encrypting log data.
dataProtectionStatus (string) --
Displays whether this log group has a protection policy, or whether it had one in the past. For more information, see PutDataProtectionPolicy.
inheritedProperties (list) --
Displays all the properties that this log group has inherited from account-level settings.
(string) --
logGroupClass (string) --
This specifies the log group class for this log group. There are two classes:
The Standard log class supports all CloudWatch Logs features.
The Infrequent Access log class supports a subset of CloudWatch Logs features and incurs lower costs.
For details about the features supported by each class, see Log classes
nextToken (string) --
The token for the next set of items to return. The token expires after 24 hours.