Amazon CloudWatch Logs

Starts an import from a data source to CloudWatch Log and creates a managed log group as the destination for the imported data. Currently, CloudTrail Event Data Store is the only supported data source.

The import task must satisfy the following constraints:

• The specified source must be in an ACTIVE state.

• The API caller must have permissions to access the data in the provided source and to perform iam:PassRole on the provided import role which has the same permissions, as described below.

• The provided IAM role must trust the "cloudtrail.amazonaws.com" principal and have the following permissions:

  • cloudtrail:GetEventDataStoreData

  • logs:CreateLogGroup

  • logs:CreateLogStream

  • logs:PutResourcePolicy

  • (If source has an associated AWS KMS Key) kms:Decrypt

  • (If source has an associated AWS KMS Key) kms:GenerateDataKey

Example IAM policy for provided import role:

[ { "Effect": "Allow", "Action": "iam:PassRole", "Resource": "arn:aws:iam::123456789012:role/apiCallerCredentials", "Condition": { "StringLike": { "iam:AssociatedResourceARN": "arn:aws:logs:us-east-1:123456789012:log-group:aws/cloudtrail/f1d45bff-d0e3-4868-b5d9-2eb678aa32fb:*" } } }, { "Effect": "Allow", "Action": [ "cloudtrail:GetEventDataStoreData" ], "Resource": [ "arn:aws:cloudtrail:us-east-1:123456789012:eventdatastore/f1d45bff-d0e3-4868-b5d9-2eb678aa32fb" ] }, { "Effect": "Allow", "Action": [ "logs:CreateImportTask", "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutResourcePolicy" ], "Resource": [ "arn:aws:logs:us-east-1:123456789012:log-group:/aws/cloudtrail/*" ] }, { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": [ "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012" ] } ]

• If the import source has a customer managed key, the "cloudtrail.amazonaws.com" principal needs permissions to perform kms:Decrypt and kms:GenerateDataKey.

• There can be no more than 3 active imports per account at a given time.

• The startEventTime must be less than or equal to endEventTime.

• The data being imported must be within the specified source's retention period.

See also: AWS API Documentation

Request Syntax

client.create_import_task(
    importSourceArn='string',
    importRoleArn='string',
    importFilter={
        'startEventTime': 123,
        'endEventTime': 123
    }
)

type importSourceArn:

string

param importSourceArn:

[REQUIRED]

The ARN of the source to import from.

type importRoleArn:

string

param importRoleArn:

[REQUIRED]

The ARN of the IAM role that grants CloudWatch Logs permission to import from the CloudTrail Lake Event Data Store.

type importFilter:

dict

param importFilter:

Optional filters to constrain the import by CloudTrail event time. Times are specified in Unix timestamp milliseconds. The range of data being imported must be within the specified source's retention period.

• startEventTime (integer) --

  The start of the time range for events to import, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

• endEventTime (integer) --

  The end of the time range for events to import, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

rtype:

dict

returns:

Response Syntax

{
    'importId': 'string',
    'importDestinationArn': 'string',
    'creationTime': 123
}

Response Structure

• (dict) --

  • importId (string) --

    A unique identifier for the import task.

  • importDestinationArn (string) --

    The ARN of the CloudWatch Logs log group created as the destination for the imported events.

  • creationTime (integer) --

    The timestamp when the import task was created, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

Cancels an active import task and stops importing data from the CloudTrail Lake Event Data Store.

See also: AWS API Documentation

Request Syntax

client.cancel_import_task(
    importId='string'
)

type importId:

string

param importId:

[REQUIRED]

The ID of the import task to cancel.

rtype:

dict

returns:

Response Syntax

{
    'importId': 'string',
    'importStatistics': {
        'bytesImported': 123
    },
    'importStatus': 'IN_PROGRESS'|'CANCELLED'|'COMPLETED'|'FAILED',
    'creationTime': 123,
    'lastUpdatedTime': 123
}

Response Structure

• (dict) --

  • importId (string) --

    The ID of the cancelled import task.

  • importStatistics (dict) --

    Statistics about the import progress at the time of cancellation.

    • bytesImported (integer) --

      The total number of bytes that have been imported to the managed log group.

  • importStatus (string) --

    The final status of the import task. This will be set to CANCELLED.

  • creationTime (integer) --

    The timestamp when the import task was created, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

  • lastUpdatedTime (integer) --

    The timestamp when the import task was cancelled, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

Lists and describes import tasks, with optional filtering by import status and source ARN.

See also: AWS API Documentation

Request Syntax

client.describe_import_tasks(
    importId='string',
    importStatus='IN_PROGRESS'|'CANCELLED'|'COMPLETED'|'FAILED',
    importSourceArn='string',
    limit=123,
    nextToken='string'
)

type importId:

string

param importId:

Optional filter to describe a specific import task by its ID.

type importStatus:

string

param importStatus:

Optional filter to list imports by their status. Valid values are IN_PROGRESS, CANCELLED, COMPLETED and FAILED.

type importSourceArn:

string

param importSourceArn:

Optional filter to list imports from a specific source

type limit:

integer

param limit:

The maximum number of import tasks to return in the response. Default: 50

type nextToken:

string

param nextToken:

The pagination token for the next set of results.

rtype:

dict

returns:

Response Syntax

{
    'imports': [
        {
            'importId': 'string',
            'importSourceArn': 'string',
            'importStatus': 'IN_PROGRESS'|'CANCELLED'|'COMPLETED'|'FAILED',
            'importDestinationArn': 'string',
            'importStatistics': {
                'bytesImported': 123
            },
            'importFilter': {
                'startEventTime': 123,
                'endEventTime': 123
            },
            'creationTime': 123,
            'lastUpdatedTime': 123,
            'errorMessage': 'string'
        },
    ],
    'nextToken': 'string'
}

Response Structure

• (dict) --

  • imports (list) --

    The list of import tasks that match the request filters.

    • (dict) --

      An import job to move data from CloudTrail Event Data Store to CloudWatch.

      • importId (string) --

        The unique identifier of the import task.

      • importSourceArn (string) --

        The ARN of the CloudTrail Lake Event Data Store being imported from.

      • importStatus (string) --

        The current status of the import task. Valid values are IN_PROGRESS, CANCELLED, COMPLETED and FAILED.

      • importDestinationArn (string) --

        The ARN of the managed CloudWatch Logs log group where the events are being imported to.

      • importStatistics (dict) --

        Statistics about the import progress

        • bytesImported (integer) --

          The total number of bytes that have been imported to the managed log group.

      • importFilter (dict) --

        The filter criteria used for this import task.

        • startEventTime (integer) --

          The start of the time range for events to import, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

        • endEventTime (integer) --

          The end of the time range for events to import, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

      • creationTime (integer) --

        The timestamp when the import task was created, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

      • lastUpdatedTime (integer) --

        The timestamp when the import task was last updated, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

      • errorMessage (string) --

        Error message related to any failed imports

  • nextToken (string) --

    The token to use when requesting the next set of results. Not present if there are no additional results to retrieve.

Gets detailed information about the individual batches within an import task, including their status and any error messages. For CloudTrail Event Data Store sources, a batch refers to a subset of stored events grouped by their eventTime.

See also: AWS API Documentation

Request Syntax

client.describe_import_task_batches(
    importId='string',
    batchImportStatus=[
        'IN_PROGRESS'|'CANCELLED'|'COMPLETED'|'FAILED',
    ],
    limit=123,
    nextToken='string'
)

type importId:

string

param importId:

[REQUIRED]

The ID of the import task to get batch information for.

type batchImportStatus:

list

param batchImportStatus:

Optional filter to list import batches by their status. Accepts multiple status values: IN_PROGRESS, CANCELLED, COMPLETED and FAILED.

• (string) --

type limit:

integer

param limit:

The maximum number of import batches to return in the response. Default: 10

type nextToken:

string

param nextToken:

The pagination token for the next set of results.

rtype:

dict

returns:

Response Syntax

{
    'importSourceArn': 'string',
    'importId': 'string',
    'importBatches': [
        {
            'batchId': 'string',
            'status': 'IN_PROGRESS'|'CANCELLED'|'COMPLETED'|'FAILED',
            'errorMessage': 'string'
        },
    ],
    'nextToken': 'string'
}

Response Structure

• (dict) --

  • importSourceArn (string) --

    The ARN of the source being imported from.

  • importId (string) --

    The ID of the import task.

  • importBatches (list) --

    The list of import batches that match the request filters.

    • (dict) --

      A collection of events being imported to CloudWatch

      • batchId (string) --

        The unique identifier of the import batch.

      • status (string) --

        The current status of the import batch. Valid values are IN_PROGRESS, CANCELLED, COMPLETED and FAILED.

      • errorMessage (string) --

        The error message if the batch failed to import. Only present when status is FAILED.

  • nextToken (string) --

    The token to use when requesting the next set of results. Not present if there are no additional results to retrieve.