2021/10/13 - Amazon Elastic Compute Cloud - 2 updated api methods
Changes This release adds support for additional VPC Flow Logs delivery options to S3, such as Apache Parquet formatted files, Hourly partitions and Hive-compatible S3 prefixes
{'DestinationOptions': {'FileFormat': 'plain-text | parquet',
'HiveCompatiblePartitions': 'boolean',
'PerHourPartition': 'boolean'}}
Creates one or more flow logs to capture information about IP traffic for a specific network interface, subnet, or VPC.
Flow log data for a monitored network interface is recorded as flow log records, which are log events consisting of fields that describe the traffic flow. For more information, see Flow log records in the Amazon Virtual Private Cloud User Guide.
When publishing to CloudWatch Logs, flow log records are published to a log group, and each network interface has a unique log stream in the log group. When publishing to Amazon S3, flow log records for all of the monitored network interfaces are published to a single log file object that is stored in the specified bucket.
For more information, see VPC Flow Logs in the Amazon Virtual Private Cloud User Guide.
See also: AWS API Documentation
Request Syntax
client.create_flow_logs(
DryRun=True|False,
ClientToken='string',
DeliverLogsPermissionArn='string',
LogGroupName='string',
ResourceIds=[
'string',
],
ResourceType='VPC'|'Subnet'|'NetworkInterface',
TrafficType='ACCEPT'|'REJECT'|'ALL',
LogDestinationType='cloud-watch-logs'|'s3',
LogDestination='string',
LogFormat='string',
TagSpecifications=[
{
'ResourceType': 'capacity-reservation'|'client-vpn-endpoint'|'customer-gateway'|'carrier-gateway'|'dedicated-host'|'dhcp-options'|'egress-only-internet-gateway'|'elastic-ip'|'elastic-gpu'|'export-image-task'|'export-instance-task'|'fleet'|'fpga-image'|'host-reservation'|'image'|'import-image-task'|'import-snapshot-task'|'instance'|'instance-event-window'|'internet-gateway'|'ipv4pool-ec2'|'ipv6pool-ec2'|'key-pair'|'launch-template'|'local-gateway'|'local-gateway-route-table'|'local-gateway-virtual-interface'|'local-gateway-virtual-interface-group'|'local-gateway-route-table-vpc-association'|'local-gateway-route-table-virtual-interface-group-association'|'natgateway'|'network-acl'|'network-interface'|'network-insights-analysis'|'network-insights-path'|'placement-group'|'prefix-list'|'replace-root-volume-task'|'reserved-instances'|'route-table'|'security-group'|'security-group-rule'|'snapshot'|'spot-fleet-request'|'spot-instances-request'|'subnet'|'traffic-mirror-filter'|'traffic-mirror-session'|'traffic-mirror-target'|'transit-gateway'|'transit-gateway-attachment'|'transit-gateway-connect-peer'|'transit-gateway-multicast-domain'|'transit-gateway-route-table'|'volume'|'vpc'|'vpc-endpoint'|'vpc-endpoint-service'|'vpc-peering-connection'|'vpn-connection'|'vpn-gateway'|'vpc-flow-log',
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
},
],
MaxAggregationInterval=123,
DestinationOptions={
'FileFormat': 'plain-text'|'parquet',
'HiveCompatiblePartitions': True|False,
'PerHourPartition': True|False
}
)
boolean
Checks whether you have the required permissions for the action, without actually making the request, and provides an error response. If you have the required permissions, the error response is DryRunOperation. Otherwise, it is UnauthorizedOperation.
string
Unique, case-sensitive identifier that you provide to ensure the idempotency of the request. For more information, see How to ensure idempotency.
string
The ARN for the IAM role that permits Amazon EC2 to publish flow logs to a CloudWatch Logs log group in your account.
If you specify LogDestinationType as s3, do not specify DeliverLogsPermissionArn or LogGroupName.
string
The name of a new or existing CloudWatch Logs log group where Amazon EC2 publishes your flow logs.
If you specify LogDestinationType as s3, do not specify DeliverLogsPermissionArn or LogGroupName.
list
[REQUIRED]
The ID of the subnet, network interface, or VPC for which you want to create a flow log.
Constraints: Maximum of 1000 resources
(string) --
string
[REQUIRED]
The type of resource for which to create the flow log. For example, if you specified a VPC ID for the ResourceId property, specify VPC for this property.
string
[REQUIRED]
The type of traffic to log. You can log traffic that the resource accepts or rejects, or all traffic.
string
The type of destination to which the flow log data is to be published. Flow log data can be published to CloudWatch Logs or Amazon S3. To publish flow log data to CloudWatch Logs, specify cloud-watch-logs. To publish flow log data to Amazon S3, specify s3.
If you specify LogDestinationType as s3, do not specify DeliverLogsPermissionArn or LogGroupName.
Default: cloud-watch-logs
string
The destination to which the flow log data is to be published. Flow log data can be published to a CloudWatch Logs log group or an Amazon S3 bucket. The value specified for this parameter depends on the value specified for LogDestinationType.
If LogDestinationType is not specified or cloud-watch-logs, specify the Amazon Resource Name (ARN) of the CloudWatch Logs log group. For example, to publish to a log group called my-logs, specify arn:aws:logs:us-east-1:123456789012:log-group:my-logs. Alternatively, use LogGroupName instead.
If LogDestinationType is s3, specify the ARN of the Amazon S3 bucket. You can also specify a subfolder in the bucket. To specify a subfolder in the bucket, use the following ARN format: bucket_ARN/subfolder_name/. For example, to specify a subfolder named my-logs in a bucket named my-bucket, use the following ARN: arn:aws:s3:::my-bucket/my-logs/. You cannot use AWSLogs as a subfolder name. This is a reserved term.
string
The fields to include in the flow log record, in the order in which they should appear. For a list of available fields, see Flow log records. If you omit this parameter, the flow log is created using the default format. If you specify this parameter, you must specify at least one field.
Specify the fields using the ${field-id} format, separated by spaces. For the CLI, surround this parameter value with single quotes on Linux or double quotes on Windows.
list
The tags to apply to the flow logs.
(dict) --
The tags to apply to a resource when the resource is being created.
ResourceType (string) --
The type of resource to tag on creation.
Tags (list) --
The tags to apply to the resource.
(dict) --
Describes a tag.
Key (string) --
The key of the tag.
Constraints: Tag keys are case-sensitive and accept a maximum of 127 Unicode characters. May not begin with aws:.
Value (string) --
The value of the tag.
Constraints: Tag values are case-sensitive and accept a maximum of 255 Unicode characters.
integer
The maximum interval of time during which a flow of packets is captured and aggregated into a flow log record. You can specify 60 seconds (1 minute) or 600 seconds (10 minutes).
When a network interface is attached to a Nitro-based instance, the aggregation interval is always 60 seconds or less, regardless of the value that you specify.
Default: 600
dict
The destination options.
FileFormat (string) --
The format for the flow log. The default is plain-text.
HiveCompatiblePartitions (boolean) --
Indicates whether to use Hive-compatible prefixes for flow logs stored in Amazon S3. The default is false.
PerHourPartition (boolean) --
Indicates whether to partition the flow log per hour. This reduces the cost and response time for queries. The default is false.
dict
Response Syntax
{
'ClientToken': 'string',
'FlowLogIds': [
'string',
],
'Unsuccessful': [
{
'Error': {
'Code': 'string',
'Message': 'string'
},
'ResourceId': 'string'
},
]
}
Response Structure
(dict) --
ClientToken (string) --
Unique, case-sensitive identifier that you provide to ensure the idempotency of the request.
FlowLogIds (list) --
The IDs of the flow logs.
(string) --
Unsuccessful (list) --
Information about the flow logs that could not be created successfully.
(dict) --
Information about items that were not successfully processed in a batch call.
Error (dict) --
Information about the error.
Code (string) --
The error code.
Message (string) --
The error message accompanying the error code.
ResourceId (string) --
The ID of the resource.
{'FlowLogs': {'DestinationOptions': {'FileFormat': 'plain-text | parquet',
'HiveCompatiblePartitions': 'boolean',
'PerHourPartition': 'boolean'}}}
Describes one or more flow logs. To view the information in your flow logs (the log streams for the network interfaces), you must use the CloudWatch Logs console or the CloudWatch Logs API.
See also: AWS API Documentation
Request Syntax
client.describe_flow_logs(
DryRun=True|False,
Filter=[
{
'Name': 'string',
'Values': [
'string',
]
},
],
FlowLogIds=[
'string',
],
MaxResults=123,
NextToken='string'
)
boolean
Checks whether you have the required permissions for the action, without actually making the request, and provides an error response. If you have the required permissions, the error response is DryRunOperation. Otherwise, it is UnauthorizedOperation.
list
One or more filters.
deliver-log-status - The status of the logs delivery ( SUCCESS | FAILED).
log-destination-type - The type of destination to which the flow log publishes data. Possible destination types include cloud-watch-logs and s3.
flow-log-id - The ID of the flow log.
log-group-name - The name of the log group.
resource-id - The ID of the VPC, subnet, or network interface.
traffic-type - The type of traffic ( ACCEPT | REJECT | ALL).
tag:<key> - The key/value combination of a tag assigned to the resource. Use the tag key in the filter name and the tag value as the filter value. For example, to find all resources that have a tag with the key Owner and the value TeamA, specify tag:Owner for the filter name and TeamA for the filter value.
tag-key - The key of a tag assigned to the resource. Use this filter to find all resources assigned a tag with a specific key, regardless of the tag value.
(dict) --
A filter name and value pair that is used to return a more specific list of results from a describe operation. Filters can be used to match a set of resources by specific criteria, such as tags, attributes, or IDs.
Name (string) --
The name of the filter. Filter names are case-sensitive.
Values (list) --
The filter values. Filter values are case-sensitive.
(string) --
list
One or more flow log IDs.
Constraint: Maximum of 1000 flow log IDs.
(string) --
integer
The maximum number of results to return with a single call. To retrieve the remaining results, make another call with the returned nextToken value.
string
The token for the next page of results.
dict
Response Syntax
{
'FlowLogs': [
{
'CreationTime': datetime(2015, 1, 1),
'DeliverLogsErrorMessage': 'string',
'DeliverLogsPermissionArn': 'string',
'DeliverLogsStatus': 'string',
'FlowLogId': 'string',
'FlowLogStatus': 'string',
'LogGroupName': 'string',
'ResourceId': 'string',
'TrafficType': 'ACCEPT'|'REJECT'|'ALL',
'LogDestinationType': 'cloud-watch-logs'|'s3',
'LogDestination': 'string',
'LogFormat': 'string',
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
],
'MaxAggregationInterval': 123,
'DestinationOptions': {
'FileFormat': 'plain-text'|'parquet',
'HiveCompatiblePartitions': True|False,
'PerHourPartition': True|False
}
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
FlowLogs (list) --
Information about the flow logs.
(dict) --
Describes a flow log.
CreationTime (datetime) --
The date and time the flow log was created.
DeliverLogsErrorMessage (string) --
Information about the error that occurred. Rate limited indicates that CloudWatch Logs throttling has been applied for one or more network interfaces, or that you've reached the limit on the number of log groups that you can create. Access error indicates that the IAM role associated with the flow log does not have sufficient permissions to publish to CloudWatch Logs. Unknown error indicates an internal error.
DeliverLogsPermissionArn (string) --
The ARN of the IAM role that posts logs to CloudWatch Logs.
DeliverLogsStatus (string) --
The status of the logs delivery ( SUCCESS | FAILED).
FlowLogId (string) --
The flow log ID.
FlowLogStatus (string) --
The status of the flow log ( ACTIVE).
LogGroupName (string) --
The name of the flow log group.
ResourceId (string) --
The ID of the resource on which the flow log was created.
TrafficType (string) --
The type of traffic captured for the flow log.
LogDestinationType (string) --
The type of destination to which the flow log data is published. Flow log data can be published to CloudWatch Logs or Amazon S3.
LogDestination (string) --
The destination to which the flow log data is published. Flow log data can be published to an CloudWatch Logs log group or an Amazon S3 bucket. If the flow log publishes to CloudWatch Logs, this element indicates the Amazon Resource Name (ARN) of the CloudWatch Logs log group to which the data is published. If the flow log publishes to Amazon S3, this element indicates the ARN of the Amazon S3 bucket to which the data is published.
LogFormat (string) --
The format of the flow log record.
Tags (list) --
The tags for the flow log.
(dict) --
Describes a tag.
Key (string) --
The key of the tag.
Constraints: Tag keys are case-sensitive and accept a maximum of 127 Unicode characters. May not begin with aws:.
Value (string) --
The value of the tag.
Constraints: Tag values are case-sensitive and accept a maximum of 255 Unicode characters.
MaxAggregationInterval (integer) --
The maximum interval of time, in seconds, during which a flow of packets is captured and aggregated into a flow log record.
When a network interface is attached to a Nitro-based instance, the aggregation interval is always 60 seconds (1 minute) or less, regardless of the specified value.
Valid Values: 60 | 600
DestinationOptions (dict) --
The destination options.
FileFormat (string) --
The format for the flow log.
HiveCompatiblePartitions (boolean) --
Indicates whether to use Hive-compatible prefixes for flow logs stored in Amazon S3.
PerHourPartition (boolean) --
Indicates whether to partition the flow log per hour.
NextToken (string) --
The token to use to retrieve the next page of results. This value is null when there are no more results to return.