Amazon Kinesis

Updates the shard count of the specified stream to the specified number of shards.

Updating the shard count is an asynchronous operation. Upon receiving the request, Amazon Kinesis returns immediately and sets the status of the stream to UPDATING . After the update is complete, Amazon Kinesis sets the status of the stream back to ACTIVE . Depending on the size of the stream, the scaling action could take a few minutes to complete. You can continue to read and write data to your stream while its status is UPDATING .

To update the shard count, Amazon Kinesis performs splits and merges and individual shards. This can cause short-lived shards to be created, in addition to the final shards. We recommend that you double or halve the shard count, as this results in the fewest number of splits or merges.

This operation has a rate limit of twice per rolling 24 hour period. You cannot scale above double your current shard count, scale below half your current shard count, or exceed the shard limits for your account.

For the default limits for an AWS account, see Streams Limits in the Amazon Kinesis Streams Developer Guide . If you need to increase a limit, contact AWS Support.

Request Syntax

client.update_shard_count(
    StreamName='string',
    TargetShardCount=123,
    ScalingType='UNIFORM_SCALING'
)

type StreamName

string

param StreamName

[REQUIRED]

The name of the stream.

type TargetShardCount

integer

param TargetShardCount

[REQUIRED]

The new number of shards.

type ScalingType

string

param ScalingType

[REQUIRED]

The scaling type. Uniform scaling creates shards of equal size.

rtype

dict

returns

Response Syntax

{
    'StreamName': 'string',
    'CurrentShardCount': 123,
    'TargetShardCount': 123
}

Response Structure

• (dict) --

  • StreamName (string) --

    The name of the stream.

  • CurrentShardCount (integer) --

    The current number of shards.

  • TargetShardCount (integer) --

    The updated number of shards.

Describes the shard limits and usage for the account.

If you update your account limits, the old limits might be returned for a few minutes.

This operation has a limit of 1 transaction per second per account.

Request Syntax

client.describe_limits()

rtype

dict

returns

Response Syntax

{
    'ShardLimit': 123,
    'OpenShardCount': 123
}

Response Structure

• (dict) --

  • ShardLimit (integer) --

    The maximum number of shards.

  • OpenShardCount (integer) --

    The number of open shards.

{'StreamDescription': {'StreamCreationTimestamp': 'timestamp'}}

Describes the specified Amazon Kinesis stream.

The information returned includes the stream name, Amazon Resource Name (ARN), creation time, enhanced metric configuration, and shard map. The shard map is an array of shard objects. For each shard object, there is the hash key and sequence number ranges that the shard spans, and the IDs of any earlier shards that played in a role in creating the shard. Every record ingested in the stream is identified by a sequence number, which is assigned when the record is put into the stream.

You can limit the number of shards returned by each call. For more information, see Retrieving Shards from a Stream in the Amazon Kinesis Streams Developer Guide .

There are no guarantees about the chronological order shards returned. To process shards in chronological order, use the ID of the parent shard to track the lineage to the oldest shard.

This operation has a limit of 10 transactions per second per account.

Request Syntax

client.describe_stream(
    StreamName='string',
    Limit=123,
    ExclusiveStartShardId='string'
)

type StreamName

string

param StreamName

[REQUIRED]

The name of the stream to describe.

type Limit

integer

param Limit

The maximum number of shards to return in a single call. The default value is 100. If you specify a value greater than 100, at most 100 shards are returned.

type ExclusiveStartShardId

string

param ExclusiveStartShardId

The shard ID of the shard to start with.

rtype

dict

returns

Response Syntax

{
    'StreamDescription': {
        'StreamName': 'string',
        'StreamARN': 'string',
        'StreamStatus': 'CREATING'|'DELETING'|'ACTIVE'|'UPDATING',
        'Shards': [
            {
                'ShardId': 'string',
                'ParentShardId': 'string',
                'AdjacentParentShardId': 'string',
                'HashKeyRange': {
                    'StartingHashKey': 'string',
                    'EndingHashKey': 'string'
                },
                'SequenceNumberRange': {
                    'StartingSequenceNumber': 'string',
                    'EndingSequenceNumber': 'string'
                }
            },
        ],
        'HasMoreShards': True|False,
        'RetentionPeriodHours': 123,
        'StreamCreationTimestamp': datetime(2015, 1, 1),
        'EnhancedMonitoring': [
            {
                'ShardLevelMetrics': [
                    'IncomingBytes'|'IncomingRecords'|'OutgoingBytes'|'OutgoingRecords'|'WriteProvisionedThroughputExceeded'|'ReadProvisionedThroughputExceeded'|'IteratorAgeMilliseconds'|'ALL',
                ]
            },
        ]
    }
}

Response Structure

• (dict) --

  Represents the output for DescribeStream .

  • StreamDescription (dict) --

    The current status of the stream, the stream ARN, an array of shard objects that comprise the stream, and whether there are more shards available.

    • StreamName (string) --

      The name of the stream being described.

    • StreamARN (string) --

      The Amazon Resource Name (ARN) for the stream being described.

    • StreamStatus (string) --

      The current status of the stream being described. The stream status is one of the following states:

      • CREATING - The stream is being created. Amazon Kinesis immediately returns and sets StreamStatus to CREATING .

      • DELETING - The stream is being deleted. The specified stream is in the DELETING state until Amazon Kinesis completes the deletion.

      • ACTIVE - The stream exists and is ready for read and write operations or deletion. You should perform read and write operations only on an ACTIVE stream.

      • UPDATING - Shards in the stream are being merged or split. Read and write operations continue to work while the stream is in the UPDATING state.

    • Shards (list) --

      The shards that comprise the stream.

      • (dict) --

        A uniquely identified group of data records in an Amazon Kinesis stream.

        • ShardId (string) --

          The unique identifier of the shard within the stream.

        • ParentShardId (string) --

          The shard ID of the shard's parent.

        • AdjacentParentShardId (string) --

          The shard ID of the shard adjacent to the shard's parent.

        • HashKeyRange (dict) --

          The range of possible hash key values for the shard, which is a set of ordered contiguous positive integers.

          • StartingHashKey (string) --

            The starting hash key of the hash key range.

          • EndingHashKey (string) --

            The ending hash key of the hash key range.

        • SequenceNumberRange (dict) --

          The range of possible sequence numbers for the shard.

          • StartingSequenceNumber (string) --

            The starting sequence number for the range.

          • EndingSequenceNumber (string) --

            The ending sequence number for the range. Shards that are in the OPEN state have an ending sequence number of null .

    • HasMoreShards (boolean) --

      If set to true , more shards in the stream are available to describe.

    • RetentionPeriodHours (integer) --

      The current retention period, in hours.

    • StreamCreationTimestamp (datetime) --

      The approximate time that the stream was created.

    • EnhancedMonitoring (list) --

      Represents the current enhanced monitoring settings of the stream.

      • (dict) --

        Represents enhanced metrics types.

        • ShardLevelMetrics (list) --

          List of shard-level metrics.

          The following are the valid shard-level metrics. The value " ALL " enhances every metric.

          • IncomingBytes

          • IncomingRecords

          • OutgoingBytes

          • OutgoingRecords

          • WriteProvisionedThroughputExceeded

          • ReadProvisionedThroughputExceeded

          • IteratorAgeMilliseconds

          • ALL

          For more information, see Monitoring the Amazon Kinesis Streams Service with Amazon CloudWatch in the Amazon Kinesis Streams Developer Guide .

          • (string) --