Amazon SageMaker Service

2021/03/04 - Amazon SageMaker Service - 3 updated api methods

Changes  This release adds the ResolvedOutputS3Uri to the DescribeFeatureGroup API to indicate the S3 prefix where offline data is stored in a feature group

CreateFeatureGroup (updated) Link ¶
Changes (request) 
{'OfflineStoreConfig': {'S3StorageConfig': {'ResolvedOutputS3Uri': 'string'}}}

Create a new FeatureGroup . A FeatureGroup is a group of Features defined in the FeatureStore to describe a Record .

The FeatureGroup defines the schema and features contained in the FeatureGroup. A FeatureGroup definition is composed of a list of Features , a RecordIdentifierFeatureName , an EventTimeFeatureName and configurations for its OnlineStore and OfflineStore . Check AWS service quotas to see the FeatureGroup s quota for your AWS account.

Warning

You must include at least one of OnlineStoreConfig and OfflineStoreConfig to create a FeatureGroup .

See also: AWS API Documentation

Request Syntax

client.create_feature_group(
    FeatureGroupName='string',
    RecordIdentifierFeatureName='string',
    EventTimeFeatureName='string',
    FeatureDefinitions=[
        {
            'FeatureName': 'string',
            'FeatureType': 'Integral'|'Fractional'|'String'
        },
    ],
    OnlineStoreConfig={
        'SecurityConfig': {
            'KmsKeyId': 'string'
        },
        'EnableOnlineStore': True|False
    },
    OfflineStoreConfig={
        'S3StorageConfig': {
            'S3Uri': 'string',
            'KmsKeyId': 'string',
            'ResolvedOutputS3Uri': 'string'
        },
        'DisableGlueTableCreation': True|False,
        'DataCatalogConfig': {
            'TableName': 'string',
            'Catalog': 'string',
            'Database': 'string'
        }
    },
    RoleArn='string',
    Description='string',
    Tags=[
        {
            'Key': 'string',
            'Value': 'string'
        },
    ]
)
type FeatureGroupName

string

param FeatureGroupName

[REQUIRED]

The name of the FeatureGroup . The name must be unique within an AWS Region in an AWS account. The name:

  • Must start and end with an alphanumeric character.

  • Can only contain alphanumeric character and hyphens. Spaces are not allowed.

type RecordIdentifierFeatureName

string

param RecordIdentifierFeatureName

[REQUIRED]

The name of the Feature whose value uniquely identifies a Record defined in the FeatureStore . Only the latest record per identifier value will be stored in the OnlineStore . RecordIdentifierFeatureName must be one of feature definitions' names.

You use the RecordIdentifierFeatureName to access data in a FeatureStore .

This name:

  • Must start and end with an alphanumeric character.

  • Can only contains alphanumeric characters, hyphens, underscores. Spaces are not allowed.

type EventTimeFeatureName

string

param EventTimeFeatureName

[REQUIRED]

The name of the feature that stores the EventTime of a Record in a FeatureGroup .

An EventTime is a point in time when a new event occurs that corresponds to the creation or update of a Record in a FeatureGroup . All Records in the FeatureGroup must have a corresponding EventTime .

An EventTime can be a String or Fractional .

  • Fractional : EventTime feature values must be a Unix timestamp in seconds.

  • String : EventTime feature values must be an ISO-8601 string in the format. The following formats are supported yyyy-MM-dd'T'HH:mm:ssZ and yyyy-MM-dd'T'HH:mm:ss.SSSZ where yyyy , MM , and dd represent the year, month, and day respectively and HH , mm , ss , and if applicable, SSS represent the hour, month, second and milliseconds respsectively. 'T' and Z are constants.

type FeatureDefinitions

list

param FeatureDefinitions

[REQUIRED]

A list of Feature names and types. Name and Type is compulsory per Feature .

Valid feature FeatureType s are Integral , Fractional and String .

FeatureName s cannot be any of the following: is_deleted , write_time , api_invocation_time

You can create up to 2,500 FeatureDefinition s per FeatureGroup .

  • (dict) --

    A list of features. You must include FeatureName and FeatureType . Valid feature FeatureType s are Integral , Fractional and String .

    • FeatureName (string) --

      The name of a feature. The type must be a string. FeatureName cannot be any of the following: is_deleted , write_time , api_invocation_time .

    • FeatureType (string) --

      The value type of a feature. Valid values are Integral, Fractional, or String.

type OnlineStoreConfig

dict

param OnlineStoreConfig

You can turn the OnlineStore on or off by specifying True for the EnableOnlineStore flag in OnlineStoreConfig ; the default value is False .

You can also include an AWS KMS key ID ( KMSKeyId ) for at-rest encryption of the OnlineStore .

  • SecurityConfig (dict) --

    Use to specify KMS Key ID ( KMSKeyId ) for at-rest encryption of your OnlineStore .

    • KmsKeyId (string) --

      The ID of the AWS Key Management Service (AWS KMS) key that SageMaker Feature Store uses to encrypt the Amazon S3 objects at rest using Amazon S3 server-side encryption.

      The caller (either IAM user or IAM role) of CreateFeatureGroup must have below permissions to the OnlineStore KmsKeyId :

      • "kms:Encrypt"

      • "kms:Decrypt"

      • "kms:DescribeKey"

      • "kms:CreateGrant"

      • "kms:RetireGrant"

      • "kms:ReEncryptFrom"

      • "kms:ReEncryptTo"

      • "kms:GenerateDataKey"

      • "kms:ListAliases"

      • "kms:ListGrants"

      • "kms:RevokeGrant"

      The caller (either IAM user or IAM role) to all DataPlane operations ( PutRecord , GetRecord , DeleteRecord ) must have the following permissions to the KmsKeyId :

      • "kms:Decrypt"

  • EnableOnlineStore (boolean) --

    Turn OnlineStore off by specifying False for the EnableOnlineStore flag. Turn OnlineStore on by specifying True for the EnableOnlineStore flag.

    The default value is False .

type OfflineStoreConfig

dict

param OfflineStoreConfig

Use this to configure an OfflineFeatureStore . This parameter allows you to specify:

  • The Amazon Simple Storage Service (Amazon S3) location of an OfflineStore .

  • A configuration for an AWS Glue or AWS Hive data cataolgue.

  • An KMS encryption key to encrypt the Amazon S3 location used for OfflineStore .

To learn more about this parameter, see OfflineStoreConfig.

  • S3StorageConfig (dict) -- [REQUIRED]

    The Amazon Simple Storage (Amazon S3) location of OfflineStore .

    • S3Uri (string) -- [REQUIRED]

      The S3 URI, or location in Amazon S3, of OfflineStore .

      S3 URIs have a format similar to the following: s3://example-bucket/prefix/ .

    • KmsKeyId (string) --

      The AWS Key Management Service (KMS) key ID of the key used to encrypt any objects written into the OfflineStore S3 location.

      The IAM roleARN that is passed as a parameter to CreateFeatureGroup must have below permissions to the KmsKeyId :

      • "kms:GenerateDataKey"

    • ResolvedOutputS3Uri (string) --

      The S3 path where offline records are written.

  • DisableGlueTableCreation (boolean) --

    Set to True to disable the automatic creation of an AWS Glue table when configuring an OfflineStore .

  • DataCatalogConfig (dict) --

    The meta data of the Glue table that is autogenerated when an OfflineStore is created.

    • TableName (string) -- [REQUIRED]

      The name of the Glue table.

    • Catalog (string) -- [REQUIRED]

      The name of the Glue table catalog.

    • Database (string) -- [REQUIRED]

      The name of the Glue table database.

type RoleArn

string

param RoleArn

The Amazon Resource Name (ARN) of the IAM execution role used to persist data into the OfflineStore if an OfflineStoreConfig is provided.

type Description

string

param Description

A free-form description of a FeatureGroup .

type Tags

list

param Tags

Tags used to identify Features in each FeatureGroup .

  • (dict) --

    Describes a tag.

    • Key (string) -- [REQUIRED]

      The tag key.

    • Value (string) -- [REQUIRED]

      The tag value.

rtype

dict

returns

Response Syntax

{
    'FeatureGroupArn': 'string'
}

Response Structure

  • (dict) --

    • FeatureGroupArn (string) --

      The Amazon Resource Name (ARN) of the FeatureGroup . This is a unique identifier for the feature group.

DescribeFeatureGroup (updated) Link ¶
Changes (response) 
{'OfflineStoreConfig': {'S3StorageConfig': {'ResolvedOutputS3Uri': 'string'}}}

Use this operation to describe a FeatureGroup . The response includes information on the creation time, FeatureGroup name, the unique identifier for each FeatureGroup , and more.

See also: AWS API Documentation

Request Syntax

client.describe_feature_group(
    FeatureGroupName='string',
    NextToken='string'
)
type FeatureGroupName

string

param FeatureGroupName

[REQUIRED]

The name of the FeatureGroup you want described.

type NextToken

string

param NextToken

A token to resume pagination of the list of Features ( FeatureDefinitions ). 2,500 Features are returned by default.

rtype

dict

returns

Response Syntax

{
    'FeatureGroupArn': 'string',
    'FeatureGroupName': 'string',
    'RecordIdentifierFeatureName': 'string',
    'EventTimeFeatureName': 'string',
    'FeatureDefinitions': [
        {
            'FeatureName': 'string',
            'FeatureType': 'Integral'|'Fractional'|'String'
        },
    ],
    'CreationTime': datetime(2015, 1, 1),
    'OnlineStoreConfig': {
        'SecurityConfig': {
            'KmsKeyId': 'string'
        },
        'EnableOnlineStore': True|False
    },
    'OfflineStoreConfig': {
        'S3StorageConfig': {
            'S3Uri': 'string',
            'KmsKeyId': 'string',
            'ResolvedOutputS3Uri': 'string'
        },
        'DisableGlueTableCreation': True|False,
        'DataCatalogConfig': {
            'TableName': 'string',
            'Catalog': 'string',
            'Database': 'string'
        }
    },
    'RoleArn': 'string',
    'FeatureGroupStatus': 'Creating'|'Created'|'CreateFailed'|'Deleting'|'DeleteFailed',
    'OfflineStoreStatus': {
        'Status': 'Active'|'Blocked'|'Disabled',
        'BlockedReason': 'string'
    },
    'FailureReason': 'string',
    'Description': 'string',
    'NextToken': 'string'
}

Response Structure

  • (dict) --

    • FeatureGroupArn (string) --

      The Amazon Resource Name (ARN) of the FeatureGroup .

    • FeatureGroupName (string) --

      he name of the FeatureGroup .

    • RecordIdentifierFeatureName (string) --

      The name of the Feature used for RecordIdentifier , whose value uniquely identifies a record stored in the feature store.

    • EventTimeFeatureName (string) --

      The name of the feature that stores the EventTime of a Record in a FeatureGroup .

      An EventTime is a point in time when a new event occurs that corresponds to the creation or update of a Record in a FeatureGroup . All Records in the FeatureGroup have a corresponding EventTime .

    • FeatureDefinitions (list) --

      A list of the Features in the FeatureGroup . Each feature is defined by a FeatureName and FeatureType .

      • (dict) --

        A list of features. You must include FeatureName and FeatureType . Valid feature FeatureType s are Integral , Fractional and String .

        • FeatureName (string) --

          The name of a feature. The type must be a string. FeatureName cannot be any of the following: is_deleted , write_time , api_invocation_time .

        • FeatureType (string) --

          The value type of a feature. Valid values are Integral, Fractional, or String.

    • CreationTime (datetime) --

      A timestamp indicating when SageMaker created the FeatureGroup .

    • OnlineStoreConfig (dict) --

      The configuration for the OnlineStore .

      • SecurityConfig (dict) --

        Use to specify KMS Key ID ( KMSKeyId ) for at-rest encryption of your OnlineStore .

        • KmsKeyId (string) --

          The ID of the AWS Key Management Service (AWS KMS) key that SageMaker Feature Store uses to encrypt the Amazon S3 objects at rest using Amazon S3 server-side encryption.

          The caller (either IAM user or IAM role) of CreateFeatureGroup must have below permissions to the OnlineStore KmsKeyId :

          • "kms:Encrypt"

          • "kms:Decrypt"

          • "kms:DescribeKey"

          • "kms:CreateGrant"

          • "kms:RetireGrant"

          • "kms:ReEncryptFrom"

          • "kms:ReEncryptTo"

          • "kms:GenerateDataKey"

          • "kms:ListAliases"

          • "kms:ListGrants"

          • "kms:RevokeGrant"

          The caller (either IAM user or IAM role) to all DataPlane operations ( PutRecord , GetRecord , DeleteRecord ) must have the following permissions to the KmsKeyId :

          • "kms:Decrypt"

      • EnableOnlineStore (boolean) --

        Turn OnlineStore off by specifying False for the EnableOnlineStore flag. Turn OnlineStore on by specifying True for the EnableOnlineStore flag.

        The default value is False .

    • OfflineStoreConfig (dict) --

      The configuration of the OfflineStore , inducing the S3 location of the OfflineStore , AWS Glue or AWS Hive data catalogue configurations, and the security configuration.

      • S3StorageConfig (dict) --

        The Amazon Simple Storage (Amazon S3) location of OfflineStore .

        • S3Uri (string) --

          The S3 URI, or location in Amazon S3, of OfflineStore .

          S3 URIs have a format similar to the following: s3://example-bucket/prefix/ .

        • KmsKeyId (string) --

          The AWS Key Management Service (KMS) key ID of the key used to encrypt any objects written into the OfflineStore S3 location.

          The IAM roleARN that is passed as a parameter to CreateFeatureGroup must have below permissions to the KmsKeyId :

          • "kms:GenerateDataKey"

        • ResolvedOutputS3Uri (string) --

          The S3 path where offline records are written.

      • DisableGlueTableCreation (boolean) --

        Set to True to disable the automatic creation of an AWS Glue table when configuring an OfflineStore .

      • DataCatalogConfig (dict) --

        The meta data of the Glue table that is autogenerated when an OfflineStore is created.

        • TableName (string) --

          The name of the Glue table.

        • Catalog (string) --

          The name of the Glue table catalog.

        • Database (string) --

          The name of the Glue table database.

    • RoleArn (string) --

      The Amazon Resource Name (ARN) of the IAM execution role used to persist data into the OfflineStore if an OfflineStoreConfig is provided.

    • FeatureGroupStatus (string) --

      The status of the feature group.

    • OfflineStoreStatus (dict) --

      The status of the OfflineStore . Notifies you if replicating data into the OfflineStore has failed. Returns either: Active or Blocked

      • Status (string) --

        An OfflineStore status.

      • BlockedReason (string) --

        The justification for why the OfflineStoreStatus is Blocked (if applicable).

    • FailureReason (string) --

      The reason that the FeatureGroup failed to be replicated in the OfflineStore . This is failure can occur because:

      • The FeatureGroup could not be created in the OfflineStore .

      • The FeatureGroup could not be deleted from the OfflineStore .

    • Description (string) --

      A free form description of the feature group.

    • NextToken (string) --

      A token to resume pagination of the list of Features ( FeatureDefinitions ).