Amazon Bedrock

2024/08/22 - Amazon Bedrock - 7 new2 updated api methods

Changes  Amazon Bedrock Evaluation BatchDeleteEvaluationJob API allows customers to delete evaluation jobs under terminated evaluation job statuses - Stopped, Failed, or Completed. Customers can submit a batch of 25 evaluation jobs to be deleted at once.

DeleteImportedModel (new) Link ¶

Deletes a custom model that you imported earlier. For more information, see Import a customized model in the Amazon Bedrock User Guide.

See also: AWS API Documentation

Request Syntax

client.delete_imported_model(
    modelIdentifier='string'
)
type modelIdentifier:

string

param modelIdentifier:

[REQUIRED]

Name of the imported model to delete.

rtype:

dict

returns:

Response Syntax

{}

Response Structure

  • (dict) --

ListModelImportJobs (new) Link ¶

Returns a list of import jobs you've submitted. You can filter the results to return based on one or more criteria. For more information, see Import a customized model in the Amazon Bedrock User Guide.

See also: AWS API Documentation

Request Syntax

client.list_model_import_jobs(
    creationTimeAfter=datetime(2015, 1, 1),
    creationTimeBefore=datetime(2015, 1, 1),
    statusEquals='InProgress'|'Completed'|'Failed',
    nameContains='string',
    maxResults=123,
    nextToken='string',
    sortBy='CreationTime',
    sortOrder='Ascending'|'Descending'
)
type creationTimeAfter:

datetime

param creationTimeAfter:

Return import jobs that were created after the specified time.

type creationTimeBefore:

datetime

param creationTimeBefore:

Return import jobs that were created before the specified time.

type statusEquals:

string

param statusEquals:

Return imported jobs with the specified status.

type nameContains:

string

param nameContains:

Return imported jobs only if the job name contains these characters.

type maxResults:

integer

param maxResults:

The maximum number of results to return in the response. If the total number of results is greater than this value, use the token returned in the response in the nextToken field when making another request to return the next batch of results.

type nextToken:

string

param nextToken:

If the total number of results is greater than the maxResults value provided in the request, enter the token returned in the nextToken field in the response in this field to return the next batch of results.

type sortBy:

string

param sortBy:

The field to sort by in the returned list of imported jobs.

type sortOrder:

string

param sortOrder:

Specifies whether to sort the results in ascending or descending order.

rtype:

dict

returns:

Response Syntax

{
    'nextToken': 'string',
    'modelImportJobSummaries': [
        {
            'jobArn': 'string',
            'jobName': 'string',
            'status': 'InProgress'|'Completed'|'Failed',
            'lastModifiedTime': datetime(2015, 1, 1),
            'creationTime': datetime(2015, 1, 1),
            'endTime': datetime(2015, 1, 1),
            'importedModelArn': 'string',
            'importedModelName': 'string'
        },
    ]
}

Response Structure

  • (dict) --

    • nextToken (string) --

      If the total number of results is greater than the maxResults value provided in the request, enter the token returned in the nextToken field in the response in this field to return the next batch of results.

    • modelImportJobSummaries (list) --

      Import job summaries.

      • (dict) --

        Information about the import job.

        • jobArn (string) --

          The Amazon Resource Name (ARN) of the import job.

        • jobName (string) --

          The name of the import job.

        • status (string) --

          The status of the imported job.

        • lastModifiedTime (datetime) --

          The time when the import job was last modified.

        • creationTime (datetime) --

          The time import job was created.

        • endTime (datetime) --

          The time when import job ended.

        • importedModelArn (string) --

          The Amazon resource Name (ARN) of the imported model.

        • importedModelName (string) --

          The name of the imported model.

GetModelImportJob (new) Link ¶

Retrieves the properties associated with import model job, including the status of the job. For more information, see Import a customized model in the Amazon Bedrock User Guide.

See also: AWS API Documentation

Request Syntax

client.get_model_import_job(
    jobIdentifier='string'
)
type jobIdentifier:

string

param jobIdentifier:

[REQUIRED]

The identifier of the import job.

rtype:

dict

returns:

Response Syntax

{
    'jobArn': 'string',
    'jobName': 'string',
    'importedModelName': 'string',
    'importedModelArn': 'string',
    'roleArn': 'string',
    'modelDataSource': {
        's3DataSource': {
            's3Uri': 'string'
        }
    },
    'status': 'InProgress'|'Completed'|'Failed',
    'failureMessage': 'string',
    'creationTime': datetime(2015, 1, 1),
    'lastModifiedTime': datetime(2015, 1, 1),
    'endTime': datetime(2015, 1, 1),
    'vpcConfig': {
        'subnetIds': [
            'string',
        ],
        'securityGroupIds': [
            'string',
        ]
    },
    'importedModelKmsKeyArn': 'string'
}

Response Structure

  • (dict) --

    • jobArn (string) --

      The Amazon Resource Name (ARN) of the import job.

    • jobName (string) --

      The name of the import job.

    • importedModelName (string) --

      The name of the imported model.

    • importedModelArn (string) --

      The Amazon Resource Name (ARN) of the imported model.

    • roleArn (string) --

      The Amazon Resource Name (ARN) of the IAM role associated with this job.

    • modelDataSource (dict) --

      The data source for the imported model.

      • s3DataSource (dict) --

        The Amazon S3 data source of the imported model.

        • s3Uri (string) --

          The URI of the Amazon S3 data source.

    • status (string) --

      The status of the job. A successful job transitions from in-progress to completed when the imported model is ready to use. If the job failed, the failure message contains information about why the job failed.

    • failureMessage (string) --

      Information about why the import job failed.

    • creationTime (datetime) --

      The time the resource was created.

    • lastModifiedTime (datetime) --

      Time the resource was last modified.

    • endTime (datetime) --

      Time that the resource transitioned to terminal state.

    • vpcConfig (dict) --

      The Virtual Private Cloud (VPC) configuration of the import model job.

      • subnetIds (list) --

        VPC configuration subnets.

        • (string) --

      • securityGroupIds (list) --

        VPC configuration security group Ids.

        • (string) --

    • importedModelKmsKeyArn (string) --

      The imported model is encrypted at rest using this key.

GetImportedModel (new) Link ¶

Gets properties associated with a customized model you imported.

See also: AWS API Documentation

Request Syntax

client.get_imported_model(
    modelIdentifier='string'
)
type modelIdentifier:

string

param modelIdentifier:

[REQUIRED]

Name or Amazon Resource Name (ARN) of the imported model.

rtype:

dict

returns:

Response Syntax

{
    'modelArn': 'string',
    'modelName': 'string',
    'jobName': 'string',
    'jobArn': 'string',
    'modelDataSource': {
        's3DataSource': {
            's3Uri': 'string'
        }
    },
    'creationTime': datetime(2015, 1, 1),
    'modelArchitecture': 'string',
    'modelKmsKeyArn': 'string'
}

Response Structure

  • (dict) --

    • modelArn (string) --

      The Amazon Resource Name (ARN) associated with this imported model.

    • modelName (string) --

      The name of the imported model.

    • jobName (string) --

      Job name associated with the imported model.

    • jobArn (string) --

      Job Amazon Resource Name (ARN) associated with the imported model.

    • modelDataSource (dict) --

      The data source for this imported model.

      • s3DataSource (dict) --

        The Amazon S3 data source of the imported model.

        • s3Uri (string) --

          The URI of the Amazon S3 data source.

    • creationTime (datetime) --

      Creation time of the imported model.

    • modelArchitecture (string) --

      The architecture of the imported model.

    • modelKmsKeyArn (string) --

      The imported model is encrypted at rest using this key.

BatchDeleteEvaluationJob (new) Link ¶

Creates a batch deletion job. A model evaluation job can only be deleted if it has following status FAILED, COMPLETED, and STOPPED. You can request up to 25 model evaluation jobs be deleted in a single request.

See also: AWS API Documentation

Request Syntax

client.batch_delete_evaluation_job(
    jobIdentifiers=[
        'string',
    ]
)
type jobIdentifiers:

list

param jobIdentifiers:

[REQUIRED]

An array of model evaluation job ARNs to be deleted.

  • (string) --

rtype:

dict

returns:

Response Syntax

{
    'errors': [
        {
            'jobIdentifier': 'string',
            'code': 'string',
            'message': 'string'
        },
    ],
    'evaluationJobs': [
        {
            'jobIdentifier': 'string',
            'jobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting'
        },
    ]
}

Response Structure

  • (dict) --

    • errors (list) --

      A JSON object containing the HTTP status codes and the ARNs of model evaluation jobs that failed to be deleted.

      • (dict) --

        A JSON array that provides the status of the model evaluation jobs being deleted.

        • jobIdentifier (string) --

          The ARN of the model evaluation job being deleted.

        • code (string) --

          A HTTP status code of the model evaluation job being deleted.

        • message (string) --

          A status message about the model evaluation job deletion.

    • evaluationJobs (list) --

      The list of model evaluation jobs to be deleted.

      • (dict) --

        An array of model evaluation jobs to be deleted, and their associated statuses.

        • jobIdentifier (string) --

          The ARN of model evaluation job to be deleted.

        • jobStatus (string) --

          The status of the job's deletion.

ListImportedModels (new) Link ¶

Returns a list of models you've imported. You can filter the results to return based on one or more criteria. For more information, see Import a customized model in the Amazon Bedrock User Guide.

See also: AWS API Documentation

Request Syntax

client.list_imported_models(
    creationTimeBefore=datetime(2015, 1, 1),
    creationTimeAfter=datetime(2015, 1, 1),
    nameContains='string',
    maxResults=123,
    nextToken='string',
    sortBy='CreationTime',
    sortOrder='Ascending'|'Descending'
)
type creationTimeBefore:

datetime

param creationTimeBefore:

Return imported models that created before the specified time.

type creationTimeAfter:

datetime

param creationTimeAfter:

Return imported models that were created after the specified time.

type nameContains:

string

param nameContains:

Return imported models only if the model name contains these characters.

type maxResults:

integer

param maxResults:

The maximum number of results to return in the response. If the total number of results is greater than this value, use the token returned in the response in the nextToken field when making another request to return the next batch of results.

type nextToken:

string

param nextToken:

If the total number of results is greater than the maxResults value provided in the request, enter the token returned in the nextToken field in the response in this field to return the next batch of results.

type sortBy:

string

param sortBy:

The field to sort by in the returned list of imported models.

type sortOrder:

string

param sortOrder:

Specifies whetehr to sort the results in ascending or descending order.

rtype:

dict

returns:

Response Syntax

{
    'nextToken': 'string',
    'modelSummaries': [
        {
            'modelArn': 'string',
            'modelName': 'string',
            'creationTime': datetime(2015, 1, 1)
        },
    ]
}

Response Structure

  • (dict) --

    • nextToken (string) --

      If the total number of results is greater than the maxResults value provided in the request, use this token when making another request in the nextToken field to return the next batch of results.

    • modelSummaries (list) --

      Model summaries.

      • (dict) --

        Information about tne imported model.

        • modelArn (string) --

          The Amazon Resource Name (ARN) of the imported model.

        • modelName (string) --

          Name of the imported model.

        • creationTime (datetime) --

          Creation time of the imported model.

CreateModelImportJob (new) Link ¶

Creates a model import job to import model that you have customized in other environments, such as Amazon SageMaker. For more information, see Import a customized model

See also: AWS API Documentation

Request Syntax

client.create_model_import_job(
    jobName='string',
    importedModelName='string',
    roleArn='string',
    modelDataSource={
        's3DataSource': {
            's3Uri': 'string'
        }
    },
    jobTags=[
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    importedModelTags=[
        {
            'key': 'string',
            'value': 'string'
        },
    ],
    clientRequestToken='string',
    vpcConfig={
        'subnetIds': [
            'string',
        ],
        'securityGroupIds': [
            'string',
        ]
    },
    importedModelKmsKeyId='string'
)
type jobName:

string

param jobName:

[REQUIRED]

The name of the import job.

type importedModelName:

string

param importedModelName:

[REQUIRED]

The name of the imported model.

type roleArn:

string

param roleArn:

[REQUIRED]

The Amazon Resource Name (ARN) of the model import job.

type modelDataSource:

dict

param modelDataSource:

[REQUIRED]

The data source for the imported model.

  • s3DataSource (dict) --

    The Amazon S3 data source of the imported model.

    • s3Uri (string) -- [REQUIRED]

      The URI of the Amazon S3 data source.

type jobTags:

list

param jobTags:

Tags to attach to this import job.

  • (dict) --

    Definition of the key/value pair for a tag.

    • key (string) -- [REQUIRED]

      Key for the tag.

    • value (string) -- [REQUIRED]

      Value for the tag.

type importedModelTags:

list

param importedModelTags:

Tags to attach to the imported model.

  • (dict) --

    Definition of the key/value pair for a tag.

    • key (string) -- [REQUIRED]

      Key for the tag.

    • value (string) -- [REQUIRED]

      Value for the tag.

type clientRequestToken:

string

param clientRequestToken:

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error. For more information, see Ensuring idempotency.

type vpcConfig:

dict

param vpcConfig:

VPC configuration parameters for the private Virtual Private Cloud (VPC) that contains the resources you are using for the import job.

  • subnetIds (list) -- [REQUIRED]

    VPC configuration subnets.

    • (string) --

  • securityGroupIds (list) -- [REQUIRED]

    VPC configuration security group Ids.

    • (string) --

type importedModelKmsKeyId:

string

param importedModelKmsKeyId:

The imported model is encrypted at rest using this key.

rtype:

dict

returns:

Response Syntax

{
    'jobArn': 'string'
}

Response Structure

  • (dict) --

    • jobArn (string) --

      The Amazon Resource Name (ARN) of the model import job.

GetEvaluationJob (updated) Link ¶
Changes (response) 
{'status': {'Deleting'}}

Retrieves the properties associated with a model evaluation job, including the status of the job. For more information, see Model evaluation.

See also: AWS API Documentation

Request Syntax

client.get_evaluation_job(
    jobIdentifier='string'
)
type jobIdentifier:

string

param jobIdentifier:

[REQUIRED]

The Amazon Resource Name (ARN) of the model evaluation job.

rtype:

dict

returns:

Response Syntax

{
    'jobName': 'string',
    'status': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting',
    'jobArn': 'string',
    'jobDescription': 'string',
    'roleArn': 'string',
    'customerEncryptionKeyId': 'string',
    'jobType': 'Human'|'Automated',
    'evaluationConfig': {
        'automated': {
            'datasetMetricConfigs': [
                {
                    'taskType': 'Summarization'|'Classification'|'QuestionAndAnswer'|'Generation'|'Custom',
                    'dataset': {
                        'name': 'string',
                        'datasetLocation': {
                            's3Uri': 'string'
                        }
                    },
                    'metricNames': [
                        'string',
                    ]
                },
            ]
        },
        'human': {
            'humanWorkflowConfig': {
                'flowDefinitionArn': 'string',
                'instructions': 'string'
            },
            'customMetrics': [
                {
                    'name': 'string',
                    'description': 'string',
                    'ratingMethod': 'string'
                },
            ],
            'datasetMetricConfigs': [
                {
                    'taskType': 'Summarization'|'Classification'|'QuestionAndAnswer'|'Generation'|'Custom',
                    'dataset': {
                        'name': 'string',
                        'datasetLocation': {
                            's3Uri': 'string'
                        }
                    },
                    'metricNames': [
                        'string',
                    ]
                },
            ]
        }
    },
    'inferenceConfig': {
        'models': [
            {
                'bedrockModel': {
                    'modelIdentifier': 'string',
                    'inferenceParams': 'string'
                }
            },
        ]
    },
    'outputDataConfig': {
        's3Uri': 'string'
    },
    'creationTime': datetime(2015, 1, 1),
    'lastModifiedTime': datetime(2015, 1, 1),
    'failureMessages': [
        'string',
    ]
}

Response Structure

  • (dict) --

    • jobName (string) --

      The name of the model evaluation job.

    • status (string) --

      The status of the model evaluation job.

    • jobArn (string) --

      The Amazon Resource Name (ARN) of the model evaluation job.

    • jobDescription (string) --

      The description of the model evaluation job.

    • roleArn (string) --

      The Amazon Resource Name (ARN) of the IAM service role used in the model evaluation job.

    • customerEncryptionKeyId (string) --

      The Amazon Resource Name (ARN) of the customer managed key specified when the model evaluation job was created.

    • jobType (string) --

      The type of model evaluation job.

    • evaluationConfig (dict) --

      Contains details about the type of model evaluation job, the metrics used, the task type selected, the datasets used, and any custom metrics you defined.

      • automated (dict) --

        Used to specify an automated model evaluation job. See AutomatedEvaluationConfig to view the required parameters.

        • datasetMetricConfigs (list) --

          Specifies the required elements for an automatic model evaluation job.

          • (dict) --

            Defines the built-in prompt datasets, built-in metric names and custom metric names, and the task type.

            • taskType (string) --

              The task type you want the model to carry out.

            • dataset (dict) --

              Specifies the prompt dataset.

              • name (string) --

                Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold, Builtin.BoolQ, Builtin.NaturalQuestions, Builtin.Gigaword, Builtin.RealToxicityPrompts, Builtin.TriviaQA, Builtin.T-Rex, Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2.

              • datasetLocation (dict) --

                For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved.

                • s3Uri (string) --

                  The S3 URI of the S3 bucket specified in the job.

            • metricNames (list) --

              The names of the metrics used. For automated model evaluation jobs valid values are "Builtin.Accuracy", "Builtin.Robustness", and "Builtin.Toxicity". In human-based model evaluation jobs the array of strings must match the name parameter specified in HumanEvaluationCustomMetric.

              • (string) --

      • human (dict) --

        Used to specify a model evaluation job that uses human workers.See HumanEvaluationConfig to view the required parameters.

        • humanWorkflowConfig (dict) --

          The parameters of the human workflow.

          • flowDefinitionArn (string) --

            The Amazon Resource Number (ARN) for the flow definition

          • instructions (string) --

            Instructions for the flow definition

        • customMetrics (list) --

          A HumanEvaluationCustomMetric object. It contains the names the metrics, how the metrics are to be evaluated, an optional description.

          • (dict) --

            In a model evaluation job that uses human workers you must define the name of the metric, and how you want that metric rated ratingMethod, and an optional description of the metric.

            • name (string) --

              The name of the metric. Your human evaluators will see this name in the evaluation UI.

            • description (string) --

              An optional description of the metric. Use this parameter to provide more details about the metric.

            • ratingMethod (string) --

              Choose how you want your human workers to evaluation your model. Valid values for rating methods are ThumbsUpDown, IndividualLikertScale, ComparisonLikertScale, ComparisonChoice, and ComparisonRank

        • datasetMetricConfigs (list) --

          Use to specify the metrics, task, and prompt dataset to be used in your model evaluation job.

          • (dict) --

            Defines the built-in prompt datasets, built-in metric names and custom metric names, and the task type.

            • taskType (string) --

              The task type you want the model to carry out.

            • dataset (dict) --

              Specifies the prompt dataset.

              • name (string) --

                Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold, Builtin.BoolQ, Builtin.NaturalQuestions, Builtin.Gigaword, Builtin.RealToxicityPrompts, Builtin.TriviaQA, Builtin.T-Rex, Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2.

              • datasetLocation (dict) --

                For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved.

                • s3Uri (string) --

                  The S3 URI of the S3 bucket specified in the job.

            • metricNames (list) --

              The names of the metrics used. For automated model evaluation jobs valid values are "Builtin.Accuracy", "Builtin.Robustness", and "Builtin.Toxicity". In human-based model evaluation jobs the array of strings must match the name parameter specified in HumanEvaluationCustomMetric.

              • (string) --

    • inferenceConfig (dict) --

      Details about the models you specified in your model evaluation job.

      • models (list) --

        Used to specify the models.

        • (dict) --

          Defines the models used in the model evaluation job.

          • bedrockModel (dict) --

            Defines the Amazon Bedrock model and inference parameters you want used.

            • modelIdentifier (string) --

              The ARN of the Amazon Bedrock model specified.

            • inferenceParams (string) --

              Each Amazon Bedrock support different inference parameters that change how the model behaves during inference.

    • outputDataConfig (dict) --

      Amazon S3 location for where output data is saved.

      • s3Uri (string) --

        The Amazon S3 URI where the results of model evaluation job are saved.

    • creationTime (datetime) --

      When the model evaluation job was created.

    • lastModifiedTime (datetime) --

      When the model evaluation job was last modified.

    • failureMessages (list) --

      An array of strings the specify why the model evaluation job has failed.

      • (string) --

ListEvaluationJobs (updated) Link ¶
Changes (request, response)
Request 
{'statusEquals': {'Deleting'}}
Response 
{'jobSummaries': {'status': {'Deleting'}}}

Lists model evaluation jobs.

See also: AWS API Documentation

Request Syntax

client.list_evaluation_jobs(
    creationTimeAfter=datetime(2015, 1, 1),
    creationTimeBefore=datetime(2015, 1, 1),
    statusEquals='InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting',
    nameContains='string',
    maxResults=123,
    nextToken='string',
    sortBy='CreationTime',
    sortOrder='Ascending'|'Descending'
)
type creationTimeAfter:

datetime

param creationTimeAfter:

A filter that includes model evaluation jobs created after the time specified.

type creationTimeBefore:

datetime

param creationTimeBefore:

A filter that includes model evaluation jobs created prior to the time specified.

type statusEquals:

string

param statusEquals:

Only return jobs where the status condition is met.

type nameContains:

string

param nameContains:

Query parameter string for model evaluation job names.

type maxResults:

integer

param maxResults:

The maximum number of results to return.

type nextToken:

string

param nextToken:

Continuation token from the previous response, for Amazon Bedrock to list the next set of results.

type sortBy:

string

param sortBy:

Allows you to sort model evaluation jobs by when they were created.

type sortOrder:

string

param sortOrder:

How you want the order of jobs sorted.

rtype:

dict

returns:

Response Syntax

{
    'nextToken': 'string',
    'jobSummaries': [
        {
            'jobArn': 'string',
            'jobName': 'string',
            'status': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting',
            'creationTime': datetime(2015, 1, 1),
            'jobType': 'Human'|'Automated',
            'evaluationTaskTypes': [
                'Summarization'|'Classification'|'QuestionAndAnswer'|'Generation'|'Custom',
            ],
            'modelIdentifiers': [
                'string',
            ]
        },
    ]
}

Response Structure

  • (dict) --

    • nextToken (string) --

      Continuation token from the previous response, for Amazon Bedrock to list the next set of results.

    • jobSummaries (list) --

      A summary of the model evaluation jobs.

      • (dict) --

        A summary of the model evaluation job.

        • jobArn (string) --

          The Amazon Resource Name (ARN) of the model evaluation job.

        • jobName (string) --

          The name of the model evaluation job.

        • status (string) --

          The current status of the model evaluation job.

        • creationTime (datetime) --

          When the model evaluation job was created.

        • jobType (string) --

          The type, either human or automatic, of model evaluation job.

        • evaluationTaskTypes (list) --

          What task type was used in the model evaluation job.

          • (string) --

        • modelIdentifiers (list) --

          The Amazon Resource Names (ARNs) of the model(s) used in the model evaluation job.

          • (string) --