2024/08/22 - Amazon Bedrock - 7 new 2 updated api methods
Changes Amazon Bedrock Evaluation BatchDeleteEvaluationJob API allows customers to delete evaluation jobs under terminated evaluation job statuses - Stopped, Failed, or Completed. Customers can submit a batch of 25 evaluation jobs to be deleted at once.
Retrieves the properties associated with import model job, including the status of the job. For more information, see Import a customized model in the Amazon Bedrock User Guide.
See also: AWS API Documentation
Request Syntax
client.get_model_import_job(
jobIdentifier='string'
)
string
[REQUIRED]
The identifier of the import job.
dict
Response Syntax
{
'jobArn': 'string',
'jobName': 'string',
'importedModelName': 'string',
'importedModelArn': 'string',
'roleArn': 'string',
'modelDataSource': {
's3DataSource': {
's3Uri': 'string'
}
},
'status': 'InProgress'|'Completed'|'Failed',
'failureMessage': 'string',
'creationTime': datetime(2015, 1, 1),
'lastModifiedTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1),
'vpcConfig': {
'subnetIds': [
'string',
],
'securityGroupIds': [
'string',
]
},
'importedModelKmsKeyArn': 'string'
}
Response Structure
(dict) --
jobArn (string) --
The Amazon Resource Name (ARN) of the import job.
jobName (string) --
The name of the import job.
importedModelName (string) --
The name of the imported model.
importedModelArn (string) --
The Amazon Resource Name (ARN) of the imported model.
roleArn (string) --
The Amazon Resource Name (ARN) of the IAM role associated with this job.
modelDataSource (dict) --
The data source for the imported model.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: s3DataSource. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
s3DataSource (dict) --
The Amazon S3 data source of the imported model.
s3Uri (string) --
The URI of the Amazon S3 data source.
status (string) --
The status of the job. A successful job transitions from in-progress to completed when the imported model is ready to use. If the job failed, the failure message contains information about why the job failed.
failureMessage (string) --
Information about why the import job failed.
creationTime (datetime) --
The time the resource was created.
lastModifiedTime (datetime) --
Time the resource was last modified.
endTime (datetime) --
Time that the resource transitioned to terminal state.
vpcConfig (dict) --
The Virtual Private Cloud (VPC) configuration of the import model job.
subnetIds (list) --
VPC configuration subnets.
(string) --
securityGroupIds (list) --
VPC configuration security group Ids.
(string) --
importedModelKmsKeyArn (string) --
The imported model is encrypted at rest using this key.
Returns a list of import jobs you've submitted. You can filter the results to return based on one or more criteria. For more information, see Import a customized model in the Amazon Bedrock User Guide.
See also: AWS API Documentation
Request Syntax
client.list_model_import_jobs(
creationTimeAfter=datetime(2015, 1, 1),
creationTimeBefore=datetime(2015, 1, 1),
statusEquals='InProgress'|'Completed'|'Failed',
nameContains='string',
maxResults=123,
nextToken='string',
sortBy='CreationTime',
sortOrder='Ascending'|'Descending'
)
datetime
Return import jobs that were created after the specified time.
datetime
Return import jobs that were created before the specified time.
string
Return imported jobs with the specified status.
string
Return imported jobs only if the job name contains these characters.
integer
The maximum number of results to return in the response. If the total number of results is greater than this value, use the token returned in the response in the nextToken field when making another request to return the next batch of results.
string
If the total number of results is greater than the maxResults value provided in the request, enter the token returned in the nextToken field in the response in this field to return the next batch of results.
string
The field to sort by in the returned list of imported jobs.
string
Specifies whether to sort the results in ascending or descending order.
dict
Response Syntax
{
'nextToken': 'string',
'modelImportJobSummaries': [
{
'jobArn': 'string',
'jobName': 'string',
'status': 'InProgress'|'Completed'|'Failed',
'lastModifiedTime': datetime(2015, 1, 1),
'creationTime': datetime(2015, 1, 1),
'endTime': datetime(2015, 1, 1),
'importedModelArn': 'string',
'importedModelName': 'string'
},
]
}
Response Structure
(dict) --
nextToken (string) --
If the total number of results is greater than the maxResults value provided in the request, enter the token returned in the nextToken field in the response in this field to return the next batch of results.
modelImportJobSummaries (list) --
Import job summaries.
(dict) --
Information about the import job.
jobArn (string) --
The Amazon Resource Name (ARN) of the import job.
jobName (string) --
The name of the import job.
status (string) --
The status of the imported job.
lastModifiedTime (datetime) --
The time when the import job was last modified.
creationTime (datetime) --
The time import job was created.
endTime (datetime) --
The time when import job ended.
importedModelArn (string) --
The Amazon resource Name (ARN) of the imported model.
importedModelName (string) --
The name of the imported model.
Creates a batch deletion job. A model evaluation job can only be deleted if it has following status FAILED , COMPLETED , and STOPPED . You can request up to 25 model evaluation jobs be deleted in a single request.
See also: AWS API Documentation
Request Syntax
client.batch_delete_evaluation_job(
jobIdentifiers=[
'string',
]
)
list
[REQUIRED]
An array of model evaluation job ARNs to be deleted.
(string) --
dict
Response Syntax
{
'errors': [
{
'jobIdentifier': 'string',
'code': 'string',
'message': 'string'
},
],
'evaluationJobs': [
{
'jobIdentifier': 'string',
'jobStatus': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting'
},
]
}
Response Structure
(dict) --
errors (list) --
A JSON object containing the HTTP status codes and the ARNs of model evaluation jobs that failed to be deleted.
(dict) --
A JSON array that provides the status of the model evaluation jobs being deleted.
jobIdentifier (string) --
The ARN of the model evaluation job being deleted.
code (string) --
A HTTP status code of the model evaluation job being deleted.
message (string) --
A status message about the model evaluation job deletion.
evaluationJobs (list) --
The list of model evaluation jobs to be deleted.
(dict) --
An array of model evaluation jobs to be deleted, and their associated statuses.
jobIdentifier (string) --
The ARN of model evaluation job to be deleted.
jobStatus (string) --
The status of the job's deletion.
Gets properties associated with a customized model you imported.
See also: AWS API Documentation
Request Syntax
client.get_imported_model(
modelIdentifier='string'
)
string
[REQUIRED]
Name or Amazon Resource Name (ARN) of the imported model.
dict
Response Syntax
{
'modelArn': 'string',
'modelName': 'string',
'jobName': 'string',
'jobArn': 'string',
'modelDataSource': {
's3DataSource': {
's3Uri': 'string'
}
},
'creationTime': datetime(2015, 1, 1),
'modelArchitecture': 'string',
'modelKmsKeyArn': 'string'
}
Response Structure
(dict) --
modelArn (string) --
The Amazon Resource Name (ARN) associated with this imported model.
modelName (string) --
The name of the imported model.
jobName (string) --
Job name associated with the imported model.
jobArn (string) --
Job Amazon Resource Name (ARN) associated with the imported model.
modelDataSource (dict) --
The data source for this imported model.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: s3DataSource. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
s3DataSource (dict) --
The Amazon S3 data source of the imported model.
s3Uri (string) --
The URI of the Amazon S3 data source.
creationTime (datetime) --
Creation time of the imported model.
modelArchitecture (string) --
The architecture of the imported model.
modelKmsKeyArn (string) --
The imported model is encrypted at rest using this key.
Creates a model import job to import model that you have customized in other environments, such as Amazon SageMaker. For more information, see Import a customized model
See also: AWS API Documentation
Request Syntax
client.create_model_import_job(
jobName='string',
importedModelName='string',
roleArn='string',
modelDataSource={
's3DataSource': {
's3Uri': 'string'
}
},
jobTags=[
{
'key': 'string',
'value': 'string'
},
],
importedModelTags=[
{
'key': 'string',
'value': 'string'
},
],
clientRequestToken='string',
vpcConfig={
'subnetIds': [
'string',
],
'securityGroupIds': [
'string',
]
},
importedModelKmsKeyId='string'
)
string
[REQUIRED]
The name of the import job.
string
[REQUIRED]
The name of the imported model.
string
[REQUIRED]
The Amazon Resource Name (ARN) of the model import job.
dict
[REQUIRED]
The data source for the imported model.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set: s3DataSource.
s3DataSource (dict) --
The Amazon S3 data source of the imported model.
s3Uri (string) -- [REQUIRED]
The URI of the Amazon S3 data source.
list
Tags to attach to this import job.
(dict) --
Definition of the key/value pair for a tag.
key (string) -- [REQUIRED]
Key for the tag.
value (string) -- [REQUIRED]
Value for the tag.
list
Tags to attach to the imported model.
(dict) --
Definition of the key/value pair for a tag.
key (string) -- [REQUIRED]
Key for the tag.
value (string) -- [REQUIRED]
Value for the tag.
string
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error. For more information, see Ensuring idempotency.
dict
VPC configuration parameters for the private Virtual Private Cloud (VPC) that contains the resources you are using for the import job.
subnetIds (list) -- [REQUIRED]
VPC configuration subnets.
(string) --
securityGroupIds (list) -- [REQUIRED]
VPC configuration security group Ids.
(string) --
string
The imported model is encrypted at rest using this key.
dict
Response Syntax
{
'jobArn': 'string'
}
Response Structure
(dict) --
jobArn (string) --
The Amazon Resource Name (ARN) of the model import job.
Deletes a custom model that you imported earlier. For more information, see Import a customized model in the Amazon Bedrock User Guide.
See also: AWS API Documentation
Request Syntax
client.delete_imported_model(
modelIdentifier='string'
)
string
[REQUIRED]
Name of the imported model to delete.
dict
Response Syntax
{}
Response Structure
(dict) --
Returns a list of models you've imported. You can filter the results to return based on one or more criteria. For more information, see Import a customized model in the Amazon Bedrock User Guide.
See also: AWS API Documentation
Request Syntax
client.list_imported_models(
creationTimeBefore=datetime(2015, 1, 1),
creationTimeAfter=datetime(2015, 1, 1),
nameContains='string',
maxResults=123,
nextToken='string',
sortBy='CreationTime',
sortOrder='Ascending'|'Descending'
)
datetime
Return imported models that created before the specified time.
datetime
Return imported models that were created after the specified time.
string
Return imported models only if the model name contains these characters.
integer
The maximum number of results to return in the response. If the total number of results is greater than this value, use the token returned in the response in the nextToken field when making another request to return the next batch of results.
string
If the total number of results is greater than the maxResults value provided in the request, enter the token returned in the nextToken field in the response in this field to return the next batch of results.
string
The field to sort by in the returned list of imported models.
string
Specifies whetehr to sort the results in ascending or descending order.
dict
Response Syntax
{
'nextToken': 'string',
'modelSummaries': [
{
'modelArn': 'string',
'modelName': 'string',
'creationTime': datetime(2015, 1, 1)
},
]
}
Response Structure
(dict) --
nextToken (string) --
If the total number of results is greater than the maxResults value provided in the request, use this token when making another request in the nextToken field to return the next batch of results.
modelSummaries (list) --
Model summaries.
(dict) --
Information about tne imported model.
modelArn (string) --
The Amazon Resource Name (ARN) of the imported model.
modelName (string) --
Name of the imported model.
creationTime (datetime) --
Creation time of the imported model.
{'status': {'Deleting'}}
Retrieves the properties associated with a model evaluation job, including the status of the job. For more information, see Model evaluation.
See also: AWS API Documentation
Request Syntax
client.get_evaluation_job(
jobIdentifier='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) of the model evaluation job.
dict
Response Syntax
{
'jobName': 'string',
'status': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting',
'jobArn': 'string',
'jobDescription': 'string',
'roleArn': 'string',
'customerEncryptionKeyId': 'string',
'jobType': 'Human'|'Automated',
'evaluationConfig': {
'automated': {
'datasetMetricConfigs': [
{
'taskType': 'Summarization'|'Classification'|'QuestionAndAnswer'|'Generation'|'Custom',
'dataset': {
'name': 'string',
'datasetLocation': {
's3Uri': 'string'
}
},
'metricNames': [
'string',
]
},
]
},
'human': {
'humanWorkflowConfig': {
'flowDefinitionArn': 'string',
'instructions': 'string'
},
'customMetrics': [
{
'name': 'string',
'description': 'string',
'ratingMethod': 'string'
},
],
'datasetMetricConfigs': [
{
'taskType': 'Summarization'|'Classification'|'QuestionAndAnswer'|'Generation'|'Custom',
'dataset': {
'name': 'string',
'datasetLocation': {
's3Uri': 'string'
}
},
'metricNames': [
'string',
]
},
]
}
},
'inferenceConfig': {
'models': [
{
'bedrockModel': {
'modelIdentifier': 'string',
'inferenceParams': 'string'
}
},
]
},
'outputDataConfig': {
's3Uri': 'string'
},
'creationTime': datetime(2015, 1, 1),
'lastModifiedTime': datetime(2015, 1, 1),
'failureMessages': [
'string',
]
}
Response Structure
(dict) --
jobName (string) --
The name of the model evaluation job.
status (string) --
The status of the model evaluation job.
jobArn (string) --
The Amazon Resource Name (ARN) of the model evaluation job.
jobDescription (string) --
The description of the model evaluation job.
roleArn (string) --
The Amazon Resource Name (ARN) of the IAM service role used in the model evaluation job.
customerEncryptionKeyId (string) --
The Amazon Resource Name (ARN) of the customer managed key specified when the model evaluation job was created.
jobType (string) --
The type of model evaluation job.
evaluationConfig (dict) --
Contains details about the type of model evaluation job, the metrics used, the task type selected, the datasets used, and any custom metrics you defined.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: automated, human. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
automated (dict) --
Used to specify an automated model evaluation job. See AutomatedEvaluationConfig to view the required parameters.
datasetMetricConfigs (list) --
Specifies the required elements for an automatic model evaluation job.
(dict) --
Defines the built-in prompt datasets, built-in metric names and custom metric names, and the task type.
taskType (string) --
The task type you want the model to carry out.
dataset (dict) --
Specifies the prompt dataset.
name (string) --
Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold , Builtin.BoolQ , Builtin.NaturalQuestions , Builtin.Gigaword , Builtin.RealToxicityPrompts , Builtin.TriviaQA , Builtin.T-Rex , Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2 .
datasetLocation (dict) --
For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: s3Uri. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
s3Uri (string) --
The S3 URI of the S3 bucket specified in the job.
metricNames (list) --
The names of the metrics used. For automated model evaluation jobs valid values are "Builtin.Accuracy" , "Builtin.Robustness" , and "Builtin.Toxicity" . In human-based model evaluation jobs the array of strings must match the name parameter specified in HumanEvaluationCustomMetric .
(string) --
human (dict) --
Used to specify a model evaluation job that uses human workers.See HumanEvaluationConfig to view the required parameters.
humanWorkflowConfig (dict) --
The parameters of the human workflow.
flowDefinitionArn (string) --
The Amazon Resource Number (ARN) for the flow definition
instructions (string) --
Instructions for the flow definition
customMetrics (list) --
A HumanEvaluationCustomMetric object. It contains the names the metrics, how the metrics are to be evaluated, an optional description.
(dict) --
In a model evaluation job that uses human workers you must define the name of the metric, and how you want that metric rated ratingMethod , and an optional description of the metric.
name (string) --
The name of the metric. Your human evaluators will see this name in the evaluation UI.
description (string) --
An optional description of the metric. Use this parameter to provide more details about the metric.
ratingMethod (string) --
Choose how you want your human workers to evaluation your model. Valid values for rating methods are ThumbsUpDown , IndividualLikertScale , ComparisonLikertScale , ComparisonChoice , and ComparisonRank
datasetMetricConfigs (list) --
Use to specify the metrics, task, and prompt dataset to be used in your model evaluation job.
(dict) --
Defines the built-in prompt datasets, built-in metric names and custom metric names, and the task type.
taskType (string) --
The task type you want the model to carry out.
dataset (dict) --
Specifies the prompt dataset.
name (string) --
Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold , Builtin.BoolQ , Builtin.NaturalQuestions , Builtin.Gigaword , Builtin.RealToxicityPrompts , Builtin.TriviaQA , Builtin.T-Rex , Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2 .
datasetLocation (dict) --
For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: s3Uri. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
s3Uri (string) --
The S3 URI of the S3 bucket specified in the job.
metricNames (list) --
The names of the metrics used. For automated model evaluation jobs valid values are "Builtin.Accuracy" , "Builtin.Robustness" , and "Builtin.Toxicity" . In human-based model evaluation jobs the array of strings must match the name parameter specified in HumanEvaluationCustomMetric .
(string) --
inferenceConfig (dict) --
Details about the models you specified in your model evaluation job.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: models. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
models (list) --
Used to specify the models.
(dict) --
Defines the models used in the model evaluation job.
Note
This is a Tagged Union structure. Only one of the following top level keys will be set: bedrockModel. If a client receives an unknown member it will set SDK_UNKNOWN_MEMBER as the top level key, which maps to the name or tag of the unknown member. The structure of SDK_UNKNOWN_MEMBER is as follows:
'SDK_UNKNOWN_MEMBER': {'name': 'UnknownMemberName'}
bedrockModel (dict) --
Defines the Amazon Bedrock model and inference parameters you want used.
modelIdentifier (string) --
The ARN of the Amazon Bedrock model specified.
inferenceParams (string) --
Each Amazon Bedrock support different inference parameters that change how the model behaves during inference.
outputDataConfig (dict) --
Amazon S3 location for where output data is saved.
s3Uri (string) --
The Amazon S3 URI where the results of model evaluation job are saved.
creationTime (datetime) --
When the model evaluation job was created.
lastModifiedTime (datetime) --
When the model evaluation job was last modified.
failureMessages (list) --
An array of strings the specify why the model evaluation job has failed.
(string) --
{'statusEquals': {'Deleting'}}
Response {'jobSummaries': {'status': {'Deleting'}}}
Lists model evaluation jobs.
See also: AWS API Documentation
Request Syntax
client.list_evaluation_jobs(
creationTimeAfter=datetime(2015, 1, 1),
creationTimeBefore=datetime(2015, 1, 1),
statusEquals='InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting',
nameContains='string',
maxResults=123,
nextToken='string',
sortBy='CreationTime',
sortOrder='Ascending'|'Descending'
)
datetime
A filter that includes model evaluation jobs created after the time specified.
datetime
A filter that includes model evaluation jobs created prior to the time specified.
string
Only return jobs where the status condition is met.
string
Query parameter string for model evaluation job names.
integer
The maximum number of results to return.
string
Continuation token from the previous response, for Amazon Bedrock to list the next set of results.
string
Allows you to sort model evaluation jobs by when they were created.
string
How you want the order of jobs sorted.
dict
Response Syntax
{
'nextToken': 'string',
'jobSummaries': [
{
'jobArn': 'string',
'jobName': 'string',
'status': 'InProgress'|'Completed'|'Failed'|'Stopping'|'Stopped'|'Deleting',
'creationTime': datetime(2015, 1, 1),
'jobType': 'Human'|'Automated',
'evaluationTaskTypes': [
'Summarization'|'Classification'|'QuestionAndAnswer'|'Generation'|'Custom',
],
'modelIdentifiers': [
'string',
]
},
]
}
Response Structure
(dict) --
nextToken (string) --
Continuation token from the previous response, for Amazon Bedrock to list the next set of results.
jobSummaries (list) --
A summary of the model evaluation jobs.
(dict) --
A summary of the model evaluation job.
jobArn (string) --
The Amazon Resource Name (ARN) of the model evaluation job.
jobName (string) --
The name of the model evaluation job.
status (string) --
The current status of the model evaluation job.
creationTime (datetime) --
When the model evaluation job was created.
jobType (string) --
The type, either human or automatic, of model evaluation job.
evaluationTaskTypes (list) --
What task type was used in the model evaluation job.
(string) --
modelIdentifiers (list) --
The Amazon Resource Names (ARNs) of the model(s) used in the model evaluation job.
(string) --