2023/04/19 - Amazon Comprehend - 4 updated api methods
Changes This release supports native document models for custom classification, in addition to plain-text models. You train native document models using documents (PDF, Word, images) in their native format.
{'Warnings': [{'Page': 'integer',
'WarnCode': 'INFERENCING_PLAINTEXT_WITH_NATIVE_TRAINED_MODEL | '
'INFERENCING_NATIVE_DOCUMENT_WITH_PLAINTEXT_TRAINED_MODEL',
'WarnMessage': 'string'}]}
Creates a new document classification request to analyze a single document in real-time, using a previously created and trained custom model and an endpoint.
You can input plain text or you can upload a single-page input document (text, PDF, Word, or image).
If the system detects errors while processing a page in the input document, the API response includes an entry in Errors that describes the errors.
If the system detects a document-level error in your input document, the API returns an InvalidRequestException error response. For details about this exception, see Errors in semi-structured documents in the Comprehend Developer Guide.
See also: AWS API Documentation
Request Syntax
client.classify_document(
Text='string',
EndpointArn='string',
Bytes=b'bytes',
DocumentReaderConfig={
'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT'|'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT'|'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': [
'TABLES'|'FORMS',
]
}
)
string
The document text to be analyzed. If you enter text using this parameter, do not use the Bytes parameter.
string
[REQUIRED]
The Amazon Resource Number (ARN) of the endpoint. For information about endpoints, see Managing endpoints.
bytes
Use the Bytes parameter to input a text, PDF, Word or image file. You can also use the Bytes parameter to input an Amazon Textract DetectDocumentText or AnalyzeDocument output file.
Provide the input document as a sequence of base64-encoded bytes. If your code uses an Amazon Web Services SDK to classify documents, the SDK may encode the document file bytes for you.
The maximum length of this field depends on the input document type. For details, see Inputs for real-time custom analysis in the Comprehend Developer Guide.
If you use the Bytes parameter, do not use the Text parameter.
dict
Provides configuration parameters to override the default actions for extracting text from PDF documents and image files.
DocumentReadAction (string) -- [REQUIRED]
This field defines the Amazon Textract API operation that Amazon Comprehend uses to extract text from PDF files and image files. Enter one of the following values:
TEXTRACT_DETECT_DOCUMENT_TEXT - The Amazon Comprehend service uses the DetectDocumentText API operation.
TEXTRACT_ANALYZE_DOCUMENT - The Amazon Comprehend service uses the AnalyzeDocument API operation.
DocumentReadMode (string) --
Determines the text extraction actions for PDF files. Enter one of the following values:
SERVICE_DEFAULT - use the Amazon Comprehend service defaults for PDF files.
FORCE_DOCUMENT_READ_ACTION - Amazon Comprehend uses the Textract API specified by DocumentReadAction for all PDF files, including digital PDF files.
FeatureTypes (list) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns information about any tables that are detected in the input document.
FORMS - Returns information and the data from any forms that are detected in the input document.
(string) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns additional information about any tables that are detected in the input document.
FORMS - Returns additional information about any forms that are detected in the input document.
dict
Response Syntax
{
'Classes': [
{
'Name': 'string',
'Score': ...,
'Page': 123
},
],
'Labels': [
{
'Name': 'string',
'Score': ...,
'Page': 123
},
],
'DocumentMetadata': {
'Pages': 123,
'ExtractedCharacters': [
{
'Page': 123,
'Count': 123
},
]
},
'DocumentType': [
{
'Page': 123,
'Type': 'NATIVE_PDF'|'SCANNED_PDF'|'MS_WORD'|'IMAGE'|'PLAIN_TEXT'|'TEXTRACT_DETECT_DOCUMENT_TEXT_JSON'|'TEXTRACT_ANALYZE_DOCUMENT_JSON'
},
],
'Errors': [
{
'Page': 123,
'ErrorCode': 'TEXTRACT_BAD_PAGE'|'TEXTRACT_PROVISIONED_THROUGHPUT_EXCEEDED'|'PAGE_CHARACTERS_EXCEEDED'|'PAGE_SIZE_EXCEEDED'|'INTERNAL_SERVER_ERROR',
'ErrorMessage': 'string'
},
],
'Warnings': [
{
'Page': 123,
'WarnCode': 'INFERENCING_PLAINTEXT_WITH_NATIVE_TRAINED_MODEL'|'INFERENCING_NATIVE_DOCUMENT_WITH_PLAINTEXT_TRAINED_MODEL',
'WarnMessage': 'string'
},
]
}
Response Structure
(dict) --
Classes (list) --
The classes used by the document being analyzed. These are used for multi-class trained models. Individual classes are mutually exclusive and each document is expected to have only a single class assigned to it. For example, an animal can be a dog or a cat, but not both at the same time.
(dict) --
Specifies the class that categorizes the document being analyzed
Name (string) --
The name of the class.
Score (float) --
The confidence score that Amazon Comprehend has this class correctly attributed.
Page (integer) --
Page number in the input document. This field is present in the response only if your request includes the Byte parameter.
Labels (list) --
The labels used the document being analyzed. These are used for multi-label trained models. Individual labels represent different categories that are related in some manner and are not mutually exclusive. For example, a movie can be just an action movie, or it can be an action movie, a science fiction movie, and a comedy, all at the same time.
(dict) --
Specifies one of the label or labels that categorize the document being analyzed.
Name (string) --
The name of the label.
Score (float) --
The confidence score that Amazon Comprehend has this label correctly attributed.
Page (integer) --
Page number where the label occurs. This field is present in the response only if your request includes the Byte parameter.
DocumentMetadata (dict) --
Extraction information about the document. This field is present in the response only if your request includes the Byte parameter.
Pages (integer) --
Number of pages in the document.
ExtractedCharacters (list) --
List of pages in the document, with the number of characters extracted from each page.
(dict) --
Array of the number of characters extracted from each page.
Page (integer) --
Page number.
Count (integer) --
Number of characters extracted from each page.
DocumentType (list) --
The document type for each page in the input document. This field is present in the response only if your request includes the Byte parameter.
(dict) --
Document type for each page in the document.
Page (integer) --
Page number.
Type (string) --
Document type.
Errors (list) --
Page-level errors that the system detected while processing the input document. The field is empty if the system encountered no errors.
(dict) --
Text extraction encountered one or more page-level errors in the input document.
The ErrorCode contains one of the following values:
TEXTRACT_BAD_PAGE - Amazon Textract cannot read the page. For more information about page limits in Amazon Textract, see Page Quotas in Amazon Textract.
TEXTRACT_PROVISIONED_THROUGHPUT_EXCEEDED - The number of requests exceeded your throughput limit. For more information about throughput quotas in Amazon Textract, see Default quotas in Amazon Textract.
PAGE_CHARACTERS_EXCEEDED - Too many text characters on the page (10,000 characters maximum).
PAGE_SIZE_EXCEEDED - The maximum page size is 10 MB.
INTERNAL_SERVER_ERROR - The request encountered a service issue. Try the API request again.
Page (integer) --
Page number where the error occurred.
ErrorCode (string) --
Error code for the cause of the error.
ErrorMessage (string) --
Text message explaining the reason for the error.
Warnings (list) --
Warnings detected while processing the input document. The response includes a warning if there is a mismatch between the input document type and the model type associated with the endpoint that you specified. The response can also include warnings for individual pages that have a mismatch.
The field is empty if the system generated no warnings.
(dict) --
The system identified one of the following warnings while processing the input document:
The document to classify is plain text, but the classifier is a native model.
The document to classify is semi-structured, but the classifier is a plain-text model.
Page (integer) --
Page number in the input document.
WarnCode (string) --
The type of warning.
WarnMessage (string) --
Text message associated with the warning.
{'InputDataConfig': {'DocumentReaderConfig': {'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT '
'| '
'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT '
'| '
'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': ['TABLES | '
'FORMS']},
'DocumentType': 'PLAIN_TEXT_DOCUMENT | '
'SEMI_STRUCTURED_DOCUMENT',
'Documents': {'S3Uri': 'string', 'TestS3Uri': 'string'}}}
Creates a new document classifier that you can use to categorize documents. To create a classifier, you provide a set of training documents that are labeled with the categories that you want to use. For more information, see Training classifier models in the Comprehend Developer Guide.
See also: AWS API Documentation
Request Syntax
client.create_document_classifier(
DocumentClassifierName='string',
VersionName='string',
DataAccessRoleArn='string',
Tags=[
{
'Key': 'string',
'Value': 'string'
},
],
InputDataConfig={
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'S3Uri': 'string',
'TestS3Uri': 'string',
'LabelDelimiter': 'string',
'AugmentedManifests': [
{
'S3Uri': 'string',
'Split': 'TRAIN'|'TEST',
'AttributeNames': [
'string',
],
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
],
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT',
'Documents': {
'S3Uri': 'string',
'TestS3Uri': 'string'
},
'DocumentReaderConfig': {
'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT'|'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT'|'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': [
'TABLES'|'FORMS',
]
}
},
OutputDataConfig={
'S3Uri': 'string',
'KmsKeyId': 'string',
'FlywheelStatsS3Prefix': 'string'
},
ClientRequestToken='string',
LanguageCode='en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
VolumeKmsKeyId='string',
VpcConfig={
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
Mode='MULTI_CLASS'|'MULTI_LABEL',
ModelKmsKeyId='string',
ModelPolicy='string'
)
string
[REQUIRED]
The name of the document classifier.
string
The version name given to the newly created classifier. Version names can have a maximum of 256 characters. Alphanumeric characters, hyphens (-) and underscores (_) are allowed. The version name must be unique among all models with the same classifier name in the Amazon Web Services account/Amazon Web Services Region.
string
[REQUIRED]
The Amazon Resource Name (ARN) of the IAM role that grants Amazon Comprehend read access to your input data.
list
Tags to associate with the document classifier. A tag is a key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with "Sales" as the key might be added to a resource to indicate its use by the sales department.
(dict) --
A key-value pair that adds as a metadata to a resource used by Amazon Comprehend. For example, a tag with the key-value pair ‘Department’:’Sales’ might be added to a resource to indicate its use by a particular department.
Key (string) -- [REQUIRED]
The initial part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the key portion of the pair, with multiple possible values such as “sales,” “legal,” and “administration.”
Value (string) --
The second part of a key-value pair that forms a tag associated with a given resource. For instance, if you want to show which resources are used by which departments, you might use “Department” as the initial (key) portion of the pair, with a value of “sales” to indicate the sales department.
dict
[REQUIRED]
Specifies the format and location of the input data for the job.
DataFormat (string) --
The format of your training data:
COMPREHEND_CSV : A two-column CSV file, where labels are provided in the first column, and documents are provided in the second. If you use this value, you must provide the S3Uri parameter in your request.
AUGMENTED_MANIFEST : A labeled dataset that is produced by Amazon SageMaker Ground Truth. This file is in JSON lines format. Each line is a complete JSON object that contains a training document and its associated labels. If you use this value, you must provide the AugmentedManifests parameter in your request.
If you don't specify a value, Amazon Comprehend uses COMPREHEND_CSV as the default.
S3Uri (string) --
The Amazon S3 URI for the input data. The S3 bucket must be in the same Region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of input files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
TestS3Uri (string) --
This specifies the Amazon S3 location where the test annotations for an entity recognizer are located. The URI must be in the same Amazon Web Services Region as the API endpoint that you are calling.
LabelDelimiter (string) --
Indicates the delimiter used to separate each label for training a multi-label classifier. The default delimiter between labels is a pipe (|). You can use a different character as a delimiter (if it's an allowed character) by specifying it under Delimiter for labels. If the training documents use a delimiter other than the default or the delimiter you specify, the labels on that line will be combined to make a single unique label, such as LABELLABELLABEL.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
This parameter is required if you set DataFormat to AUGMENTED_MANIFEST .
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
S3Uri (string) -- [REQUIRED]
The Amazon S3 location of the augmented manifest file.
Split (string) --
The purpose of the data you've provided in the augmented manifest. You can either train or test this data. If you don't specify, the default is train.
TRAIN - all of the documents in the manifest will be used for training. If no test documents are provided, Amazon Comprehend will automatically reserve a portion of the training documents for testing.
TEST - all of the documents in the manifest will be used for testing.
AttributeNames (list) -- [REQUIRED]
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. PlainTextDocument or SemiStructuredDocument. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
SEMI_STRUCTURED_DOCUMENT A document type with positional and structural context, like a PDF. For training with Amazon Comprehend, only PDFs are supported. For inference, Amazon Comprehend support PDFs, DOCX and TXT.
DocumentType (string) --
The type of input documents for training the model. Provide plain-text documents to create a plain-text model, and provide semi-structured documents to create a native model.
Documents (dict) --
The S3 location of the training documents. This parameter is required in a request to create a native classifier model.
S3Uri (string) -- [REQUIRED]
The S3 URI location of the training documents specified in the S3Uri CSV file.
TestS3Uri (string) --
The S3 URI location of the test documents included in the TestS3Uri CSV file. This field is not required if you do not specify a test CSV file.
DocumentReaderConfig (dict) --
Provides configuration parameters to override the default actions for extracting text from PDF documents and image files.
By default, Amazon Comprehend performs the following actions to extract text from files, based on the input file type:
Word files - Amazon Comprehend parser extracts the text.
Digital PDF files - Amazon Comprehend parser extracts the text.
Image files and scanned PDF files - Amazon Comprehend uses the Amazon Textract DetectDocumentText API to extract the text.
DocumentReaderConfig does not apply to plain text files or Word files.
For image files and PDF documents, you can override these default actions using the fields listed below. For more information, see Setting text extraction options in the Comprehend Developer Guide.
DocumentReadAction (string) -- [REQUIRED]
This field defines the Amazon Textract API operation that Amazon Comprehend uses to extract text from PDF files and image files. Enter one of the following values:
TEXTRACT_DETECT_DOCUMENT_TEXT - The Amazon Comprehend service uses the DetectDocumentText API operation.
TEXTRACT_ANALYZE_DOCUMENT - The Amazon Comprehend service uses the AnalyzeDocument API operation.
DocumentReadMode (string) --
Determines the text extraction actions for PDF files. Enter one of the following values:
SERVICE_DEFAULT - use the Amazon Comprehend service defaults for PDF files.
FORCE_DOCUMENT_READ_ACTION - Amazon Comprehend uses the Textract API specified by DocumentReadAction for all PDF files, including digital PDF files.
FeatureTypes (list) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns information about any tables that are detected in the input document.
FORMS - Returns information and the data from any forms that are detected in the input document.
(string) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns additional information about any tables that are detected in the input document.
FORMS - Returns additional information about any forms that are detected in the input document.
dict
Specifies the location for the output files from a custom classifier job. This parameter is required for a request that creates a native classifier model.
S3Uri (string) --
When you use the OutputDataConfig object while creating a custom classifier, you specify the Amazon S3 location where you want to write the confusion matrix and other output files. The URI must be in the same Region as the API endpoint that you are calling. The location is used as the prefix for the actual location of this output file.
When the custom classifier job is finished, the service creates the output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the confusion matrix.
KmsKeyId (string) --
ID for the Amazon Web Services Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
FlywheelStatsS3Prefix (string) --
The Amazon S3 prefix for the data lake location of the flywheel statistics.
string
A unique identifier for the request. If you don't set the client request token, Amazon Comprehend generates one.
This field is autopopulated if not provided.
string
[REQUIRED]
The language of the input documents. You can specify any of the languages supported by Amazon Comprehend. All documents must be in the same language.
string
ID for the Amazon Web Services Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
dict
Configuration parameters for an optional private Virtual Private Cloud (VPC) containing the resources you are using for your custom classifier. For more information, see Amazon VPC.
SecurityGroupIds (list) -- [REQUIRED]
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC.
(string) --
Subnets (list) -- [REQUIRED]
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s Region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets.
(string) --
string
Indicates the mode in which the classifier will be trained. The classifier can be trained in multi-class mode, which identifies one and only one class for each document, or multi-label mode, which identifies one or more labels for each document. In multi-label mode, multiple labels for an individual document are separated by a delimiter. The default delimiter between labels is a pipe (|).
string
ID for the KMS key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
string
The resource-based policy to attach to your custom document classifier model. You can use this policy to allow another Amazon Web Services account to import your custom model.
Provide your policy as a JSON body that you enter as a UTF-8 encoded string without line breaks. To provide valid JSON, enclose the attribute names and values in double quotes. If the JSON body is also enclosed in double quotes, then you must escape the double quotes that are inside the policy:
"{\"attribute\": \"value\", \"attribute\": [\"value\"]}"
To avoid escaping quotes, you can use single quotes to enclose the policy and double quotes to enclose the JSON names and values:
'{"attribute": "value", "attribute": ["value"]}'
dict
Response Syntax
{
'DocumentClassifierArn': 'string'
}
Response Structure
(dict) --
DocumentClassifierArn (string) --
The Amazon Resource Name (ARN) that identifies the document classifier.
{'DocumentClassifierProperties': {'InputDataConfig': {'DocumentReaderConfig': {'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT '
'| '
'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT '
'| '
'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': ['TABLES '
'| '
'FORMS']},
'DocumentType': 'PLAIN_TEXT_DOCUMENT '
'| '
'SEMI_STRUCTURED_DOCUMENT',
'Documents': {'S3Uri': 'string',
'TestS3Uri': 'string'}}}}
Gets the properties associated with a document classifier.
See also: AWS API Documentation
Request Syntax
client.describe_document_classifier(
DocumentClassifierArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) that identifies the document classifier. The CreateDocumentClassifier operation returns this identifier in its response.
dict
Response Syntax
{
'DocumentClassifierProperties': {
'DocumentClassifierArn': 'string',
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED'|'TRAINED_WITH_WARNING',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'InputDataConfig': {
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'S3Uri': 'string',
'TestS3Uri': 'string',
'LabelDelimiter': 'string',
'AugmentedManifests': [
{
'S3Uri': 'string',
'Split': 'TRAIN'|'TEST',
'AttributeNames': [
'string',
],
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
],
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT',
'Documents': {
'S3Uri': 'string',
'TestS3Uri': 'string'
},
'DocumentReaderConfig': {
'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT'|'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT'|'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': [
'TABLES'|'FORMS',
]
}
},
'OutputDataConfig': {
'S3Uri': 'string',
'KmsKeyId': 'string',
'FlywheelStatsS3Prefix': 'string'
},
'ClassifierMetadata': {
'NumberOfLabels': 123,
'NumberOfTrainedDocuments': 123,
'NumberOfTestDocuments': 123,
'EvaluationMetrics': {
'Accuracy': 123.0,
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0,
'MicroPrecision': 123.0,
'MicroRecall': 123.0,
'MicroF1Score': 123.0,
'HammingLoss': 123.0
}
},
'DataAccessRoleArn': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'Mode': 'MULTI_CLASS'|'MULTI_LABEL',
'ModelKmsKeyId': 'string',
'VersionName': 'string',
'SourceModelArn': 'string',
'FlywheelArn': 'string'
}
}
Response Structure
(dict) --
DocumentClassifierProperties (dict) --
An object that contains the properties associated with a document classifier.
DocumentClassifierArn (string) --
The Amazon Resource Name (ARN) that identifies the document classifier.
LanguageCode (string) --
The language code for the language of the documents that the classifier was trained on.
Status (string) --
The status of the document classifier. If the status is TRAINED the classifier is ready to use. If the status is TRAINED_WITH_WARNINGS the classifier training succeeded, but you should review the warnings returned in the CreateDocumentClassifier response.
If the status is FAILED you can see additional information about why the classifier wasn't trained in the Message field.
Message (string) --
Additional information about the status of the classifier.
SubmitTime (datetime) --
The time that the document classifier was submitted for training.
EndTime (datetime) --
The time that training the document classifier completed.
TrainingStartTime (datetime) --
Indicates the time when the training starts on documentation classifiers. You are billed for the time interval between this time and the value of TrainingEndTime.
TrainingEndTime (datetime) --
The time that training of the document classifier was completed. Indicates the time when the training completes on documentation classifiers. You are billed for the time interval between this time and the value of TrainingStartTime.
InputDataConfig (dict) --
The input data configuration that you supplied when you created the document classifier for training.
DataFormat (string) --
The format of your training data:
COMPREHEND_CSV : A two-column CSV file, where labels are provided in the first column, and documents are provided in the second. If you use this value, you must provide the S3Uri parameter in your request.
AUGMENTED_MANIFEST : A labeled dataset that is produced by Amazon SageMaker Ground Truth. This file is in JSON lines format. Each line is a complete JSON object that contains a training document and its associated labels. If you use this value, you must provide the AugmentedManifests parameter in your request.
If you don't specify a value, Amazon Comprehend uses COMPREHEND_CSV as the default.
S3Uri (string) --
The Amazon S3 URI for the input data. The S3 bucket must be in the same Region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of input files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
TestS3Uri (string) --
This specifies the Amazon S3 location where the test annotations for an entity recognizer are located. The URI must be in the same Amazon Web Services Region as the API endpoint that you are calling.
LabelDelimiter (string) --
Indicates the delimiter used to separate each label for training a multi-label classifier. The default delimiter between labels is a pipe (|). You can use a different character as a delimiter (if it's an allowed character) by specifying it under Delimiter for labels. If the training documents use a delimiter other than the default or the delimiter you specify, the labels on that line will be combined to make a single unique label, such as LABELLABELLABEL.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
This parameter is required if you set DataFormat to AUGMENTED_MANIFEST .
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
S3Uri (string) --
The Amazon S3 location of the augmented manifest file.
Split (string) --
The purpose of the data you've provided in the augmented manifest. You can either train or test this data. If you don't specify, the default is train.
TRAIN - all of the documents in the manifest will be used for training. If no test documents are provided, Amazon Comprehend will automatically reserve a portion of the training documents for testing.
TEST - all of the documents in the manifest will be used for testing.
AttributeNames (list) --
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. PlainTextDocument or SemiStructuredDocument. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
SEMI_STRUCTURED_DOCUMENT A document type with positional and structural context, like a PDF. For training with Amazon Comprehend, only PDFs are supported. For inference, Amazon Comprehend support PDFs, DOCX and TXT.
DocumentType (string) --
The type of input documents for training the model. Provide plain-text documents to create a plain-text model, and provide semi-structured documents to create a native model.
Documents (dict) --
The S3 location of the training documents. This parameter is required in a request to create a native classifier model.
S3Uri (string) --
The S3 URI location of the training documents specified in the S3Uri CSV file.
TestS3Uri (string) --
The S3 URI location of the test documents included in the TestS3Uri CSV file. This field is not required if you do not specify a test CSV file.
DocumentReaderConfig (dict) --
Provides configuration parameters to override the default actions for extracting text from PDF documents and image files.
By default, Amazon Comprehend performs the following actions to extract text from files, based on the input file type:
Word files - Amazon Comprehend parser extracts the text.
Digital PDF files - Amazon Comprehend parser extracts the text.
Image files and scanned PDF files - Amazon Comprehend uses the Amazon Textract DetectDocumentText API to extract the text.
DocumentReaderConfig does not apply to plain text files or Word files.
For image files and PDF documents, you can override these default actions using the fields listed below. For more information, see Setting text extraction options in the Comprehend Developer Guide.
DocumentReadAction (string) --
This field defines the Amazon Textract API operation that Amazon Comprehend uses to extract text from PDF files and image files. Enter one of the following values:
TEXTRACT_DETECT_DOCUMENT_TEXT - The Amazon Comprehend service uses the DetectDocumentText API operation.
TEXTRACT_ANALYZE_DOCUMENT - The Amazon Comprehend service uses the AnalyzeDocument API operation.
DocumentReadMode (string) --
Determines the text extraction actions for PDF files. Enter one of the following values:
SERVICE_DEFAULT - use the Amazon Comprehend service defaults for PDF files.
FORCE_DOCUMENT_READ_ACTION - Amazon Comprehend uses the Textract API specified by DocumentReadAction for all PDF files, including digital PDF files.
FeatureTypes (list) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns information about any tables that are detected in the input document.
FORMS - Returns information and the data from any forms that are detected in the input document.
(string) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns additional information about any tables that are detected in the input document.
FORMS - Returns additional information about any forms that are detected in the input document.
OutputDataConfig (dict) --
Provides output results configuration parameters for custom classifier jobs.
S3Uri (string) --
When you use the OutputDataConfig object while creating a custom classifier, you specify the Amazon S3 location where you want to write the confusion matrix and other output files. The URI must be in the same Region as the API endpoint that you are calling. The location is used as the prefix for the actual location of this output file.
When the custom classifier job is finished, the service creates the output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the confusion matrix.
KmsKeyId (string) --
ID for the Amazon Web Services Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
FlywheelStatsS3Prefix (string) --
The Amazon S3 prefix for the data lake location of the flywheel statistics.
ClassifierMetadata (dict) --
Information about the document classifier, including the number of documents used for training the classifier, the number of documents used for test the classifier, and an accuracy rating.
NumberOfLabels (integer) --
The number of labels in the input data.
NumberOfTrainedDocuments (integer) --
The number of documents in the input data that were used to train the classifier. Typically this is 80 to 90 percent of the input documents.
NumberOfTestDocuments (integer) --
The number of documents in the input data that were used to test the classifier. Typically this is 10 to 20 percent of the input documents, up to 10,000 documents.
EvaluationMetrics (dict) --
Describes the result metrics for the test data associated with an documentation classifier.
Accuracy (float) --
The fraction of the labels that were correct recognized. It is computed by dividing the number of labels in the test documents that were correctly recognized by the total number of labels in the test documents.
Precision (float) --
A measure of the usefulness of the classifier results in the test data. High precision means that the classifier returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the classifier results are for the test data. High recall means that the classifier returned most of the relevant results.
F1Score (float) --
A measure of how accurate the classifier results are for the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. The highest score is 1, and the worst score is 0.
MicroPrecision (float) --
A measure of the usefulness of the recognizer results in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones. Unlike the Precision metric which comes from averaging the precision of all available labels, this is based on the overall score of all precision scores added together.
MicroRecall (float) --
A measure of how complete the classifier results are for the test data. High recall means that the classifier returned most of the relevant results. Specifically, this indicates how many of the correct categories in the text that the model can predict. It is a percentage of correct categories in the text that can found. Instead of averaging the recall scores of all labels (as with Recall), micro Recall is based on the overall score of all recall scores added together.
MicroF1Score (float) --
A measure of how accurate the classifier results are for the test data. It is a combination of the Micro Precision and Micro Recall values. The Micro F1Score is the harmonic mean of the two scores. The highest score is 1, and the worst score is 0.
HammingLoss (float) --
Indicates the fraction of labels that are incorrectly predicted. Also seen as the fraction of wrong labels compared to the total number of labels. Scores closer to zero are better.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that grants Amazon Comprehend read access to your input data.
VolumeKmsKeyId (string) --
ID for the Amazon Web Services Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VpcConfig (dict) --
Configuration parameters for a private Virtual Private Cloud (VPC) containing the resources you are using for your custom classifier. For more information, see Amazon VPC.
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC.
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s Region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets.
(string) --
Mode (string) --
Indicates the mode in which the specific classifier was trained. This also indicates the format of input documents and the format of the confusion matrix. Each classifier can only be trained in one mode and this cannot be changed once the classifier is trained.
ModelKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VersionName (string) --
The version name that you assigned to the document classifier.
SourceModelArn (string) --
The Amazon Resource Name (ARN) of the source model. This model was imported from a different Amazon Web Services account to create the document classifier model in your Amazon Web Services account.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
{'DocumentClassifierPropertiesList': {'InputDataConfig': {'DocumentReaderConfig': {'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT '
'| '
'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT '
'| '
'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': ['TABLES '
'| '
'FORMS']},
'DocumentType': 'PLAIN_TEXT_DOCUMENT '
'| '
'SEMI_STRUCTURED_DOCUMENT',
'Documents': {'S3Uri': 'string',
'TestS3Uri': 'string'}}}}
Gets a list of the document classifiers that you have created.
See also: AWS API Documentation
Request Syntax
client.list_document_classifiers(
Filter={
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED'|'TRAINED_WITH_WARNING',
'DocumentClassifierName': 'string',
'SubmitTimeBefore': datetime(2015, 1, 1),
'SubmitTimeAfter': datetime(2015, 1, 1)
},
NextToken='string',
MaxResults=123
)
dict
Filters the jobs that are returned. You can filter jobs on their name, status, or the date and time that they were submitted. You can only set one filter at a time.
Status (string) --
Filters the list of classifiers based on status.
DocumentClassifierName (string) --
The name that you assigned to the document classifier
SubmitTimeBefore (datetime) --
Filters the list of classifiers based on the time that the classifier was submitted for processing. Returns only classifiers submitted before the specified time. Classifiers are returned in ascending order, oldest to newest.
SubmitTimeAfter (datetime) --
Filters the list of classifiers based on the time that the classifier was submitted for processing. Returns only classifiers submitted after the specified time. Classifiers are returned in descending order, newest to oldest.
string
Identifies the next page of results to return.
integer
The maximum number of results to return in each page. The default is 100.
dict
Response Syntax
{
'DocumentClassifierPropertiesList': [
{
'DocumentClassifierArn': 'string',
'LanguageCode': 'en'|'es'|'fr'|'de'|'it'|'pt'|'ar'|'hi'|'ja'|'ko'|'zh'|'zh-TW',
'Status': 'SUBMITTED'|'TRAINING'|'DELETING'|'STOP_REQUESTED'|'STOPPED'|'IN_ERROR'|'TRAINED'|'TRAINED_WITH_WARNING',
'Message': 'string',
'SubmitTime': datetime(2015, 1, 1),
'EndTime': datetime(2015, 1, 1),
'TrainingStartTime': datetime(2015, 1, 1),
'TrainingEndTime': datetime(2015, 1, 1),
'InputDataConfig': {
'DataFormat': 'COMPREHEND_CSV'|'AUGMENTED_MANIFEST',
'S3Uri': 'string',
'TestS3Uri': 'string',
'LabelDelimiter': 'string',
'AugmentedManifests': [
{
'S3Uri': 'string',
'Split': 'TRAIN'|'TEST',
'AttributeNames': [
'string',
],
'AnnotationDataS3Uri': 'string',
'SourceDocumentsS3Uri': 'string',
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT'
},
],
'DocumentType': 'PLAIN_TEXT_DOCUMENT'|'SEMI_STRUCTURED_DOCUMENT',
'Documents': {
'S3Uri': 'string',
'TestS3Uri': 'string'
},
'DocumentReaderConfig': {
'DocumentReadAction': 'TEXTRACT_DETECT_DOCUMENT_TEXT'|'TEXTRACT_ANALYZE_DOCUMENT',
'DocumentReadMode': 'SERVICE_DEFAULT'|'FORCE_DOCUMENT_READ_ACTION',
'FeatureTypes': [
'TABLES'|'FORMS',
]
}
},
'OutputDataConfig': {
'S3Uri': 'string',
'KmsKeyId': 'string',
'FlywheelStatsS3Prefix': 'string'
},
'ClassifierMetadata': {
'NumberOfLabels': 123,
'NumberOfTrainedDocuments': 123,
'NumberOfTestDocuments': 123,
'EvaluationMetrics': {
'Accuracy': 123.0,
'Precision': 123.0,
'Recall': 123.0,
'F1Score': 123.0,
'MicroPrecision': 123.0,
'MicroRecall': 123.0,
'MicroF1Score': 123.0,
'HammingLoss': 123.0
}
},
'DataAccessRoleArn': 'string',
'VolumeKmsKeyId': 'string',
'VpcConfig': {
'SecurityGroupIds': [
'string',
],
'Subnets': [
'string',
]
},
'Mode': 'MULTI_CLASS'|'MULTI_LABEL',
'ModelKmsKeyId': 'string',
'VersionName': 'string',
'SourceModelArn': 'string',
'FlywheelArn': 'string'
},
],
'NextToken': 'string'
}
Response Structure
(dict) --
DocumentClassifierPropertiesList (list) --
A list containing the properties of each job returned.
(dict) --
Provides information about a document classifier.
DocumentClassifierArn (string) --
The Amazon Resource Name (ARN) that identifies the document classifier.
LanguageCode (string) --
The language code for the language of the documents that the classifier was trained on.
Status (string) --
The status of the document classifier. If the status is TRAINED the classifier is ready to use. If the status is TRAINED_WITH_WARNINGS the classifier training succeeded, but you should review the warnings returned in the CreateDocumentClassifier response.
If the status is FAILED you can see additional information about why the classifier wasn't trained in the Message field.
Message (string) --
Additional information about the status of the classifier.
SubmitTime (datetime) --
The time that the document classifier was submitted for training.
EndTime (datetime) --
The time that training the document classifier completed.
TrainingStartTime (datetime) --
Indicates the time when the training starts on documentation classifiers. You are billed for the time interval between this time and the value of TrainingEndTime.
TrainingEndTime (datetime) --
The time that training of the document classifier was completed. Indicates the time when the training completes on documentation classifiers. You are billed for the time interval between this time and the value of TrainingStartTime.
InputDataConfig (dict) --
The input data configuration that you supplied when you created the document classifier for training.
DataFormat (string) --
The format of your training data:
COMPREHEND_CSV : A two-column CSV file, where labels are provided in the first column, and documents are provided in the second. If you use this value, you must provide the S3Uri parameter in your request.
AUGMENTED_MANIFEST : A labeled dataset that is produced by Amazon SageMaker Ground Truth. This file is in JSON lines format. Each line is a complete JSON object that contains a training document and its associated labels. If you use this value, you must provide the AugmentedManifests parameter in your request.
If you don't specify a value, Amazon Comprehend uses COMPREHEND_CSV as the default.
S3Uri (string) --
The Amazon S3 URI for the input data. The S3 bucket must be in the same Region as the API endpoint that you are calling. The URI can point to a single input file or it can provide the prefix for a collection of input files.
For example, if you use the URI S3://bucketName/prefix , if the prefix is a single file, Amazon Comprehend uses that file as input. If more than one file begins with the prefix, Amazon Comprehend uses all of them as input.
This parameter is required if you set DataFormat to COMPREHEND_CSV .
TestS3Uri (string) --
This specifies the Amazon S3 location where the test annotations for an entity recognizer are located. The URI must be in the same Amazon Web Services Region as the API endpoint that you are calling.
LabelDelimiter (string) --
Indicates the delimiter used to separate each label for training a multi-label classifier. The default delimiter between labels is a pipe (|). You can use a different character as a delimiter (if it's an allowed character) by specifying it under Delimiter for labels. If the training documents use a delimiter other than the default or the delimiter you specify, the labels on that line will be combined to make a single unique label, such as LABELLABELLABEL.
AugmentedManifests (list) --
A list of augmented manifest files that provide training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
This parameter is required if you set DataFormat to AUGMENTED_MANIFEST .
(dict) --
An augmented manifest file that provides training data for your custom model. An augmented manifest file is a labeled dataset that is produced by Amazon SageMaker Ground Truth.
S3Uri (string) --
The Amazon S3 location of the augmented manifest file.
Split (string) --
The purpose of the data you've provided in the augmented manifest. You can either train or test this data. If you don't specify, the default is train.
TRAIN - all of the documents in the manifest will be used for training. If no test documents are provided, Amazon Comprehend will automatically reserve a portion of the training documents for testing.
TEST - all of the documents in the manifest will be used for testing.
AttributeNames (list) --
The JSON attribute that contains the annotations for your training documents. The number of attribute names that you specify depends on whether your augmented manifest file is the output of a single labeling job or a chained labeling job.
If your file is the output of a single labeling job, specify the LabelAttributeName key that was used when the job was created in Ground Truth.
If your file is the output of a chained labeling job, specify the LabelAttributeName key for one or more jobs in the chain. Each LabelAttributeName key provides the annotations from an individual job.
(string) --
AnnotationDataS3Uri (string) --
The S3 prefix to the annotation files that are referred in the augmented manifest file.
SourceDocumentsS3Uri (string) --
The S3 prefix to the source files (PDFs) that are referred to in the augmented manifest file.
DocumentType (string) --
The type of augmented manifest. PlainTextDocument or SemiStructuredDocument. If you don't specify, the default is PlainTextDocument.
PLAIN_TEXT_DOCUMENT A document type that represents any unicode text that is encoded in UTF-8.
SEMI_STRUCTURED_DOCUMENT A document type with positional and structural context, like a PDF. For training with Amazon Comprehend, only PDFs are supported. For inference, Amazon Comprehend support PDFs, DOCX and TXT.
DocumentType (string) --
The type of input documents for training the model. Provide plain-text documents to create a plain-text model, and provide semi-structured documents to create a native model.
Documents (dict) --
The S3 location of the training documents. This parameter is required in a request to create a native classifier model.
S3Uri (string) --
The S3 URI location of the training documents specified in the S3Uri CSV file.
TestS3Uri (string) --
The S3 URI location of the test documents included in the TestS3Uri CSV file. This field is not required if you do not specify a test CSV file.
DocumentReaderConfig (dict) --
Provides configuration parameters to override the default actions for extracting text from PDF documents and image files.
By default, Amazon Comprehend performs the following actions to extract text from files, based on the input file type:
Word files - Amazon Comprehend parser extracts the text.
Digital PDF files - Amazon Comprehend parser extracts the text.
Image files and scanned PDF files - Amazon Comprehend uses the Amazon Textract DetectDocumentText API to extract the text.
DocumentReaderConfig does not apply to plain text files or Word files.
For image files and PDF documents, you can override these default actions using the fields listed below. For more information, see Setting text extraction options in the Comprehend Developer Guide.
DocumentReadAction (string) --
This field defines the Amazon Textract API operation that Amazon Comprehend uses to extract text from PDF files and image files. Enter one of the following values:
TEXTRACT_DETECT_DOCUMENT_TEXT - The Amazon Comprehend service uses the DetectDocumentText API operation.
TEXTRACT_ANALYZE_DOCUMENT - The Amazon Comprehend service uses the AnalyzeDocument API operation.
DocumentReadMode (string) --
Determines the text extraction actions for PDF files. Enter one of the following values:
SERVICE_DEFAULT - use the Amazon Comprehend service defaults for PDF files.
FORCE_DOCUMENT_READ_ACTION - Amazon Comprehend uses the Textract API specified by DocumentReadAction for all PDF files, including digital PDF files.
FeatureTypes (list) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns information about any tables that are detected in the input document.
FORMS - Returns information and the data from any forms that are detected in the input document.
(string) --
Specifies the type of Amazon Textract features to apply. If you chose TEXTRACT_ANALYZE_DOCUMENT as the read action, you must specify one or both of the following values:
TABLES - Returns additional information about any tables that are detected in the input document.
FORMS - Returns additional information about any forms that are detected in the input document.
OutputDataConfig (dict) --
Provides output results configuration parameters for custom classifier jobs.
S3Uri (string) --
When you use the OutputDataConfig object while creating a custom classifier, you specify the Amazon S3 location where you want to write the confusion matrix and other output files. The URI must be in the same Region as the API endpoint that you are calling. The location is used as the prefix for the actual location of this output file.
When the custom classifier job is finished, the service creates the output file in a directory specific to the job. The S3Uri field contains the location of the output file, called output.tar.gz . It is a compressed archive that contains the confusion matrix.
KmsKeyId (string) --
ID for the Amazon Web Services Key Management Service (KMS) key that Amazon Comprehend uses to encrypt the output results from an analysis job. The KmsKeyId can be one of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
KMS Key Alias: "alias/ExampleAlias"
ARN of a KMS Key Alias: "arn:aws:kms:us-west-2:111122223333:alias/ExampleAlias"
FlywheelStatsS3Prefix (string) --
The Amazon S3 prefix for the data lake location of the flywheel statistics.
ClassifierMetadata (dict) --
Information about the document classifier, including the number of documents used for training the classifier, the number of documents used for test the classifier, and an accuracy rating.
NumberOfLabels (integer) --
The number of labels in the input data.
NumberOfTrainedDocuments (integer) --
The number of documents in the input data that were used to train the classifier. Typically this is 80 to 90 percent of the input documents.
NumberOfTestDocuments (integer) --
The number of documents in the input data that were used to test the classifier. Typically this is 10 to 20 percent of the input documents, up to 10,000 documents.
EvaluationMetrics (dict) --
Describes the result metrics for the test data associated with an documentation classifier.
Accuracy (float) --
The fraction of the labels that were correct recognized. It is computed by dividing the number of labels in the test documents that were correctly recognized by the total number of labels in the test documents.
Precision (float) --
A measure of the usefulness of the classifier results in the test data. High precision means that the classifier returned substantially more relevant results than irrelevant ones.
Recall (float) --
A measure of how complete the classifier results are for the test data. High recall means that the classifier returned most of the relevant results.
F1Score (float) --
A measure of how accurate the classifier results are for the test data. It is derived from the Precision and Recall values. The F1Score is the harmonic average of the two scores. The highest score is 1, and the worst score is 0.
MicroPrecision (float) --
A measure of the usefulness of the recognizer results in the test data. High precision means that the recognizer returned substantially more relevant results than irrelevant ones. Unlike the Precision metric which comes from averaging the precision of all available labels, this is based on the overall score of all precision scores added together.
MicroRecall (float) --
A measure of how complete the classifier results are for the test data. High recall means that the classifier returned most of the relevant results. Specifically, this indicates how many of the correct categories in the text that the model can predict. It is a percentage of correct categories in the text that can found. Instead of averaging the recall scores of all labels (as with Recall), micro Recall is based on the overall score of all recall scores added together.
MicroF1Score (float) --
A measure of how accurate the classifier results are for the test data. It is a combination of the Micro Precision and Micro Recall values. The Micro F1Score is the harmonic mean of the two scores. The highest score is 1, and the worst score is 0.
HammingLoss (float) --
Indicates the fraction of labels that are incorrectly predicted. Also seen as the fraction of wrong labels compared to the total number of labels. Scores closer to zero are better.
DataAccessRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that grants Amazon Comprehend read access to your input data.
VolumeKmsKeyId (string) --
ID for the Amazon Web Services Key Management Service (KMS) key that Amazon Comprehend uses to encrypt data on the storage volume attached to the ML compute instance(s) that process the analysis job. The VolumeKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VpcConfig (dict) --
Configuration parameters for a private Virtual Private Cloud (VPC) containing the resources you are using for your custom classifier. For more information, see Amazon VPC.
SecurityGroupIds (list) --
The ID number for a security group on an instance of your private VPC. Security groups on your VPC function serve as a virtual firewall to control inbound and outbound traffic and provides security for the resources that you’ll be accessing on the VPC. This ID number is preceded by "sg-", for instance: "sg-03b388029b0a285ea". For more information, see Security Groups for your VPC.
(string) --
Subnets (list) --
The ID for each subnet being used in your private VPC. This subnet is a subset of the a range of IPv4 addresses used by the VPC and is specific to a given availability zone in the VPC’s Region. This ID number is preceded by "subnet-", for instance: "subnet-04ccf456919e69055". For more information, see VPCs and Subnets.
(string) --
Mode (string) --
Indicates the mode in which the specific classifier was trained. This also indicates the format of input documents and the format of the confusion matrix. Each classifier can only be trained in one mode and this cannot be changed once the classifier is trained.
ModelKmsKeyId (string) --
ID for the KMS key that Amazon Comprehend uses to encrypt trained custom models. The ModelKmsKeyId can be either of the following formats:
KMS Key ID: "1234abcd-12ab-34cd-56ef-1234567890ab"
Amazon Resource Name (ARN) of a KMS Key: "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
VersionName (string) --
The version name that you assigned to the document classifier.
SourceModelArn (string) --
The Amazon Resource Name (ARN) of the source model. This model was imported from a different Amazon Web Services account to create the document classifier model in your Amazon Web Services account.
FlywheelArn (string) --
The Amazon Resource Number (ARN) of the flywheel
NextToken (string) --
Identifies the next page of results to return.