2018/02/06 - AWS Glue - 4 updated api methods
Changes Update glue client to latest version
{'JsonClassifier': {'JsonPath': 'string', 'Name': 'string'}}
Creates a classifier in the user's account. This may be a GrokClassifier , an XMLClassifier , or abbrev JsonClassifier , depending on which field of the request is present.
See also: AWS API Documentation
Request Syntax
client.create_classifier( GrokClassifier={ 'Classification': 'string', 'Name': 'string', 'GrokPattern': 'string', 'CustomPatterns': 'string' }, XMLClassifier={ 'Classification': 'string', 'Name': 'string', 'RowTag': 'string' }, JsonClassifier={ 'Name': 'string', 'JsonPath': 'string' } )
dict
A GrokClassifier object specifying the classifier to create.
Classification (string) -- [REQUIRED]
An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.
Name (string) -- [REQUIRED]
The name of the new classifier.
GrokPattern (string) -- [REQUIRED]
The grok pattern used by this classifier.
CustomPatterns (string) --
Optional custom grok patterns used by this classifier.
dict
An XMLClassifier object specifying the classifier to create.
Classification (string) -- [REQUIRED]
An identifier of the data format that the classifier matches.
Name (string) -- [REQUIRED]
The name of the classifier.
RowTag (string) --
The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot identify a self-closing element (closed by /> ). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example, <row item_a="A" item_b="B"></row> is okay, but <row item_a="A" item_b="B" /> is not).
dict
A JsonClassifier object specifying the classifier to create.
Name (string) -- [REQUIRED]
The name of the classifier.
JsonPath (string) -- [REQUIRED]
A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.
dict
Response Syntax
{}
Response Structure
(dict) --
{'Classifier': {'JsonClassifier': {'CreationTime': 'timestamp', 'JsonPath': 'string', 'LastUpdated': 'timestamp', 'Name': 'string', 'Version': 'long'}}}
Retrieve a classifier by name.
See also: AWS API Documentation
Request Syntax
client.get_classifier( Name='string' )
string
[REQUIRED]
Name of the classifier to retrieve.
dict
Response Syntax
{ 'Classifier': { 'GrokClassifier': { 'Name': 'string', 'Classification': 'string', 'CreationTime': datetime(2015, 1, 1), 'LastUpdated': datetime(2015, 1, 1), 'Version': 123, 'GrokPattern': 'string', 'CustomPatterns': 'string' }, 'XMLClassifier': { 'Name': 'string', 'Classification': 'string', 'CreationTime': datetime(2015, 1, 1), 'LastUpdated': datetime(2015, 1, 1), 'Version': 123, 'RowTag': 'string' }, 'JsonClassifier': { 'Name': 'string', 'CreationTime': datetime(2015, 1, 1), 'LastUpdated': datetime(2015, 1, 1), 'Version': 123, 'JsonPath': 'string' } } }
Response Structure
(dict) --
Classifier (dict) --
The requested classifier.
GrokClassifier (dict) --
A GrokClassifier object.
Name (string) --
The name of the classifier.
Classification (string) --
An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, and so on.
CreationTime (datetime) --
The time this classifier was registered.
LastUpdated (datetime) --
The time this classifier was last updated.
Version (integer) --
The version of this classifier.
GrokPattern (string) --
The grok pattern applied to a data store by this classifier. For more information, see built-in patterns in Writing Custom Classifers.
CustomPatterns (string) --
Optional custom grok patterns defined by this classifier. For more information, see custom patterns in Writing Custom Classifers.
XMLClassifier (dict) --
An XMLClassifier object.
Name (string) --
The name of the classifier.
Classification (string) --
An identifier of the data format that the classifier matches.
CreationTime (datetime) --
The time this classifier was registered.
LastUpdated (datetime) --
The time this classifier was last updated.
Version (integer) --
The version of this classifier.
RowTag (string) --
The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot identify a self-closing element (closed by /> ). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example, <row item_a="A" item_b="B"></row> is okay, but <row item_a="A" item_b="B" /> is not).
JsonClassifier (dict) --
A JsonClassifier object.
Name (string) --
The name of the classifier.
CreationTime (datetime) --
The time this classifier was registered.
LastUpdated (datetime) --
The time this classifier was last updated.
Version (integer) --
The version of this classifier.
JsonPath (string) --
A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.
{'Classifiers': {'JsonClassifier': {'CreationTime': 'timestamp', 'JsonPath': 'string', 'LastUpdated': 'timestamp', 'Name': 'string', 'Version': 'long'}}}
Lists all classifier objects in the Data Catalog.
See also: AWS API Documentation
Request Syntax
client.get_classifiers( MaxResults=123, NextToken='string' )
integer
Size of the list to return (optional).
string
An optional continuation token.
dict
Response Syntax
{ 'Classifiers': [ { 'GrokClassifier': { 'Name': 'string', 'Classification': 'string', 'CreationTime': datetime(2015, 1, 1), 'LastUpdated': datetime(2015, 1, 1), 'Version': 123, 'GrokPattern': 'string', 'CustomPatterns': 'string' }, 'XMLClassifier': { 'Name': 'string', 'Classification': 'string', 'CreationTime': datetime(2015, 1, 1), 'LastUpdated': datetime(2015, 1, 1), 'Version': 123, 'RowTag': 'string' }, 'JsonClassifier': { 'Name': 'string', 'CreationTime': datetime(2015, 1, 1), 'LastUpdated': datetime(2015, 1, 1), 'Version': 123, 'JsonPath': 'string' } }, ], 'NextToken': 'string' }
Response Structure
(dict) --
Classifiers (list) --
The requested list of classifier objects.
(dict) --
Classifiers are written in Python and triggered during a crawl task. You can write your own classifiers to best categorize your data sources and specify the appropriate schemas to use for them. A classifier checks whether a given file is in a format it can handle, and if it is, the classifier creates a schema in the form of a StructType object that matches that data format.
A classifier can be a grok classifier, an XML classifier, or a JSON classifier, asspecified in one of the fields in the Classifier object.
GrokClassifier (dict) --
A GrokClassifier object.
Name (string) --
The name of the classifier.
Classification (string) --
An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, and so on.
CreationTime (datetime) --
The time this classifier was registered.
LastUpdated (datetime) --
The time this classifier was last updated.
Version (integer) --
The version of this classifier.
GrokPattern (string) --
The grok pattern applied to a data store by this classifier. For more information, see built-in patterns in Writing Custom Classifers.
CustomPatterns (string) --
Optional custom grok patterns defined by this classifier. For more information, see custom patterns in Writing Custom Classifers.
XMLClassifier (dict) --
An XMLClassifier object.
Name (string) --
The name of the classifier.
Classification (string) --
An identifier of the data format that the classifier matches.
CreationTime (datetime) --
The time this classifier was registered.
LastUpdated (datetime) --
The time this classifier was last updated.
Version (integer) --
The version of this classifier.
RowTag (string) --
The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot identify a self-closing element (closed by /> ). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example, <row item_a="A" item_b="B"></row> is okay, but <row item_a="A" item_b="B" /> is not).
JsonClassifier (dict) --
A JsonClassifier object.
Name (string) --
The name of the classifier.
CreationTime (datetime) --
The time this classifier was registered.
LastUpdated (datetime) --
The time this classifier was last updated.
Version (integer) --
The version of this classifier.
JsonPath (string) --
A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.
NextToken (string) --
A continuation token.
{'JsonClassifier': {'JsonPath': 'string', 'Name': 'string'}}
Modifies an existing classifier (a GrokClassifier , XMLClassifier , or JsonClassifier , depending on which field is present).
See also: AWS API Documentation
Request Syntax
client.update_classifier( GrokClassifier={ 'Name': 'string', 'Classification': 'string', 'GrokPattern': 'string', 'CustomPatterns': 'string' }, XMLClassifier={ 'Name': 'string', 'Classification': 'string', 'RowTag': 'string' }, JsonClassifier={ 'Name': 'string', 'JsonPath': 'string' } )
dict
A GrokClassifier object with updated fields.
Name (string) -- [REQUIRED]
The name of the GrokClassifier .
Classification (string) --
An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.
GrokPattern (string) --
The grok pattern used by this classifier.
CustomPatterns (string) --
Optional custom grok patterns used by this classifier.
dict
An XMLClassifier object with updated fields.
Name (string) -- [REQUIRED]
The name of the classifier.
Classification (string) --
An identifier of the data format that the classifier matches.
RowTag (string) --
The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot identify a self-closing element (closed by /> ). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example, <row item_a="A" item_b="B"></row> is okay, but <row item_a="A" item_b="B" /> is not).
dict
A JsonClassifier object with updated fields.
Name (string) -- [REQUIRED]
The name of the classifier.
JsonPath (string) --
A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.
dict
Response Syntax
{}
Response Structure
(dict) --