2021/04/22 - Amazon Forecast Service - 3 updated api methods
Changes This release adds EstimatedTimeRemaining minutes field to the DescribeDatasetImportJob, DescribePredictor, DescribeForecast API response which denotes the time remaining to complete the job IN_PROGRESS.
{'EstimatedTimeRemainingInMinutes': 'long'}
Describes a dataset import job created using the CreateDatasetImportJob operation.
In addition to listing the parameters provided in the CreateDatasetImportJob request, this operation includes the following properties:
CreationTime
LastModificationTime
DataSize
FieldStatistics
Status
Message - If an error occurred, information about the error.
See also: AWS API Documentation
Request Syntax
client.describe_dataset_import_job(
DatasetImportJobArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) of the dataset import job.
dict
Response Syntax
{
'DatasetImportJobName': 'string',
'DatasetImportJobArn': 'string',
'DatasetArn': 'string',
'TimestampFormat': 'string',
'TimeZone': 'string',
'UseGeolocationForTimeZone': True|False,
'GeolocationFormat': 'string',
'DataSource': {
'S3Config': {
'Path': 'string',
'RoleArn': 'string',
'KMSKeyArn': 'string'
}
},
'EstimatedTimeRemainingInMinutes': 123,
'FieldStatistics': {
'string': {
'Count': 123,
'CountDistinct': 123,
'CountNull': 123,
'CountNan': 123,
'Min': 'string',
'Max': 'string',
'Avg': 123.0,
'Stddev': 123.0
}
},
'DataSize': 123.0,
'Status': 'string',
'Message': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModificationTime': datetime(2015, 1, 1)
}
Response Structure
(dict) --
DatasetImportJobName (string) --
The name of the dataset import job.
DatasetImportJobArn (string) --
The ARN of the dataset import job.
DatasetArn (string) --
The Amazon Resource Name (ARN) of the dataset that the training data was imported to.
TimestampFormat (string) --
The format of timestamps in the dataset. The format that you specify depends on the DataFrequency specified when the dataset was created. The following formats are supported
"yyyy-MM-dd" For the following data frequencies: Y, M, W, and D
"yyyy-MM-dd HH:mm:ss" For the following data frequencies: H, 30min, 15min, and 1min; and optionally, for: Y, M, W, and D
TimeZone (string) --
The single time zone applied to every item in the dataset
UseGeolocationForTimeZone (boolean) --
Whether TimeZone is automatically derived from the geolocation attribute.
GeolocationFormat (string) --
The format of the geolocation attribute. Valid Values: "LAT_LONG" and "CC_POSTALCODE" .
DataSource (dict) --
The location of the training data to import and an AWS Identity and Access Management (IAM) role that Amazon Forecast can assume to access the data.
If encryption is used, DataSource includes an AWS Key Management Service (KMS) key.
S3Config (dict) --
The path to the training data stored in an Amazon Simple Storage Service (Amazon S3) bucket along with the credentials to access the data.
Path (string) --
The path to an Amazon Simple Storage Service (Amazon S3) bucket or file(s) in an Amazon S3 bucket.
RoleArn (string) --
The ARN of the AWS Identity and Access Management (IAM) role that Amazon Forecast can assume to access the Amazon S3 bucket or files. If you provide a value for the KMSKeyArn key, the role must allow access to the key.
Passing a role across AWS accounts is not allowed. If you pass a role that isn't in your account, you get an InvalidInputException error.
KMSKeyArn (string) --
The Amazon Resource Name (ARN) of an AWS Key Management Service (KMS) key.
EstimatedTimeRemainingInMinutes (integer) --
The estimated time in minutes for the dataset import job to complete.
FieldStatistics (dict) --
Statistical information about each field in the input data.
(string) --
(dict) --
Provides statistics for each data field imported into to an Amazon Forecast dataset with the CreateDatasetImportJob operation.
Count (integer) --
The number of values in the field.
CountDistinct (integer) --
The number of distinct values in the field.
CountNull (integer) --
The number of null values in the field.
CountNan (integer) --
The number of NAN (not a number) values in the field.
Min (string) --
For a numeric field, the minimum value in the field.
Max (string) --
For a numeric field, the maximum value in the field.
Avg (float) --
For a numeric field, the average value in the field.
Stddev (float) --
For a numeric field, the standard deviation.
DataSize (float) --
The size of the dataset in gigabytes (GB) after the import job has finished.
Status (string) --
The status of the dataset import job. States include:
ACTIVE
CREATE_PENDING , CREATE_IN_PROGRESS , CREATE_FAILED
DELETE_PENDING , DELETE_IN_PROGRESS , DELETE_FAILED
CREATE_STOPPING , CREATE_STOPPED
Message (string) --
If an error occurred, an informational message about the error.
CreationTime (datetime) --
When the dataset import job was created.
LastModificationTime (datetime) --
The last time the resource was modified. The timestamp depends on the status of the job:
CREATE_PENDING - The CreationTime .
CREATE_IN_PROGRESS - The current timestamp.
CREATE_STOPPING - The current timestamp.
CREATE_STOPPED - When the job stopped.
ACTIVE or CREATE_FAILED - When the job finished or failed.
{'EstimatedTimeRemainingInMinutes': 'long'}
Describes a forecast created using the CreateForecast operation.
In addition to listing the properties provided in the CreateForecast request, this operation lists the following properties:
DatasetGroupArn - The dataset group that provided the training data.
CreationTime
LastModificationTime
Status
Message - If an error occurred, information about the error.
See also: AWS API Documentation
Request Syntax
client.describe_forecast(
ForecastArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) of the forecast.
dict
Response Syntax
{
'ForecastArn': 'string',
'ForecastName': 'string',
'ForecastTypes': [
'string',
],
'PredictorArn': 'string',
'DatasetGroupArn': 'string',
'EstimatedTimeRemainingInMinutes': 123,
'Status': 'string',
'Message': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModificationTime': datetime(2015, 1, 1)
}
Response Structure
(dict) --
ForecastArn (string) --
The forecast ARN as specified in the request.
ForecastName (string) --
The name of the forecast.
ForecastTypes (list) --
The quantiles at which probabilistic forecasts were generated.
(string) --
PredictorArn (string) --
The ARN of the predictor used to generate the forecast.
DatasetGroupArn (string) --
The ARN of the dataset group that provided the data used to train the predictor.
EstimatedTimeRemainingInMinutes (integer) --
The estimated time in minutes for the forecast job to complete.
Status (string) --
The status of the forecast. States include:
ACTIVE
CREATE_PENDING , CREATE_IN_PROGRESS , CREATE_FAILED
CREATE_STOPPING , CREATE_STOPPED
DELETE_PENDING , DELETE_IN_PROGRESS , DELETE_FAILED
Note
The Status of the forecast must be ACTIVE before you can query or export the forecast.
Message (string) --
If an error occurred, an informational message about the error.
CreationTime (datetime) --
When the forecast creation task was created.
LastModificationTime (datetime) --
The last time the resource was modified. The timestamp depends on the status of the job:
CREATE_PENDING - The CreationTime .
CREATE_IN_PROGRESS - The current timestamp.
CREATE_STOPPING - The current timestamp.
CREATE_STOPPED - When the job stopped.
ACTIVE or CREATE_FAILED - When the job finished or failed.
{'EstimatedTimeRemainingInMinutes': 'long'}
Describes a predictor created using the CreatePredictor operation.
In addition to listing the properties provided in the CreatePredictor request, this operation lists the following properties:
DatasetImportJobArns - The dataset import jobs used to import training data.
AutoMLAlgorithmArns - If AutoML is performed, the algorithms that were evaluated.
CreationTime
LastModificationTime
Status
Message - If an error occurred, information about the error.
See also: AWS API Documentation
Request Syntax
client.describe_predictor(
PredictorArn='string'
)
string
[REQUIRED]
The Amazon Resource Name (ARN) of the predictor that you want information about.
dict
Response Syntax
{
'PredictorArn': 'string',
'PredictorName': 'string',
'AlgorithmArn': 'string',
'ForecastHorizon': 123,
'ForecastTypes': [
'string',
],
'PerformAutoML': True|False,
'PerformHPO': True|False,
'TrainingParameters': {
'string': 'string'
},
'EvaluationParameters': {
'NumberOfBacktestWindows': 123,
'BackTestWindowOffset': 123
},
'HPOConfig': {
'ParameterRanges': {
'CategoricalParameterRanges': [
{
'Name': 'string',
'Values': [
'string',
]
},
],
'ContinuousParameterRanges': [
{
'Name': 'string',
'MaxValue': 123.0,
'MinValue': 123.0,
'ScalingType': 'Auto'|'Linear'|'Logarithmic'|'ReverseLogarithmic'
},
],
'IntegerParameterRanges': [
{
'Name': 'string',
'MaxValue': 123,
'MinValue': 123,
'ScalingType': 'Auto'|'Linear'|'Logarithmic'|'ReverseLogarithmic'
},
]
}
},
'InputDataConfig': {
'DatasetGroupArn': 'string',
'SupplementaryFeatures': [
{
'Name': 'string',
'Value': 'string'
},
]
},
'FeaturizationConfig': {
'ForecastFrequency': 'string',
'ForecastDimensions': [
'string',
],
'Featurizations': [
{
'AttributeName': 'string',
'FeaturizationPipeline': [
{
'FeaturizationMethodName': 'filling',
'FeaturizationMethodParameters': {
'string': 'string'
}
},
]
},
]
},
'EncryptionConfig': {
'RoleArn': 'string',
'KMSKeyArn': 'string'
},
'PredictorExecutionDetails': {
'PredictorExecutions': [
{
'AlgorithmArn': 'string',
'TestWindows': [
{
'TestWindowStart': datetime(2015, 1, 1),
'TestWindowEnd': datetime(2015, 1, 1),
'Status': 'string',
'Message': 'string'
},
]
},
]
},
'EstimatedTimeRemainingInMinutes': 123,
'DatasetImportJobArns': [
'string',
],
'AutoMLAlgorithmArns': [
'string',
],
'Status': 'string',
'Message': 'string',
'CreationTime': datetime(2015, 1, 1),
'LastModificationTime': datetime(2015, 1, 1)
}
Response Structure
(dict) --
PredictorArn (string) --
The ARN of the predictor.
PredictorName (string) --
The name of the predictor.
AlgorithmArn (string) --
The Amazon Resource Name (ARN) of the algorithm used for model training.
ForecastHorizon (integer) --
The number of time-steps of the forecast. The forecast horizon is also called the prediction length.
ForecastTypes (list) --
The forecast types used during predictor training. Default value is ["0.1","0.5","0.9"]
(string) --
PerformAutoML (boolean) --
Whether the predictor is set to perform AutoML.
PerformHPO (boolean) --
Whether the predictor is set to perform hyperparameter optimization (HPO).
TrainingParameters (dict) --
The default training parameters or overrides selected during model training. When running AutoML or choosing HPO with CNN-QR or DeepAR+, the optimized values for the chosen hyperparameters are returned. For more information, see aws-forecast-choosing-recipes.
(string) --
(string) --
EvaluationParameters (dict) --
Used to override the default evaluation parameters of the specified algorithm. Amazon Forecast evaluates a predictor by splitting a dataset into training data and testing data. The evaluation parameters define how to perform the split and the number of iterations.
NumberOfBacktestWindows (integer) --
The number of times to split the input data. The default is 1. Valid values are 1 through 5.
BackTestWindowOffset (integer) --
The point from the end of the dataset where you want to split the data for model training and testing (evaluation). Specify the value as the number of data points. The default is the value of the forecast horizon. BackTestWindowOffset can be used to mimic a past virtual forecast start date. This value must be greater than or equal to the forecast horizon and less than half of the TARGET_TIME_SERIES dataset length.
ForecastHorizon <= BackTestWindowOffset < 1/2 * TARGET_TIME_SERIES dataset length
HPOConfig (dict) --
The hyperparameter override values for the algorithm.
ParameterRanges (dict) --
Specifies the ranges of valid values for the hyperparameters.
CategoricalParameterRanges (list) --
Specifies the tunable range for each categorical hyperparameter.
(dict) --
Specifies a categorical hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) --
The name of the categorical hyperparameter to tune.
Values (list) --
A list of the tunable categories for the hyperparameter.
(string) --
ContinuousParameterRanges (list) --
Specifies the tunable range for each continuous hyperparameter.
(dict) --
Specifies a continuous hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) --
The name of the hyperparameter to tune.
MaxValue (float) --
The maximum tunable value of the hyperparameter.
MinValue (float) --
The minimum tunable value of the hyperparameter.
ScalingType (string) --
The scale that hyperparameter tuning uses to search the hyperparameter range. Valid values:
Auto
Amazon Forecast hyperparameter tuning chooses the best scale for the hyperparameter.
Linear
Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale.
Logarithmic
Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale.
Logarithmic scaling works only for ranges that have values greater than 0.
ReverseLogarithmic
hyperparameter tuning searches the values in the hyperparameter range by using a reverse logarithmic scale.
Reverse logarithmic scaling works only for ranges that are entirely within the range 0 <= x < 1.0.
For information about choosing a hyperparameter scale, see Hyperparameter Scaling. One of the following values:
IntegerParameterRanges (list) --
Specifies the tunable range for each integer hyperparameter.
(dict) --
Specifies an integer hyperparameter and it's range of tunable values. This object is part of the ParameterRanges object.
Name (string) --
The name of the hyperparameter to tune.
MaxValue (integer) --
The maximum tunable value of the hyperparameter.
MinValue (integer) --
The minimum tunable value of the hyperparameter.
ScalingType (string) --
The scale that hyperparameter tuning uses to search the hyperparameter range. Valid values:
Auto
Amazon Forecast hyperparameter tuning chooses the best scale for the hyperparameter.
Linear
Hyperparameter tuning searches the values in the hyperparameter range by using a linear scale.
Logarithmic
Hyperparameter tuning searches the values in the hyperparameter range by using a logarithmic scale.
Logarithmic scaling works only for ranges that have values greater than 0.
ReverseLogarithmic
Not supported for IntegerParameterRange .
Reverse logarithmic scaling works only for ranges that are entirely within the range 0 <= x < 1.0.
For information about choosing a hyperparameter scale, see Hyperparameter Scaling. One of the following values:
InputDataConfig (dict) --
Describes the dataset group that contains the data to use to train the predictor.
DatasetGroupArn (string) --
The Amazon Resource Name (ARN) of the dataset group.
SupplementaryFeatures (list) --
An array of supplementary features. The only supported feature is a holiday calendar.
(dict) --
Describes a supplementary feature of a dataset group. This object is part of the InputDataConfig object. Forecast supports the Weather Index and Holidays built-in featurizations.
Weather Index
The Amazon Forecast Weather Index is a built-in featurization that incorporates historical and projected weather information into your model. The Weather Index supplements your datasets with over two years of historical weather data and up to 14 days of projected weather data. For more information, see Amazon Forecast Weather Index.
Holidays
Holidays is a built-in featurization that incorporates a feature-engineered dataset of national holiday information into your model. It provides native support for the holiday calendars of 66 countries. To view the holiday calendars, refer to the Jollyday library. For more information, see Holidays Featurization.
Name (string) --
The name of the feature. Valid values: "holiday" and "weather" .
Value (string) --
Weather Index
To enable the Weather Index, set the value to "true"
Holidays
To enable Holidays, specify a country with one of the following two-letter country codes:
"AL" - ALBANIA
"AR" - ARGENTINA
"AT" - AUSTRIA
"AU" - AUSTRALIA
"BA" - BOSNIA HERZEGOVINA
"BE" - BELGIUM
"BG" - BULGARIA
"BO" - BOLIVIA
"BR" - BRAZIL
"BY" - BELARUS
"CA" - CANADA
"CL" - CHILE
"CO" - COLOMBIA
"CR" - COSTA RICA
"HR" - CROATIA
"CZ" - CZECH REPUBLIC
"DK" - DENMARK
"EC" - ECUADOR
"EE" - ESTONIA
"ET" - ETHIOPIA
"FI" - FINLAND
"FR" - FRANCE
"DE" - GERMANY
"GR" - GREECE
"HU" - HUNGARY
"IS" - ICELAND
"IN" - INDIA
"IE" - IRELAND
"IT" - ITALY
"JP" - JAPAN
"KZ" - KAZAKHSTAN
"KR" - KOREA
"LV" - LATVIA
"LI" - LIECHTENSTEIN
"LT" - LITHUANIA
"LU" - LUXEMBOURG
"MK" - MACEDONIA
"MT" - MALTA
"MX" - MEXICO
"MD" - MOLDOVA
"ME" - MONTENEGRO
"NL" - NETHERLANDS
"NZ" - NEW ZEALAND
"NI" - NICARAGUA
"NG" - NIGERIA
"NO" - NORWAY
"PA" - PANAMA
"PY" - PARAGUAY
"PE" - PERU
"PL" - POLAND
"PT" - PORTUGAL
"RO" - ROMANIA
"RU" - RUSSIA
"RS" - SERBIA
"SK" - SLOVAKIA
"SI" - SLOVENIA
"ZA" - SOUTH AFRICA
"ES" - SPAIN
"SE" - SWEDEN
"CH" - SWITZERLAND
"UA" - UKRAINE
"AE" - UNITED ARAB EMIRATES
"US" - UNITED STATES
"UK" - UNITED KINGDOM
"UY" - URUGUAY
"VE" - VENEZUELA
FeaturizationConfig (dict) --
The featurization configuration.
ForecastFrequency (string) --
The frequency of predictions in a forecast.
Valid intervals are Y (Year), M (Month), W (Week), D (Day), H (Hour), 30min (30 minutes), 15min (15 minutes), 10min (10 minutes), 5min (5 minutes), and 1min (1 minute). For example, "Y" indicates every year and "5min" indicates every five minutes.
The frequency must be greater than or equal to the TARGET_TIME_SERIES dataset frequency.
When a RELATED_TIME_SERIES dataset is provided, the frequency must be equal to the RELATED_TIME_SERIES dataset frequency.
ForecastDimensions (list) --
An array of dimension (field) names that specify how to group the generated forecast.
For example, suppose that you are generating a forecast for item sales across all of your stores, and your dataset contains a store_id field. If you want the sales forecast for each item by store, you would specify store_id as the dimension.
All forecast dimensions specified in the TARGET_TIME_SERIES dataset don't need to be specified in the CreatePredictor request. All forecast dimensions specified in the RELATED_TIME_SERIES dataset must be specified in the CreatePredictor request.
(string) --
Featurizations (list) --
An array of featurization (transformation) information for the fields of a dataset.
(dict) --
Provides featurization (transformation) information for a dataset field. This object is part of the FeaturizationConfig object.
For example:
{
"AttributeName": "demand",
FeaturizationPipeline [ {
"FeaturizationMethodName": "filling",
"FeaturizationMethodParameters": {"aggregation": "avg", "backfill": "nan"}
} ]
}
AttributeName (string) --
The name of the schema attribute that specifies the data field to be featurized. Amazon Forecast supports the target field of the TARGET_TIME_SERIES and the RELATED_TIME_SERIES datasets. For example, for the RETAIL domain, the target is demand , and for the CUSTOM domain, the target is target_value . For more information, see howitworks-missing-values.
FeaturizationPipeline (list) --
An array of one FeaturizationMethod object that specifies the feature transformation method.
(dict) --
Provides information about the method that featurizes (transforms) a dataset field. The method is part of the FeaturizationPipeline of the Featurization object.
The following is an example of how you specify a FeaturizationMethod object.
{
"FeaturizationMethodName": "filling",
"FeaturizationMethodParameters": {"aggregation": "sum", "middlefill": "zero", "backfill": "zero"}
}
FeaturizationMethodName (string) --
The name of the method. The "filling" method is the only supported method.
FeaturizationMethodParameters (dict) --
The method parameters (key-value pairs), which are a map of override parameters. Specify these parameters to override the default values. Related Time Series attributes do not accept aggregation parameters.
The following list shows the parameters and their valid values for the "filling" featurization method for a Target Time Series dataset. Bold signifies the default value.
aggregation : sum , avg , first , min , max
frontfill : none
middlefill : zero , nan (not a number), value , median , mean , min , max
backfill : zero , nan , value , median , mean , min , max
The following list shows the parameters and their valid values for a Related Time Series featurization method (there are no defaults):
middlefill : zero , value , median , mean , min , max
backfill : zero , value , median , mean , min , max
futurefill : zero , value , median , mean , min , max
To set a filling method to a specific value, set the fill parameter to value and define the value in a corresponding _value parameter. For example, to set backfilling to a value of 2, include the following: "backfill": "value" and "backfill_value":"2" .
(string) --
(string) --
EncryptionConfig (dict) --
An AWS Key Management Service (KMS) key and the AWS Identity and Access Management (IAM) role that Amazon Forecast can assume to access the key.
RoleArn (string) --
The ARN of the IAM role that Amazon Forecast can assume to access the AWS KMS key.
Passing a role across AWS accounts is not allowed. If you pass a role that isn't in your account, you get an InvalidInputException error.
KMSKeyArn (string) --
The Amazon Resource Name (ARN) of the KMS key.
PredictorExecutionDetails (dict) --
Details on the the status and results of the backtests performed to evaluate the accuracy of the predictor. You specify the number of backtests to perform when you call the operation.
PredictorExecutions (list) --
An array of the backtests performed to evaluate the accuracy of the predictor against a particular algorithm. The NumberOfBacktestWindows from the object determines the number of windows in the array.
(dict) --
The algorithm used to perform a backtest and the status of those tests.
AlgorithmArn (string) --
The ARN of the algorithm used to test the predictor.
TestWindows (list) --
An array of test windows used to evaluate the algorithm. The NumberOfBacktestWindows from the object determines the number of windows in the array.
(dict) --
The status, start time, and end time of a backtest, as well as a failure reason if applicable.
TestWindowStart (datetime) --
The time at which the test began.
TestWindowEnd (datetime) --
The time at which the test ended.
Status (string) --
The status of the test. Possible status values are:
ACTIVE
CREATE_IN_PROGRESS
CREATE_FAILED
Message (string) --
If the test failed, the reason why it failed.
EstimatedTimeRemainingInMinutes (integer) --
The estimated time in minutes for the predictor training job to complete.
DatasetImportJobArns (list) --
An array of the ARNs of the dataset import jobs used to import training data for the predictor.
(string) --
AutoMLAlgorithmArns (list) --
When PerformAutoML is specified, the ARN of the chosen algorithm.
(string) --
Status (string) --
The status of the predictor. States include:
ACTIVE
CREATE_PENDING , CREATE_IN_PROGRESS , CREATE_FAILED
DELETE_PENDING , DELETE_IN_PROGRESS , DELETE_FAILED
CREATE_STOPPING , CREATE_STOPPED
Note
The Status of the predictor must be ACTIVE before you can use the predictor to create a forecast.
Message (string) --
If an error occurred, an informational message about the error.
CreationTime (datetime) --
When the model training task was created.
LastModificationTime (datetime) --
The last time the resource was modified. The timestamp depends on the status of the job:
CREATE_PENDING - The CreationTime .
CREATE_IN_PROGRESS - The current timestamp.
CREATE_STOPPING - The current timestamp.
CREATE_STOPPED - When the job stopped.
ACTIVE or CREATE_FAILED - When the job finished or failed.