2016/04/21 - Amazon Elastic MapReduce - 2 updated api methods
{'InstanceGroups': {'ShrinkPolicy': {'DecommissionTimeout': 'integer',
'InstanceResizePolicy': {'InstanceTerminationTimeout': 'integer',
'InstancesToProtect': ['string'],
'InstancesToTerminate': ['string']}}}}
Provides all available details about the instance groups in a cluster.
Request Syntax
client.list_instance_groups(
ClusterId='string',
Marker='string'
)
string
[REQUIRED]
The identifier of the cluster for which to list the instance groups.
string
The pagination token that indicates the next set of results to retrieve.
dict
Response Syntax
{
'InstanceGroups': [
{
'Id': 'string',
'Name': 'string',
'Market': 'ON_DEMAND'|'SPOT',
'InstanceGroupType': 'MASTER'|'CORE'|'TASK',
'BidPrice': 'string',
'InstanceType': 'string',
'RequestedInstanceCount': 123,
'RunningInstanceCount': 123,
'Status': {
'State': 'PROVISIONING'|'BOOTSTRAPPING'|'RUNNING'|'RESIZING'|'SUSPENDED'|'TERMINATING'|'TERMINATED'|'ARRESTED'|'SHUTTING_DOWN'|'ENDED',
'StateChangeReason': {
'Code': 'INTERNAL_ERROR'|'VALIDATION_ERROR'|'INSTANCE_FAILURE'|'CLUSTER_TERMINATED',
'Message': 'string'
},
'Timeline': {
'CreationDateTime': datetime(2015, 1, 1),
'ReadyDateTime': datetime(2015, 1, 1),
'EndDateTime': datetime(2015, 1, 1)
}
},
'Configurations': [
{
'Classification': 'string',
'Configurations': {'... recursive ...'},
'Properties': {
'string': 'string'
}
},
],
'EbsBlockDevices': [
{
'VolumeSpecification': {
'VolumeType': 'string',
'Iops': 123,
'SizeInGB': 123
},
'Device': 'string'
},
],
'EbsOptimized': True|False,
'ShrinkPolicy': {
'DecommissionTimeout': 123,
'InstanceResizePolicy': {
'InstancesToTerminate': [
'string',
],
'InstancesToProtect': [
'string',
],
'InstanceTerminationTimeout': 123
}
}
},
],
'Marker': 'string'
}
Response Structure
(dict) --
This input determines which instance groups to retrieve.
InstanceGroups (list) --
The list of instance groups for the cluster and given filters.
(dict) --
This entity represents an instance group, which is a group of instances that have common purpose. For example, CORE instance group is used for HDFS.
Id (string) --
The identifier of the instance group.
Name (string) --
The name of the instance group.
Market (string) --
The marketplace to provision instances for this group. Valid values are ON_DEMAND or SPOT.
InstanceGroupType (string) --
The type of the instance group. Valid values are MASTER, CORE or TASK.
BidPrice (string) --
The bid price for each EC2 instance in the instance group when launching nodes as Spot Instances, expressed in USD.
InstanceType (string) --
The EC2 instance type for all instances in the instance group.
RequestedInstanceCount (integer) --
The target number of instances for the instance group.
RunningInstanceCount (integer) --
The number of instances currently running in this instance group.
Status (dict) --
The current status of the instance group.
State (string) --
The current state of the instance group.
StateChangeReason (dict) --
The status change reason details for the instance group.
Code (string) --
The programmable code for the state change reason.
Message (string) --
The status change reason description.
Timeline (dict) --
The timeline of the instance group status over time.
CreationDateTime (datetime) --
The creation date and time of the instance group.
ReadyDateTime (datetime) --
The date and time when the instance group became ready to perform tasks.
EndDateTime (datetime) --
The date and time when the instance group terminated.
Configurations (list) --
Note
Amazon EMR releases 4.x or later.
The list of configurations supplied for an EMR cluster instance group. You can specify a separate configuration for each instance group (master, core, and task).
(dict) --
Note
Amazon EMR releases 4.x or later.
Specifies a hardware and software configuration of the EMR cluster. This includes configurations for applications and software bundled with Amazon EMR. The Configuration object is a JSON object which is defined by a classification and a set of properties. Configurations can be nested, so a configuration may have its own Configuration objects listed.
Classification (string) --
The classification of a configuration. For more information see, Amazon EMR Configurations.
Configurations (list) --
A list of configurations you apply to this configuration object.
Properties (dict) --
A set of properties supplied to the Configuration object.
(string) --
(string) --
EbsBlockDevices (list) --
The EBS block devices that are mapped to this instance group.
(dict) --
Configuration of requested EBS block device associated with the instance group.
VolumeSpecification (dict) --
EBS volume specifications such as volume type, IOPS, and size(GiB) that will be requested for the EBS volume attached to an EC2 instance in the cluster.
VolumeType (string) --
The volume type. Volume types supported are gp2, io1, standard.
Iops (integer) --
The number of I/O operations per second (IOPS) that the volume supports.
SizeInGB (integer) --
The volume size, in gibibytes (GiB). This can be a number from 1 – 1024. If the volume type is EBS-optimized, the minimum value is 10.
Device (string) --
The device name that is exposed to the instance, such as /dev/sdh.
EbsOptimized (boolean) --
If the instance group is EBS-optimized. An Amazon EBS–optimized instance uses an optimized configuration stack and provides additional, dedicated capacity for Amazon EBS I/O.
ShrinkPolicy (dict) --
Policy for customizing shrink operations.
DecommissionTimeout (integer) --
The desired timeout for decommissioning an instance. Overrides the default YARN decommissioning timeout.
InstanceResizePolicy (dict) --
Custom policy for requesting termination protection or termination of specific instances when shrinking an instance group.
InstancesToTerminate (list) --
Specific list of instances to be terminated when shrinking an instance group.
(string) --
InstancesToProtect (list) --
Specific list of instances to be protected when shrinking an instance group.
(string) --
InstanceTerminationTimeout (integer) --
Decommissioning timeout override for the specific list of instances to be terminated.
Marker (string) --
The pagination token that indicates the next set of results to retrieve.
{'InstanceGroups': {'ShrinkPolicy': {'DecommissionTimeout': 'integer',
'InstanceResizePolicy': {'InstanceTerminationTimeout': 'integer',
'InstancesToProtect': ['string'],
'InstancesToTerminate': ['string']}}}}
ModifyInstanceGroups modifies the number of nodes and configuration settings of an instance group. The input parameters include the new target instance count for the group and the instance group ID. The call will either succeed or fail atomically.
Request Syntax
client.modify_instance_groups(
InstanceGroups=[
{
'InstanceGroupId': 'string',
'InstanceCount': 123,
'EC2InstanceIdsToTerminate': [
'string',
],
'ShrinkPolicy': {
'DecommissionTimeout': 123,
'InstanceResizePolicy': {
'InstancesToTerminate': [
'string',
],
'InstancesToProtect': [
'string',
],
'InstanceTerminationTimeout': 123
}
}
},
]
)
list
Instance groups to change.
(dict) --
Modify an instance group size.
InstanceGroupId (string) -- [REQUIRED]
Unique ID of the instance group to expand or shrink.
InstanceCount (integer) --
Target size for the instance group.
EC2InstanceIdsToTerminate (list) --
The EC2 InstanceIds to terminate. Once you terminate the instances, the instance group will not return to its original requested size.
(string) --
ShrinkPolicy (dict) --
Policy for customizing shrink operations.
DecommissionTimeout (integer) --
The desired timeout for decommissioning an instance. Overrides the default YARN decommissioning timeout.
InstanceResizePolicy (dict) --
Custom policy for requesting termination protection or termination of specific instances when shrinking an instance group.
InstancesToTerminate (list) --
Specific list of instances to be terminated when shrinking an instance group.
(string) --
InstancesToProtect (list) --
Specific list of instances to be protected when shrinking an instance group.
(string) --
InstanceTerminationTimeout (integer) --
Decommissioning timeout override for the specific list of instances to be terminated.
None