Package org.apache.sysds.resource.enumeration

# Class Enumerator

java.lang.Object
org.apache.sysds.resource.enumeration.Enumerator

Direct Known Subclasses:
  `GridBasedEnumerator`, `InterestBasedEnumerator`, `PruneBasedEnumerator`

public abstract class Enumerator extends Object

## Nested Class Summary

Nested Classes
Modifier and Type
Class
Description
`static class`
`Enumerator.Builder`
`static enum`
`Enumerator.EnumerationStrategy`
`static enum`
`Enumerator.OptimizationStrategy`

## Field Summary

Fields
Modifier and Type
Field
Description
`static final double`
`COST_DELTA_FRACTION`
Time/Monetary delta for considering optimal solutions as fraction
`static final int`
`DEFAULT_MAX_EXECUTORS`
A reasonable upper bound for the possible number of executors is required to set limits for the search space and to avoid evaluating cluster configurations that most probably would have too high distribution overhead
`static final int`
`DEFAULT_MIN_EXECUTORS`

## Constructor Summary

Constructors
Constructor
Description
`Enumerator(Enumerator.Builder builder)`

## Method Summary

Modifier and Type
Method
Description
`abstract ArrayList<Integer>`
`estimateRangeExecutors(int driverCores, long executorMemory, int executorCores)`
Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations
`abstract boolean`
`evaluateSingleNodeExecution(long driverMemory, int cores)`
`double`
`getCostsWeightFactor()`
Meant to be used for testing purposes
`EnumerationUtils.InstanceSearchSpace`
`getDriverSpace()`
Meant to be used for testing purposes
`Enumerator.EnumerationStrategy`
`getEnumStrategy()`
Meant to be used for testing purposes
`EnumerationUtils.InstanceSearchSpace`
`getExecutorSpace()`
Meant to be used for testing purposes
`HashMap<String,CloudInstance>`
`getInstances()`
Meant to be used for testing purposes
`double`
`getMaxPrice()`
Meant to be used for testing purposes
`double`
`getMaxTime()`
Meant to be used for testing purposes
`EnumerationUtils.SolutionPoint`
`getOptimalSolution()`
Meant to be used for testing purposes
`Enumerator.OptimizationStrategy`
`getOptStrategy()`
Meant to be used for testing purposes
`EnumerationUtils.SolutionPoint`
`postprocessing()`
Retrieving the estimated optimal configurations after processing.
`abstract void`
`preprocessing()`
Called once to enumerate the search space for VM instances for driver or executor nodes.
`void`
`processing()`
Called once after preprocessing to fill the pool with optimal solutions by parsing the enumerated search space.
`static void`
`setCostsWeightFactor(double newFactor)`
`static void`
`setCpuQuota(int newQuotaValue)`
`void`
`setDriverSpace(EnumerationUtils.InstanceSearchSpace inputSpace)`
Meant to be used for testing purposes
`void`
`setExecutorSpace(EnumerationUtils.InstanceSearchSpace inputSpace)`
Meant to be used for testing purposes
`static void`
`setMinPrice(double maxPrice)`
`static void`
`setMinTime(double maxTime)`
`void`
`updateOptimalSolution(double newTimeEstimate, double newMonetaryEstimate, EnumerationUtils.ConfigurationPoint newPoint)`
Invokes the estimation of the time and monetary cost based on the compiled program and the given cluster configurations.

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### DEFAULT_MIN_EXECUTORS

public static final int DEFAULT_MIN_EXECUTORS

See Also:
  - Constant Field Values

### DEFAULT_MAX_EXECUTORS

public static final int DEFAULT_MAX_EXECUTORS
A reasonable upper bound for the possible number of executors is required to set limits for the search space and to avoid evaluating cluster configurations that most probably would have too high distribution overhead

See Also:
  - Constant Field Values

### COST_DELTA_FRACTION

public static final double COST_DELTA_FRACTION
Time/Monetary delta for considering optimal solutions as fraction

See Also:
  - Constant Field Values

## Constructor Details

### Enumerator

public Enumerator(Enumerator.Builder builder)

## Method Details

### setCostsWeightFactor

public static void setCostsWeightFactor(double newFactor)

### setMinTime

public static void setMinTime(double maxTime)

### setMinPrice

public static void setMinPrice(double maxPrice)

### setCpuQuota

public static void setCpuQuota(int newQuotaValue)

### preprocessing

public abstract void preprocessing()
Called once to enumerate the search space for VM instances for driver or executor nodes. These instances are being represented as

### processing

public void processing()
Called once after preprocessing to fill the pool with optimal solutions by parsing the enumerated search space. Within its execution the number of potential executor nodes is being estimated (enumerated) dynamically for each parsed executor instance.

### postprocessing

public EnumerationUtils.SolutionPoint postprocessing()
Retrieving the estimated optimal configurations after processing.

Returns:
  optimal cluster configuration and corresponding costs

### evaluateSingleNodeExecution

public abstract boolean evaluateSingleNodeExecution(long driverMemory, int cores)

### estimateRangeExecutors

public abstract ArrayList<Integer> estimateRangeExecutors(int driverCores, long executorMemory, int executorCores)
Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations

Parameters:
  `driverCores` - CPU cores for the currently evaluated driver node
  `executorMemory` - memory of currently evaluated executor node
  `executorCores` - CPU cores of currently evaluated executor node
Returns:
  - [min, max]

### updateOptimalSolution

public void updateOptimalSolution(double newTimeEstimate, double newMonetaryEstimate, EnumerationUtils.ConfigurationPoint newPoint)
Invokes the estimation of the time and monetary cost based on the compiled program and the given cluster configurations. Following the optimization strategy, the given current optimal solution and the new cost estimation, it decides if the given cluster configuration can be potential optimal solution having lower cost or such a cost that is negligibly higher than the current lowest one.

Parameters:
  `newTimeEstimate` - estimated time cost for the given configurations
  `newMonetaryEstimate` - estimated monetary cost for the given configurations
  `newPoint` - new cluster configuration for estimation

### getInstances

public HashMap<String,CloudInstance> getInstances()
Meant to be used for testing purposes

Returns:
  the available instances for enumeration

### getDriverSpace

public EnumerationUtils.InstanceSearchSpace getDriverSpace()
Meant to be used for testing purposes

Returns:
  the object representing the driver search space

### setDriverSpace

public void setDriverSpace(EnumerationUtils.InstanceSearchSpace inputSpace)
Meant to be used for testing purposes

Parameters:
  `inputSpace` - the object representing the driver search space

### getExecutorSpace

public EnumerationUtils.InstanceSearchSpace getExecutorSpace()
Meant to be used for testing purposes

Returns:
  the object representing the executor search space

### setExecutorSpace

public void setExecutorSpace(EnumerationUtils.InstanceSearchSpace inputSpace)
Meant to be used for testing purposes

Parameters:
  `inputSpace` - the object representing the executor search space

### getEnumStrategy

public Enumerator.EnumerationStrategy getEnumStrategy()
Meant to be used for testing purposes

Returns:
  applied enumeration strategy

### getOptStrategy

public Enumerator.OptimizationStrategy getOptStrategy()
Meant to be used for testing purposes

Returns:
  applied optimization strategy

### getCostsWeightFactor

public double getCostsWeightFactor()
Meant to be used for testing purposes

Returns:
  configured weight factor optimization function 'costs'

### getMaxTime

public double getMaxTime()
Meant to be used for testing purposes

Returns:
  configured max time for consideration (seconds)

### getMaxPrice

public double getMaxPrice()
Meant to be used for testing purposes

Returns:
  configured max price for consideration (dollars)

### getOptimalSolution

public EnumerationUtils.SolutionPoint getOptimalSolution()
Meant to be used for testing purposes

Returns:
  current optimal solution