Package org.apache.sysds.resource.enumeration

# Class PruneBasedEnumerator

java.lang.Object
org.apache.sysds.resource.enumeration.Enumerator
org.apache.sysds.resource.enumeration.PruneBasedEnumerator

public class PruneBasedEnumerator extends Enumerator

## Nested Class Summary

## Nested classes/interfaces inherited from class org.apache.sysds.resource.enumeration.Enumerator

`Enumerator.Builder, Enumerator.EnumerationStrategy, Enumerator.OptimizationStrategy`

## Field Summary

### Fields inherited from class org.apache.sysds.resource.enumeration.Enumerator

`COST_DELTA_FRACTION, DEFAULT_MAX_EXECUTORS, DEFAULT_MIN_EXECUTORS`

## Constructor Summary

Constructors
Constructor
Description
`PruneBasedEnumerator(Enumerator.Builder builder)`

## Method Summary

Modifier and Type
Method
Description
`static long`
`combineHash(long executorMemory, int cores)`
Ensures unique mapping for a combination of node memory and number of executor cores due to the discrete nature of the node memory given in bytes.
`ArrayList<Integer>`
`estimateRangeExecutors(int driverCores, long executorMemory, int executorCores)`
Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations
`boolean`
`evaluateSingleNodeExecution(long driverMemory, int cores)`
`static boolean`
`hasSparkInstructions(Program program)`
Checks for Spark instruction in the given runtime program.
`static boolean`
`isInvalidConfiguration(double[] estimates)`
`void`
`preprocessing()`
Called once to enumerate the search space for VM instances for driver or executor nodes.
`void`
`processing()`
Called once after preprocessing to fill the pool with optimal solutions by parsing the enumerated search space.

### Methods inherited from class org.apache.sysds.resource.enumeration.Enumerator

`getCostsWeightFactor, getDriverSpace, getEnumStrategy, getExecutorSpace, getInstances, getMaxPrice, getMaxTime, getOptimalSolution, getOptStrategy, postprocessing, setCostsWeightFactor, setCpuQuota, setDriverSpace, setExecutorSpace, setMinPrice, setMinTime, updateOptimalSolution`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Constructor Details

### PruneBasedEnumerator

public PruneBasedEnumerator(Enumerator.Builder builder)

## Method Details

### preprocessing

public void preprocessing()
Description copied from class: `Enumerator`
Called once to enumerate the search space for VM instances for driver or executor nodes. These instances are being represented as

Specified by:
  `preprocessing` in class `Enumerator`

### processing

public void processing()
Description copied from class: `Enumerator`
Called once after preprocessing to fill the pool with optimal solutions by parsing the enumerated search space. Within its execution the number of potential executor nodes is being estimated (enumerated) dynamically for each parsed executor instance.

Overrides:
  `processing` in class `Enumerator`

### evaluateSingleNodeExecution

public boolean evaluateSingleNodeExecution(long driverMemory, int cores)

Specified by:
  `evaluateSingleNodeExecution` in class `Enumerator`

### estimateRangeExecutors

public ArrayList<Integer> estimateRangeExecutors(int driverCores, long executorMemory, int executorCores)
Description copied from class: `Enumerator`
Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations

Specified by:
  `estimateRangeExecutors` in class `Enumerator`
Parameters:
  `driverCores` - CPU cores for the currently evaluated driver node
  `executorMemory` - memory of currently evaluated executor node
  `executorCores` - CPU cores of currently evaluated executor node
Returns:
  - [min, max]

### combineHash

public static long combineHash(long executorMemory, int cores)
Ensures unique mapping for a combination of node memory and number of executor cores due to the discrete nature of the node memory given in bytes. The smallest margin for cloud instances would be around 500MB ~ 500*1024^2 bytes, what is by far larger than the maximum number of executors core physically possible.

Parameters:
  `executorMemory` - node memory in bytes
  `cores` - number virtual cores (physical threads) for the node
Returns:
  hash value

### isInvalidConfiguration

public static boolean isInvalidConfiguration(double[] estimates)

### hasSparkInstructions

public static boolean hasSparkInstructions(Program program)
Checks for Spark instruction in the given runtime program. It excludes from the check instructions for reblock operations and for caching since these are not always remove from the runtime program even their outputs are never used and ignored at execution.

Parameters:
  `program` - runtime program
Returns:
  boolean to mark if the execution would execute any Spark operation