Package org.apache.sysds.resource.enumeration

# Class InterestBasedEnumerator

java.lang.Object
org.apache.sysds.resource.enumeration.Enumerator
org.apache.sysds.resource.enumeration.InterestBasedEnumerator

public class InterestBasedEnumerator extends Enumerator

## Nested Class Summary

## Nested classes/interfaces inherited from class org.apache.sysds.resource.enumeration.Enumerator

`Enumerator.Builder, Enumerator.EnumerationStrategy, Enumerator.OptimizationStrategy`

## Field Summary

Fields
Modifier and Type
Field
Description
`static final double`
`BROADCAST_MEMORY_FACTOR`
`static final double`
`CACHE_MEMORY_FACTOR`
`static final double`
`MEMORY_DELTA_FRACTION`
`static final double`
`MEMORY_FACTOR`
`static final long`
`MINIMUM_RELEVANT_MEM_ESTIMATE`
`static final boolean`
`USE_MEMORY_DELTA`
different instance families can have slightly different memory characteristics and using memory delta allows not ignoring equivalent instances of different families (e.g.

### Fields inherited from class org.apache.sysds.resource.enumeration.Enumerator

`COST_DELTA_FRACTION, DEFAULT_MAX_EXECUTORS, DEFAULT_MIN_EXECUTORS`

## Constructor Summary

Constructors
Constructor
Description
`InterestBasedEnumerator(Enumerator.Builder builder, boolean interestLargestEstimate, boolean fitDriverMemory, boolean interestBroadcastVars, boolean interestOutputCaching)`

## Method Summary

Modifier and Type
Method
Description
`ArrayList<Integer>`
`estimateRangeExecutors(int driverCores, long executorMemory, int executorCores)`
Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations
`boolean`
`evaluateSingleNodeExecution(long driverMemory, int cores)`
`static TreeSet<Long>`
`getMemoryEstimates(Program currentProgram, boolean outputOnly, double memoryFactor)`
Extracts the memory estimates which original size is larger than `MINIMUM_RELEVANT_MEM_ESTIMATE`
`boolean`
`interestBroadcastVars()`
`boolean`
`interestEstimatesInCPEnabled()`
`boolean`
`interestLargestEstimateEnabled()`
`boolean`
`interestOutputCachingEnabled()`
`void`
`preprocessing()`
Called once to enumerate the search space for VM instances for driver or executor nodes.

### Methods inherited from class org.apache.sysds.resource.enumeration.Enumerator

`getCostsWeightFactor, getDriverSpace, getEnumStrategy, getExecutorSpace, getInstances, getMaxPrice, getMaxTime, getOptimalSolution, getOptStrategy, postprocessing, processing, setCostsWeightFactor, setCpuQuota, setDriverSpace, setExecutorSpace, setMinPrice, setMinTime, updateOptimalSolution`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### MINIMUM_RELEVANT_MEM_ESTIMATE

public static final long MINIMUM_RELEVANT_MEM_ESTIMATE

See Also:
  - Constant Field Values

### USE_MEMORY_DELTA

public static final boolean USE_MEMORY_DELTA
different instance families can have slightly different memory characteristics and using memory delta allows not ignoring equivalent instances of different families (e.g. newer generation use 7.5/15.25/30.5/... GB memory instead of 8/16/32/...)

See Also:
  - Constant Field Values

### MEMORY_DELTA_FRACTION

public static final double MEMORY_DELTA_FRACTION

See Also:
  - Constant Field Values

### MEMORY_FACTOR

public static final double MEMORY_FACTOR

### BROADCAST_MEMORY_FACTOR

public static final double BROADCAST_MEMORY_FACTOR

See Also:
  - Constant Field Values

### CACHE_MEMORY_FACTOR

public static final double CACHE_MEMORY_FACTOR

See Also:
  - Constant Field Values

## Constructor Details

### InterestBasedEnumerator

public InterestBasedEnumerator(Enumerator.Builder builder, boolean interestLargestEstimate, boolean fitDriverMemory, boolean interestBroadcastVars, boolean interestOutputCaching)

## Method Details

### preprocessing

public void preprocessing()
Description copied from class: `Enumerator`
Called once to enumerate the search space for VM instances for driver or executor nodes. These instances are being represented as

Specified by:
  `preprocessing` in class `Enumerator`

### evaluateSingleNodeExecution

public boolean evaluateSingleNodeExecution(long driverMemory, int cores)

Specified by:
  `evaluateSingleNodeExecution` in class `Enumerator`

### estimateRangeExecutors

public ArrayList<Integer> estimateRangeExecutors(int driverCores, long executorMemory, int executorCores)
Description copied from class: `Enumerator`
Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations

Specified by:
  `estimateRangeExecutors` in class `Enumerator`
Parameters:
  `driverCores` - CPU cores for the currently evaluated driver node
  `executorMemory` - memory of currently evaluated executor node
  `executorCores` - CPU cores of currently evaluated executor node
Returns:
  - [min, max]

### getMemoryEstimates

public static TreeSet<Long> getMemoryEstimates(Program currentProgram, boolean outputOnly, double memoryFactor)
Extracts the memory estimates which original size is larger than `MINIMUM_RELEVANT_MEM_ESTIMATE`

Parameters:
  `currentProgram` - program for extracting the memory estimates from
  `outputOnly` - `true` - output estimate only; `false` - sum of input, intermediate and output estimates
  `memoryFactor` - factor for reverse scaling the estimates to avoid scaling the search space parameters representing the nodes' memory budget
Returns:
  memory estimates in ascending order ensured by the `TreeSet` data structure

### interestEstimatesInCPEnabled

public boolean interestEstimatesInCPEnabled()

### interestBroadcastVars

public boolean interestBroadcastVars()

### interestLargestEstimateEnabled

public boolean interestLargestEstimateEnabled()

### interestOutputCachingEnabled

public boolean interestOutputCachingEnabled()