Package org.apache.sysds.resource

# Class ResourceCompiler

java.lang.Object
org.apache.sysds.resource.ResourceCompiler

public class ResourceCompiler extends Object
This class does full or partial program recompilation based on given runtime program. It uses the methods provided by `hops.recompile.Recompiler`). It keeps a state of the current recompilation phase in order to decide when to do full recompilation and when not.

## Field Summary

Fields
Modifier and Type
Field
Description
`static final long`
`DEFAULT_DRIVER_MEMORY`
`static final int`
`DEFAULT_DRIVER_THREADS`
`static final long`
`DEFAULT_EXECUTOR_MEMORY`
`static final int`
`DEFAULT_EXECUTOR_THREADS`
`static final int`
`DEFAULT_NUMBER_EXECUTORS`

## Constructor Summary

Constructors
Constructor
Description
`ResourceCompiler()`

## Method Summary

Modifier and Type
Method
Description
`static Program`
`compile(String filePath, Map<String,String> args)`
`static Program`
`compile(String filePath, Map<String,String> args, HashMap<String,String> replaceVars)`
`static Program`
`doFullRecompilation(Program program)`
Recompiling a given program for resource optimization.
`static void`
`replaceFilename(DMLProgram dmlp, HashMap<String,String> replaceVars)`
`static void`
`setSingleNodeResourceConfigs(long nodeMemory, int nodeCores)`
Sets resource configurations for executions in single-node mode including the hardware configurations for the node running the CP.
`static void`
`setSparkClusterResourceConfigs(long driverMemory, int driverCores, int numExecutors, long executorMemory, int executorCores)`
Sets resource configurations for executions in hybrid mode including the hardware configurations for the node running the CP and the worker nodes running Spark executors

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### DEFAULT_DRIVER_MEMORY

public static final long DEFAULT_DRIVER_MEMORY

See Also:
  - Constant Field Values

### DEFAULT_DRIVER_THREADS

public static final int DEFAULT_DRIVER_THREADS

See Also:
  - Constant Field Values

### DEFAULT_EXECUTOR_MEMORY

public static final long DEFAULT_EXECUTOR_MEMORY

See Also:
  - Constant Field Values

### DEFAULT_EXECUTOR_THREADS

public static final int DEFAULT_EXECUTOR_THREADS

See Also:
  - Constant Field Values

### DEFAULT_NUMBER_EXECUTORS

public static final int DEFAULT_NUMBER_EXECUTORS

See Also:
  - Constant Field Values

## Constructor Details

### ResourceCompiler

public ResourceCompiler()

## Method Details

### compile

public static Program compile(String filePath, Map<String,String> args) throws IOException

Throws:
  `IOException`

### compile

public static Program compile(String filePath, Map<String,String> args, HashMap<String,String> replaceVars) throws IOException

Throws:
  `IOException`

### replaceFilename

public static void replaceFilename(DMLProgram dmlp, HashMap<String,String> replaceVars)

### doFullRecompilation

public static Program doFullRecompilation(Program program)
Recompiling a given program for resource optimization. This method should always be called after setting the target resources to `InfrastructureAnalyzer` and `SparkExecutionContext`

Parameters:
  `program` - program to be recompiled
Returns:
  the recompiled program as a new `Program` instance

### setSingleNodeResourceConfigs

public static void setSingleNodeResourceConfigs(long nodeMemory, int nodeCores)
Sets resource configurations for executions in single-node mode including the hardware configurations for the node running the CP.

Parameters:
  `nodeMemory` - memory budget for the node running CP
  `nodeCores` - number of CPU cores for the node running CP

### setSparkClusterResourceConfigs

public static void setSparkClusterResourceConfigs(long driverMemory, int driverCores, int numExecutors, long executorMemory, int executorCores)
Sets resource configurations for executions in hybrid mode including the hardware configurations for the node running the CP and the worker nodes running Spark executors

Parameters:
  `driverMemory` - memory budget for the node running CP
  `driverCores` - number of CPU cores for the node running CP
  `numExecutors` - number of nodes in cluster
  `executorMemory` - memory budget for the nodes running executors
  `executorCores` - number of CPU cores for the nodes running executors