Package org.apache.sysds.runtime.controlprogram.parfor

# Class RemoteParForUtils

java.lang.Object
org.apache.sysds.runtime.controlprogram.parfor.RemoteParForUtils

public class RemoteParForUtils extends Object
Common functionalities for parfor workers in MR jobs. Used by worker wrappers in mappers (base RemoteParFor) and reducers (fused data partitioning and parfor)

## Constructor Summary

Constructors
Constructor
Description
`RemoteParForUtils()`

## Method Summary

Modifier and Type
Method
Description
`static void`
`cleanupWorkingDirectories()`
Cleanup all temporary files created by this SystemDS process.
`static void`
`cleanupWorkingDirectoriesOnShutdown()`
Cleanup all temporary files created by this SystemDS process, on shutdown via exit or interrupt.
`static void`
`exportLineageItems(long workerID, LocalVariableMap vars, ArrayList<ParForStatementBlock.ResultVar> resultVars, Lineage lineage)`
Export lineage for remote Spark parfor workers.
`static ArrayList<String>`
`exportResultVariables(long workerID, LocalVariableMap vars, ArrayList<ParForStatementBlock.ResultVar> resultVars)`
For remote Spark parfor workers.
`static Lineage[]`
`getLineages(LocalVariableMap[] results)`
`static LocalVariableMap[]`
`getResults(List<scala.Tuple2<Long,String>> out, org.apache.commons.logging.Log LOG)`
`static void`
`setupBufferPool(long workerID, boolean isLocal)`
Init and register-cleanup of buffer pool

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Constructor Details

### RemoteParForUtils

public RemoteParForUtils()

## Method Details

### exportResultVariables

public static ArrayList<String> exportResultVariables(long workerID, LocalVariableMap vars, ArrayList<ParForStatementBlock.ResultVar> resultVars) throws IOException
For remote Spark parfor workers. This is a simplified version compared to MR.

Parameters:
  `workerID` - worker id
  `vars` - local variable map
  `resultVars` - list of result variables
Returns:
  list of result variables
Throws:
  `IOException` - if IOException occurs

### exportLineageItems

public static void exportLineageItems(long workerID, LocalVariableMap vars, ArrayList<ParForStatementBlock.ResultVar> resultVars, Lineage lineage) throws IOException
Export lineage for remote Spark parfor workers.

Parameters:
  `workerID` - worker id
  `vars` - local variable map
  `resultVars` - list of result variables
  `lineage` - lineage object
Throws:
  `IOException` - if IOException occurs

### cleanupWorkingDirectories

public static void cleanupWorkingDirectories()
Cleanup all temporary files created by this SystemDS process.

### cleanupWorkingDirectoriesOnShutdown

public static void cleanupWorkingDirectoriesOnShutdown()
Cleanup all temporary files created by this SystemDS process, on shutdown via exit or interrupt.

### getLineages

public static Lineage[] getLineages(LocalVariableMap[] results)

### getResults

public static LocalVariableMap[] getResults(List<scala.Tuple2<Long,String>> out, org.apache.commons.logging.Log LOG)

### setupBufferPool

public static void setupBufferPool(long workerID, boolean isLocal) throws IOException
Init and register-cleanup of buffer pool

Parameters:
  `workerID` - worker id
  `isLocal` - in local spark mode (single JVM)
Throws:
  `IOException` - exception