Package org.apache.sysds.api.mlcontext

# Class MLContextUtil

java.lang.Object
org.apache.sysds.api.mlcontext.MLContextUtil

public final class MLContextUtil extends Object
Utility class containing methods for working with the MLContext API.

## Field Summary

Fields
Modifier and Type
Field
Description
`static final Class[]`
`ALL_SUPPORTED_DATA_TYPES`
All data types supported by the MLContext API.
`static final Class[]`
`BASIC_DATA_TYPES`
Basic data types supported by the MLContext API.
`static final String`
`BUILD_TIME_NOT_AVAILABLE`
Build time not available message.
`static final Class[]`
`COMPLEX_DATA_TYPES`
Complex data types supported by the MLContext API.
`static final String`
`VERSION_NOT_AVAILABLE`
Version not available message.

## Constructor Summary

Constructors
Constructor
Description
`MLContextUtil()`

## Method Summary

Modifier and Type
Method
Description
`static void`
`checkInputParameterType(String parameterName, Object parameterValue)`
Verify that the type of input parameter value is supported.
`static void`
`checkInputValueType(String name, Object value)`
Verify that the type of input value is supported.
`static void`
`checkInputValueTypes(Map<String,Object> inputs)`
Verify that the types of input values are supported.
`static Map<String,String>`
`convertInputParametersForParser(Map<String,Object> basicInputParameterMap)`
Converts non-string basic input parameter values to strings to pass to the parser.
`static Data`
`convertInputType(String parameterName, Object parameterValue)`
Convert input types to internal SystemDS representations
`static Data`
`convertInputType(String parameterName, Object parameterValue, Metadata metadata)`
Convert input types to internal SystemDS representations
`static void`
`deleteRemoveVariableInstructions(Program progam)`
Delete the 'remove variable' instructions from a runtime program.
`static String`
`determineOutputTypeAsString(LocalVariableMap symbolTable, String outputName)`
Obtain a symbol table output type as a String
`static String`
`displayInputs(String name, Map<String,Object> map, LocalVariableMap symbolTable)`
Obtain a display of script inputs.
`static String`
`displayMap(String mapName, Map<String,Object> map)`
Display the keys and values in a Map
`static String`
`displayOutputs(String name, Set<String> outputNames, LocalVariableMap symbolTable)`
Obtain a display of the script outputs.
`static String`
`displayOutputs(Set<String> outputNames, LocalVariableMap symbolTable)`
Obtain a display of the script outputs.
`static String`
`displaySet(String setName, Set<String> set)`
Display the values in a Set
`static String`
`displaySymbolTable(String name, LocalVariableMap symbolTable)`
Display the keys and values in the symbol table
`static String`
`displaySymbolTable(LocalVariableMap symbolTable)`
Display the keys and values in the symbol table
`static boolean`
`doesDataFrameLookLikeMatrix(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)`
Examine the DataFrame schema to determine whether the data appears to be a matrix.
`static boolean`
`doesSymbolTableContainFrameObject(LocalVariableMap symbolTable, String variableName)`
Determine if the symbol table contains a FrameObject with the given variable name.
`static boolean`
`doesSymbolTableContainMatrixObject(LocalVariableMap symbolTable, String variableName)`
Determine if the symbol table contains a MatrixObject with the given variable name.
`static String`
`getBasicTypeString(Object object)`
Obtain the SystemDS scalar value type string equivalent of an accepted basic type (Integer, Boolean, Double, String)
`static org.apache.spark.api.java.JavaSparkContext`
`getJavaSparkContext(MLContext mlContext)`
Obtain the Java Spark Context
`static org.apache.spark.api.java.JavaSparkContext`
`getJavaSparkContextFromProxy()`
Obtain the Java Spark Context from the MLContextProxy
`static org.apache.spark.SparkContext`
`getSparkContext(MLContext mlContext)`
Obtain the Spark Context
`static org.apache.spark.SparkContext`
`getSparkContextFromProxy()`
Obtain the Spark Context from the MLContextProxy
`static org.apache.spark.sql.SparkSession`
`getSparkSessionFromProxy()`
Obtain the Spark Session from the MLContextProxy
`static boolean`
`isBasicType(Object object)`
Is the object one of the supported basic data types? (Integer, Boolean, Double, String)
`static boolean`
`isComplexType(Object object)`
Is the object one of the supported complex data types? (JavaRDD, RDD, DataFrame, Matrix, double[][], MatrixBlock, URL)
`static boolean`
`isCSVLineAllNumbers(String line)`
If no metadata is supplied for an RDD or JavaRDD, this method can be used to determine whether the data appears to be matrix (or a frame)
`static boolean`
`isSparkVersionSupported(String sparkVersion, String minimumRecommendedSparkVersion)`
Determine whether the Spark version is supported.
`static String`
`quotedString(String str)`
Return a double-quoted string with inner single and double quotes escaped.
`static void`
`setCompilerConfig()`
Set SystemDS compiler configuration properties for MLContext
`static void`
`setConfig(String configFilePath)`
Set SystemDS configuration properties based on a configuration file.
`static void`
`setDefaultConfig()`
Set default SystemDS configuration properties.
`static void`
`verifySparkVersionSupported(org.apache.spark.sql.SparkSession spark)`
Check that the Spark version is supported.
`static String`
`welcomeMessage()`
The SystemDS welcome message

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### VERSION_NOT_AVAILABLE

public static final String VERSION_NOT_AVAILABLE
Version not available message.

See Also:
  - Constant Field Values

### BUILD_TIME_NOT_AVAILABLE

public static final String BUILD_TIME_NOT_AVAILABLE
Build time not available message.

See Also:
  - Constant Field Values

### BASIC_DATA_TYPES

public static final Class[] BASIC_DATA_TYPES
Basic data types supported by the MLContext API.

### COMPLEX_DATA_TYPES

public static final Class[] COMPLEX_DATA_TYPES
Complex data types supported by the MLContext API.

### ALL_SUPPORTED_DATA_TYPES

public static final Class[] ALL_SUPPORTED_DATA_TYPES
All data types supported by the MLContext API.

## Constructor Details

### MLContextUtil

public MLContextUtil()

## Method Details

### isSparkVersionSupported

public static boolean isSparkVersionSupported(String sparkVersion, String minimumRecommendedSparkVersion)
Determine whether the Spark version is supported.

Parameters:
  `sparkVersion` - Spark version string (ie, "1.5.0").
  `minimumRecommendedSparkVersion` - Minimum recommended Spark version string (ie, "2.1.0").
Returns:
  `true` if Spark version supported; otherwise `false`.

### verifySparkVersionSupported

public static void verifySparkVersionSupported(org.apache.spark.sql.SparkSession spark)
Check that the Spark version is supported. If it isn't supported, throw an MLContextException.

Parameters:
  `spark` - SparkSession
Throws:
  `MLContextException` - thrown if Spark version isn't supported

### setDefaultConfig

public static void setDefaultConfig()
Set default SystemDS configuration properties.

### setConfig

public static void setConfig(String configFilePath)
Set SystemDS configuration properties based on a configuration file.

Parameters:
  `configFilePath` - Path to configuration file.
Throws:
  `MLContextException` - if configuration file was not found or a parse exception occurred

### setCompilerConfig

public static void setCompilerConfig()
Set SystemDS compiler configuration properties for MLContext

### checkInputValueTypes

public static void checkInputValueTypes(Map<String,Object> inputs)
Verify that the types of input values are supported.

Parameters:
  `inputs` - Map of String/Object pairs
Throws:
  `MLContextException` - if an input value type is not supported

### checkInputValueType

public static void checkInputValueType(String name, Object value)
Verify that the type of input value is supported.

Parameters:
  `name` - The name of the input
  `value` - The value of the input
Throws:
  `MLContextException` - if the input value type is not supported

### checkInputParameterType

public static void checkInputParameterType(String parameterName, Object parameterValue)
Verify that the type of input parameter value is supported.

Parameters:
  `parameterName` - The name of the input parameter
  `parameterValue` - The value of the input parameter
Throws:
  `MLContextException` - if the input parameter value type is not supported

### isBasicType

public static boolean isBasicType(Object object)
Is the object one of the supported basic data types? (Integer, Boolean, Double, String)

Parameters:
  `object` - the object type to be examined
Returns:
  `true` if type is a basic data type; otherwise `false`.

### getBasicTypeString

public static String getBasicTypeString(Object object)
Obtain the SystemDS scalar value type string equivalent of an accepted basic type (Integer, Boolean, Double, String)

Parameters:
  `object` - the object type to be examined
Returns:
  a String representing the type as a SystemDS scalar value type

### isComplexType

public static boolean isComplexType(Object object)
Is the object one of the supported complex data types? (JavaRDD, RDD, DataFrame, Matrix, double[][], MatrixBlock, URL)

Parameters:
  `object` - the object type to be examined
Returns:
  `true` if type is a complex data type; otherwise `false`.

### convertInputParametersForParser

public static Map<String,String> convertInputParametersForParser(Map<String,Object> basicInputParameterMap)
Converts non-string basic input parameter values to strings to pass to the parser.

Parameters:
  `basicInputParameterMap` - map of input parameters
Returns:
  map of String/String name/value pairs

### convertInputType

public static Data convertInputType(String parameterName, Object parameterValue)
Convert input types to internal SystemDS representations

Parameters:
  `parameterName` - The name of the input parameter
  `parameterValue` - The value of the input parameter
Returns:
  input in SystemDS data representation

### convertInputType

public static Data convertInputType(String parameterName, Object parameterValue, Metadata metadata)
Convert input types to internal SystemDS representations

Parameters:
  `parameterName` - The name of the input parameter
  `parameterValue` - The value of the input parameter
  `metadata` - matrix/frame metadata
Returns:
  input in SystemDS data representation

### isCSVLineAllNumbers

public static boolean isCSVLineAllNumbers(String line)
If no metadata is supplied for an RDD or JavaRDD, this method can be used to determine whether the data appears to be matrix (or a frame)

Parameters:
  `line` - a line of the RDD
Returns:
  `true` if all the csv-separated values are numbers, `false` otherwise

### doesDataFrameLookLikeMatrix

public static boolean doesDataFrameLookLikeMatrix(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)
Examine the DataFrame schema to determine whether the data appears to be a matrix.

Parameters:
  `df` - the DataFrame
Returns:
  `true` if the DataFrame appears to be a matrix, `false` otherwise

### quotedString

public static String quotedString(String str)
Return a double-quoted string with inner single and double quotes escaped.

Parameters:
  `str` - the original string
Returns:
  double-quoted string with inner single and double quotes escaped

### displayMap

public static String displayMap(String mapName, Map<String,Object> map)
Display the keys and values in a Map

Parameters:
  `mapName` - the name of the map
  `map` - Map of String keys and Object values
Returns:
  the keys and values in the Map as a String

### displaySet

public static String displaySet(String setName, Set<String> set)
Display the values in a Set

Parameters:
  `setName` - the name of the Set
  `set` - Set of String values
Returns:
  the values in the Set as a String

### displaySymbolTable

public static String displaySymbolTable(String name, LocalVariableMap symbolTable)
Display the keys and values in the symbol table

Parameters:
  `name` - the name of the symbol table
  `symbolTable` - the LocalVariableMap
Returns:
  the keys and values in the symbol table as a String

### displaySymbolTable

public static String displaySymbolTable(LocalVariableMap symbolTable)
Display the keys and values in the symbol table

Parameters:
  `symbolTable` - the LocalVariableMap
Returns:
  the keys and values in the symbol table as a String

### determineOutputTypeAsString

public static String determineOutputTypeAsString(LocalVariableMap symbolTable, String outputName)
Obtain a symbol table output type as a String

Parameters:
  `symbolTable` - the symbol table
  `outputName` - the name of the output variable
Returns:
  the symbol table output type for a variable as a String

### displayInputs

public static String displayInputs(String name, Map<String,Object> map, LocalVariableMap symbolTable)
Obtain a display of script inputs.

Parameters:
  `name` - the title to display for the inputs
  `map` - the map of inputs
  `symbolTable` - the symbol table
Returns:
  the script inputs represented as a String

### displayOutputs

public static String displayOutputs(String name, Set<String> outputNames, LocalVariableMap symbolTable)
Obtain a display of the script outputs.

Parameters:
  `name` - the title to display for the outputs
  `outputNames` - the names of the output variables
  `symbolTable` - the symbol table
Returns:
  the script outputs represented as a String

### displayOutputs

public static String displayOutputs(Set<String> outputNames, LocalVariableMap symbolTable)
Obtain a display of the script outputs.

Parameters:
  `outputNames` - the names of the output variables
  `symbolTable` - the symbol table
Returns:
  the script outputs represented as a String

### welcomeMessage

public static String welcomeMessage()
The SystemDS welcome message

Returns:
  the SystemDS welcome message

### getSparkContext

public static org.apache.spark.SparkContext getSparkContext(MLContext mlContext)
Obtain the Spark Context

Parameters:
  `mlContext` - the SystemDS MLContext
Returns:
  the Spark Context

### getJavaSparkContext

public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContext(MLContext mlContext)
Obtain the Java Spark Context

Parameters:
  `mlContext` - the SystemDS MLContext
Returns:
  the Java Spark Context

### getSparkContextFromProxy

public static org.apache.spark.SparkContext getSparkContextFromProxy()
Obtain the Spark Context from the MLContextProxy

Returns:
  the Spark Context

### getJavaSparkContextFromProxy

public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContextFromProxy()
Obtain the Java Spark Context from the MLContextProxy

Returns:
  the Java Spark Context

### getSparkSessionFromProxy

public static org.apache.spark.sql.SparkSession getSparkSessionFromProxy()
Obtain the Spark Session from the MLContextProxy

Returns:
  the Spark Session

### doesSymbolTableContainFrameObject

public static boolean doesSymbolTableContainFrameObject(LocalVariableMap symbolTable, String variableName)
Determine if the symbol table contains a FrameObject with the given variable name.

Parameters:
  `symbolTable` - the LocalVariableMap
  `variableName` - the variable name
Returns:
  `true` if the variable in the symbol table is a FrameObject, `false` otherwise.

### doesSymbolTableContainMatrixObject

public static boolean doesSymbolTableContainMatrixObject(LocalVariableMap symbolTable, String variableName)
Determine if the symbol table contains a MatrixObject with the given variable name.

Parameters:
  `symbolTable` - the LocalVariableMap
  `variableName` - the variable name
Returns:
  `true` if the variable in the symbol table is a MatrixObject, `false` otherwise.

### deleteRemoveVariableInstructions

public static void deleteRemoveVariableInstructions(Program progam)
Delete the 'remove variable' instructions from a runtime program.

Parameters:
  `progam` - runtime program