Package org.apache.sysds.api.jmlc

# Class Connection

java.lang.Object
org.apache.sysds.api.jmlc.Connection

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public class Connection extends Object implements Closeable
Interaction with SystemDS using the JMLC (Java Machine Learning Connector) API is initiated with a `Connection` object. The JMLC API is designed after JDBC. A DML script is precompiled by calling the `prepareScript(String, String[], String[])` method or the `prepareScript(String, Map, String[], String[])` method on the `Connection` object, which returns a `PreparedScript` object. Note that this is similar to calling a `prepareStatement` method on a JDBC `Connection` object.

Following this, input variable data is passed to the script by calling the `setFrame`, `setMatrix`, and `setScalar` methods of the `PreparedScript` object. The script is executed via `PreparedScript`'s `executeScript` method, which returns a `ResultVariables` object, which is similar to a JDBC `ResultSet`. Data can be read from a `ResultVariables` object by calling its `getFrame` and `getMatrix` methods.

## Constructor Summary

Constructors
Constructor
Description
`Connection()`
Connection constructor, the starting point for any other JMLC API calls.
`Connection(CompilerConfig.ConfigType... cconfigs)`
Connection constructor, the starting point for any other JMLC API calls.
`Connection(DMLConfig dmlconfig)`
Connection constructor, the starting point for any other JMLC API calls.
`Connection(DMLConfig dmlconfig, CompilerConfig.ConfigType... cconfigs)`
Connection constructor, the starting point for any other JMLC API calls.

## Method Summary

Modifier and Type
Method
Description
`void`
`close()`
Close connection to SystemDS, which clears the thread-local DML and compiler configurations.
`double[][]`
`convertToDoubleMatrix(InputStream input, int rows, int cols)`
Converts an input stream of a string matrix in textcell format into a dense double array.
`double[][]`
`convertToDoubleMatrix(InputStream input, int rows, int cols, String format)`
Converts an input stream of a string matrix in csv or textcell format into a dense double array.
`double[][]`
`convertToDoubleMatrix(String input, int rows, int cols)`
Converts an input string representation of a matrix in textcell format into a dense double array.
`double[][]`
`convertToDoubleMatrix(String input, String meta)`
Converts an input string representation of a matrix in csv or textcell format into a dense double array.
`FrameBlock`
`convertToFrame(InputStream input, int rows, int cols)`
Converts an input stream of a string frame in textcell format into a frame block.
`FrameBlock`
`convertToFrame(InputStream input, int rows, int cols, String format)`
Converts an input stream of a frame in csv or textcell format into a frame block.
`FrameBlock`
`convertToFrame(InputStream input, String meta)`
Converts an input stream of a string frame in csv or textcell format into a frame block.
`FrameBlock`
`convertToFrame(String input, int rows, int cols)`
Converts an input string representation of a frame in textcell format into a frame block.
`FrameBlock`
`convertToFrame(String input, String meta)`
Converts an input string representation of a frame in csv or textcell format into a frame block.
`MatrixBlock`
`convertToMatrix(InputStream input, int rows, int cols)`
Converts an input stream of a string matrix in text format into a matrix block.
`MatrixBlock`
`convertToMatrix(InputStream input, int rows, int cols, String format)`
Converts an input stream of a string matrix in csv or text format into a matrix block.
`MatrixBlock`
`convertToMatrix(InputStream input, String meta)`
Converts an input stream of a string matrix in csv or textcell format into a matrix block.
`MatrixBlock`
`convertToMatrix(String input, int rows, int cols)`
Converts an input string representation of a matrix in textcell format into a matrix block.
`MatrixBlock`
`convertToMatrix(String input, String meta)`
Converts an input string representation of a matrix in csv or textcell format into a matrix block.
`String[][]`
`convertToStringFrame(InputStream input, int rows, int cols)`
Converts an input stream of a string frame in textcell format into a dense string array.
`String[][]`
`convertToStringFrame(InputStream input, int rows, int cols, String format)`
Converts an input stream of a string frame in csv or text format into a dense string array.
`String[][]`
`convertToStringFrame(String input, int rows, int cols)`
Converts an input stream of a string frame in textcell format into a dense string array.
`String[][]`
`convertToStringFrame(String input, String meta)`
Converts an input string representation of a frame in csv or textcell format into a dense string array.
`void`
`gatherMemStats(boolean stats)`
Sets a boolean flag indicating if memory profiling statistics should be gathered.
`PreparedScript`
`prepareScript(String script, String[] inputs, String[] outputs)`
Prepares (precompiles) a script and registers input and output variables.
`PreparedScript`
`prepareScript(String script, Map<String,String> args, String[] inputs, String[] outputs)`
Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
`PreparedScript`
`prepareScript(String script, Map<String,String> nsscripts, Map<String,String> args, String[] inputs, String[] outputs)`
Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
`double[][]`
`readDoubleMatrix(String fname)`
Reads an input matrix in arbitrary format from HDFS into a dense double array.
`double[][]`
`readDoubleMatrix(String fname, Types.FileFormat fmt, long rows, long cols, int blen, long nnz)`
Reads an input matrix in arbitrary format from HDFS into a dense double array.
`String`
`readScript(String fname)`
Read a DML or PyDML file as a string.
`String[][]`
`readStringFrame(String fname)`
Reads an input frame in arbitrary format from HDFS into a dense string array.
`String[][]`
`readStringFrame(String fname, Types.FileFormat fmt, long rows, long cols)`
Reads an input frame in arbitrary format from HDFS into a dense string array.
`FrameBlock`
`readTransformMetaDataFromFile(String spec, String metapath)`
Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
`FrameBlock`
`readTransformMetaDataFromFile(String spec, String metapath, String colDelim)`
Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
`FrameBlock`
`readTransformMetaDataFromPath(String metapath)`
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
`FrameBlock`
`readTransformMetaDataFromPath(String spec, String metapath)`
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
`FrameBlock`
`readTransformMetaDataFromPath(String spec, String metapath, String colDelim)`
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
`void`
`setConfigTypes(boolean activate, CompilerConfig.ConfigType... cconfigs)`
Sets compiler configs.
`void`
`setLineage(boolean lt)`
Sets a boolean flag indicating if lineage trace should be captured
`void`
`setStatistics(boolean stats)`
Sets a boolean flag indicating if runtime statistics should be gathered Same behavior as in "MLContext.setStatistics()"

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Constructor Details

### Connection

public Connection()
Connection constructor, the starting point for any other JMLC API calls.

### Connection

public Connection(CompilerConfig.ConfigType... cconfigs)
Connection constructor, the starting point for any other JMLC API calls. This variant allows to enable a set of boolean compiler configurations.

Parameters:
  `cconfigs` - one or many boolean compiler configurations to enable.

### Connection

public Connection(DMLConfig dmlconfig, CompilerConfig.ConfigType... cconfigs)
Connection constructor, the starting point for any other JMLC API calls. This variant allows to pass a global dml configuration and enable a set of boolean compiler configurations.

Parameters:
  `dmlconfig` - a dml configuration.
  `cconfigs` - one or many boolean compiler configurations to enable.

### Connection

public Connection(DMLConfig dmlconfig)
Connection constructor, the starting point for any other JMLC API calls. This variant allows to pass a global dml configuration.

Parameters:
  `dmlconfig` - a dml configuration.

## Method Details

### setConfigTypes

public void setConfigTypes(boolean activate, CompilerConfig.ConfigType... cconfigs)
Sets compiler configs.

Parameters:
  `activate` - activate or disable
  `cconfigs` - the configs to set

### setStatistics

public void setStatistics(boolean stats)
Sets a boolean flag indicating if runtime statistics should be gathered Same behavior as in "MLContext.setStatistics()"

Parameters:
  `stats` - boolean value with true indicating statistics should be gathered

### setLineage

public void setLineage(boolean lt)
Sets a boolean flag indicating if lineage trace should be captured

Parameters:
  `lt` - boolean value with true indicating lineage should be captured

### gatherMemStats

public void gatherMemStats(boolean stats)
Sets a boolean flag indicating if memory profiling statistics should be gathered. The option is false by default.

Parameters:
  `stats` - boolean value with true indicating memory statistics should be gathered

### prepareScript

public PreparedScript prepareScript(String script, String[] inputs, String[] outputs)
Prepares (precompiles) a script and registers input and output variables.

Parameters:
  `script` - string representing the DML or PyDML script
  `inputs` - string array of input variables to register
  `outputs` - string array of output variables to register
Returns:
  PreparedScript object representing the precompiled script

### prepareScript

public PreparedScript prepareScript(String script, Map<String,String> args, String[] inputs, String[] outputs)
Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.

Parameters:
  `script` - string representing the DML or PyDML script
  `args` - map of input parameters ($) and their values
  `inputs` - string array of input variables to register
  `outputs` - string array of output variables to register
Returns:
  PreparedScript object representing the precompiled script

### prepareScript

public PreparedScript prepareScript(String script, Map<String,String> nsscripts, Map<String,String> args, String[] inputs, String[] outputs)
Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.

Parameters:
  `script` - string representing of the DML or PyDML script
  `nsscripts` - map (name, script) of the DML or PyDML namespace scripts
  `args` - map of input parameters ($) and their values
  `inputs` - string array of input variables to register
  `outputs` - string array of output variables to register
Returns:
  PreparedScript object representing the precompiled script

### close

public void close()
Close connection to SystemDS, which clears the thread-local DML and compiler configurations.

Specified by:
  `close` in interface `AutoCloseable`
Specified by:
  `close` in interface `Closeable`

### readScript

public String readScript(String fname) throws IOException
Read a DML or PyDML file as a string.

Parameters:
  `fname` - the filename of the script
Returns:
  string content of the script file
Throws:
  `IOException` - if IOException occurs

### readDoubleMatrix

public double[][] readDoubleMatrix(String fname) throws IOException
Reads an input matrix in arbitrary format from HDFS into a dense double array. NOTE: this call currently only supports default configurations for CSV.

Parameters:
  `fname` - the filename of the input matrix
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

### readDoubleMatrix

public double[][] readDoubleMatrix(String fname, Types.FileFormat fmt, long rows, long cols, int blen, long nnz) throws IOException
Reads an input matrix in arbitrary format from HDFS into a dense double array. NOTE: this call currently only supports default configurations for CSV.

Parameters:
  `fname` - the filename of the input matrix
  `fmt` - file format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
  `blen` - block length
  `nnz` - number of non-zero values, -1 indicates unknown
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

### convertToDoubleMatrix

public double[][] convertToDoubleMatrix(String input, String meta) throws IOException
Converts an input string representation of a matrix in csv or textcell format into a dense double array. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - string matrix in csv or textcell format
  `meta` - string representing SystemDS matrix metadata in JSON format
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

### convertToDoubleMatrix

public double[][] convertToDoubleMatrix(String input, int rows, int cols) throws IOException
Converts an input string representation of a matrix in textcell format into a dense double array.

Parameters:
  `input` - string matrix in textcell format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

### convertToDoubleMatrix

public double[][] convertToDoubleMatrix(InputStream input, int rows, int cols) throws IOException
Converts an input stream of a string matrix in textcell format into a dense double array.

Parameters:
  `input` - InputStream to a string matrix in textcell format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

### convertToDoubleMatrix

public double[][] convertToDoubleMatrix(InputStream input, int rows, int cols, String format) throws IOException
Converts an input stream of a string matrix in csv or textcell format into a dense double array.

Parameters:
  `input` - InputStream to a string matrix in csv or textcell format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
  `format` - input format of the given stream
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

### convertToMatrix

public MatrixBlock convertToMatrix(String input, String meta) throws IOException
Converts an input string representation of a matrix in csv or textcell format into a matrix block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - string matrix in csv or textcell format
  `meta` - string representing SystemDS matrix metadata in JSON format
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

### convertToMatrix

public MatrixBlock convertToMatrix(InputStream input, String meta) throws IOException
Converts an input stream of a string matrix in csv or textcell format into a matrix block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - InputStream to a string matrix in csv or textcell format
  `meta` - string representing SystemDS matrix metadata in JSON format
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

### convertToMatrix

public MatrixBlock convertToMatrix(String input, int rows, int cols) throws IOException
Converts an input string representation of a matrix in textcell format into a matrix block.

Parameters:
  `input` - string matrix in textcell format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

### convertToMatrix

public MatrixBlock convertToMatrix(InputStream input, int rows, int cols) throws IOException
Converts an input stream of a string matrix in text format into a matrix block.

Parameters:
  `input` - InputStream to a string matrix in text format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

### convertToMatrix

public MatrixBlock convertToMatrix(InputStream input, int rows, int cols, String format) throws IOException
Converts an input stream of a string matrix in csv or text format into a matrix block.

Parameters:
  `input` - InputStream to a string matrix in csv or text format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
  `format` - input format of the given stream
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

### readStringFrame

public String[][] readStringFrame(String fname) throws IOException
Reads an input frame in arbitrary format from HDFS into a dense string array. NOTE: this call currently only supports default configurations for CSV.

Parameters:
  `fname` - the filename of the input frame
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

### readStringFrame

public String[][] readStringFrame(String fname, Types.FileFormat fmt, long rows, long cols) throws IOException
Reads an input frame in arbitrary format from HDFS into a dense string array. NOTE: this call currently only supports default configurations for CSV.

Parameters:
  `fname` - the filename of the input frame
  `fmt` - file format type
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

### convertToStringFrame

public String[][] convertToStringFrame(String input, String meta) throws IOException
Converts an input string representation of a frame in csv or textcell format into a dense string array. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - string frame in csv or textcell format
  `meta` - string representing SystemDS frame metadata in JSON format
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

### convertToStringFrame

public String[][] convertToStringFrame(String input, int rows, int cols) throws IOException
Converts an input stream of a string frame in textcell format into a dense string array.

Parameters:
  `input` - string frame in textcell format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

### convertToStringFrame

public String[][] convertToStringFrame(InputStream input, int rows, int cols) throws IOException
Converts an input stream of a string frame in textcell format into a dense string array.

Parameters:
  `input` - InputStream to a string frame in text format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

### convertToStringFrame

public String[][] convertToStringFrame(InputStream input, int rows, int cols, String format) throws IOException
Converts an input stream of a string frame in csv or text format into a dense string array.

Parameters:
  `input` - InputStream to a string frame in csv or text format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
  `format` - input format of the given stream
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

### convertToFrame

public FrameBlock convertToFrame(String input, String meta) throws IOException
Converts an input string representation of a frame in csv or textcell format into a frame block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - string frame in csv or textcell format
  `meta` - string representing SystemDS frame metadata in JSON format
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

### convertToFrame

public FrameBlock convertToFrame(InputStream input, String meta) throws IOException
Converts an input stream of a string frame in csv or textcell format into a frame block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - InputStream to a string frame in csv or textcell format
  `meta` - string representing SystemDS frame metadata in JSON format
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

### convertToFrame

public FrameBlock convertToFrame(String input, int rows, int cols) throws IOException
Converts an input string representation of a frame in textcell format into a frame block.

Parameters:
  `input` - string frame in textcell format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

### convertToFrame

public FrameBlock convertToFrame(InputStream input, int rows, int cols) throws IOException
Converts an input stream of a string frame in textcell format into a frame block.

Parameters:
  `input` - InputStream to a string frame in textcell format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

### convertToFrame

public FrameBlock convertToFrame(InputStream input, int rows, int cols, String format) throws IOException
Converts an input stream of a frame in csv or textcell format into a frame block.

Parameters:
  `input` - InputStream to a string frame in csv or textcell format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
  `format` - input format of the given stream
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

### readTransformMetaDataFromFile

public FrameBlock readTransformMetaDataFromFile(String spec, String metapath) throws IOException
Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.

Parameters:
  `spec` - transform specification as json string
  `metapath` - hdfs file path to meta data directory
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

### readTransformMetaDataFromFile

public FrameBlock readTransformMetaDataFromFile(String spec, String metapath, String colDelim) throws IOException
Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.

Parameters:
  `spec` - transform specification as json string
  `metapath` - hdfs file path to meta data directory
  `colDelim` - separator for processing column names in the meta data file 'column.names'
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

### readTransformMetaDataFromPath

public FrameBlock readTransformMetaDataFromPath(String metapath) throws IOException
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.

Parameters:
  `metapath` - resource path to meta data directory
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

### readTransformMetaDataFromPath

public FrameBlock readTransformMetaDataFromPath(String spec, String metapath) throws IOException
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.

Parameters:
  `spec` - transform specification as json string
  `metapath` - resource path to meta data directory
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

### readTransformMetaDataFromPath

public FrameBlock readTransformMetaDataFromPath(String spec, String metapath, String colDelim) throws IOException
Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.

Parameters:
  `spec` - transform specification as json string
  `metapath` - resource path to meta data directory
  `colDelim` - separator for processing column names in the meta data file 'column.names'
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs