# Uses of Class
org.apache.sysds.runtime.meta.DataCharacteristics

Packages that use DataCharacteristics
Package
Description
org.apache.sysds.api.mlcontext
org.apache.sysds.hops
org.apache.sysds.hops.estim
org.apache.sysds.hops.recompile
org.apache.sysds.hops.rewrite
org.apache.sysds.lops
org.apache.sysds.resource.cost
org.apache.sysds.runtime.compress.io
org.apache.sysds.runtime.controlprogram
org.apache.sysds.runtime.controlprogram.caching
org.apache.sysds.runtime.controlprogram.context
org.apache.sysds.runtime.controlprogram.federated
org.apache.sysds.runtime.controlprogram.parfor
org.apache.sysds.runtime.data
org.apache.sysds.runtime.frame.data
org.apache.sysds.runtime.functionobjects
org.apache.sysds.runtime.instructions.cp
org.apache.sysds.runtime.instructions.ooc
org.apache.sysds.runtime.instructions.spark
org.apache.sysds.runtime.instructions.spark.data
org.apache.sysds.runtime.instructions.spark.functions
org.apache.sysds.runtime.instructions.spark.utils
org.apache.sysds.runtime.io
org.apache.sysds.runtime.lineage
org.apache.sysds.runtime.matrix.data
org.apache.sysds.runtime.meta
org.apache.sysds.runtime.ooc.stream
org.apache.sysds.runtime.ooc.util
org.apache.sysds.runtime.util

## Uses of DataCharacteristics in org.apache.sysds.api.mlcontext

Constructors in org.apache.sysds.api.mlcontext with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`FrameMetadata(DataCharacteristics dataCharacteristics)`
Constructor to create a FrameMetadata object based on a DataCharacteristics object.
``
`MatrixMetadata(DataCharacteristics dataCharacteristics)`
Constructor to create a MatrixMetadata object based on a DataCharacteristics object.

## Uses of DataCharacteristics in org.apache.sysds.hops

Methods in org.apache.sysds.hops that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics[]`
MemoTable.`getAllInputStats(List<Hop> inputs)`
`DataCharacteristics`
MemoTable.`getAllInputStats(Hop input)`
`DataCharacteristics`
Hop.`getDataCharacteristics()`
Methods in org.apache.sysds.hops with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static boolean`
OptimizerUtils.`checkSparkCollectMemoryBudget(DataCharacteristics dc, long memPinned)`
`static boolean`
OptimizerUtils.`checkSparkCollectMemoryBudget(DataCharacteristics dc, long memPinned, boolean checkBP)`
`static boolean`
OptimizerUtils.`checkSparseBlockCSRConversion(DataCharacteristics dcIn)`
`static long`
OptimizerUtils.`estimatePartitionedSizeExactSparsity(DataCharacteristics dc)`
Estimates the footprint (in bytes) for a partitioned in-memory representation of a matrix with the given matrix characteristics
`static long`
OptimizerUtils.`estimatePartitionedSizeExactSparsity(DataCharacteristics dc, boolean outputEmptyBlocks)`
`static long`
OptimizerUtils.`estimateSize(DataCharacteristics dc)`
`static long`
OptimizerUtils.`estimateSizeExactSparsity(DataCharacteristics dc)`
`static double`
OptimizerUtils.`getSparsity(DataCharacteristics dc)`
`static boolean`
OptimizerUtils.`isIndexingRangeBlockAligned(IndexRange ixrange, DataCharacteristics mc)`
Indicates if the given indexing range is block aligned, i.e., it does not require global aggregation of blocks.
`static boolean`
OptimizerUtils.`isValidCPDimensions(DataCharacteristics mc)`
`void`
MemoTable.`memoizeStatistics(long hopID, DataCharacteristics dc)`

## Uses of DataCharacteristics in org.apache.sysds.hops.estim

Methods in org.apache.sysds.hops.estim that return DataCharacteristics
Modifier and Type
Method
Description
`static DataCharacteristics`
EstimatorMatrixHistogram.MatrixHistogram.`deriveOutputCharacteristics(EstimatorMatrixHistogram.MatrixHistogram h1, EstimatorMatrixHistogram.MatrixHistogram h2, double spOut, SparsityEstimator.OpCode op, long[] misc)`
`static DataCharacteristics`
EstimatorRowWise.`deriveOutputCharacteristics(MMNode node, double spOut)`
`DataCharacteristics`
EstimatorBasicAvg.`estim(MMNode root)`
`DataCharacteristics`
EstimatorBasicWorst.`estim(MMNode root)`
`DataCharacteristics`
EstimatorBitsetMM.`estim(MMNode root)`
`DataCharacteristics`
EstimatorDensityMap.`estim(MMNode root)`
`DataCharacteristics`
EstimatorLayeredGraph.`estim(MMNode root)`
`DataCharacteristics`
EstimatorMatrixHistogram.`estim(MMNode root)`
`DataCharacteristics`
EstimatorMatrixHistogram.`estim(MMNode root, boolean topLevel)`
`DataCharacteristics`
EstimatorRowWise.`estim(MMNode root)`
`DataCharacteristics`
EstimatorSample.`estim(MMNode root)`
`DataCharacteristics`
EstimatorSampleRa.`estim(MMNode root)`
`abstract DataCharacteristics`
SparsityEstimator.`estim(MMNode root)`
Estimates the output sparsity of a DAG of matrix multiplications for the given operator graph of a single root node.
`DataCharacteristics`
MMNode.`getDataCharacteristics()`
`DataCharacteristics`
MMNode.`setDataCharacteristics(DataCharacteristics mc)`
Methods in org.apache.sysds.hops.estim with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
MMNode.`setDataCharacteristics(DataCharacteristics mc)`

## Uses of DataCharacteristics in org.apache.sysds.hops.recompile

Methods in org.apache.sysds.hops.recompile that return types with arguments of type DataCharacteristics
Modifier and Type
Method
Description
`Map<String,DataCharacteristics>`
RecompileStatus.`getTWriteStats()`
Methods in org.apache.sysds.hops.recompile with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static boolean`
Recompiler.`checkCPCheckpoint(DataCharacteristics dc)`

## Uses of DataCharacteristics in org.apache.sysds.hops.rewrite

Methods in org.apache.sysds.hops.rewrite with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static long`
HopRewriteUtils.`getMaxInputDim(DataCharacteristics[] dc, boolean dim1)`
`static long`
HopRewriteUtils.`getSumValidInputDims(DataCharacteristics[] mc, boolean dim1)`
`static long`
HopRewriteUtils.`getSumValidInputNnz(DataCharacteristics[] mc, boolean worstcase)`
`static boolean`
HopRewriteUtils.`hasValidInputDims(DataCharacteristics[] mc, boolean dim1)`
`static boolean`
HopRewriteUtils.`hasValidInputNnz(DataCharacteristics[] mc, boolean worstcase)`

## Uses of DataCharacteristics in org.apache.sysds.lops

Methods in org.apache.sysds.lops with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`void`
OutputParameters.`setDimensions(DataCharacteristics dc)`

## Uses of DataCharacteristics in org.apache.sysds.resource.cost

Constructors in org.apache.sysds.resource.cost with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`VarStats(String name, DataCharacteristics dc)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.compress.io

Methods in org.apache.sysds.runtime.compress.io with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static void`
WriterCompressed.`writeRDDToHDFS(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> src, String path, int blen, DataCharacteristics mc)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram

Methods in org.apache.sysds.runtime.controlprogram with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`long`
ParForProgramBlock.PartitionFormat.`getNumColumns(DataCharacteristics mc)`
`long`
ParForProgramBlock.PartitionFormat.`getNumParts(DataCharacteristics mc)`
`long`
ParForProgramBlock.PartitionFormat.`getNumRows(DataCharacteristics mc)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram.caching

Methods in org.apache.sysds.runtime.controlprogram.caching that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
CacheableData.`getDataCharacteristics()`
`DataCharacteristics`
CacheBlock.`getDataCharacteristics()`
Methods in org.apache.sysds.runtime.controlprogram.caching with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`void`
MatrixObject.`updateDataCharacteristics(DataCharacteristics dc)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram.context

Methods in org.apache.sysds.runtime.controlprogram.context that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
ExecutionContext.`getDataCharacteristics(String varname)`
Methods in org.apache.sysds.runtime.controlprogram.context with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static FrameObject`
ExecutionContext.`createFrameObject(DataCharacteristics dc)`
`static MatrixObject`
ExecutionContext.`createMatrixObject(DataCharacteristics dc)`
`static TensorBlock`
SparkExecutionContext.`toTensorBlock(org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> rdd, DataCharacteristics dc)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram.federated

Methods in org.apache.sysds.runtime.controlprogram.federated that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
MatrixLineagePair.`getDataCharacteristics()`

## Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram.parfor

Constructors in org.apache.sysds.runtime.controlprogram.parfor with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`DataPartitionerRemoteSparkMapper(DataCharacteristics mc, Types.FileFormat fmt, ParForProgramBlock.PDataPartitionFormat dpf, int n)`
``
`RemoteDPParForSparkWorker(String program, boolean isLocal, HashMap<String,byte[]> clsMap, String inputVar, String iterVar, boolean cpCaching, DataCharacteristics mc, boolean tSparseCol, ParForProgramBlock.PartitionFormat dpf, Types.FileFormat fmt, org.apache.spark.util.LongAccumulator atasks, org.apache.spark.util.LongAccumulator aiters)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.data

Methods in org.apache.sysds.runtime.data that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
TensorBlock.`getDataCharacteristics()`
Methods in org.apache.sysds.runtime.data with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static List<IndexedTensorBlock>`
LibTensorReorg.`reshape(IndexedTensorBlock in, DataCharacteristics mcIn, DataCharacteristics mcOut, boolean rowwise, boolean outputEmptyBlocks)`
MR/SPARK reshape interface - for reshape we cannot view blocks independently, and hence, there are different CP and MR interfaces.

## Uses of DataCharacteristics in org.apache.sysds.runtime.frame.data

Methods in org.apache.sysds.runtime.frame.data that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
FrameBlock.`getDataCharacteristics()`

## Uses of DataCharacteristics in org.apache.sysds.runtime.functionobjects

Methods in org.apache.sysds.runtime.functionobjects with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`boolean`
DiagIndex.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`abstract boolean`
IndexFunction.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
OffsetColumnIndex.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
ReduceAll.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
ReduceCol.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
ReduceDiag.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
ReduceRow.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
RevIndex.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
RollIndex.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
SortIndex.`computeDimension(DataCharacteristics in, DataCharacteristics out)`
`boolean`
SwapIndex.`computeDimension(DataCharacteristics in, DataCharacteristics out)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.cp

Methods in org.apache.sysds.runtime.instructions.cp that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
Encrypted.`getDataCharacteristics()`
Methods in org.apache.sysds.runtime.instructions.cp with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static Instruction`
VariableCPInstruction.`prepCreatevarInstruction(String varName, String fileName, boolean fNameOverride, Types.DataType dt, String format, DataCharacteristics mc, MatrixObject.UpdateType update)`
`static Instruction`
VariableCPInstruction.`prepCreatevarInstruction(String varName, String fileName, boolean fNameOverride, Types.DataType dt, String format, DataCharacteristics mc, MatrixObject.UpdateType update, boolean hasHeader, String delim, boolean sparse)`
`void`
Data.`updateDataCharacteristics(DataCharacteristics mc)`
Constructors in org.apache.sysds.runtime.instructions.cp with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`CiphertextMatrix(int[] dims, DataCharacteristics dc, byte[] data)`
``
`Encrypted(int[] dims, DataCharacteristics dc, byte[] data, Types.DataType dt)`
``
`PlaintextMatrix(int[] dims, DataCharacteristics dc, byte[] data)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.ooc

Methods in org.apache.sysds.runtime.instructions.ooc that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
CachingStream.`getDataCharacteristics()`
`DataCharacteristics`
OOCStreamable.`getDataCharacteristics()`
`DataCharacteristics`
PlaybackStream.`getDataCharacteristics()`
`DataCharacteristics`
SubscribableTaskQueue.`getDataCharacteristics()`

## Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.spark

Methods in org.apache.sysds.runtime.instructions.spark with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static MatrixBlock`
MatrixIndexingSPInstruction.`inmemoryIndexing(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in1, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)`
`static boolean`
MatrixIndexingSPInstruction.`isMultiBlockLookup(org.apache.spark.api.java.JavaPairRDD<?,?> in, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)`
Indicates if the given index range and input matrix exhibit the following properties: (1) existing hash partitioner, (2) out-of-core input matrix (larger than aggregated memory), (3) aligned indexing range (which does not required aggregation), and (4) the output fits twice in memory (in order to collect the result).
`static boolean`
MatrixIndexingSPInstruction.`isSingleBlockLookup(DataCharacteristics mcIn, IndexRange ixrange)`
Indicates if the given index range only covers a single blocks of the inputs matrix.
`void`
ParameterizedBuiltinSPInstruction.`setOutputCharacteristicsForGroupedAgg(DataCharacteristics mc1, DataCharacteristics mcOut, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> out)`
Constructors in org.apache.sysds.runtime.instructions.spark with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`MergeWithShiftedBlocks(DataCharacteristics mc1, DataCharacteristics mc2, boolean cbind)`
``
`PadBlocksFunction(DataCharacteristics mcOut)`
``
`ShiftMatrix(DataCharacteristics mc1, DataCharacteristics mc2, boolean cbind)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.spark.data

Methods in org.apache.sysds.runtime.instructions.spark.data that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
PartitionedBroadcast.`getDataCharacteristics()`
`DataCharacteristics`
RDDObject.`getDataCharacteristics()`
Methods in org.apache.sysds.runtime.instructions.spark.data with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`void`
RDDObject.`setDataCharacteristics(DataCharacteristics dc)`
Constructors in org.apache.sysds.runtime.instructions.spark.data with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`BlockPartitioner(DataCharacteristics mc, int numParts)`
``
`PartitionedBroadcast(org.apache.spark.broadcast.Broadcast<PartitionedBlock<T>>[] broadcasts, DataCharacteristics dc)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.spark.functions

Constructors in org.apache.sysds.runtime.instructions.spark.functions with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`ExtractBlockForBinaryReblock(DataCharacteristics mcIn, DataCharacteristics mcOut)`
``
`IsBlockInList(long[] cols, DataCharacteristics mc)`
``
`IsBlockInRange(long rl, long ru, long cl, long cu, DataCharacteristics mc)`
``
`IsFrameBlockInRange(long rl, long ru, DataCharacteristics mcOut)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.spark.utils

Methods in org.apache.sysds.runtime.instructions.spark.utils that return DataCharacteristics
Modifier and Type
Method
Description
`static DataCharacteristics`
SparkUtils.`computeDataCharacteristics(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input)`
Utility to compute dimensions and non-zeros in a given RDD of binary cells.
Methods in org.apache.sysds.runtime.instructions.spark.utils with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtils.`binaryBlockToBinaryBlock(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mcIn, DataCharacteristics mcOut)`
`static org.apache.spark.api.java.JavaRDD<String>`
FrameRDDConverterUtils.`binaryBlockToCsv(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mcIn, FileFormatPropertiesCSV props, boolean strict)`
`static org.apache.spark.api.java.JavaRDD<String>`
RDDConverterUtils.`binaryBlockToCsv(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mcIn, FileFormatPropertiesCSV props, boolean strict)`
`static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>`
FrameRDDConverterUtils.`binaryBlockToDataFrame(org.apache.spark.sql.SparkSession sparkSession, org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mc, Types.ValueType[] schema)`
`static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>`
FrameRDDConverterUtils.`binaryBlockToDataFrame(org.apache.spark.sql.SQLContext sqlContext, org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mc, Types.ValueType[] schema)`
Deprecated.
`static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>`
RDDConverterUtils.`binaryBlockToDataFrame(org.apache.spark.sql.SparkSession sparkSession, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mc, boolean toVector)`
`static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>`
RDDConverterUtils.`binaryBlockToDataFrame(org.apache.spark.sql.SQLContext sqlContext, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mc, boolean toVector)`
Deprecated.
`static org.apache.spark.api.java.JavaRDD<String>`
RDDConverterUtils.`binaryBlockToLibsvm(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mcIn, FileFormatPropertiesLIBSVM props, boolean strict)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
FrameRDDConverterUtils.`binaryBlockToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input, DataCharacteristics mcIn, DataCharacteristics mcOut)`
`static org.apache.spark.api.java.JavaRDD<String>`
FrameRDDConverterUtils.`binaryBlockToTextCell(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input, DataCharacteristics mcIn)`
`static org.apache.spark.api.java.JavaRDD<String>`
RDDConverterUtils.`binaryBlockToTextCell(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mc)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtils.`binaryCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input, DataCharacteristics mcOut, boolean outputEmptyBlocks)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtilsExt.`coordinateMatrixToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.mllib.linalg.distributed.CoordinateMatrix input, DataCharacteristics mcIn, boolean outputEmptyBlocks)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtilsExt.`coordinateMatrixToBinaryBlock(org.apache.spark.SparkContext sc, org.apache.spark.mllib.linalg.distributed.CoordinateMatrix input, DataCharacteristics mcIn, boolean outputEmptyBlocks)`
`static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>`
FrameRDDConverterUtils.`csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mc, Types.ValueType[] schema, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)`
`static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>`
FrameRDDConverterUtils.`csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaRDD<String> input, DataCharacteristics mcOut, Types.ValueType[] schema, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtils.`csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mc, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtils.`csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaRDD<String> input, DataCharacteristics mcOut, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)`
`static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>`
FrameRDDConverterUtils.`dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df, DataCharacteristics mc, boolean containsID)`
`static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>`
FrameRDDConverterUtils.`dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df, DataCharacteristics mc, boolean containsID, Pair<String[],Types.ValueType[]> out)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtils.`dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df, DataCharacteristics mc, boolean containsID, boolean isVector)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
SparkUtils.`getEmptyBlockRDD(org.apache.spark.api.java.JavaSparkContext sc, DataCharacteristics mc)`
Creates an RDD of empty blocks according to the given matrix characteristics.
`static int`
SparkUtils.`getNumPreferredPartitions(DataCharacteristics dc)`
`static int`
SparkUtils.`getNumPreferredPartitions(DataCharacteristics dc, boolean outputEmptyBlocks)`
`static int`
SparkUtils.`getNumPreferredPartitions(DataCharacteristics dc, org.apache.spark.api.java.JavaPairRDD<?,?> in)`
`static void`
RDDConverterUtils.`libsvmToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, String pathIn, String pathX, String pathY, DataCharacteristics mcOutX)`
Converts a libsvm text input file into two binary block matrices for features and labels, and saves these to the specified output files.
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtils.`libsvmToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mc, String delim, String indexDelim)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtils.`matrixBlockToAlignedMatrixBlock(org.apache.spark.api.java.JavaPairRDD<Long,MatrixBlock> input, DataCharacteristics mcIn, DataCharacteristics mcOut)`
`static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,FrameBlock>`
FrameRDDConverterUtils.`matrixBlockToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input, DataCharacteristics mcIn)`
`static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>`
FrameRDDConverterUtils.`matrixBlockToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input, DataCharacteristics dcIn)`
`static void`
SparkUtils.`postprocessUltraSparseOutput(MatrixObject mo, DataCharacteristics mcOut)`
`static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>`
FrameRDDConverterUtils.`textCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> in, DataCharacteristics mcOut, Types.ValueType[] schema)`
`static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>`
RDDConverterUtils.`textCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mcOut, boolean outputEmptyBlocks, FileFormatPropertiesMM mmProps)`
`static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>`
FrameRDDConverterUtils.`textCellToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<Long,org.apache.hadoop.io.Text> input, DataCharacteristics mc, Types.ValueType[] schema)`
Constructors in org.apache.sysds.runtime.instructions.spark.utils with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`BinaryCellToBinaryBlockFunction(DataCharacteristics mc)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.io

Methods in org.apache.sysds.runtime.io with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static List<org.apache.commons.lang3.tuple.Pair<FederatedRange,FederatedData>>`
ReaderWriterFederated.`read(String file, DataCharacteristics mc)`
Read a federated map from disk, It is not initialized before it is used in: org.apache.sysds.runtime.instructions.fed.InitFEDInstruction

## Uses of DataCharacteristics in org.apache.sysds.runtime.lineage

Methods in org.apache.sysds.runtime.lineage that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
LineageCacheEntry.`getDataCharacteristics()`

## Uses of DataCharacteristics in org.apache.sysds.runtime.matrix.data

Methods in org.apache.sysds.runtime.matrix.data that return DataCharacteristics
Modifier and Type
Method
Description
`final DataCharacteristics`
MatrixBlock.`getDataCharacteristics()`
Methods in org.apache.sysds.runtime.matrix.data with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static long`
MatrixBlock.`estimateSizeInMemory(DataCharacteristics dc)`
`static boolean`
MatrixBlock.`evalSparseFormatInMemory(DataCharacteristics dc)`
`static List<IndexedMatrixValue>`
LibMatrixReorg.`reshape(IndexedMatrixValue in, DataCharacteristics mcIn, DataCharacteristics mcOut, boolean rowwise, boolean outputEmptyBlocks)`
MR/SPARK reshape interface - for reshape we cannot view blocks independently, and hence, there are different CP and MR interfaces.

## Uses of DataCharacteristics in org.apache.sysds.runtime.meta

Subclasses of DataCharacteristics in org.apache.sysds.runtime.meta
Modifier and Type
Class
Description
`class`
`MatrixCharacteristics`
`class`
`TensorCharacteristics`
Methods in org.apache.sysds.runtime.meta that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
MetaData.`getDataCharacteristics()`
`DataCharacteristics`
MetaDataAll.`getDataCharacteristics()`
`abstract DataCharacteristics`
DataCharacteristics.`set(long[] dims, int blocksize)`
`abstract DataCharacteristics`
DataCharacteristics.`set(long[] dims, int blocksize, long nnz)`
`abstract DataCharacteristics`
DataCharacteristics.`set(long nr, long nc, int blen)`
`abstract DataCharacteristics`
DataCharacteristics.`set(long nr, long nc, int blen, long nnz)`
`abstract DataCharacteristics`
DataCharacteristics.`set(DataCharacteristics that)`
`DataCharacteristics`
MatrixCharacteristics.`set(long[] dims, int blocksize)`
`DataCharacteristics`
MatrixCharacteristics.`set(long[] dims, int blocksize, long nnz)`
`DataCharacteristics`
MatrixCharacteristics.`set(long nr, long nc, int blen)`
`DataCharacteristics`
MatrixCharacteristics.`set(long nr, long nc, int blen, long nnz)`
`DataCharacteristics`
MatrixCharacteristics.`set(DataCharacteristics that)`
`DataCharacteristics`
TensorCharacteristics.`set(long[] dims, int blocksize)`
`DataCharacteristics`
TensorCharacteristics.`set(long[] dims, int blocksize, long nnz)`
`DataCharacteristics`
TensorCharacteristics.`set(long nr, long nc, int blen)`
`DataCharacteristics`
TensorCharacteristics.`set(long nr, long nc, int blen, long nnz)`
`DataCharacteristics`
TensorCharacteristics.`set(DataCharacteristics that)`
`DataCharacteristics`
DataCharacteristics.`setBlocksize(int blen)`
`abstract DataCharacteristics`
DataCharacteristics.`setCols(long clen)`
`DataCharacteristics`
MatrixCharacteristics.`setCols(long clen)`
`DataCharacteristics`
TensorCharacteristics.`setCols(long clen)`
`abstract DataCharacteristics`
DataCharacteristics.`setDim(int i, long dim)`
`DataCharacteristics`
MatrixCharacteristics.`setDim(int i, long dim)`
`DataCharacteristics`
TensorCharacteristics.`setDim(int i, long dim)`
`abstract DataCharacteristics`
DataCharacteristics.`setDimension(long nr, long nc)`
`DataCharacteristics`
MatrixCharacteristics.`setDimension(long nr, long nc)`
`DataCharacteristics`
TensorCharacteristics.`setDimension(long nr, long nc)`
`abstract DataCharacteristics`
DataCharacteristics.`setDims(long[] dims)`
`DataCharacteristics`
MatrixCharacteristics.`setDims(long[] dims)`
`DataCharacteristics`
TensorCharacteristics.`setDims(long[] dims)`
`DataCharacteristics`
DataCharacteristics.`setNoEmptyBlocks(boolean flag)`
`abstract DataCharacteristics`
DataCharacteristics.`setNonZeros(long nnz)`
`DataCharacteristics`
MatrixCharacteristics.`setNonZeros(long nnz)`
`DataCharacteristics`
TensorCharacteristics.`setNonZeros(long nnz)`
`abstract DataCharacteristics`
DataCharacteristics.`setNonZerosBound(long nnz)`
`DataCharacteristics`
MatrixCharacteristics.`setNonZerosBound(long nnz)`
`DataCharacteristics`
TensorCharacteristics.`setNonZerosBound(long nnz)`
`abstract DataCharacteristics`
DataCharacteristics.`setRows(long rlen)`
`DataCharacteristics`
MatrixCharacteristics.`setRows(long rlen)`
`DataCharacteristics`
TensorCharacteristics.`setRows(long rlen)`
Methods in org.apache.sysds.runtime.meta with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`abstract DataCharacteristics`
DataCharacteristics.`set(DataCharacteristics that)`
`DataCharacteristics`
MatrixCharacteristics.`set(DataCharacteristics that)`
`DataCharacteristics`
TensorCharacteristics.`set(DataCharacteristics that)`
`static void`
MetaDataUtils.`updateAppendDataCharacteristics(DataCharacteristics mc1, DataCharacteristics mc2, DataCharacteristics mcOut, boolean cbind)`
Constructors in org.apache.sysds.runtime.meta with parameters of type DataCharacteristics
Modifier
Constructor
Description
``
`MatrixCharacteristics(DataCharacteristics that)`
``
`MetaData(DataCharacteristics dc)`
``
`MetaDataFormat(DataCharacteristics dc, Types.FileFormat fmt)`
``
`TensorCharacteristics(DataCharacteristics that)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.ooc.stream

Methods in org.apache.sysds.runtime.ooc.stream that return DataCharacteristics
Modifier and Type
Method
Description
`DataCharacteristics`
FilteredOOCStream.`getDataCharacteristics()`
`DataCharacteristics`
MergedOOCStream.`getDataCharacteristics()`
`DataCharacteristics`
SourceOOCStreamable.`getDataCharacteristics()`
`DataCharacteristics`
SplittingOOCStream.`getDataCharacteristics()`
`DataCharacteristics`
SubOOCStream.`getDataCharacteristics()`

## Uses of DataCharacteristics in org.apache.sysds.runtime.ooc.util

Methods in org.apache.sysds.runtime.ooc.util with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static long`
OOCUtils.`getNumBlocks(DataCharacteristics dc)`

## Uses of DataCharacteristics in org.apache.sysds.runtime.util

Methods in org.apache.sysds.runtime.util with parameters of type DataCharacteristics
Modifier and Type
Method
Description
`static IndexedMatrixValue`
UtilFunctions.`createIndexedMatrixBlock(MatrixBlock mb, DataCharacteristics mc, long ix)`
`static String`
HDFSTool.`metaDataToString(Types.ValueType vt, Types.ValueType[] schema, Types.DataType dt, DataCharacteristics dc, Types.FileFormat fmt, FileFormatProperties formatProperties)`
`static String`
HDFSTool.`metaDataToString(Types.ValueType vt, Types.ValueType[] schema, Types.DataType dt, DataCharacteristics dc, Types.FileFormat fmt, FileFormatProperties formatProperties, String privacyConstraints)`
`static void`
DataConverter.`writeMatrixToHDFS(MatrixBlock mat, String dir, Types.FileFormat fmt, DataCharacteristics dc)`
`static void`
DataConverter.`writeMatrixToHDFS(MatrixBlock mat, String dir, Types.FileFormat fmt, DataCharacteristics dc, int replication, FileFormatProperties formatProperties)`
`static void`
DataConverter.`writeMatrixToHDFS(MatrixBlock mat, String dir, Types.FileFormat fmt, DataCharacteristics dc, int replication, FileFormatProperties formatProperties, boolean diag)`
`static void`
HDFSTool.`writeMetaDataFile(String mtdfile, Types.ValueType vt, Types.ValueType[] schema, Types.DataType dt, DataCharacteristics mc, Types.FileFormat fmt)`
`static void`
HDFSTool.`writeMetaDataFile(String mtdfile, Types.ValueType vt, Types.ValueType[] schema, Types.DataType dt, DataCharacteristics dc, Types.FileFormat fmt, FileFormatProperties formatProperties)`
`static void`
HDFSTool.`writeMetaDataFile(String mtdfile, Types.ValueType vt, Types.ValueType[] schema, Types.DataType dt, DataCharacteristics dc, Types.FileFormat fmt, FileFormatProperties formatProperties, String privacyConstraints)`
`static void`
HDFSTool.`writeMetaDataFile(String mtdfile, Types.ValueType vt, DataCharacteristics mc, Types.FileFormat fmt)`
`static void`
HDFSTool.`writeMetaDataFile(String mtdfile, Types.ValueType vt, DataCharacteristics dc, Types.FileFormat fmt, FileFormatProperties formatProperties)`
`static void`
HDFSTool.`writeMetaDataFileFrame(String mtdfile, Types.ValueType[] schema, DataCharacteristics dc, Types.FileFormat fmt)`
`static void`
DataConverter.`writeTensorToHDFS(TensorBlock tensor, String dir, Types.FileFormat fmt, DataCharacteristics dc)`