Package org.apache.sysds.runtime.matrix.data

# Class LibMatrixAgg

java.lang.Object
org.apache.sysds.runtime.matrix.data.LibMatrixAgg

public class LibMatrixAgg extends Object
MB: Library for matrix aggregations including ak+, uak+ for all combinations of dense and sparse representations, and corrections. Those are performance-critical operations because they are used on combiners/reducers of important operations like tsmm, mvmult, indexing, but also basic sum/min/max/mean, row*, col*, etc. Specific handling is especially required for all non sparse-safe operations in order to prevent unnecessary worse asymptotic behavior. This library currently covers the following opcodes: ak+, uak+, uark+, uack+, uasqk+, uarsqk+, uacsqk+, uamin, uarmin, uacmin, uamax, uarmax, uacmax, ua*, uamean, uarmean, uacmean, uavar, uarvar, uacvar, uarimax, uaktrace, cumk+, cummin, cummax, cum*, tak+, cm, cov TODO next opcode extensions: a+, colindexmax

## Field Summary

Fields
Modifier and Type
Field
Description
`static final long`
`PAR_NUMCELL_THRESHOLD1`

## Method Summary

Modifier and Type
Method
Description
`static void`
`aggregateBinaryMatrix(MatrixBlock in, MatrixBlock aggVal, MatrixBlock aggCorr, boolean deep)`
Core incremental matrix aggregate (ak+) as used in mapmult, tsmm, cpmm, etc.
`static void`
`aggregateBinaryMatrix(MatrixBlock in, MatrixBlock aggVal, AggregateOperator aop)`
Core incremental matrix aggregate (ak+) as used for uack+ and acrk+.
`static CmCovObject`
`aggregateCmCov(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, ValueFunction fn)`
Single threaded Covariance and Central Moment operations CM = Central Moment COV = Covariance
`static CmCovObject`
`aggregateCmCov(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, ValueFunction fn, int k)`
Multi threaded Covariance and Central Moment operations CM = Central Moment COV = Covariance
`static MatrixBlock`
`aggregateTernary(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, MatrixBlock ret, AggregateTernaryOperator op)`
`static MatrixBlock`
`aggregateTernary(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, MatrixBlock ret, AggregateTernaryOperator op, int k)`
`static void`
`aggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop)`
`static void`
`aggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop, boolean allowReformatToSparse)`
`static void`
`aggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop, int k)`
`static MatrixBlock`
`aggregateUnaryMatrix(AggregateUnaryOperator op, MatrixBlock in, MatrixValue result, int blen, MatrixIndexes indexesIn, boolean inCP)`
`static void`
`countAgg(double[] a, int[] c, int[] aix, int ai, int len)`
`static void`
`countAgg(double[] a, int[] c, int ai, int len)`
`static MatrixBlock`
`cumaggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, UnaryOperator uop)`
`static MatrixBlock`
`cumaggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, UnaryOperator uop, double[] agg)`
`static MatrixBlock`
`cumaggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, UnaryOperator uop, int k)`
`static void`
`groupedAggregate(MatrixBlock groups, MatrixBlock target, MatrixBlock weights, MatrixBlock result, int numGroups, Operator op)`
`static void`
`groupedAggregate(MatrixBlock groups, MatrixBlock target, MatrixBlock weights, MatrixBlock result, int numGroups, Operator op, int k)`
`static boolean`
`isSupportedUnaryAggregateOperator(AggregateUnaryOperator op)`
`static boolean`
`isSupportedUnaryOperator(UnaryOperator op)`
`static MatrixBlock`
`prepareAggregateUnaryOutput(MatrixBlock in, AggregateUnaryOperator op, MatrixValue result, int blen)`
`static void`
`recomputeIndexes(MatrixBlock out, AggregateUnaryOperator op, int blen, MatrixIndexes ix)`
Recompute outputs (e.g., maxindex or minindex) according to block indexes from MR.
`static boolean`
`satisfiesMultiThreadingConstraints(MatrixBlock in, int k)`
`static boolean`
`satisfiesMultiThreadingConstraints(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop, int k)`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### PAR_NUMCELL_THRESHOLD1

public static final long PAR_NUMCELL_THRESHOLD1

See Also:
  - Constant Field Values

## Method Details

### aggregateBinaryMatrix

public static void aggregateBinaryMatrix(MatrixBlock in, MatrixBlock aggVal, MatrixBlock aggCorr, boolean deep)
Core incremental matrix aggregate (ak+) as used in mapmult, tsmm, cpmm, etc. Note that we try to keep the current aggVal and aggCorr in dense format in order to allow efficient access according to the dense/sparse input.

Parameters:
  `in` - input matrix
  `aggVal` - current aggregate values (in/out)
  `aggCorr` - current aggregate correction (in/out)
  `deep` - deep copy flag

### aggregateBinaryMatrix

public static void aggregateBinaryMatrix(MatrixBlock in, MatrixBlock aggVal, AggregateOperator aop)
Core incremental matrix aggregate (ak+) as used for uack+ and acrk+. Embedded correction values. DOES NOT EVALUATE SPARSITY SINCE IT IS USED IN INCREMENTAL AGGREGATION

Parameters:
  `in` - matrix block
  `aggVal` - aggregate operator
  `aop` - aggregate operator

### aggregateUnaryMatrix

public static MatrixBlock aggregateUnaryMatrix(AggregateUnaryOperator op, MatrixBlock in, MatrixValue result, int blen, MatrixIndexes indexesIn, boolean inCP)

### aggregateUnaryMatrix

public static void aggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop)

### aggregateUnaryMatrix

public static void aggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop, boolean allowReformatToSparse)

### aggregateUnaryMatrix

public static void aggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop, int k)

### cumaggregateUnaryMatrix

public static MatrixBlock cumaggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, UnaryOperator uop)

### cumaggregateUnaryMatrix

public static MatrixBlock cumaggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, UnaryOperator uop, double[] agg)

### cumaggregateUnaryMatrix

public static MatrixBlock cumaggregateUnaryMatrix(MatrixBlock in, MatrixBlock out, UnaryOperator uop, int k)

### aggregateCmCov

public static CmCovObject aggregateCmCov(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, ValueFunction fn)
Single threaded Covariance and Central Moment operations CM = Central Moment COV = Covariance

Parameters:
  `in1` - Main input matrix
  `in2` - Second input matrix
  `in3` - Third input matrix (not output since output is returned)
  `fn` - Value function to apply
Returns:
  Central Moment or Covariance object

### aggregateCmCov

public static CmCovObject aggregateCmCov(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, ValueFunction fn, int k)
Multi threaded Covariance and Central Moment operations CM = Central Moment COV = Covariance

Parameters:
  `in1` - Main input matrix
  `in2` - Second input matrix
  `in3` - Third input matrix (not output since output is returned)
  `fn` - Value function to apply
  `k` - Parallelization degree
Returns:
  Central Moment or Covariance object

### aggregateTernary

public static MatrixBlock aggregateTernary(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, MatrixBlock ret, AggregateTernaryOperator op)

### aggregateTernary

public static MatrixBlock aggregateTernary(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, MatrixBlock ret, AggregateTernaryOperator op, int k)

### groupedAggregate

public static void groupedAggregate(MatrixBlock groups, MatrixBlock target, MatrixBlock weights, MatrixBlock result, int numGroups, Operator op)

### groupedAggregate

public static void groupedAggregate(MatrixBlock groups, MatrixBlock target, MatrixBlock weights, MatrixBlock result, int numGroups, Operator op, int k)

### isSupportedUnaryAggregateOperator

public static boolean isSupportedUnaryAggregateOperator(AggregateUnaryOperator op)

### isSupportedUnaryOperator

public static boolean isSupportedUnaryOperator(UnaryOperator op)

### satisfiesMultiThreadingConstraints

public static boolean satisfiesMultiThreadingConstraints(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop, int k)

### satisfiesMultiThreadingConstraints

public static boolean satisfiesMultiThreadingConstraints(MatrixBlock in, int k)

### recomputeIndexes

public static void recomputeIndexes(MatrixBlock out, AggregateUnaryOperator op, int blen, MatrixIndexes ix)
Recompute outputs (e.g., maxindex or minindex) according to block indexes from MR. TODO: this should not be part of block operations but of the MR instruction.

Parameters:
  `out` - matrix block
  `op` - aggregate unary operator
  `blen` - number of rows/cols in a block
  `ix` - matrix indexes

### countAgg

public static void countAgg(double[] a, int[] c, int[] aix, int ai, int len)

### countAgg

public static void countAgg(double[] a, int[] c, int ai, int len)

### prepareAggregateUnaryOutput

public static MatrixBlock prepareAggregateUnaryOutput(MatrixBlock in, AggregateUnaryOperator op, MatrixValue result, int blen)