Package org.apache.sysds.runtime.transform.encode

# Class ColumnEncoderComposite

java.lang.Object
org.apache.sysds.runtime.transform.encode.ColumnEncoder
org.apache.sysds.runtime.transform.encode.ColumnEncoderComposite

All Implemented Interfaces:
  `Externalizable`, `Serializable`, `Comparable<ColumnEncoder>`, `Encoder`

public class ColumnEncoderComposite extends ColumnEncoder
Simple composite encoder that applies a list of encoders in specified order. By implementing the default encoder API it can be used as a drop-in replacement for any other encoder.

See Also:
  - Serialized Form

## Nested Class Summary

## Nested classes/interfaces inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

`ColumnEncoder.EncoderType`

## Field Summary

### Fields inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

`APPLY_ROW_BLOCKS_PER_COLUMN, BUILD_ROW_BLOCKS_PER_COLUMN`

## Constructor Summary

Constructors
Constructor
Description
`ColumnEncoderComposite()`
`ColumnEncoderComposite(List<ColumnEncoder> columnEncoders)`
`ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, int colID)`
`ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, FrameBlock meta)`
`ColumnEncoderComposite(ColumnEncoder columnEncoder)`

## Method Summary

Modifier and Type
Method
Description
`void`
`addEncoder(ColumnEncoder other)`
`void`
`allocateMetaData(FrameBlock meta)`
Pre-allocate a FrameBlock for metadata collection.
`MatrixBlock`
`apply(CacheBlock<?> in, MatrixBlock out, int outputCol, int rowStart, int blk)`
`void`
`build(CacheBlock<?> in)`
Build the transform meta data for the given block input.
`void`
`build(CacheBlock<?> in, Map<Integer,double[]> equiHeightMaxs)`
`void`
`buildPartial(FrameBlock in)`
Partial build of internal data structures (e.g., in distributed spark operations).
`void`
`computeMapSizeEstimate(CacheBlock<?> in, int[] sampleIndices)`
`boolean`
`equals(Object o)`
`List<DependencyTask<?>>`
`getApplyTasks(CacheBlock<?> in, MatrixBlock out, int outputCol, int[] sparseRowPointerOffsets)`
`long`
`getAvgEntrySize()`
`List<DependencyTask<?>>`
`getBuildTasks(CacheBlock<?> in)`
`int`
`getDomainSize()`
`<T extends ColumnEncoder>
T`
`getEncoder(Class<T> type)`
`List<ColumnEncoder>`
`getEncoders()`
`FrameBlock`
`getMetaData(FrameBlock out)`
Construct a frame block out of the transform meta data.
`<T extends ColumnEncoder>
boolean`
`hasBuild()`
`<T extends ColumnEncoder>
boolean`
`hasEncoder(Class<T> type)`
`int`
`hashCode()`
`void`
`initEmbeddings(MatrixBlock embeddings)`
`void`
`initMetaData(FrameBlock out)`
Sets up the required meta data for a subsequent call to apply.
`boolean`
`isBin()`
`boolean`
`isBinToDummy()`
`boolean`
`isEncoder(int colID, Class<?> type)`
`boolean`
`isHash()`
`boolean`
`isHashToDummy()`
`boolean`
`isPassThrough()`
`boolean`
`isRecode()`
`boolean`
`isRecodeToDummy()`
`void`
`mergeAt(ColumnEncoder other)`
Merges another encoder, of a compatible type, in after a certain position.
`void`
`prepareBuildPartial()`
Allocates internal data structures for partial build.
`void`
`readExternal(ObjectInput in)`
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.
`void`
`setNumPartitions(int nBuild, int nApply)`
`void`
`shiftCol(int columnOffset)`
`String`
`toString()`
`void`
`updateAllDCEncoders()`
`void`
`updateIndexRanges(long[] beginDims, long[] endDims, int colOffset)`
Update index-ranges to after encoding.
`void`
`writeExternal(ObjectOutput out)`
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

### Methods inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

`apply, build, compareTo, getBuildTask, getColID, getColMapping, getEstMetaSize, getEstNumDistincts, getPartialBuildTask, getPartialMergeBuildTask, isApplicable, isApplicable, setColID, setEstMetaSize, setEstNumDistincts`

### Methods inherited from class java.lang.Object

`getClass, notify, notifyAll, wait, wait, wait`

## Constructor Details

### ColumnEncoderComposite

public ColumnEncoderComposite()

### ColumnEncoderComposite

public ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, FrameBlock meta)

### ColumnEncoderComposite

public ColumnEncoderComposite(List<ColumnEncoder> columnEncoders)

### ColumnEncoderComposite

public ColumnEncoderComposite(List<ColumnEncoder> columnEncoders, int colID)

### ColumnEncoderComposite

public ColumnEncoderComposite(ColumnEncoder columnEncoder)

## Method Details

### getEncoders

public List<ColumnEncoder> getEncoders()

### getEncoder

public <T extends ColumnEncoder> T getEncoder(Class<T> type)

### isEncoder

public boolean isEncoder(int colID, Class<?> type)

### build

public void build(CacheBlock<?> in)
Description copied from interface: `Encoder`
Build the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.

Parameters:
  `in` - input frame block

### build

public void build(CacheBlock<?> in, Map<Integer,double[]> equiHeightMaxs)

Overrides:
  `build` in class `ColumnEncoder`

### getApplyTasks

public List<DependencyTask<?>> getApplyTasks(CacheBlock<?> in, MatrixBlock out, int outputCol, int[] sparseRowPointerOffsets)

Overrides:
  `getApplyTasks` in class `ColumnEncoder`

### getBuildTasks

public List<DependencyTask<?>> getBuildTasks(CacheBlock<?> in)

Overrides:
  `getBuildTasks` in class `ColumnEncoder`

### prepareBuildPartial

public void prepareBuildPartial()
Description copied from class: `ColumnEncoder`
Allocates internal data structures for partial build.

Specified by:
  `prepareBuildPartial` in interface `Encoder`
Overrides:
  `prepareBuildPartial` in class `ColumnEncoder`

### buildPartial

public void buildPartial(FrameBlock in)
Description copied from class: `ColumnEncoder`
Partial build of internal data structures (e.g., in distributed spark operations).

Specified by:
  `buildPartial` in interface `Encoder`
Overrides:
  `buildPartial` in class `ColumnEncoder`
Parameters:
  `in` - input frame block

### apply

public MatrixBlock apply(CacheBlock<?> in, MatrixBlock out, int outputCol, int rowStart, int blk)

Overrides:
  `apply` in class `ColumnEncoder`

### equals

public boolean equals(Object o)

Overrides:
  `equals` in class `Object`

### hashCode

public int hashCode()

Overrides:
  `hashCode` in class `Object`

### mergeAt

public void mergeAt(ColumnEncoder other)
Description copied from class: `ColumnEncoder`
Merges another encoder, of a compatible type, in after a certain position. Resizes as necessary. `ColumnEncoders` are compatible with themselves and `EncoderComposite` is compatible with every other `ColumnEncoders`. `MultiColumnEncoders` are compatible with every encoder

Overrides:
  `mergeAt` in class `ColumnEncoder`
Parameters:
  `other` - the encoder that should be merged in

### updateAllDCEncoders

public void updateAllDCEncoders()

### addEncoder

public void addEncoder(ColumnEncoder other)

### updateIndexRanges

public void updateIndexRanges(long[] beginDims, long[] endDims, int colOffset)
Description copied from class: `ColumnEncoder`
Update index-ranges to after encoding. Note that only Dummycoding changes the ranges.

Specified by:
  `updateIndexRanges` in interface `Encoder`
Overrides:
  `updateIndexRanges` in class `ColumnEncoder`
Parameters:
  `beginDims` - begin dimensions of range
  `endDims` - end dimensions of range
  `colOffset` - is applied to begin and endDims

### allocateMetaData

public void allocateMetaData(FrameBlock meta)
Description copied from interface: `Encoder`
Pre-allocate a FrameBlock for metadata collection.

Parameters:
  `meta` - frame block

### getMetaData

public FrameBlock getMetaData(FrameBlock out)
Description copied from interface: `Encoder`
Construct a frame block out of the transform meta data.

Parameters:
  `out` - output frame block
Returns:
  output frame block?

### initMetaData

public void initMetaData(FrameBlock out)
Description copied from interface: `Encoder`
Sets up the required meta data for a subsequent call to apply.

Parameters:
  `out` - frame block

### initEmbeddings

public void initEmbeddings(MatrixBlock embeddings)

Overrides:
  `initEmbeddings` in class `ColumnEncoder`

### toString

public String toString()

Overrides:
  `toString` in class `Object`

### writeExternal

public void writeExternal(ObjectOutput out) throws IOException
Description copied from class: `ColumnEncoder`
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

Specified by:
  `writeExternal` in interface `Externalizable`
Overrides:
  `writeExternal` in class `ColumnEncoder`
Parameters:
  `out` - object output
Throws:
  `IOException` - if IOException occurs

### readExternal

public void readExternal(ObjectInput in) throws IOException
Description copied from class: `ColumnEncoder`
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.

Specified by:
  `readExternal` in interface `Externalizable`
Overrides:
  `readExternal` in class `ColumnEncoder`
Parameters:
  `in` - object input
Throws:
  `IOException` - if IOException occur

### hasEncoder

public <T extends ColumnEncoder> boolean hasEncoder(Class<T> type)

### hasBuild

public <T extends ColumnEncoder> boolean hasBuild()

### computeMapSizeEstimate

public void computeMapSizeEstimate(CacheBlock<?> in, int[] sampleIndices)

Overrides:
  `computeMapSizeEstimate` in class `ColumnEncoder`

### getAvgEntrySize

public long getAvgEntrySize()

### setNumPartitions

public void setNumPartitions(int nBuild, int nApply)

### shiftCol

public void shiftCol(int columnOffset)

Overrides:
  `shiftCol` in class `ColumnEncoder`

### getDomainSize

public int getDomainSize()

Overrides:
  `getDomainSize` in class `ColumnEncoder`

### isRecodeToDummy

public boolean isRecodeToDummy()

### isRecode

public boolean isRecode()

### isPassThrough

public boolean isPassThrough()

### isBin

public boolean isBin()

### isBinToDummy

public boolean isBinToDummy()

### isHash

public boolean isHash()

### isHashToDummy

public boolean isHashToDummy()