Package org.apache.sysds.runtime.transform.encode

# Class ColumnEncoderBagOfWords

java.lang.Object
org.apache.sysds.runtime.transform.encode.ColumnEncoder
org.apache.sysds.runtime.transform.encode.ColumnEncoderBagOfWords

All Implemented Interfaces:
  `Externalizable`, `Serializable`, `Comparable<ColumnEncoder>`, `Encoder`

public class ColumnEncoderBagOfWords extends ColumnEncoder

See Also:
  - Serialized Form

## Nested Class Summary

## Nested classes/interfaces inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

`ColumnEncoder.EncoderType`

## Field Summary

Fields
Modifier and Type
Field
Description
`static int`
`NUM_SAMPLES_MAP_ESTIMATION`

### Fields inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

`APPLY_ROW_BLOCKS_PER_COLUMN, BUILD_ROW_BLOCKS_PER_COLUMN`

## Constructor Summary

Constructors
Constructor
Description
`ColumnEncoderBagOfWords()`
`ColumnEncoderBagOfWords(ColumnEncoderBagOfWords enc)`

## Method Summary

Modifier and Type
Method
Description
`void`
`allocateMetaData(FrameBlock meta)`
Pre-allocate a FrameBlock for metadata collection.
`void`
`build(CacheBlock<?> in)`
Build the transform meta data for the given block input.
`void`
`buildPartial(FrameBlock in)`
Partial build of internal data structures (e.g., in distributed spark operations).
`void`
`computeMapSizeEstimate(CacheBlock<?> in, int[] sampleIndices)`
`double`
`computeNnzEstimate(CacheBlock<?> in, int[] sampleIndices)`
`void`
`computeNnzPerRow(CacheBlock<?> in, int start, int end)`
`Callable<Object>`
`getBuildTask(CacheBlock<?> in)`
`int`
`getDomainSize()`
`FrameBlock`
`getMetaData(FrameBlock out)`
Construct a frame block out of the transform meta data.
`Callable<Object>`
`getPartialBuildTask(CacheBlock<?> in, int startRow, int blockSize, HashMap<Integer,Object> ret, int pos)`
`Callable<Object>`
`getPartialMergeBuildTask(HashMap<Integer,?> ret)`
`HashSet<Object>`
`getPartialTokenDictionary()`
`Map<Object,Integer>`
`getTokenDictionary()`
`void`
`initMetaData(FrameBlock meta)`
Sets up the required meta data for a subsequent call to apply.
`void`
`prepareBuildPartial()`
Allocates internal data structures for partial build.
`void`
`readExternal(ObjectInput in)`
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.
`void`
`setTokenDictionary(HashMap<Object,Integer> dict)`
`static String[]`
`tokenize(String current, boolean caseSensitive, String seperatorRegex)`
`void`
`writeExternal(ObjectOutput out)`
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

### Methods inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

`apply, apply, build, build, compareTo, getApplyTasks, getBuildTasks, getColID, getColMapping, getEstMetaSize, getEstNumDistincts, initEmbeddings, isApplicable, isApplicable, mergeAt, setColID, setEstMetaSize, setEstNumDistincts, shiftCol, updateIndexRanges`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### NUM_SAMPLES_MAP_ESTIMATION

public static int NUM_SAMPLES_MAP_ESTIMATION

## Constructor Details

### ColumnEncoderBagOfWords

public ColumnEncoderBagOfWords()

### ColumnEncoderBagOfWords

public ColumnEncoderBagOfWords(ColumnEncoderBagOfWords enc)

## Method Details

### setTokenDictionary

public void setTokenDictionary(HashMap<Object,Integer> dict)

### getTokenDictionary

public Map<Object,Integer> getTokenDictionary()

### computeNnzEstimate

public double computeNnzEstimate(CacheBlock<?> in, int[] sampleIndices)

### computeMapSizeEstimate

public void computeMapSizeEstimate(CacheBlock<?> in, int[] sampleIndices)

Overrides:
  `computeMapSizeEstimate` in class `ColumnEncoder`

### computeNnzPerRow

public void computeNnzPerRow(CacheBlock<?> in, int start, int end)

### tokenize

public static String[] tokenize(String current, boolean caseSensitive, String seperatorRegex)

### getDomainSize

public int getDomainSize()

Overrides:
  `getDomainSize` in class `ColumnEncoder`

### getBuildTask

public Callable<Object> getBuildTask(CacheBlock<?> in)

Overrides:
  `getBuildTask` in class `ColumnEncoder`

### build

public void build(CacheBlock<?> in)
Description copied from interface: `Encoder`
Build the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.

Parameters:
  `in` - input frame block

### getPartialBuildTask

public Callable<Object> getPartialBuildTask(CacheBlock<?> in, int startRow, int blockSize, HashMap<Integer,Object> ret, int pos)

Overrides:
  `getPartialBuildTask` in class `ColumnEncoder`

### getPartialMergeBuildTask

public Callable<Object> getPartialMergeBuildTask(HashMap<Integer,?> ret)

Overrides:
  `getPartialMergeBuildTask` in class `ColumnEncoder`

### prepareBuildPartial

public void prepareBuildPartial()
Description copied from class: `ColumnEncoder`
Allocates internal data structures for partial build.

Specified by:
  `prepareBuildPartial` in interface `Encoder`
Overrides:
  `prepareBuildPartial` in class `ColumnEncoder`

### getPartialTokenDictionary

public HashSet<Object> getPartialTokenDictionary()

### buildPartial

public void buildPartial(FrameBlock in)
Description copied from class: `ColumnEncoder`
Partial build of internal data structures (e.g., in distributed spark operations).

Specified by:
  `buildPartial` in interface `Encoder`
Overrides:
  `buildPartial` in class `ColumnEncoder`
Parameters:
  `in` - input frame block

### allocateMetaData

public void allocateMetaData(FrameBlock meta)
Description copied from interface: `Encoder`
Pre-allocate a FrameBlock for metadata collection.

Parameters:
  `meta` - frame block

### getMetaData

public FrameBlock getMetaData(FrameBlock out)
Description copied from interface: `Encoder`
Construct a frame block out of the transform meta data.

Parameters:
  `out` - output frame block
Returns:
  output frame block?

### initMetaData

public void initMetaData(FrameBlock meta)
Description copied from interface: `Encoder`
Sets up the required meta data for a subsequent call to apply.

Parameters:
  `meta` - frame block

### writeExternal

public void writeExternal(ObjectOutput out) throws IOException
Description copied from class: `ColumnEncoder`
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

Specified by:
  `writeExternal` in interface `Externalizable`
Overrides:
  `writeExternal` in class `ColumnEncoder`
Parameters:
  `out` - object output
Throws:
  `IOException` - if IOException occurs

### readExternal

public void readExternal(ObjectInput in) throws IOException
Description copied from class: `ColumnEncoder`
Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.

Specified by:
  `readExternal` in interface `Externalizable`
Overrides:
  `readExternal` in class `ColumnEncoder`
Parameters:
  `in` - object input
Throws:
  `IOException` - if IOException occur