Package org.apache.sysds.runtime.transform.tokenize.applier

# Class TokenizerApplierHash

java.lang.Object
org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier
org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplierHash

All Implemented Interfaces:
  `Serializable`

public class TokenizerApplierHash extends TokenizerApplier

See Also:
  - Serialized Form

## Field Summary

Fields
Modifier and Type
Field
Description
`int`
`num_features`

### Fields inherited from class org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

`PADDING_STRING`

## Constructor Summary

Constructors
Constructor
Description
`TokenizerApplierHash(int numIdCols, int maxTokens, boolean wideFormat, boolean applyPadding, org.apache.wink.json4j.JSONObject params)`

## Method Summary

Modifier and Type
Method
Description
`void`
`allocateInternalMeta(int numDocuments)`
`int`
`applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)`
`void`
`build(DocumentRepresentation[] internalRepresentation, int inputRowStart, int blk)`
`int`
`getNumRows(DocumentRepresentation[] internalRepresentation)`
`Types.ValueType[]`
`getOutSchema()`

### Methods inherited from class org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

`applyInternalRepresentation, getApplyTasks, getBuildTasks, getMaxNumRows, getMaxTokens, getNumCols, getOutputRow, getOutputRow, hasPadding, isWideFormat`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### num_features

public int num_features

## Constructor Details

### TokenizerApplierHash

public TokenizerApplierHash(int numIdCols, int maxTokens, boolean wideFormat, boolean applyPadding, org.apache.wink.json4j.JSONObject params) throws org.apache.wink.json4j.JSONException

Throws:
  `org.apache.wink.json4j.JSONException`

## Method Details

### getNumRows

public int getNumRows(DocumentRepresentation[] internalRepresentation)

Specified by:
  `getNumRows` in class `TokenizerApplier`

### allocateInternalMeta

public void allocateInternalMeta(int numDocuments)

Overrides:
  `allocateInternalMeta` in class `TokenizerApplier`

### build

public void build(DocumentRepresentation[] internalRepresentation, int inputRowStart, int blk)

Overrides:
  `build` in class `TokenizerApplier`

### applyInternalRepresentation

public int applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)

### getOutSchema

public Types.ValueType[] getOutSchema()

Specified by:
  `getOutSchema` in class `TokenizerApplier`