Package org.apache.sysds.runtime.transform.tokenize.applier

# Class TokenizerApplierPosition

java.lang.Object
org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier
org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplierPosition

All Implemented Interfaces:
  `Serializable`

public class TokenizerApplierPosition extends TokenizerApplier

See Also:
  - Serialized Form

## Field Summary

### Fields inherited from class org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

`PADDING_STRING`

## Constructor Summary

Constructors
Constructor
Description
`TokenizerApplierPosition(int numIdCols, int maxTokens, boolean wideFormat, boolean applyPadding)`

## Method Summary

Modifier and Type
Method
Description
`int`
`appendTokensLong(int row, List<Object> keys, List<Token> tokenList, FrameBlock out)`
`int`
`appendTokensWide(int row, List<Object> keys, List<Token> tokenList, FrameBlock out)`
`int`
`applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)`
`int`
`getNumRows(DocumentRepresentation[] internalRepresentation)`
`Types.ValueType[]`
`getOutSchema()`

### Methods inherited from class org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

`allocateInternalMeta, applyInternalRepresentation, build, getApplyTasks, getBuildTasks, getMaxNumRows, getMaxTokens, getNumCols, getOutputRow, getOutputRow, hasPadding, isWideFormat`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Constructor Details

### TokenizerApplierPosition

public TokenizerApplierPosition(int numIdCols, int maxTokens, boolean wideFormat, boolean applyPadding)

## Method Details

### getNumRows

public int getNumRows(DocumentRepresentation[] internalRepresentation)

Specified by:
  `getNumRows` in class `TokenizerApplier`

### applyInternalRepresentation

public int applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)

### appendTokensLong

public int appendTokensLong(int row, List<Object> keys, List<Token> tokenList, FrameBlock out)

### appendTokensWide

public int appendTokensWide(int row, List<Object> keys, List<Token> tokenList, FrameBlock out)

### getOutSchema

public Types.ValueType[] getOutSchema()

Specified by:
  `getOutSchema` in class `TokenizerApplier`