Package org.apache.sysds.runtime.transform.tokenize.builder

# Class TokenizerBuilderWhitespaceSplit

java.lang.Object
org.apache.sysds.runtime.transform.tokenize.builder.TokenizerBuilder
org.apache.sysds.runtime.transform.tokenize.builder.TokenizerBuilderWhitespaceSplit

All Implemented Interfaces:
  `Serializable`

Direct Known Subclasses:
  `TokenizerBuilderNgram`

public class TokenizerBuilderWhitespaceSplit extends TokenizerBuilder

See Also:
  - Serialized Form

## Field Summary

Fields
Modifier and Type
Field
Description
`String`
`regex`

## Constructor Summary

Constructors
Constructor
Description
`TokenizerBuilderWhitespaceSplit(int[] idCols, int tokenizeCol, org.apache.wink.json4j.JSONObject params)`

## Method Summary

Modifier and Type
Method
Description
`void`
`createInternalRepresentation(FrameBlock in, DocumentRepresentation[] internalRepresentation, int rowStart, int blk)`
`List<Token>`
`splitToTokens(String text)`

### Methods inherited from class org.apache.sysds.runtime.transform.tokenize.builder.TokenizerBuilder

`createInternalRepresentation, getTasks`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### regex

public String regex

## Constructor Details

### TokenizerBuilderWhitespaceSplit

public TokenizerBuilderWhitespaceSplit(int[] idCols, int tokenizeCol, org.apache.wink.json4j.JSONObject params) throws org.apache.wink.json4j.JSONException

Throws:
  `org.apache.wink.json4j.JSONException`

## Method Details

### splitToTokens

public List<Token> splitToTokens(String text)

### createInternalRepresentation

public void createInternalRepresentation(FrameBlock in, DocumentRepresentation[] internalRepresentation, int rowStart, int blk)

Specified by:
  `createInternalRepresentation` in class `TokenizerBuilder`