# Uses of Class
org.apache.sysds.runtime.transform.tokenize.Token

Packages that use Token
Package
Description
org.apache.sysds.runtime.transform.tokenize
org.apache.sysds.runtime.transform.tokenize.builder

## Uses of Token in org.apache.sysds.runtime.transform.tokenize

Fields in org.apache.sysds.runtime.transform.tokenize with type parameters of type Token
Modifier and Type
Field
Description
`List<Token>`
DocumentRepresentation.`tokens`
Methods in org.apache.sysds.runtime.transform.tokenize that return types with arguments of type Token
Modifier and Type
Method
Description
`List<Token>`
DocumentRepresentation.`getTokens()`
Constructor parameters in org.apache.sysds.runtime.transform.tokenize with type arguments of type Token
Modifier
Constructor
Description
``
`Token(List<Token> subList)`

## Uses of Token in org.apache.sysds.runtime.transform.tokenize.builder

Methods in org.apache.sysds.runtime.transform.tokenize.builder that return types with arguments of type Token
Modifier and Type
Method
Description
`List<Token>`
TokenizerBuilderNgram.`splitIntoNgrams(Token token, int minGram, int maxGram)`
`List<Token>`
TokenizerBuilderWhitespaceSplit.`splitToTokens(String text)`
Methods in org.apache.sysds.runtime.transform.tokenize.builder with parameters of type Token
Modifier and Type
Method
Description
`List<Token>`
TokenizerBuilderNgram.`splitIntoNgrams(Token token, int minGram, int maxGram)`