tokenize
English
/ˈtoʊ.kən.aɪz/
verb
Definitions
- (transitive) To reduce to a set of tokens by lexical analysis.
- (transitive) To substitute sensitive data with meaningless placeholders.
- (transitive) To treat as a token minority.
Etymology
Suffix from English token.
Origin
English
token
Gloss
Timeline
Distribution of cognates by language
Geogrpahic distribution of cognates
Cognates and derived terms
- betoken English
- detokenize English
- detokenizer English
- foretoken English
- honeytoken English
- retokenize English
- subtoken English
- token English
- tokeneth English
- tokening English
- tokenisation English
- tokenism English
- tokenist English
- tokenistic English
- tokenizable English
- tokenization English
- tokenizer English
- tokenless English
- tokenlike English
- tokenwise English
- token Dutch, Flemish
- *deyḱ- Proto-Indo-European
- トークン Japanese
- token Middle English