Login Preferences
About News Publications Statistics Downloads API Guide License How to use Preferences
bn:13890904n
Noun Concept
Categories: Tasks of natural language processing, Computing stubs
EN
tokenization  tokenisation
EN
In lexical analysis, tokenization is the process of breaking a stream of text up into words, phrases, symbols, or other meaningful elements called tokens. Wikipedia
Definitions
Relations
Sources
EN
In lexical analysis, tokenization is the process of breaking a stream of text up into words, phrases, symbols, or other meaningful elements called tokens. Wikipedia
IS A
natural language processing
Wikipedia
EN
tokenization (lexical analysis)
Wikidata
EN
tokenization
Wikidata Alias
EN
tokenisation

Please note that, in order to improve your browsing experience on this website, BabelNet® uses various types of cookies, including: browsing functionality, performance and statistical cookies.

By continuing to browse the site you are agreeing to our use of cookies. OK

License