IBM
Developer
Explore
Articles
Blogs
Courses
Learning paths
Open projects
Series
Tutorials
Products
IBM Bob
Granite models
Open Liberty
watsonx.ai
watsonx.data
Docling
Languages
IBM Semeru Runtimes
Java
Python
Node.js
JavaScript
COBOL
Technologies
Artificial intelligence
Data Science
Messaging
Machine Learning
Observability
Security
Events
Events
IBM Hackathons
IBM Community Events
TechXchange Conference
Resources
External Resources
IBM Documentation
IBM Support
IBM Developer Videos
IBM Technology Videos
Open Source @ IBM
TechXchange
Home
Explore
Articles
Blogs
Courses
Learning paths
Open projects
Series
Tutorials
Products
IBM Bob
Granite models
Open Liberty
watsonx.ai
watsonx.data
Docling
Languages
IBM Semeru Runtimes
Java
Python
Node.js
JavaScript
COBOL
Technologies
Artificial intelligence
Data Science
Messaging
Machine Learning
Observability
Security
Events
Events
IBM Hackathons
IBM Community Events
TechXchange Conference
Resources
External Resources
IBM Documentation
IBM Support
IBM Developer Videos
IBM Technology Videos
Open Source @ IBM
TechXchange
Subscribe
Options
Tutorial
Tokenizing text in Python
Use tokenizers from the Python NLTK to complete a standard text normalization technique
By
Jacob Murel, Ph.D.
Save
On this page
Overview