RunKit +

Try any Node.js package right in your browser

This is a playground to test code. It runs a full Node.js environment and already has all of npm’s 1,000,000+ packages pre-installed, including tokenizer2 with all npm packages installed. Try it out:

var tokenizer2 = require("tokenizer2")

all documents on RunKit are public
require() any package directly from npm
use arrow functions, classes, template strings, and most of ES6
await any promise instead of using callbacks (example)
create your own embedded node.js snippets

This service is provided by RunKit and is not affiliated with npm, Inc or the package authors.

tokenizer2 v2.0.1

tokenize any text stream given some basic regex rules to match tokens

Overview Browse Files