keywords:tokenizer - npm search

js-tokens

Tiny JavaScript tokenizer.

published version 10.0.0, 6 months ago5822 dependents licensed under $MIT

786,335,683

css-tree

A tool set for CSS: fast detailed parser (CSS → AST), walker (AST traversal), generator (AST → CSS) and lexer (validation and matching) based on specs and browser implementations

lahmatiy

published version 3.2.1, 3 months ago1081 dependents licensed under $MIT

350,196,989

strtok3

A promise based streaming tokenizer

borewit

published version 10.3.5, 3 months ago64 dependents licensed under $MIT

101,382,382

micromark

small commonmark compliant markdown parser with positional info and concrete tokens

wooorm

published version 4.0.2, a year ago185 dependents licensed under $MIT

165,950,985

@csstools/css-tokenizer

Tokenize CSS

romainmenke

published version 4.0.0, 5 months ago62 dependents licensed under $MIT

187,045,922

ret

Tokenizes a string that represents a regular expression.

fent

published version 0.5.0, 3 years ago674 dependents licensed under $MIT

116,518,800

@tokenizer/token

TypeScript definition for strtok3 token

borewit

published version 0.3.0, 5 years ago15 dependents licensed under $MIT

82,291,747

@tokenizer/inflate

Tokenized zip support

borewit

published version 0.4.1, 7 months ago5 dependents licensed under $MIT

67,851,461

A micro-library of stream components for building custom JSON and JSONC processing pipelines with a minimal memory footprint — parse, filter, and transform JSON far larger than available memory with a SAX-inspired token API, on Node.js or Web Streams.

elazutkin

published version 3.4.0, 6 days ago440 dependents licensed under $BSD-3-Clause

29,476,979

leac

Lexer / tokenizer

killy.mxi

published version 0.7.0, 9 months ago5 dependents licensed under $MIT

40,550,644

chevrotain

Chevrotain is a high performance fault tolerant javascript parsing DSL for building recursive decent parsers

GitHub Actions

published version 12.0.0, 3 months ago459 dependents licensed under $Apache-2.0

38,682,482

unbash

Fast 0-deps bash parser written in TypeScript

webpro

published version 4.0.1, 4 days ago12 dependents licensed under $ISC

23,596,670

scss-tokenizer

A tokenzier for Sass' SCSS syntax

xzyfer

published version 0.4.3, 4 years ago35 dependents licensed under $MIT

3,824,583

@eslint/css-tree

A tool set for CSS: fast detailed parser (CSS → AST), walker (AST traversal), generator (AST → CSS) and lexer (validation and matching) based on specs and browser implementations

GitHub Actions

published version 4.0.4, 24 days ago9 dependents licensed under $MIT

3,700,553

natural

General natural language (tokenizing, stemming (English, Russian, Spanish), part-of-speech tagging, sentiment analysis, classification, inflection, phonetics, tfidf, WordNet, jaro-winkler, Levenshtein distance, Dice's Coefficient) facilities for node.