glsl-tokenizer

Maps GLSL string data into GLSL tokens, either synchronously or using a streaming API.

vartokenString=require('glsl-tokenizer/string')vartokenStream=require('glsl-tokenizer/stream')varfs=require('fs')// Synchronously:vartokens=tokenString(fs.readFileSync('some.glsl'))// Streaming API:fs.createReadStream('some.glsl').pipe(tokenStream()).on('data',function(token){console.log(token.data,token.position,token.type)})

API

tokens = require('glsl-tokenizer/string')(src, [opt])

Returns an array of tokens given the GLSL source string src

You can specify opt.version string to use different keywords/builtins, such as '300 es' for WebGL2. Otherwise, will assume GLSL 100 (WebGL1).

vartokens=tokenizer(src,{version: '300 es'})

stream = require('glsl-tokenizer/stream')([opt])

Emits 'data' events whenever a token is parsed with a token object as output.

As above, you can specify opt.version.

Tokens

{'type': TOKEN_TYPE,'data': "string of constituent data",'position': integerpositionwithintheGLSLsource,'line': linenumberwithintheGLSLsource,'column': columnnumberwithintheGLSLsource}

The available token types are:

• block-comment: /* ... */
• line-comment: // ... \n
• preprocessor: # ... \n
• operator: Any operator. If it looks like punctuation, it's an operator.
• float: Optionally suffixed with f
• ident: User defined identifier.
• builtin: Builtin function.
• eof: Emitted on end; data will === '(eof)'.
• integer
• whitespace
• keyword

License

MIT, see LICENSE.md for further information.