jsonlines-web

Web stream based jsonlines decoder/encoder

• ✅Deno
• ✅browser
• ✅Node.js

This library supports JSON in the following formats:

• Line-delimited JSON (JSONLinesParseStream)
  • NDJSON
  • JSON lines
• Record separator-delimited JSON (JSONLinesParseStream)
• Concatenated JSON (ConcatenatedJSONParseStream)

See wikipedia for the specifications of each JSON.

install or import

Deno

https://deno.land/x/jsonlines/https://doc.deno.land/https://deno.land/x/jsonlines/mod.ts

import{ConcatenatedJSONParseStream,ConcatenatedJSONStringifyStream,JSONLinesParseStream,JSONLinesStringifyStream,}from"https://deno.land/x/jsonlines@v1.2.1/mod.ts";

browser

import{ConcatenatedJSONParseStream,ConcatenatedJSONStringifyStream,JSONLinesParseStream,JSONLinesStringifyStream,}from"https://deno.land/x/jsonlines@v1.2.1/js/mod.js";

Node.js

https://www.npmjs.com/package/jsonlines-web

npm install jsonlines-web

import{ConcatenatedJSONParseStream,ConcatenatedJSONStringifyStream,JSONLinesParseStream,JSONLinesStringifyStream,}from"jsonlines-web";// if you need// import { TextDecoderStream, TextEncoderStream } from "node:stream/web";// import { fetch } from "undici";

Usage

A working example can be found at ./testdata/test.ts.

How to parse JSON Lines

./json-lines.jsonl

{"some":"thing"}
{"foo":17,"bar":false,"quux":true}
{"may":{"include":"nested","objects":["and","arrays"]}}

import{JSONLinesParseStream}from"https://deno.land/x/jsonlines@v1.2.1/mod.ts";const{ body }=awaitfetch("https://deno.land/x/jsonlines@v1.2.1/testdata/json-lines.jsonl",);constreadable=body!.pipeThrough(newTextDecoderStream()).pipeThrough(newJSONLinesParseStream());forawait(constdataofreadable){console.log(data);}

How to parse json-seq

./json-seq.json-seq

�{"some":"thing\n"}�{"may": {
"include": "nested",
"objects": [
"and",
"arrays"
]
}
}

import{JSONLinesParseStream}from"https://deno.land/x/jsonlines@v1.2.1/mod.ts";const{ body }=awaitfetch("https://deno.land/x/jsonlines@v1.2.1/testdata/json-seq.json-seq",);constrecordSeparator="\x1E";constreadable=body!.pipeThrough(newTextDecoderStream()).pipeThrough(newJSONLinesParseStream({separator: recordSeparator}));forawait(constdataofreadable){console.log(data);}

How to parse concat-json

./concat-json.concat-json

{"foo":"bar"}{"qux":"corge"}{"baz":{"waldo":"thud"}}

import{ConcatenatedJSONParseStream}from"https://deno.land/x/jsonlines@v1.2.1/mod.ts";const{ body }=awaitfetch("https://deno.land/x/jsonlines@v1.2.1/testdata/concat-json.concat-json",);constreadable=body!.pipeThrough(newTextDecoderStream()).pipeThrough(newConcatenatedJSONParseStream());forawait(constdataofreadable){console.log(data);}

How to stringify JSON Lines

import{readableStreamFromIterable}from"https://deno.land/std@0.138.0/streams/mod.ts";import{JSONLinesStringifyStream}from"https://deno.land/x/jsonlines@v1.2.1/mod.ts";constfile=awaitDeno.open(newURL("./tmp.concat-json",import.meta.url),{create: true,write: true,});readableStreamFromIterable([{foo: "bar"},{baz: 100}]).pipeThrough(newJSONLinesStringifyStream()).pipeThrough(newTextEncoderStream()).pipeTo(file.writable).then(()=>console.log("write success"));

How to stringify json-seq

import{readableStreamFromIterable}from"https://deno.land/std@0.138.0/streams/mod.ts";import{JSONLinesStringifyStream}from"https://deno.land/x/jsonlines@v1.2.1/mod.ts";constrecordSeparator="\x1E";constfile=awaitDeno.open(newURL("./tmp.concat-json",import.meta.url),{create: true,write: true,});readableStreamFromIterable([{foo: "bar"},{baz: 100}]).pipeThrough(newJSONLinesStringifyStream({separator: recordSeparator})).pipeThrough(newTextEncoderStream()).pipeTo(file.writable).then(()=>console.log("write success"));

How to stringify concat-json

import{readableStreamFromIterable}from"https://deno.land/std@0.138.0/streams/mod.ts";import{ConcatenatedJSONStringifyStream}from"https://deno.land/x/jsonlines@v1.2.1/mod.ts";constfile=awaitDeno.open(newURL("./tmp.concat-json",import.meta.url),{create: true,write: true,});readableStreamFromIterable([{foo: "bar"},{baz: 100}]).pipeThrough(newConcatenatedJSONStringifyStream()).pipeThrough(newTextEncoderStream()).pipeTo(file.writable).then(()=>console.log("write success"));

note

This library contains ReadableStream.prototype[Symbol.asyncIterator] polyfills. Importing this library will automatically enable ReadableStream.prototype[Symbol.asyncIterator].

develop

need to manually deno task transpile before release.