What's dmr-source

Dmr-Source provides unified methods to achieve stream of data sources, such as local / remote (http / ftp) ...

• Source
• FileSource
• HttpSource
• FtpSource
• MultiSource
• SftpSource
• HadoopSource

Usage

Get file from ftp server to local

constfs=newFtpSource({host: "127.0.0.1",path: "/source-ftp-test.dict",port: 21,});constwriter=newFileSource().createWritableStream({path: "/home/work/a.log"});fs.createReadableStream().pipe(writer);

Copy big file by FileSource

constfs=newFileSource({encoding: "utf8"});constreader=fs.createReadableStream({path: "/home/work/a.log"});constwriter=fs.createWritableStream((option)=>{option.path="/home/work/b.log";returnoption;});reader.pipe(writer);

Link readables to one readable stream

conststream1=newHttpSource({host: "localhost",path: "/a.log"}).createReadableStream();conststream2=newFileSource({host: "localhost",path: "/b.log"}).createReadableStream();constms=newMultiSource().add(stream1).add(stream2).add(()=>{returnnewFileSource({host: "localhost",path: "/c.log"}).createReadableStream();});ms.createReadableStream().on("data",(chunk)=>console.log(chunk));

Events

conststream=newHttpSource({host: "localhost",path: "/404.page",timeout: 6000}).createReadableStream();stream.on("error",(err)=>{console.log("should be 404 error",err);});stream.on("end",(err)=>{console.log("never arrive end");});

API

• https://node-dmr.github.io/dmr-source/