BatchJS-Data

Extension of Batch JS adding data storage support for databases.

---

• BatchJS-Data
• Download
  • NPM
  • Yarn
• Usage
• Documentation
• Collaborators welcome!

---

Download

NPM

npm install batchjs-data
npm install mariadb #For MariaDB implementation
npm install mysql2 #For MySQL implementation
npm install pg @types/pg #For PostgreSQL implementation

Yarn

yarn add batchjs-data --no-optional
yarn add mariadb #For MariaDB implementation
yarn add mysql2 #For MySQL implementation
yarn add pg @types/pg #For PostgreSQL implementation

Usage

1. Create your reader

   import{DatabaseSync}from"node:sqlite";import{SqliteBatchEntityReader}from"batchjs-data/sqlite";import{UserDTO}from"./UserDTO";exportclassUserBatchReaderextendsSqliteBatchEntityReader<UserDTO,UserDTO>{constructor(options: {batchSize: number;query?: string}){super({batchSize: options.batchSize,dbConnectionFactory: async()=>Promise.resolve(newDatabaseSync("./database.db")),query: options.query||"SELECT id, username FROM users",rowToEntity: (row: UserDTO)=>row,});}}

2. Create your writer

   import{DatabaseSync,StatementSync}from"node:sqlite";import{SqliteBatchEntityWriter}from"batchjs-data/sqlite";import{UserDTO}from"./UserDTO";exportclassUserBatchWriterextendsSqliteBatchEntityWriter<UserDTO>{constructor(options: {batchSize: number}){super({batchSize: options.batchSize,dbConnectionFactory: async()=>Promise.resolve(newDatabaseSync("./database.db")),prepareStatement: "INSERT INTO users (id, username) VALUES (?, ?)",saveEntity: async(entity: UserDTO,stmt: StatementSync)=>{stmt.run(entity.id,entity.username);},});}}

3. Use them in your BatchJS Job

   import{Job,Step}from"batchjs";// Implement a stepclassStepImplementationextendsStep{// Set a name to the stepconstructor(name: string="DemoStep"){super(name);}// Implement the reader to load step data sourceprotected_reader(){returnnewUserBatchReader({batchSize: 2});}// Implement the processors to transform data sequently using our streams or your own streamsprotected_processors(){constopts: TransformOptions={objectMode: true,transform(chunk: unknown,encoding: BufferEncoding,callback: TransformCallback,){this.push(chunk);callback();},};return[newTransform(opts),newTransform(opts)];}// Implement the write to stock final step dataprotected_writer(){returnnewUserBatchWriter({batchSize: 2});}}// Implement a JobclassJobImplementationextendsJob{// Implement to set the steps to be sequently executed.protected_steps(){return[newStepImplementation(),newStepImplementation()];}}// Instance the Jobconstjob=newJobImplementation("My job");// Set events listenerjob.on("stepStart",(step: step)=>{console.log(`Starting step ${step.name}`);});// Launch the jobjob.run().then(()=>{console.log("Job completed successfully");}).catch((error)=>{console.log("Job completed with errors");});

Documentation

• Documentation site
• Core API
• MariaDB API
• MySQL API
• PostgreSQL API
• SQLite API

To preview the documentation locally, run:

npm run docs

Collaborators welcome!

• ¿Do you like the project? Give us a ⭐ in GitHub.
• 🆘 ¿Do you need some help? Open a discussion in GitHub help wanted
• 🐛 ¿Do you find a bug? Open a issue in GitHub bug report
• 💡 ¿Do you have a great idea? Open a issue in GitHub feature request
• 💻 ¿Do you know how to fix a bug? Open a pull request in GitHub pull request.
• ¿Do you know a security issue? Take a read to our security strategy.

Subscribe our code of conduct and follow the Contribution Guidelines.