Skip to content

Latest commit

History

History
731 lines (566 loc) · 17.9 KB

File metadata and controls

731 lines (566 loc) · 17.9 KB

Apache IoTDB Node.js Client - Tree Model User Guide

Version: 1.0.0
Last Updated: 2024

Table of Contents

1. Introduction

1.1 Overview

The Apache IoTDB Node.js Client provides native support for the tree model (timeseries data model), enabling efficient management of time-series data using hierarchical device paths. This guide covers the SessionPool API for tree model operations with connection pooling and high-concurrency support.

1.2 Tree Model Features

The tree model in IoTDB organizes data hierarchically:

  • Path-based Organization: root.{storage_group}.{device}.{measurement}
  • Timeseries Management: Create and manage individual timeseries with specific data types
  • Efficient Batch Insertion: Use insertTablet for high-performance batch writes
  • Flexible Queries: Support for path patterns and wildcard queries
  • Connection Pooling: SessionPool for high-concurrency scenarios

1.3 Key Concepts

  • Storage Group: Top-level data organization unit (e.g., root.test)
  • Device: Physical or logical entity that generates data (e.g., root.test.device1)
  • Measurement: Sensor or metric name (e.g., temperature, humidity)
  • Timeseries: Full path with data type (e.g., root.test.device1.temperature FLOAT)

2. Installation

2.1 Install from npm

npm install @iotdb/client

Requirements:

  • Node.js >= 14.0.0
  • Apache IoTDB >= 1.0.0

2.2 Import in Your Project

TypeScript:

import{SessionPool,PoolConfigBuilder,TreeTablet,TSDataType}from'@iotdb/client';

JavaScript:

const{ SessionPool, PoolConfigBuilder, TreeTablet, TSDataType }=require('@iotdb/client');

3. Quick Start

3.1 SessionPool Example

import{SessionPool,TreeTablet}from'@iotdb/client';asyncfunctionquickStart(){// Create and initialize poolconstpool=newSessionPool('localhost',6667,{username: 'root',password: 'root',maxPoolSize: 10,minPoolSize: 2,});awaitpool.init();try{// Create storage groupawaitpool.executeNonQueryStatement('CREATE DATABASE root.test');// Create timeseriesawaitpool.executeNonQueryStatement('CREATE TIMESERIES root.test.device1.temperature WITH DATATYPE=FLOAT, ENCODING=RLE');// Insert data using TreeTablet class with addRowconsttablet=newTreeTablet('root.test.device1',['temperature'],[3]// FLOAT);tablet.addRow(Date.now(),[25.5]);awaitpool.insertTablet(tablet);// Query dataconstresult=awaitpool.executeQueryStatement('SELECT temperature FROM root.test.device1');console.log('Query result:',result);console.log('Pool size:',pool.getPoolSize());console.log('Available:',pool.getAvailableSize());}finally{awaitpool.close();}}quickStart();

4. SessionPool API

4.1 Overview

SessionPool provides connection pooling for high-concurrency scenarios. It automatically manages multiple sessions, distributes load across nodes, and recycles idle connections.

Key Features:

  • Round-robin load balancing across multiple nodes
  • Configurable pool size (min/max)
  • Automatic idle connection cleanup
  • Wait queue for connection requests
  • Thread-safe operations

4.2 Constructor

Option 1: Traditional API (Same Port)

constpool=newSessionPool(['node1','node2','node3'],// Hosts6667,// Port{username: 'root',password: 'root',maxPoolSize: 20,minPoolSize: 5,});

Option 2: Using nodeUrls (Different Ports)

constpool=newSessionPool({nodeUrls: ['node1:6667','node2:6668','node3:6669',],username: 'root',password: 'root',maxPoolSize: 20,minPoolSize: 5,});

Option 3: Using Builder Pattern (Recommended)

import{PoolConfigBuilder}from'@iotdb/client';constpool=newSessionPool(newPoolConfigBuilder().nodeUrls(['node1:6667','node2:6667','node3:6667']).username('root').password('root').maxPoolSize(20).minPoolSize(5).maxIdleTime(60000).waitTimeout(60000).build());

4.3 Pool Configuration Options

OptionTypeDefaultDescription
maxPoolSizenumber10Maximum number of sessions in pool
minPoolSizenumber1Minimum number of sessions maintained
maxIdleTimenumber60000Max idle time before cleanup (ms)
waitTimeoutnumber60000Max wait time for available session (ms)

4.4 Methods

4.4.1 Pool Management

async init(): Promise<void>

Initializes the connection pool and creates minimum sessions.

Example:

awaitpool.init();
async close(): Promise<void>

Closes all sessions in the pool and releases resources.

Example:

awaitpool.close();

4.4.2 Automatic Session Management

The pool automatically acquires and releases sessions for these operations:

async executeQueryStatement(sql: string, timeoutMs?: number): Promise<QueryResult>

Executes a query using an available session from the pool.

Example:

constresult=awaitpool.executeQueryStatement('SELECT * FROM root.test.**');
async executeNonQueryStatement(sql: string): Promise<void>

Executes a non-query statement using an available session.

Example:

awaitpool.executeNonQueryStatement('CREATE DATABASE root.test');
async insertTablet(tablet: Tablet): Promise<void>

Inserts data using an available session.

Example:

awaitpool.insertTablet({deviceId: 'root.test.device1',measurements: ['temperature'],dataTypes: [3],timestamps: [Date.now()],values: [[25.5]],});

4.4.3 Explicit Session Management

For multiple operations on the same session:

async getSession(): Promise<Session>

Acquires a session from the pool. Must be released after use.

Example:

constsession=awaitpool.getSession();try{awaitsession.executeNonQueryStatement('CREATE DATABASE root.test');awaitsession.insertTablet({/* data */});constresult=awaitsession.executeQueryStatement('SELECT ...');}finally{pool.releaseSession(session);}
releaseSession(session: Session): void

Releases a session back to the pool.

Example:

pool.releaseSession(session);

4.4.4 Pool Statistics

getPoolSize(): number

Returns the current total number of sessions in the pool.

getAvailableSize(): number

Returns the number of available (idle) sessions.

getInUseSize(): number

Returns the number of sessions currently in use.

Example:

console.log(`Total: ${pool.getPoolSize()}`);console.log(`Available: ${pool.getAvailableSize()}`);console.log(`In Use: ${pool.getInUseSize()}`);

5. Configuration Builders

5.1 PoolConfigBuilder

Fluent API for building SessionPool configurations.

Available Methods:

  • host(host: string): this
  • port(port: number): this
  • nodeUrls(urls: string[]): this
  • username(username: string): this
  • password(password: string): this
  • database(database: string): this
  • timezone(timezone: string): this
  • fetchSize(size: number): this
  • enableSSL(enable: boolean): this
  • sslOptions(options: SSLOptions): this
  • maxPoolSize(size: number): this
  • minPoolSize(size: number): this
  • maxIdleTime(time: number): this
  • waitTimeout(timeout: number): this
  • build(): PoolConfig

Example:

constpoolConfig=newPoolConfigBuilder().nodeUrls(['node1:6667','node2:6667']).username('root').password('root').maxPoolSize(20).minPoolSize(5).maxIdleTime(60000).waitTimeout(60000).build();constpool=newSessionPool(poolConfig);

6. Data Types

6.1 Supported Data Types

The tree model supports all IoTDB data types:

CodeTypeJavaScript TypeDescription
0BOOLEANbooleanTrue or false
1INT32number32-bit signed integer
2INT64number/string64-bit signed integer (use string for large values)
3FLOATnumber32-bit floating point
4DOUBLEnumber64-bit floating point
5TEXTstringUTF-8 string
8TIMESTAMPnumber/DateMilliseconds since epoch
9DATEnumber/DateCalendar date as yyyyMMdd integer (e.g. 20240101 for 2024-01-01)
10BLOBBufferBinary data
11STRINGstringSame as TEXT

6.2 Using Data Types in insertTablet

Example with Multiple Types:

awaitpool.insertTablet({deviceId: 'root.test.sensor1',measurements: ['temp','humidity','status','description','reading_time'],dataTypes: [3,4,0,5,8],// FLOAT, DOUBLE, BOOLEAN, TEXT, TIMESTAMPtimestamps: [Date.now()],values: [[25.5,// FLOAT60.123456,// DOUBLEtrue,// BOOLEAN'Normal operation',// TEXTDate.now(),// TIMESTAMP]],});

6.3 Handling INT64

For INT64 values larger than JavaScript's safe integer range (2^53 - 1), use strings:

awaitpool.insertTablet({deviceId: 'root.test.device1',measurements: ['largeCounter'],dataTypes: [2],// INT64timestamps: [Date.now()],values: [['9223372036854775807']],// String for large INT64});

7. Code Examples

7.1 Complete CRUD Example

import{SessionPool}from'@iotdb/client';asyncfunctioncrudExample(){constpool=newSessionPool('localhost',6667,{username: 'root',password: 'root',maxPoolSize: 10,minPoolSize: 2,});awaitpool.init();try{// CREATEawaitpool.executeNonQueryStatement('CREATE DATABASE root.factory');awaitpool.executeNonQueryStatement('CREATE TIMESERIES root.factory.workshop1.temperature WITH DATATYPE=FLOAT');// INSERTawaitpool.insertTablet({deviceId: 'root.factory.workshop1',measurements: ['temperature'],dataTypes: [3],timestamps: [Date.now()-3000,Date.now()-2000,Date.now()-1000],values: [[25.5],[26.0],[25.8]],});// READconstresult=awaitpool.executeQueryStatement('SELECT temperature FROM root.factory.workshop1');console.log('Temperature readings:',result);// UPDATE (Delete and re-insert)awaitpool.executeNonQueryStatement(`DELETE FROM root.factory.workshop1.temperature WHERE time <= ${Date.now()-2500}`);// DELETEawaitpool.executeNonQueryStatement('DELETE DATABASE root.factory');}finally{awaitpool.close();}}crudExample();

7.2 Multi-Node SessionPool Example

import{SessionPool,PoolConfigBuilder}from'@iotdb/client';asyncfunctionmultiNodeExample(){constpool=newSessionPool(newPoolConfigBuilder().nodeUrls(['iotdb-node1:6667','iotdb-node2:6667','iotdb-node3:6667',]).username('root').password('root').maxPoolSize(30).minPoolSize(10).build());awaitpool.init();try{// Simulate concurrent operationsconstoperations=[];for(leti=0;i<100;i++){operations.push(pool.insertTablet({deviceId: `root.test.device${i%10}`,measurements: ['value'],dataTypes: [3],timestamps: [Date.now()],values: [[Math.random()*100]],}));}awaitPromise.all(operations);console.log('All operations completed');// Pool statisticsconsole.log('Pool Statistics:');console.log(` Total Sessions: ${pool.getPoolSize()}`);console.log(` Available: ${pool.getAvailableSize()}`);console.log(` In Use: ${pool.getInUseSize()}`);}finally{awaitpool.close();}}multiNodeExample();

7.3 Time Range Query Example

asyncfunctiontimeRangeQuery(pool: SessionPool){constnow=Date.now();consthourAgo=now-3600000;constresult=awaitpool.executeQueryStatement(`SELECT temperature, humidity FROM root.test.** WHERE time >= ${hourAgo} AND time <= ${now}`);console.log('Query result:',result);returnresult;}

7.4 Batch Insert with Multiple Devices

asyncfunctionbatchInsertMultipleDevices(pool: SessionPool){constdevices=['device1','device2','device3'];consttimestamps=[];constnow=Date.now();// Generate timestamps for last 10 minutesfor(leti=0;i<600;i++){timestamps.push(now-(600-i)*1000);}for(constdeviceofdevices){constvalues=timestamps.map(()=>[20+Math.random()*10,// temperature50+Math.random()*30,// humidity]);awaitpool.insertTablet({deviceId: `root.test.${device}`,measurements: ['temperature','humidity'],dataTypes: [3,3],
timestamps,
values,});}console.log(`Inserted ${timestamps.length} records for ${devices.length} devices`);}

8. Best Practices

8.1 Resource Management

Always close pool resources:

// SessionPooltry{awaitpool.init();// ... operations}finally{awaitpool.close();}

8.2 Batch Insertion

Optimize batch size:

  • Use insertTablet instead of individual inserts
  • Batch 100-1000 rows per tablet
  • Consider memory vs network trade-offs

Example:

// Good: Batch insertawaitpool.insertTablet({deviceId: 'root.test.device1',measurements: ['temperature'],dataTypes: [3],timestamps: timestamps,// 100-1000 timestampsvalues: values,// 100-1000 values});// Bad: Individual insertsfor(leti=0;i<1000;i++){awaitpool.executeNonQueryStatement(`INSERT INTO root.test.device1(timestamp, temperature) VALUES(${timestamps[i]}, ${values[i]})`);}

8.3 Error Handling

try{awaitpool.init();awaitpool.executeNonQueryStatement('CREATE DATABASE root.test');}catch(error){if(error.message.includes('already exists')){console.log('Database already exists, continuing...');}else{console.error('Failed to create database:',error);throwerror;}}finally{awaitpool.close();}

8.4 Connection Pool Sizing

Guidelines:

  • Set minPoolSize to average concurrent load
  • Set maxPoolSize to peak load + buffer (20-30%)
  • Monitor pool statistics in production
  • Adjust based on server capacity

Example:

constpool=newSessionPool({nodeUrls: ['localhost:6667'],maxPoolSize: 50,// Peak load: 40 clients + 25% bufferminPoolSize: 20,// Average load: 20 clientsmaxIdleTime: 60000,// Clean up idle after 1 minutewaitTimeout: 30000,// Wait max 30s for available session});

9. Troubleshooting

9.1 Common Issues

Connection Refused

Symptoms:

Error: connect ECONNREFUSED 127.0.0.1:6667

Solutions:

  1. Verify IoTDB is running: jps | grep IoTDB
  2. Check port configuration in iotdb-datanode.properties
  3. Verify firewall allows connections
  4. Test with telnet: telnet localhost 6667

Pool Timeout

Symptoms:

Error: Timeout waiting for available session

Solutions:

  1. Increase waitTimeout in pool configuration
  2. Increase maxPoolSize if server can handle it
  3. Verify sessions are being released properly
  4. Check for connection leaks (forgotten releaseSession())

Out of Memory

Symptoms:

FATAL ERROR: Reached heap limit

Solutions:

  1. Reduce fetchSize in pool configuration
  2. Process query results in batches
  3. Increase Node.js heap: node --max-old-space-size=4096 app.js

9.2 Debugging Tips

Enable debug logging:

// Set environment variableprocess.env.LOG_LEVEL='debug';// Or use logger directlyimport{logger}from'@iotdb/client';logger.setLevel('debug');

Check connection status:

console.log('Pool size:',pool.getPoolSize());console.log('Available:',pool.getAvailableSize());console.log('In Use:',pool.getInUseSize());

Test query execution time:

conststart=Date.now();constresult=awaitpool.executeQueryStatement('SELECT ...');console.log(`Query took ${Date.now()-start}ms`);

9.3 Performance Optimization

  1. Use connection pooling: For concurrent operations
  2. Batch inserts: Use insertTablet with 100-1000 rows
  3. Multi-node setup: Distribute load across nodes
  4. Monitor resources: Watch CPU, memory, network
  5. Adjust pool size: Set min/max pool size based on workload

9.4 Getting Help

Appendix A: Complete Type Reference

See data-types.md for comprehensive data type documentation.

Appendix B: API Quick Reference

SessionPool Methods

  • init() - Initialize pool
  • close() - Close all sessions
  • executeQueryStatement(sql, timeout?) - Execute query
  • executeNonQueryStatement(sql) - Execute DDL/DML
  • insertTablet(tablet) - Batch insert
  • getSession() - Get session from pool
  • releaseSession(session) - Return session to pool
  • getPoolSize() - Total sessions
  • getAvailableSize() - Available sessions
  • getInUseSize() - Active sessions

Version: 1.0.0
Last Updated: January 2024
License: Apache License 2.0