Skip to content

Repository files navigation

dClimate Client JS

Foundation for a JavaScript/TypeScript client that loads dClimate Zarr datasets from IPFS using jaxray.

The goal is to mirror the functionality of the Python dclimate-zarr-client package while staying within the JavaScript ecosystem and avoiding any Ethereum/Web3 dependencies. This initial pass focuses on resolving dataset CIDs, opening sharded Zarr stores with jaxray, and exposing a small geotemporal helper API for downstream applications.

Features

  • Dataset loader with curated catalog that resolves datasets from HTTP endpoints or direct CIDs
  • Smart variant concatenation – automatically merges finalized and non-finalized data for complete time series
  • IPFS integration powered by @dclimate/jaxray for efficient Zarr store access
  • GeoTemporalDataset wrapper with rich selection capabilities:
    • Single point selection with nearest-neighbor matching
    • Multiple points selection
    • Circular region selection (radius-based)
    • Rectangular bounding box selection
    • Time range filtering
    • Chained selections for complex queries
  • No blockchain dependencies – all resolution through HTTP APIs and IPFS
  • TypeScript support with full type definitions
  • Dual build targets for Node.js and browser environments
  • OpenTelemetry hooks for IPFS/Zarr open latency, status, and gateway attribution

Installation

npm install @dclimate/dclimate-client-js

Getting Started

For contributors

# Clone and build from source
git clone https://github.com/dClimate/dclimate-client-js.git
cd dclimate-client-js
npm install
npm run build

Usage

import{DClimateClient}from"@dclimate/dclimate-client-js";constclient=newDClimateClient();// Load a dataset by collection, dataset name, and variant.// Returns a tuple: [dataset, metadata]const[dataset,metadata]=awaitclient.loadDataset({request: {collection: "aifs",dataset: "temperature",variant: "ensemble"}});// Check metadata about what was loadedconsole.log(`Loaded: ${metadata.path}`);console.log(`CID: ${metadata.cid}`);console.log(`Timestamp: ${metadata.timestamp}`);// Unix timestamp in millisecondsconsole.log(`Source: ${metadata.source}`);// 'catalog' or 'direct_cid'console.log(`URL: ${metadata.url}`);// URL if fetched from endpoint// Narrow to a single location (nearest neighbour) and time range.constpoint=awaitdataset.point(40.75,-73.99);constslice=awaitpoint.timeRange({start: "2023-01-01T00:00:00Z",end: "2023-01-07T00:00:00Z",});console.log(awaitslice.toRecords("precipitation"));

Entity data usage

Gridded Zarr datasets come from loadDataset. Point-observation entity datasets (GHCND and friends) live under client.entities, and read the same way: degrees, ISO timestamps, chained selections.

"Entity" is the format's word for whatever a row belongs to — a weather station in GHCND, a buoy in NDBC.

// `columnKey` maps schema field names to the column names queries use. It is a// property of the dataset's publishing profile, not of the stored blocks, so// the caller states it: GHCND stores `tmax` and publishes `TMAX`. Without it,// column names default to the schema's own (lowercase, for GHCND).constentities=awaitclient.entities.load({cid: "bafyr4i...",columnKey: (field)=>field.name.toUpperCase(),// GHCND's mapping});// Every entity, with position and coverage window.for(consteofawaitentities.listEntities()){console.log(e.entityId,e.latitude,e.longitude,e.start,e.end);}// Windows the dataset states it does *not* know, as distinct from windows where// nothing happened. `[start, end]` is only an outer envelope: a station can go// dark in the middle of it, and without this a missing row and an unfetched one// look identical. Carried on every info row at no extra cost.for(constgapof(awaitentities.infoFor("USW00094728")).gaps){console.log(gap.beginUs,gap.endUs,gap.reason);// e.g. "awdb: HTTP 500"}// Every column the dataset defines, with its stated unit.console.log(entities.columns());// [{ name: "TMAX", units: "degC_tenths" }, ...]// Entities within 50 km of a point, over one week.constrecords=awaitentities.circle(40.75,-73.99,50).timeRange({start: "2023-01-01",end: "2023-01-07"}).toRecords("TMAX");

Selections return new instances, so a partial selection can be branched:

constweek=entities.timeRange({start: "2023-01-01",end: "2023-01-07"});constnyc=awaitweek.select("USW00094728").rows();constlax=awaitweek.select("USW00023174").rows();

Two things differ from GeoTemporalDataset, because the data model differs:

  • nearest(lat, lon, { maxKm }) instead of point(). A grid always has a cell under any coordinate; stations are irregular, so the nearest one may be far away. Pass maxKm to make that a hard bound rather than a surprise. Use findNearestEntity(lat, lon) when you need the distance itself — it returns { entityId, km, latitude, longitude } instead of a chainable dataset.
  • where(...) has no gridded counterpart. Row-level predicates are pushed down to fragment statistics, so most fragments are skipped without being read:
// `nearest` reads the entity index to find the match, so it is async --// unlike the synchronous selections above, it has to be awaited before the// chain continues.consthotDays=await(awaitentities.nearest(29.98,-95.36)).timeRange({start: "2025-01-01",end: "2025-12-31"}).where({element: "TMAX",op: "gt",value: 350})// tenths of °C, so 35 °C.rows();

A third difference is worth stating on its own, because it changes how a result should be read:

  • An empty result may mean "unknown", not "nothing happened".plan() reports the known gaps overlapping the range you asked for, so a query that lands in a hole says so instead of returning a plausible, silent zero rows. Only overlapping gaps are listed — asking about 2024 stays quiet about a hole in 1996.
constrange=entities.select("2001:NE:SCAN").timeRange({start: "1996-06-01",end: "1996-07-01"});const{ gaps }=awaitrange.plan();for(const{ entityId,gaps: windows}ofgaps){for(constgofwindows){console.warn(`${entityId}: no data for ${g.beginUs}..${g.endUs} (${g.reason})`);}}constrows=awaitrange.rows();// may be empty *because* of the gap above

Reads go over the IPFS HTTP gateway, so no local daemon is required and the same code runs in a browser. Resolution is by CID for now; STAC catalog support will follow.

Dataset version history

For datasets that advertise version history in STAC, the client follows the item's dclimate:versions_api URL. STAC therefore selects Hydrogen, Tritium, or a future version service without a client-side dataset routing table.

constversions=awaitclient.listDatasetVersions({collection: "noaa_aigfs",dataset: "wind_u_forecast",variant: "operational",filters: {anchored: true,isCitable: true,versionLabel: "2026-08",},});for(constreleaseofversions.versions){console.log(release.versionLabel,release.cid);}constexactVersion=awaitclient.getDatasetVersion({collection: "noaa_aigfs",dataset: "wind_u_forecast",variant: "operational",commitId: "commit-id",});console.log(exactVersion.cid);

The low-level listVersionsFromUrl, getExactVersionFromUrl, and getCitationFromUrl helpers are also exported for applications that already have the complete URLs. Items backed by hard-coded CIDs may not advertise a version-history service.

Multiresolution datasets

Pyramidal datasets require an explicit resolution (recommended) or raw Zarr group. The client reports the available resolutions instead of silently choosing between different precision, chunking, and fetching strategies.

const[data,metadata]=awaitclient.loadDataset({request: {collection: "copernicus_clms",dataset: "fpar",resolution: "2km",},});console.log(metadata.resolution,metadata.zarrGroup);

FPAR advertises 500m → group "0", 2km → group "1", and 8km → group "2". Change request.resolution to select any of those levels. A raw options.zarrGroup is supported for storage-aware callers, but must not be combined with request.resolution.

During migration, STAC may also contain a legacy assets.data alias for the 500 m asset. The client ignores it when building the three choices, and it is neither a fourth resolution nor a default. Consumers relying on assets.data or implicit group "0" should migrate before the alias is removed in a future breaking release.

Direct CID requests have no STAC resolution mapping and must use options.zarrGroup when the store contains multiple groups; a human-readable resolution is rejected. STAC's internal metadataGroup controls only catalog metadata extraction and never selects a client resolution.

Siren REST API usage

Use Siren methods by configuring siren in the client options.

import{DClimateClient}from"@dclimate/dclimate-client-js";constclient=newDClimateClient({siren: {auth: {type: "apiKey"},// reads SIREN_API_KEY + SIREN_ACCOUNT_ID from env when omitted},});constmetrics=awaitclient.siren.listMetrics();// returns string[] of available metric namesconstregions=awaitclient.siren.listRegions();constdata=awaitclient.siren.getMetricData({regionId: regions[0].id,metric: metrics[0],startDate: "2025-01-01",endDate: "2025-01-31",});

Siren lives under the client.siren namespace, kept separate from the core dataset API. Accessing it without configuring siren throws SirenNotConfiguredError. You can also use the standalone SirenClient directly if you don't need the dataset client at all.

Credentials can be passed explicitly instead of via environment variables:

constclient=newDClimateClient({siren: {auth: {type: "apiKey",apiKey: "sk-...",accountId: "acc-..."},// baseUrl: "https://production-api-siren.dclimate.net/api", // override if needed},});

Automatic variant concatenation

For datasets with multiple variants (e.g., ERA5 with "finalized" and "non-finalized" data), the client automatically merges them into a complete time series when no specific variant is requested:

// Automatically loads and concatenates finalized + non-finalized variantsconst[dataset,metadata]=awaitclient.loadDataset({request: {organization: "ecmwf",collection: "era5",dataset: "temperature_2m"// No variant specified - triggers auto-concatenation if possible}});// Returns a dataset with:// - Finalized data (high-quality, historical)// - Non-finalized data (recent, after finalized ends)// - No duplicate timestampsconsole.log(metadata.concatenatedVariants);// Output: ["finalized", "non-finalized"]console.log(dataset.info.concatenatedVariants);// Output: ["finalized", "non-finalized"]

The concatenation:

  • Prioritizes finalized data – Higher quality historical data comes first
  • Avoids duplicates – Non-finalized data is automatically sliced to start after the last finalized timestamp
  • Works with any variants – Supports finalized/non-finalized, analysis/reanalysis, or any custom variant combinations
  • Configurable – Controlled by concatPriority in the dataset catalog

To load a specific variant without concatenation:

// Load only finalized dataconst[finalized,metadata]=awaitclient.loadDataset({request: {organization: "ecmwf",collection: "era5",dataset: "temperature_2m",variant: "finalized"// Specific variant - no concatenation}});

ERA5 land datasets

ERA5 and ERA5-Land datasets are separate dataset IDs within the ECMWF ERA5 collection. Use listAvailableDatasets() to inspect the exact names before loading.

// Non-land ERA5 total precipitationconst[precipitation,precipitationMetadata]=awaitclient.loadDataset({request: {organization: "ecmwf",collection: "era5",dataset: "precipitation_total",variant: "finalized"}});// ERA5-Land total precipitationconst[landPrecipitation,landPrecipitationMetadata]=awaitclient.loadDataset({request: {organization: "ecmwf",collection: "era5",dataset: "precipitation_total_land",variant: "finalized"}});// ERA5-Land wind datasets follow the same pattern:// dataset: "wind_u_10m_land" or dataset: "wind_v_10m_land"

Selecting while loading

const[subset,metadata]=awaitclient.selectDataset({request: {organization: "ecmwf",collection: "era5",dataset: "temperature_2m",variant: "finalized"},selection: {point: {latitude: 40.75,longitude: -73.99},timeRange: {start: newDate("2023-02-01T00:00:00Z"),end: newDate("2023-02-05T00:00:00Z"),},}});

Use bounds for rectangular lon/lat selections. Tuple bounds are [west, south, east, north].

const[westernEurope,metadata]=awaitclient.selectDataset({request: {organization: "ecmwf",collection: "era5",dataset: "temperature_2m",variant: "finalized"},selection: {bounds: [-12,35,16,60],timeRange: {start: "2024-01-01T00:00:00Z",end: "2024-01-07T23:00:00Z",},}});

Geographic shape selections

The client supports advanced geographic selections beyond single points:

Multiple points

// Select data at multiple specific coordinatesconstpointsData=awaitdataset.points([40.75,41.0,42.5],// latitudes[-73.99,-74.5,-75.0],// longitudes{epsgCrs: 4326,snapToGrid: true,tolerance: 0.1});

Circle selection

// Select all data within a circular regionconstcircleData=awaitdataset.circle(40.75,// center latitude-73.99,// center longitude50,// radius in kilometers{latitudeKey: "latitude",longitudeKey: "longitude"});

Rectangle selection

// Select all data within a rectangular bounding boxconstrectangleData=awaitdataset.rectangle(40.0,// min latitude (south)-75.0,// min longitude (west)41.0,// max latitude (north)-73.0,// max longitude (east){latitudeKey: "latitude",longitudeKey: "longitude"});

Discovering available datasets

constcatalog=client.listAvailableDatasets();catalog.forEach(({ collection, datasets })=>{console.log(collection);datasets.forEach(({ dataset, variants })=>{variants.forEach(({ variant, cid, url })=>{console.log(` ${dataset} (${variant})`);if(cid)console.log(` CID: ${cid}`);if(url)console.log(` URL: ${url}`);});});});

Resolving a CID directly

Use the public STAC resolver when you need the selected CID and variant without loading the dataset. The API is natively asynchronous and uses the platform's pooled fetch implementation.

import{resolveCidFromStacServer}from"@dclimate/dclimate-client-js";constresolved=awaitresolveCidFromStacServer("ecmwf_aifs","temperature_forecast","single");console.log(resolved.cid,resolved.variant);

Configuration

Client options

constclient=newDClimateClient({gatewayUrl: "https://custom-ipfs-gateway.com"// Optional, defaults to public gateway});

Dataset loading options

constdataset=awaitclient.loadDataset({request: {collection: "aifs",dataset: "temperature",variant: "ensemble"},options: {gatewayUrl: "https://custom-gateway.com",// Optional: override client gatewaycid: "bafyr4ia...",// Optional: load directly from CIDzarrGroup: "0",// Optional: open a specific Zarr groupshardReadMode: "sparse",// Optional: decode only requested shard entriesreturnJaxrayDataset: false,// Optional: return raw jaxray DatasetautoConcatenate: true// Optional: auto-merge variants (default: false)}});
  • Dataset catalog – includes both HTTP-backed dataset endpoints and direct CID entries. Use listAvailableDatasets() to explore all available datasets.
  • Gateway – set gatewayUrl on the client constructor or per-call in loadDataset options.
  • Direct CID access – supply cid in options to skip catalog resolution and load directly from IPFS.
  • Grouped Zarr stores – set zarrGroup when loading grouped sharded Zarr v2 stores such as pyramid level "0".
  • Sparse shard decoding – read-only loads use shardReadMode: "sparse" by default to decode only requested shard entries. Set shardReadMode: "full" for dense reads that should reuse a fully decoded shard cache.

OpenTelemetry

The client emits OpenTelemetry API spans and metrics around IPFS/Zarr dataset opens. This is passive by default: no telemetry is exported unless the application configures an OpenTelemetry SDK/provider.

Emitted names include:

  • Span dclimate_client.ipfs.load_zarr_dataset
  • Span dclimate_client.ipfs.open_jaxray_store
  • Counter dclimate_client.ipfs.dataset_open.requests
  • Histogram dclimate_client.ipfs.dataset_open.duration
  • Counter dclimate_client.ipfs.store_open.requests
  • Histogram dclimate_client.ipfs.store_open.duration

Metric attributes include the gateway URL, store type, and status. The dataset CID is only attached to the trace span to avoid high-cardinality metric labels.

API Reference

DClimateClient

  • constructor(options?: ClientOptions) - Create a new client instance
  • loadDataset({ request, options }) - Load a dataset from the catalog
  • selectDataset({ request, selection, options }) - Load and apply selections in one call
  • listAvailableDatasets() - Get the full dataset catalog
  • siren - Namespaced Siren REST API client (getter; throws SirenNotConfiguredError unless siren is configured)

STAC utilities

  • resolveCidFromStacServer(collection, dataset, variant?, serverUrl?) - Resolve a CID and selected variant without loading the dataset
  • resolveDatasetCidFromStacServer(collection, dataset, variant?, serverUrl?) - Resolve only the CID string
  • listAvailableDatasetsFromStacServer(serverUrl?) - List collections, datasets, and variants directly from the paginated STAC API

SirenClient (via client.siren or standalone)

  • getMetricData(query) - Fetch metric data for a region over a date range
  • listRegions() - List available regions (auto-paginates)
  • listMetrics() - List available metric names

GeoTemporalDataset

  • point(latitude, longitude, options?) - Select nearest point
  • points(latitudes, longitudes, options?) - Select multiple points
  • circle(centerLat, centerLon, radiusKm, options?) - Select circular region
  • rectangle(minLat, minLon, maxLat, maxLon, options?) - Select rectangular region
  • timeRange(range, dimension?) - Filter by time range
  • select(options) - Apply combined point and time selections
  • toRecords(varName, options?) - Convert to array of records
  • getVariable(name) - Access a specific variable
  • variables - List all data variables
  • coords - Access coordinate arrays
  • info - Get dataset metadata

Roadmap

  • Aggregation helpers (spatial and temporal statistics)
  • S3 storage backend support
  • Advanced caching controls and persistent catalog storage
  • Additional coordinate reference system (CRS) transformations
  • Expanded test coverage and integration fixtures

About

A javascript client to interact with the dclimate ecosystem

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages