TypesMem

TypesMem is a TypeScript translation memory engine backed by SQLite. It stores translation units (TUs) with full TMX 1.4 fidelity, imports and exports TMX files, and provides fuzzy and concordance search over the stored content.

Why TypesMem

• No native bindings. Storage is built on node:sqlite, which ships with Node.js itself — no native module to compile, no ABI mismatches under Electron or across platforms.
• CJK-friendly fuzzy matching. Matching uses a custom 3-character sliding-window (trigram) index instead of SQLite's FTS5, since FTS5's tokenizers don't segment Chinese, Japanese, or Korean text well. The index is language-scoped, so each language's trigrams are matched independently.
• Synchronous API. Every operation is backed by node:sqlite's synchronous DatabaseSync, so there's no Promise/async ceremony for what are, in practice, sub-millisecond local calls.
• Full attribute fidelity. Every optional attribute TMX 1.4 allows on <tu> and <tuv> (creationdate, creationid, changedate, changeid, usagecount, o-encoding, o-tmf, and the rest) round-trips exactly — nothing is fabricated on import, and nothing is silently dropped on export.
• Internationalized. Error messages are localized (English and Spanish included) via a small resource-bundle-based I18n class.

Installation

npm install typesmem

Quick Start

Import a TMX file, search it, export it back out

import{TranslationMemory}from"typesmem";consttm=newTranslationMemory("myMemory","/path/to/memories");constcount=tm.storeTMX("/path/to/file.tmx",undefined,(imported)=>{console.log("Imported "+imported+" translation units so far");});console.log("Import finished: "+count+" translation units");constmatches=tm.searchTranslation("Hello world","en-US","es-ES",70,10);for(constmatchofmatches){console.log(match.similarity+"% - "+match.target.toString());}tm.exportMemory("/path/to/export.tmx",["en-US","es-ES"],"en-US");tm.close();

Store and retrieve a single translation unit

import{TranslationMemory}from"typesmem";import{XMLAttribute,XMLElement}from"typesxml";consttm=newTranslationMemory("myMemory","/path/to/memories");consttu=newXMLElement("tu");constsourceTuv=newXMLElement("tuv");sourceTuv.setAttribute(newXMLAttribute("xml:lang","en-US"));constsourceSeg=newXMLElement("seg");sourceSeg.addString("Hello world");sourceTuv.addElement(sourceSeg);tu.addElement(sourceTuv);consttargetTuv=newXMLElement("tuv");targetTuv.setAttribute(newXMLAttribute("xml:lang","es-ES"));consttargetSeg=newXMLElement("seg");targetSeg.addString("Hola mundo");targetTuv.addElement(targetSeg);tu.addElement(targetTuv);consttuid=tm.storeTu(tu,{project: "demo"});conststored=tm.getTu(tuid);console.log(stored?.toString());tm.removeTu(tuid);tm.close();

API Overview

Lifecycle

Method	Description
new TranslationMemory(name, workFolder, lang?)	Opens (or creates) a memory named name under workFolder. lang selects the language for TypesMem's own error messages ("en" or "es", default "en") — it has nothing to do with the languages stored in the memory.
getName() / getType()	Returns the memory's name, and "Local" as its engine type.
close()	Closes the underlying database connection.
deleteDatabase()	Closes the connection and deletes the memory's folder. Don't call close() first — this already closes it.

Storage

Method	Description
storeTu(tu, metadata?)	Stores a <tu> element (a typesxmlXMLElement), returning its tuid. If the element already has a tuid matching an existing entry, that entry is fully replaced (including all its <tuv>s, properties, and notes); otherwise a new entry is created, generating a tuid if the element doesn't have one. metadata supplies default <prop> values for keys not already present on the TU.
getTu(tuid)	Returns the full <tu> element for a given tuid, or undefined.
removeTu(tuid)	Deletes a translation unit and all of its associated data.

Search

Method	Description
searchTranslation(text, srcLang, tgtLang, similarity, limit?, caseSensitive?)	Fuzzy-matches text against srcLang segments (similarity is 0-100) and returns Match[] — each with source/target<tuv> elements, a similarity score, origin, and properties — for translation units that also have a tgtLang variant.
searchAll(text, srcLang, similarity, limit?, caseSensitive?)	Same fuzzy matching as searchTranslation, but only requires a srcLang match; returns the full <tu> element for each hit regardless of what other languages it has.
concordanceSearch(text, srcLang, limit?, isRegexp?, caseSensitive?)	Substring or regex search (not fuzzy) against stored srcLang text, returning full <tu> elements. When isRegexp is true, text is compiled as a regular expression and matching is always case-sensitive, regardless of caseSensitive.
getMetadataValues(field)	Returns the distinct values stored for a given metadata/property field name, across the whole memory.

TMX import/export

Method	Description
storeTMX(tmxFile, metadata?, onProgress?)	Imports a TMX file, returning the number of translation units stored. onProgress is called periodically (every 500 TUs) with the running count.
exportMemory(tmxFile, langs, srcLang)	Writes a TMX 1.4 file containing only the given langs, skipping any translation unit left with fewer than two language variants after filtering.

Design notes

A memory is a single SQLite database file (database.db) under its own folder. Fuzzy matching is powered by a per-language trigram index: while importing, ngram occurrences are written to a fast (ngram, id) table and periodically compacted into a compact packed form, so bulk imports stay fast without leaving the index bloated; a single storeTu call outside of a bulk import writes directly into the compact form, since there's no bulk-write pressure to justify the temporary structure.

License

Eclipse Public License 1.0 — see LICENSE.