Extract

Extract turns Mercury Parser into an authenticated web service that can run on a VM without platform-specific dependencies.

How it works

One Express application authenticates each request and uses Mercury Parser to fetch and extract the requested page:

app/app.js HTTP routes, authentication, and parsing
app/server.js Node/Bun entry point and graceful shutdown

Installation

Install Node.js 26

git clone https://github.com/feedbin/extract.git
cd extract
npm ci

Run the server in development:

PORT=8889 node app/server.js

Run both compatibility suites with:

npm test

Configuration

Users are defined in a YAML mapping where each key is a username and each value is that user's non-empty secret:

username: secret

Set EXTRACT_USERS to the file path before starting the server:

EXTRACT_USERS=users.yml PORT=8889 node app/server.js

The file is read once at boot, so changes require a restart. Development falls back to a demo user with secret demo when EXTRACT_USERS is unset. Production requires EXTRACT_USERS and refuses to start without it.

API

The service exposes:

GET /health_check
GET /parser/:username/:signature?base64_url=:base64_url

signature is the hexadecimal HMAC-SHA1 of the decoded URL using the user's secret. base64_url is canonical RFC 4648 URL-safe Base64, with optional valid padding and no whitespace.

This Node example constructs a request URL:

constcrypto=require("node:crypto")constusername="username"constsecret="secret"consturl="https://feedbin.com/blog/2018/09/11/private-by-default/"constsignature=crypto.createHmac("sha1",secret).update(url).digest("hex")constencodedUrl=Buffer.from(url).toString("base64url")constrequestUrl=newURL(`/parser/${username}/${signature}`,"http://localhost:8889")requestUrl.searchParams.set("base64_url",encodedUrl)console.log(requestUrl.toString())