diff --git a/.github/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..8a8a7e75 --- /dev/null +++ b/.github/CODE_OF_CONDUCT.md @@ -0,0 +1,41 @@ +# Code of Conduct + +OpenChainBench publishes performance numbers about real services run by real people. The conversations around those numbers can get heated — especially when a provider's reputation is tied to the data. This document is short and direct so the bar is clear. + +## Expected behavior + +- **Critique numbers, not people.** It's fair to question a methodology, a sample window, a regional bias. It's not fair to suggest someone is incompetent or dishonest. +- **Cite, don't assert.** If you think a published figure is wrong, file a [provider correction](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=provider-correction.yml) with a reproducer. "This number is bullshit" without a reproducer is a non-issue. +- **Disclose conflicts.** If you work for a provider that's measured here, say so when you participate in discussions about that provider. It doesn't disqualify your input — it just keeps the conversation honest. +- **Respect contributors' time.** Maintainers are not obligated to debate. If a thread is going in circles, a maintainer may close it with a dated decision and that decision stands until new evidence is filed. + +## Unacceptable behavior + +- Harassment, personal attacks, or sustained disruption of discussion. +- Posting private communications without consent. +- Deliberately misleading benchmarks or harness submissions designed to flatter a provider you're affiliated with. +- Targeting individual contributors based on their employer, nationality, or any protected characteristic. + +## Enforcement + +A maintainer may, at their discretion: + +- Edit, lock, or delete issues, PRs, comments, and discussions that violate this code. +- Block users from the repository. +- Revert merged PRs that contain manipulated data or undisclosed conflicts of interest. + +Decisions are recorded in the affected thread. + +## Reporting + +For private reports — including suspected manipulation, harassment, or undisclosed conflicts of interest — email **conduct@openchainbench.xyz** or open a [private security advisory](https://github.com/OpenChainBench/OpenChainBench/security/advisories/new) (the same channel works for non-security disclosures). + +We treat reporters' identities as confidential by default. + +## Scope + +This Code of Conduct applies to all OpenChainBench spaces — issues, PRs, discussions, the Discord/Telegram if those exist, and any in-person event held under the project's name. + +--- + +Loosely adapted from the [Contributor Covenant 2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct/), trimmed for a small, technical project. diff --git a/.github/DISCUSSIONS_AND_PROJECT_SETUP.md b/.github/DISCUSSIONS_AND_PROJECT_SETUP.md new file mode 100644 index 00000000..a4a1370b --- /dev/null +++ b/.github/DISCUSSIONS_AND_PROJECT_SETUP.md @@ -0,0 +1,77 @@ +# Discussions & Project board — manual setup + +This file documents the GitHub UI configuration that lives outside the repo (Discussions categories, the Project board). It exists so a maintainer can recreate the setup if it gets lost or migrated. + +## Discussions + +**Setup path:** Repo → Discussions tab → ⚙️ "Manage categories" + +| Category | Type | Description | +| --- | --- | --- | +| 📣 Announcements | Announcement (maintainers only) | New benchmarks shipped, methodology changes, infrastructure migrations. | +| 💡 Ideas | Open-ended discussion | Brainstorm new benchmarks, providers, or methodology improvements before writing them up. | +| 🙋 Q&A | Question / Answer | Ask how a spec / harness / metric works. Pick a best answer when one is given. | +| 📊 Show & tell | Open-ended discussion | Share a fork, a derived dashboard, a benchmark you ran locally. | + +Default categories like "General" and "Polls" should be deleted to keep the surface focused. + +## Roadmap Project + +**Setup path:** Org → Projects tab → New project → "From scratch" → Board template + +- **Name**: `OpenChainBench Roadmap` +- **Visibility**: Public +- **Linked repository**: `OpenChainBench/OpenChainBench` + +### Columns + +``` +Requested → Approved → In progress → Live → Archived +``` + +| Column | Meaning | +| --- | --- | +| Requested | A bench has been proposed (issue with `bench-request` label or a Discussion thread). Methodology not yet validated by maintainers. | +| Approved | Methodology agreed on. Spec / harness work can start. | +| In progress | A PR is open implementing the benchmark. | +| Live | Merged + harness wired into Railway + emitting data. | +| Archived | Deprecated, retired, or rejected with a dated note. | + +### Workflows + +In Project Settings → Workflows, enable: + +- **Auto-add to project** — for issues in `OpenChainBench/OpenChainBench` matching the filter `is:issue label:bench-request`. They land in `Requested` automatically. +- **Item closed → Archived** — when an issue is closed without a `live` label, move it to `Archived`. +- **PR merged → Live** (manual; GitHub doesn't have a native trigger, so a maintainer moves the card after merge). + +### Custom fields (optional) + +- `Category` (single-select): RPC / Aggregator / Bridge / Price feed / Wallet / Other. +- `Priority` (single-select): P0 / P1 / P2. +- `Owner` (text or person reference). + +## Labels + +These labels back the issue templates and the project workflow. Create them under Repo → Issues → Labels: + +| Label | Color | Description | +| --- | --- | --- | +| `bench-request` | `#0e8a16` (green) | Used by `new-benchmark.yml` template; auto-adds to the Project board. | +| `data-quality` | `#fbca04` (yellow) | Used by `data-quality.yml`; site / data anomaly. | +| `correction` | `#d93f0b` (orange) | Provider-submitted correction. | +| `provider` | `#5319e7` (purple) | Combined with `correction` to mark provider-affiliated issues. | +| `live` | `#0366d6` (blue) | Bench is published and emitting data; survives an issue's lifetime. | +| `harness-bug` | `#b60205` (red) | A harness has stopped emitting or is reporting wrong values. | +| `methodology` | `#5319e7` (purple) | Discussion about how a benchmark is computed. | +| `good first issue` | `#7057ff` (purple) | Curated easy-entry tasks for new contributors. | +| `help wanted` | `#008672` (teal) | Maintainers explicitly want outside contribution. | + +## Re-running this setup + +If the Project or Discussions are wiped: + +1. Re-create the Discussions categories per the table above. +2. Re-create the Project with the columns + workflows. +3. Add the labels listed in the Labels section. +4. Push a tiny change to this file documenting the date of the rebuild so the next maintainer knows when the configuration was last verified. diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 00000000..00392232 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,14 @@ +blank_issues_enabled: false +contact_links: + - name: 💡 Idea or rough proposal + url: https://github.com/OpenChainBench/OpenChainBench/discussions/new?category=ideas + about: Have an idea but not ready for a formal benchmark proposal? Start a discussion in Ideas. + - name: 🙋 Question (How do I…?) + url: https://github.com/OpenChainBench/OpenChainBench/discussions/new?category=q-a + about: Ask a question about a spec, harness, methodology, or how the site works. + - name: 📊 Show & tell + url: https://github.com/OpenChainBench/OpenChainBench/discussions/new?category=show-and-tell + about: Show a fork, a dashboard, or a benchmark you ran locally. + - name: 🔐 Security disclosure + url: https://github.com/OpenChainBench/OpenChainBench/security/advisories/new + about: Privately report a vulnerability. Do not file a public issue for security problems. diff --git a/.github/ISSUE_TEMPLATE/correction.md b/.github/ISSUE_TEMPLATE/correction.md deleted file mode 100644 index 1ff34e3a..00000000 --- a/.github/ISSUE_TEMPLATE/correction.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -name: Number you can't reproduce -about: Report a correction to a published benchmark -title: "Correction: · " -labels: ["correction"] ---- - -### Which report - -Bench № — `` - -### Which provider / number - - - -### What you measured - - - -### What we published - - - -### Reproducer - -```bash -# how you ran your harness -``` - - diff --git a/.github/ISSUE_TEMPLATE/data-quality.yml b/.github/ISSUE_TEMPLATE/data-quality.yml new file mode 100644 index 00000000..cb362922 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/data-quality.yml @@ -0,0 +1,62 @@ +name: 🐞 Data quality issue +description: Report a number on the site that looks wrong, broken, or stale. For provider-specific corrections (you measured a different number for your own service), use the "Provider correction" template instead. +title: "Data: · " +labels: ["data-quality"] +body: + - type: markdown + attributes: + value: | + Thanks for flagging this. Be specific so we can investigate quickly. + - type: dropdown + id: bench + attributes: + label: Which benchmark + options: + - "№ 001 · aggregator-head-lag" + - "№ 002 · bridge-quote-latency" + - "№ 003 · bridge-fee" + - "Other (specify in description)" + validations: + required: true + - type: dropdown + id: kind + attributes: + label: What kind of issue + options: + - "Number looks wrong (out of plausible range)" + - "Provider missing from the page" + - "Sparkline / chart not updating" + - "Last-run timestamp is stale" + - "Region label looks incorrect" + - "Page rendering / formatting bug" + - "Other" + validations: + required: true + - type: textarea + id: observed + attributes: + label: What you see + description: Paste the number, screenshot, or describe the symptom. Include the URL of the report. + validations: + required: true + - type: textarea + id: expected + attributes: + label: What you expected + description: Why does this look wrong to you? A reference number, a sanity check, an external dashboard. + validations: + required: false + - type: input + id: when + attributes: + label: Approximate time observed (UTC) + placeholder: e.g. "2026-04-30 14:20 UTC" + validations: + required: false + - type: textarea + id: anything + attributes: + label: Anything else + description: Context that might help narrow it down — region you're checking from, browser, recent harness changes, related issues. + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/new-benchmark.md b/.github/ISSUE_TEMPLATE/new-benchmark.md deleted file mode 100644 index b9deb5b3..00000000 --- a/.github/ISSUE_TEMPLATE/new-benchmark.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -name: New benchmark -about: Propose a new benchmark to add to the journal -title: "Bench: " -labels: ["new-benchmark"] ---- - -### What should be measured - - - -### Why it matters - - - -### Proposed methodology - -- Inputs (workload, sample, region): -- Cadence: -- Timeout: -- Success criteria: - -### Providers in scope - - - -### Where will the data come from? - -- Harness language / runtime: -- Prometheus instance (if any) : -- Existing metric names (if any) : - -### Anything else diff --git a/.github/ISSUE_TEMPLATE/new-benchmark.yml b/.github/ISSUE_TEMPLATE/new-benchmark.yml new file mode 100644 index 00000000..609bf46a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/new-benchmark.yml @@ -0,0 +1,101 @@ +name: 📊 Propose a benchmark +description: Suggest a new benchmark for OpenChainBench. Sketch the metric, the providers, the methodology — the goal is to align before code is written. +title: "Bench: " +labels: ["bench-request"] +body: + - type: markdown + attributes: + value: | + Thanks for proposing a benchmark. + + OpenChainBench prefers to align on **what** is measured and **how** before any code is written. Fill out the sections below — fields marked required are the bare minimum needed for the maintainers to give meaningful feedback. If you're not sure on some of them, write what you have and leave a note. + + Have a rough idea but not ready to fill all this? Open a thread in [Discussions → Ideas](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/ideas) instead. + - type: input + id: metric + attributes: + label: What should be measured (one sentence) + description: The metric and the providers. Be specific. + placeholder: e.g. "Time-to-finality of an Ethereum block, measured by RPC providers across 3 regions." + validations: + required: true + - type: textarea + id: motivation + attributes: + label: Why it matters + description: Who cares about this number, and what decision does it inform? + placeholder: e.g. "Wallet UX teams choose RPCs based on confirmation speed; today no one publishes a like-for-like number." + validations: + required: true + - type: textarea + id: providers + attributes: + label: Providers in scope + description: One per line. Mark "(?)" next to any you want feedback on including. + placeholder: | + Alchemy + QuickNode + Infura + Ankr + Public RPCs (?) + validations: + required: true + - type: textarea + id: methodology + attributes: + label: Proposed methodology + description: How will the harness run? Cadence, timeout, success criteria, regions. + value: | + - Workload / inputs: + - Cadence: + - Timeout: + - Regions: + - What counts as success: + - What's excluded from latency aggregates: + validations: + required: true + - type: dropdown + id: hosting + attributes: + label: Where will the harness run? + description: Light = single HTTP loop, no wallets, no signing. Heavy = wallets, on-chain transactions, capital-at-risk. + options: + - "Light — happy to have OpenChainBench Railway host it" + - "Heavy — I'll host on my own infra and push to a public Prom endpoint" + - "Not sure yet" + validations: + required: true + - type: input + id: language + attributes: + label: Harness language / runtime (planned) + placeholder: e.g. Go 1.24, Bun, Python 3.13, Rust + validations: + required: false + - type: textarea + id: existing + attributes: + label: Existing data sources + description: Are you already running a similar harness? Got Prometheus metrics ready? Link/paste anything relevant. + placeholder: | + - Existing harness repo (private or public): + - Existing Prometheus instance: + - Existing metric names: + validations: + required: false + - type: textarea + id: anything + attributes: + label: Anything else + description: Caveats, prior art, related benchmarks, conflicts of interest. + validations: + required: false + - type: checkboxes + id: terms + attributes: + label: Confirm + options: + - label: I understand that the harness code, methodology, and any corrections will be published under the project's MIT/CC-BY-4.0 license. + required: true + - label: I have searched [existing benchmarks](https://github.com/OpenChainBench/OpenChainBench/tree/main/benchmarks) and [open issues](https://github.com/OpenChainBench/OpenChainBench/issues?q=is%3Aissue+label%3Abench-request) for duplicates. + required: true diff --git a/.github/ISSUE_TEMPLATE/provider-correction.yml b/.github/ISSUE_TEMPLATE/provider-correction.yml new file mode 100644 index 00000000..840a7b04 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/provider-correction.yml @@ -0,0 +1,75 @@ +name: ✏️ Provider correction +description: Submitted by a provider (or someone running their stack) who measured a different number than what the site publishes. +title: "Correction: · " +labels: ["correction", "provider"] +body: + - type: markdown + attributes: + value: | + OpenChainBench treats every provider equally and corrections in the open. If your numbers disagree with what's published, open this issue — material corrections result in a dated note on the affected report. + + Please be specific and reproducible. We will not update a number based on screenshots without a method we can re-run. + - type: dropdown + id: bench + attributes: + label: Which benchmark + options: + - "№ 001 · aggregator-head-lag" + - "№ 002 · bridge-quote-latency" + - "№ 003 · bridge-fee" + - "Other (specify in description)" + validations: + required: true + - type: input + id: provider + attributes: + label: Provider (your service) + placeholder: e.g. Alchemy + validations: + required: true + - type: input + id: relationship + attributes: + label: Your relationship with this provider + description: For transparency. e.g. "Founding engineer", "Head of DevRel", "External user". + placeholder: e.g. "I'm on the platform team at " + validations: + required: true + - type: textarea + id: published + attributes: + label: What we published + description: The figure on the site, with link to the report. + validations: + required: true + - type: textarea + id: measured + attributes: + label: What you measured + description: Your aggregate (p50/p99/etc), your window, your region, your sample size. + validations: + required: true + - type: textarea + id: reproducer + attributes: + label: Reproducer + description: How can we reproduce your number? A Prometheus query, a script, a notebook, an exact command. We will run it before correcting. + render: shell + validations: + required: true + - type: textarea + id: hypothesis + attributes: + label: Why you think the published number is off + description: Different region? Stale data? A bug in our harness? Different methodology? + validations: + required: false + - type: checkboxes + id: terms + attributes: + label: Confirm + options: + - label: I understand that any correction will be published openly with a dated note on the report. + required: true + - label: My reproducer is genuinely runnable by maintainers without provider-specific credentials we don't already have, OR I will provide credentials privately if needed. + required: true diff --git a/.github/SECURITY.md b/.github/SECURITY.md new file mode 100644 index 00000000..b0f8bfb6 --- /dev/null +++ b/.github/SECURITY.md @@ -0,0 +1,45 @@ +# Security policy + +## Reporting a vulnerability + +Open a [private security advisory](https://github.com/OpenChainBench/OpenChainBench/security/advisories/new) on this repository. Do **not** open a public issue for a security report — the advisory mechanism keeps the disclosure private until a fix is in place. + +If GitHub is not an option, email **security@openchainbench.xyz** with: + +- A description of the vulnerability and the affected component (site, harness, infrastructure). +- A reproducer or proof of concept. +- Your contact details for follow-up (email is fine). + +We aim to acknowledge reports within 72 hours and provide a remediation timeline within 7 days. + +## Scope + +In-scope: + +- The Next.js site (`src/`). +- The published harnesses (`harnesses/`). +- The Prometheus / Grafana / Alertmanager configurations under `harnesses//deploy/`. +- The benchmark specs (`benchmarks/*.yml`) — particularly any spec that could exfiltrate secrets through a malformed Prometheus URL. + +Out-of-scope: + +- Provider-side vulnerabilities (we benchmark these services; we don't operate them). Report those to the provider directly. +- Issues in third-party dependencies that have already been disclosed upstream. +- Self-XSS, missing security headers without an exploit chain, and similar low-severity findings unless tied to a concrete impact. + +## What gets a CVE + +We assign CVEs (via GitHub) for: + +- Remote code execution in the harnesses or the site. +- Authentication / authorization bypass on any deployed endpoint. +- Secret leakage from the published codebase or CI logs. +- Tampering vectors that would let an attacker influence published numbers. + +## What's already disclosed + +If you find a hardcoded secret in the git history (e.g. an old API key, a wallet token), please open a private advisory anyway — the secret should be rotated even if the value has been replaced in HEAD. We monitor `git log -p` for these but report them in any case. + +## Preferred languages + +English or French. diff --git a/.github/SUPPORT.md b/.github/SUPPORT.md new file mode 100644 index 00000000..45c43552 --- /dev/null +++ b/.github/SUPPORT.md @@ -0,0 +1,34 @@ +# Getting help + +OpenChainBench is community-run. Before opening an issue, pick the right channel — it gets you faster answers and keeps the issue tracker focused on actionable work. + +## I want to… + +| Goal | Where to go | +| --- | --- | +| Propose a new benchmark formally | [New issue → 📊 Propose a benchmark](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=new-benchmark.yml) | +| Float an idea before writing it up | [Discussions → Ideas](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/ideas) | +| Ask a question about a spec, harness, methodology, or the site | [Discussions → Q&A](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/q-a) | +| Show a fork or a dashboard you built | [Discussions → Show & tell](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/show-and-tell) | +| Report a number that looks wrong on the site | [New issue → 🐞 Data quality](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=data-quality.yml) | +| Submit a provider correction (you measured a different number) | [New issue → ✏️ Provider correction](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=provider-correction.yml) | +| Privately report a security vulnerability | [Security advisories](https://github.com/OpenChainBench/OpenChainBench/security/advisories/new) | +| See what's planned and where to contribute code | [Roadmap project board](https://github.com/orgs/OpenChainBench/projects) | +| Read about how the data is produced | [`benchmarks/README.md`](../benchmarks/README.md) and [`harnesses/README.md`](../harnesses/README.md) | + +## Response time + +- **Discussions**: best-effort, usually within a few days. +- **Issues**: triaged once a week. Critical (data clearly wrong / site down) is faster. +- **Pull requests**: reviewed within ~5 days for new benchmarks, ~2 days for fixes. +- **Security advisories**: acknowledged within 72 hours. + +## What we won't help with + +- Provider-specific support (e.g. "my Alchemy quota is exhausted"). Contact the provider. +- Generic Prometheus / Grafana questions unrelated to OpenChainBench. +- Asking us to add your private node as a benchmarked provider. We benchmark public, generally-available infrastructure. + +## Commercial support + +There is no commercial support offering. The maintainers contribute on a best-effort basis. If your team depends on continuous, SLA-backed visibility into a particular metric, run your own harness — the contract documented in [`harnesses/README.md`](../harnesses/README.md) is small and the existing harnesses are MIT-licensed reference implementations. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 83e796d9..794f5313 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -13,12 +13,24 @@ docs/ Methodology, ADRs, style guide .github/ CI, issue templates, PR template ``` +## Where to start + +| Goal | Right channel | +| --- | --- | +| Float a rough idea | [Discussions → Ideas](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/ideas) | +| Ask a methodology / harness question | [Discussions → Q&A](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/q-a) | +| Propose a benchmark formally | [Issue → 📊 Propose a benchmark](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=new-benchmark.yml) | +| Report a number that looks wrong | [Issue → 🐞 Data quality](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=data-quality.yml) | +| Submit a provider correction | [Issue → ✏️ Provider correction](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=provider-correction.yml) | +| See what's planned | [Roadmap project board](https://github.com/orgs/OpenChainBench/projects) | + ## Submitting a benchmark -1. **Open an issue** with the [new-benchmark template](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=new-benchmark.md). Sketch the metric, providers, methodology — gets you feedback before you write code. -2. **Write the spec**. Drop a YAML at `benchmarks/.yml`. The format is described in [`benchmarks/README.md`](./benchmarks/README.md) and validated by `src/lib/spec-schema.ts`. -3. **Build the harness**. Anything that pushes Prometheus metrics with the labels your spec references. See [`harnesses/README.md`](./harnesses/README.md) for the contract. -4. **Open a PR**. CI runs `pnpm validate` (schema lint), `pnpm typecheck`, `pnpm lint`, and `pnpm build`. Once green and reviewed, merge → ISR pulls live numbers within 60 seconds. +1. **Open an issue** with the [📊 Propose a benchmark template](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=new-benchmark.yml). Sketch the metric, providers, methodology, hosting plan — get feedback before you write code. The issue lands in the `Requested` column of the roadmap. +2. **Write the spec.** Drop a YAML at `benchmarks/.yml`. The format is described in [`benchmarks/README.md`](./benchmarks/README.md) and validated by `src/lib/spec-schema.ts`. +3. **Build the harness** at `harnesses//`. Anything that pushes Prometheus metrics with the labels your spec references. See [`harnesses/README.md`](./harnesses/README.md) for the contract and the existing Go harnesses as reference implementations. +4. **Open a PR** referencing the issue (`Closes #N`). CI runs `pnpm validate` (schema lint), `pnpm typecheck`, `pnpm lint`, and `pnpm build`. Once green and reviewed, merge → site picks up the spec on next ISR cycle (≤60 s). +5. **Wire the harness on Railway** (maintainer task). Light harnesses run on the project's shared Railway. Heavier harnesses (wallets, signing) are hosted by the contributor and push to a publicly-reachable Prom endpoint. ## Local development @@ -40,7 +52,12 @@ pnpm build # production build ## Corrections -If you can't reproduce a number, file a [correction issue](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=correction.md). Material errors are corrected in place with a dated note on the report. +If you can't reproduce a number, file a [provider correction](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=provider-correction.yml) (you measured a different value for your service) or a [data quality issue](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=data-quality.yml) (the site is showing something obviously wrong or stale). Material errors are corrected in place with a dated note on the report. + +## Code of conduct & security + +- [Code of conduct](./.github/CODE_OF_CONDUCT.md) — short, applies to all project spaces (issues, PRs, discussions). +- [Security policy](./.github/SECURITY.md) — private vulnerability reporting via GitHub advisories. ## License diff --git a/README.md b/README.md index cb7eabed..d4f57788 100644 --- a/README.md +++ b/README.md @@ -94,7 +94,7 @@ Set `prom_url` in the corresponding YAML to your local Prom (`http://localhost:9 Full guide in [CONTRIBUTING.md](./CONTRIBUTING.md). Short version: -1. **Open an issue** with the [new-benchmark template](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=new-benchmark.md). Sketch the metric, providers, methodology — get feedback before you build. +1. **Open an issue** with the [📊 Propose a benchmark template](https://github.com/OpenChainBench/OpenChainBench/issues/new?template=new-benchmark.yml). Sketch the metric, providers, methodology — get feedback before you build. Want to brainstorm first? Use [Discussions → Ideas](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/ideas) instead. 2. **Write the spec** at `benchmarks/.yml`. Format documented in [`benchmarks/README.md`](./benchmarks/README.md), validated by `src/lib/spec-schema.ts`. 3. **Build the harness** in `harnesses//`. Any language works as long as it pushes Prometheus metrics with the labels your spec references. See the existing harnesses as reference. 4. **Open a PR.** CI runs schema validation, typecheck, lint, and build. Once green and merged: the site picks up the new spec automatically; a maintainer wires the harness into Railway (one-time setup per benchmark). @@ -118,6 +118,15 @@ Hosting trade-off: light harnesses (one HTTP poll loop, no secrets) can be deplo - Prometheus HTTP API (instant + range queries) - Go 1.24 for the existing harnesses (any language is acceptable) +## Community + +- 💡 [Discussions → Ideas](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/ideas) — brainstorm new benchmarks before writing them up +- 🙋 [Discussions → Q&A](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/q-a) — methodology / harness / spec questions +- 📊 [Discussions → Show & tell](https://github.com/OpenChainBench/OpenChainBench/discussions/categories/show-and-tell) — share forks and dashboards +- 🗺️ [Roadmap](https://github.com/orgs/OpenChainBench/projects) — what's planned and what's live +- 🐞 [New issue](https://github.com/OpenChainBench/OpenChainBench/issues/new/choose) — formal benchmark proposal, data-quality flag, or provider correction +- See [SUPPORT.md](./.github/SUPPORT.md) for the full triage matrix. + ## Links - Site — [openchainbench.xyz](https://openchainbench.xyz)