Skip to content

Repository files navigation

Means

S3-Compatible Object Storage — Enterprise-Grade, Self-Deployable, .NET-Powered.

Means is a self-deployable S3-compatible object storage service built on ASP.NET Core. It uses the custom XlFs storage backend (inspired by MinIO's architecture — single-node multi-disk, object manifest, quorum-based writes, LogDb metadata indexing), powered entirely by the self-developed MeansLogDb metadata engine.

As of 2026-05-09: This document reflects the current code state of the repository, not aspirational design goals.


Table of Contents


Architecture Overview

LayerProjectPurpose
Host / APIsrc/MeansASP.NET Core net10.0 application entry point, middleware pipeline, endpoints, DI composition
Protocolsrc/Means.Protocol.S3S3-compatible address resolution, SigV4 signature validation, S3 XML response serialization
Storage Enginesrc/Means.Infrastructure.XlFsDefault XlFs storage backend (multi-disk, manifest, quorum, MeansLogDb metadata)
Core Abstractionssrc/Means.CoreDomain models, storage interfaces, policies, error definitions, placement strategies
Managementsrc/Means/Endpoints/ConsoleCookie-authenticated Console JSON API
Web UIsrc/Means/wwwrootReact-based management dashboard (built from web/)
SDK - C#SDKs/csharpFull-featured C# client SDK
SDK - TypeScriptSDKs/typescriptBrowser-safe TS SDK + Node extension (SigV4)
Teststests/Unit, integration, and contract tests

Solution Structure (Means.slnx)

Means.slnx
├── src/
│ ├── Means/ # Host application
│ ├── Means.Core/ # Core abstractions & domain
│ ├── Means.Infrastructure.XlFs/ # XlFs storage engine
│ └── Means.Protocol.S3/ # S3 protocol implementation
│
├── tests/
│ ├── Means.UnitTests/ # Unit tests
│ ├── Means.IntegrationTests/ # Integration tests
│ └── Means.ContractTests/ # SDK spec compliance tests
│
├── SDKs/
│ ├── csharp/ # C# SDK
│ ├── typescript/ # TypeScript SDK
│ └── spec/ # Machine-readable protocol spec
│
├── web/ # React frontend (Vite)
└── docs/ # Documentation site

Implemented Capabilities

S3 Data Plane (v1 Baseline)

CategoryOperationsStatus
ServiceListBuckets
BucketCreateBucket, HeadBucket, DeleteBucket
ObjectPutObject, GetObject, HeadObject, DeleteObject
ListingListObjectsV2 (prefix, delimiter, continuation-token, max-keys)
CopyCopyObject (x-amz-copy-source, COPY/REPLACE metadata directive)
MultipartInitiate, UploadPart, UploadPartCopy, Complete, Abort, ListParts, ListMultipartUploads
Versioning?versioning, ?versions, GET/HEAD/DELETE by versionId, delete markers
Tagging?tagging (current & specific version)
Lifecycle?lifecycle (expiration, noncurrent cleanup, AbortIncompleteMultipartUpload)
CORS?cors config CRUD, OPTIONS preflight
Notification?notification config persistence (reserved interface)
Policy?policy sub-resource (GET/PUT/DELETE)
Pre-signed URLSigV4 pre-signed GET, PUT, multipart UploadPart

Key Implementation Details

  • Address Resolution: Supports both path-style and virtual-hosted-style S3 addressing.
  • Response Format: Unified S3-compatible XML (listings, errors, copy results).
  • Range Reads:Range header → 206 Partial Content; invalid ranges → 416 InvalidRange with Content-Range header.
  • Compression: Content-negotiated (br / gzip) on responses; disabled for Range requests.
  • Atomic Writes:PutObject is atomic — objects become visible only after the metadata transaction commits.
  • Multipart Rules:
    • partNumber: 110000
    • Minimum part size (except last): 5 MiB
    • Final ETag: md5(concat(part-md5-bytes))-part-count
    • Means:RequestLimits:MaxUploadSizeBytes applies per-part, not total assembled size.
  • Concurrency Control: S3 PUT and multipart UploadPart share a global concurrency limit (default: 64). Exceeding returns 503 SlowDown with Retry-After: 1.
  • Checksum Verification: Disabled by default on reads to avoid double I/O for large objects. Enable via VerifyChecksumOnRead. Background scrub always checksums and enqueues repairs.

Console Management Plane

Built-in management API (/api/console, JSON) and web dashboard (React):

FeatureEndpoint / Details
AuthenticationCookie-based login/logout/session check
Bucket ManagementCreate, delete, browse objects
Bucket PolicyView and edit bucket policies
Pre-signed URLsGenerate upload/download links
Large File UploadMultipart from 5 MiB, 16 MiB parts, 3 concurrent
AccessKey ManagementCreate, delete, list access keys
System SettingsConfigure max upload size
Audit & MetricsAudit log, hourly request statistics dashboard
Cluster StatusNode/disk health, diagnostics export (/api/console/cluster, /api/console/diagnostics)
Monitoring ExportPrometheus /metrics, optional OpenTelemetry (OTLP)
Background TasksUnified management with manual triggers — heartbeat, disk health, metadata consistency, storage GC, repair, rebalance, lifecycle, replication worker
Rate LimitingFixed-window limits for Console login, Console API, and S3 data plane

Not Yet Implemented

The following features are not present in the current codebase:

  • 🔲 Replication — Cross-bucket/bucket replication rules
  • 🔲 Object Lock / Retention — WORM compliance
  • 🔲 IAM / STS — Full identity and access management model
  • 🔲 Distributed Sharding / EC — Multi-node data sharding and erasure coding across nodes

Quick Start

1) Prerequisites

DependencyVersionNotes
.NET SDK10.0Required for building and running
Node.js20+Only needed for frontend development

2) Start the Service

# Restore dependencies
dotnet restore Means.slnx
# Build the solution
dotnet build Means.slnx
# Run with the 'http' launch profile
dotnet run --project src/Means/Means.csproj --launch-profile http

The development server starts at http://localhost:5178.

2.1) Docker Compose

Single-node (default XlFs):

docker compose up -d --build

Access: http://localhost:5178

Default credentials (override via .env or environment variables for production):

  • Console: meansadmin / meansadmin-local
  • S3: meansadmin / meansadmin-local-secret

Multi-node (experimental):

docker compose -f compose.multinode.yaml up -d --build
NodeAddress
means-node1http://localhost:5181
means-node2http://localhost:5182
means-node3http://localhost:5183

⚠️ Important: The multi-node setup is for topology and operations page validation only. Each node has an independent XlFs namespace. Do not place a data-plane load balancer in front of these nodes until distributed metadata/RPC is implemented.

3) Access the Console

Open http://localhost:5178 and sign in with the default development credentials:

FieldValue
Usernameadmin
Passwordmeansadmin

4) Local S3 Access Methods

Same-origin alias (development):

http://localhost:5178/s3/{bucket}/{key}

Standard host styles (requires DNS/hosts configuration):

StyleURL Pattern
Path-stylehttp(s)://api.means.local/{bucket}/{key}
Virtual-hosted-stylehttp(s)://{bucket}.means.local/{key}

Configuration Reference

All configuration is in src/Means/appsettings.json under the Means section.

S3 Settings

KeyDefaultDescription
Means:S3:ServiceHostapi.means.localPath-style hostname
Means:S3:DomainSuffixmeans.localVirtual-hosted-style domain suffix
Means:S3:AliasPrefix/s3Same-origin S3 alias prefix

Storage Settings

KeyDefaultDescription
Means:Storage:ObjectsPathdata/objectsSingle-disk directory (when Disks not configured)
Means:Storage:Disks/data/xlfs/disk1...XlFs multi-disk root directories; each gets .means.sys/format.json
Means:Storage:ErasureDataShards2Reed-Solomon data shards; used when online disks satisfy data+parity
Means:Storage:ErasureParityShards2Reed-Solomon parity shards; falls back to full-copy quorum when insufficient disks
Means:Storage:WriteQuorum3Minimum disks required for write success
Means:Storage:ReadQuorum1Minimum disks required for read success
Means:Storage:MetaSyncModeAlwaysMetadata WAL flush strategy
Means:Storage:VerifyChecksumOnReadfalseSynchronous SHA256 verification on read (adds full I/O)
Means:Storage:DefaultAccessKeymeansadminDefault S3 access key
Means:Storage:DefaultSecretKeymeansadminsecretDefault S3 secret key
Means:Storage:MultipartUploadCleanupAgeHours24Age threshold for incomplete multipart upload cleanup
Means:Storage:MultipartUploadCleanupIntervalMinutes60Cleanup background task interval
Means:Storage:GarbageCollectionIntervalSeconds3600Orphan file GC scan interval
Means:Storage:GarbageCollectionBatchSize1000Max candidates per GC run
Means:Storage:GarbageCollectionTempFileAgeMinutes60Min age for temp/unreferenced file GC eligibility
Means:Storage:ReplicationIntervalSeconds3600Replication worker interval (reports status only when no rules configured)

Request Limits

KeyDefaultDescription
Means:RequestLimits:MaxUploadSizeBytes1073741824 (1 GiB)Max per-request upload size
Means:RequestLimits:MaxConcurrentUploadRequests64Global concurrency for PUT/UploadPart; returns 503 SlowDown when exceeded

Rate Limiting

KeyDefaultDescription
Means:RateLimits:EnabledtrueEnable fixed-window rate limiting
Means:RateLimits:ConsoleLoginPermitLimit10Requests per console login window
Means:RateLimits:ConsoleLoginWindowSeconds60Console login window (seconds)
Means:RateLimits:ConsoleApiPermitLimit600Requests per console API window
Means:RateLimits:ConsoleApiWindowSeconds60Console API window (seconds)
Means:RateLimits:S3PermitLimit1200Requests per S3 data plane window
Means:RateLimits:S3WindowSeconds60S3 data plane window (seconds)

Telemetry

KeyDefaultDescription
Means:Telemetry:EnabledfalseEnable OpenTelemetry tracing
Means:Telemetry:ServiceNameMeansTracing resource service name
Means:Telemetry:OtlpEndpoint(empty)OTLP exporter endpoint
Means:Telemetry:SampleRatio1.0Trace sampling ratio [0, 1]

Cluster

KeyDefaultDescription
Means:Cluster:InternalAuthToken(empty)Inter-node shard RPC token; empty = /api/internal/cluster returns 404
Means:Cluster:MaxShardTransferBytes5368709120 (5 GiB)Per-shard streaming transfer limit

Console

KeyDefaultDescription
Means:Console:AdminUseradminAdmin username
Means:Console:AdminPasswordmeansadminAdmin password
Means:Console:SessionHours8Session duration (hours)

Important Notes

  • Production security: The service refuses to start if default console credentials are detected. Always replace AdminUser/AdminPassword and DefaultAccessKey/DefaultSecretKey for production.
  • Multipart cleanup: Background tasks automatically clean up incomplete upload metadata and part files. Frontend cancellation best-effort calls abort, but disk reclamation does not depend on browser request success.
  • Read verification: SHA256 verification is off by default on reads. Enable VerifyChecksumOnRead for strong verification. Background scrub always checksums.
  • Rate limiting responses: Console → JSON SlowDown; S3 → XML SlowDown; both include Retry-After header.
  • Prometheus: Scrape /metrics. Recommended alerting rules in docs/operations/deployment-observability.md.
  • OpenTelemetry: Disabled by default. Set Means:Telemetry:Enabled=true to collect ASP.NET Core and background task spans. Configure OtlpEndpoint for collector export.

Development & Testing

Running Tests

dotnet test Means.slnx
Test ProjectFocus
Means.UnitTestsAddress resolution, naming rules, policy evaluation, compression logic
Means.IntegrationTestsS3 full-chain workflows, pre-signed URLs, Console API end-to-end
Means.ContractTestsSDK protocol YAML spec vs. fixture completeness

Frontend Development

cd web
npm install
npm run dev
  • Vite dev server proxies /api and /s3 to http://localhost:5178.
  • Build for production: npm run build → outputs to src/Means/wwwroot.

SDKs & Specification

PackageLocationDescription
C# SDKSDKs/csharpFull-featured .NET client SDK (Means.Client.csproj)
TypeScript SDKSDKs/typescript/packages/sdkBrowser-safe S3 client
TypeScript Node ExtensionSDKs/typescript/packages/sdk-nodeSigV4 signing & pre-signing for Node.js
Protocol Spec (YAML)SDKs/spec/means-sdk-v1.yamlMachine-readable API specification
Protocol Spec (Markdown)SDKs/spec/means-sdk-v1.mdHuman-readable protocol documentation
SDK ExamplesSDKs/examples/Usage examples in C# and TypeScript

Project Map

Means/
├── CHANGELOG.md
├── IMPLEMENTATION_PLAN.md
├── Means.slnx # .NET solution file
├── compose.yaml # Single-node Docker Compose
├── compose.multinode.yaml # Multi-node experimental Compose
├── README.md # This file (English)
├── README.zh.md # Chinese documentation
│
├── src/
│ ├── Means/ # ASP.NET Core host
│ │ ├── Program.cs # Application entry point
│ │ ├── Composition/ # DI composition & service registration
│ │ ├── Configuration/ # Configuration binding & validation
│ │ ├── Endpoints/ # HTTP API endpoints (Console, etc.)
│ │ ├── Middleware/ # ASP.NET Core middleware pipeline
│ │ ├── Services/ # Application services
│ │ ├── Security/ # Authentication & authorization
│ │ ├── Serialization/ # JSON serialization configuration
│ │ ├── Properties/ # Launch profiles, assembly info
│ │ └── wwwroot/ # Built frontend artifacts
│ │
│ ├── Means.Core/ # Core domain & abstractions
│ │ ├── Abstractions/ # Storage interfaces & contracts
│ │ ├── Constants/ # Well-known constants
│ │ ├── Errors/ # Domain error types
│ │ ├── Models/ # Domain models
│ │ ├── Placement/ # Disk placement strategies
│ │ ├── Policies/ # Storage policies
│ │ └── Requests/ # Request model definitions
│ │
│ ├── Means.Infrastructure.XlFs/ # XlFs storage engine
│ │ ├── Store/ # Core store implementation
│ │ ├── LogDb/ # MeansLogDb metadata engine
│ │ ├── Models/ # XlFs-specific models
│ │ └── XlFsOptions.cs # Storage options
│ │
│ └── Means.Protocol.S3/ # S3 protocol layer
│ ├── Addressing/ # Path-style & vhost-style resolution
│ ├── Compression/ # Content negotiation & compression
│ ├── Serialization/ # S3 XML response serialization
│ ├── Signing/ # SigV4 signature validation
│ └── Validation/ # Request validation
│
├── tests/
│ ├── Means.UnitTests/
│ ├── Means.IntegrationTests/
│ └── Means.ContractTests/
│
├── SDKs/
│ ├── csharp/
│ ├── typescript/
│ ├── spec/
│ └── examples/
│
├── web/ # React frontend (Vite + TypeScript)
│ ├── src/ # React components & pages
│ ├── public/ # Static assets
│ ├── vite.config.ts # Vite configuration
│ └── package.json
│
├── docs/ # Documentation site (Next.js)
│ ├── content/docs/ # Documentation content
│ ├── app/ # Next.js app router pages
│ └── source.config.ts
│
└── scripts/ # Utility scripts
├── benchmarks/ # S3 benchmark scripts
└── compatibility/ # S3 client compatibility matrix

License

MIT — see LICENSE for details.

About

Means 是一个面向应用文件与静态资源场景的、架构优先的自建分布式对象存储项目。

Resources

Stars

87 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages