feat: switch to s3 storage - #926

Merged
coodos merged 7 commits into
mainfrom
feat/object-store-for-files
Mar 16, 2026
Merged

feat: switch to s3 storage#926
coodos merged 7 commits into
mainfrom
feat/object-store-for-files

Conversation

@coodos

@coodoscoodos commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description of change

Change from db to object store

Issue Number

Type of change

  • Update (a change which updates existing functionality)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • New Features
    • Integrated S3-compatible cloud storage for file uploads
    • Implemented presigned upload flow for direct client-to-cloud transfers
    • Files now accessible via direct URLs for improved performance on downloads and previews

@coderabbitai

coderabbitaiBot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 752c3fcb-d800-43ee-8d82-ef7eadc43d95

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce20f5 and 5eb8c47.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (24)
  • platforms/esigner/api/package.json
  • platforms/esigner/api/src/controllers/FileController.ts
  • platforms/esigner/api/src/controllers/WebhookController.ts
  • platforms/esigner/api/src/database/entities/File.ts
  • platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts
  • platforms/esigner/api/src/index.ts
  • platforms/esigner/api/src/services/FileService.ts
  • platforms/esigner/api/src/services/S3Service.ts
  • platforms/esigner/api/src/web3adapter/mappings/file.mapping.json
  • platforms/esigner/client/src/lib/stores/files.ts
  • platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte
  • platforms/file-manager/api/package.json
  • platforms/file-manager/api/src/controllers/FileController.ts
  • platforms/file-manager/api/src/controllers/WebhookController.ts
  • platforms/file-manager/api/src/database/entities/File.ts
  • platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
  • platforms/file-manager/api/src/index.ts
  • platforms/file-manager/api/src/services/FileService.ts
  • platforms/file-manager/api/src/services/S3Service.ts
  • platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
  • platforms/file-manager/client/src/lib/stores/files.ts
  • platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
  • platforms/file-manager/client/src/routes/(protected)/files/[id]/+page.svelte
  • services/ontology/schemas/file.json

📝 Walkthrough

Walkthrough

This PR implements S3-compatible object storage integration for file uploads across both esigner and file-manager platforms. It introduces presigned upload endpoints, a new S3Service class for AWS SDK operations, modifies the File entity to add a url field and make data nullable, updates database schemas via migrations, and refactors client-side upload flows from server-side multipart to direct S3 uploads with presigned URLs.

Changes

Cohort / File(s)Summary
Package Dependencies
platforms/esigner/api/package.json, platforms/file-manager/api/package.json
Added AWS SDK dependencies (@aws-sdk/client-s3, @aws-sdk/s3-request-presigner) and repositioned signature-validator within the dependencies block.
S3 Service Integration
platforms/esigner/api/src/services/S3Service.ts, platforms/file-manager/api/src/services/S3Service.ts
New S3Service class encapsulating AWS SDK v3 operations with lazy initialization, presigned URL generation, bucket/region derivation, and methods for upload URLs, public URLs, object metadata, and streaming (identical implementations across both platforms).
File Controller Upload Endpoints
platforms/esigner/api/src/controllers/FileController.ts, platforms/file-manager/api/src/controllers/FileController.ts
Added presignUpload and confirmUpload methods to handle S3 presigned URL generation and upload confirmation; presignUpload generates uploadUrl and fileId, while confirmUpload validates inputs, fetches S3 object metadata (MD5 hash), and creates file records.
File Service S3 Integration
platforms/esigner/api/src/services/FileService.ts, platforms/file-manager/api/src/services/FileService.ts
Added s3Service instance, createFileWithUrl method to create file records with S3 URLs, and updated getDocumentsWithStatus/getFileMetadataById to include url field in response payloads.
Webhook File Handling
platforms/esigner/api/src/controllers/WebhookController.ts, platforms/file-manager/api/src/controllers/WebhookController.ts
Updated file creation and update paths to handle URL assignment from local.data.url and initialize file data directly with base64 decoding (when present) instead of temporary buffer construction.
File Entity & Schema
platforms/esigner/api/src/database/entities/File.ts, platforms/file-manager/api/src/database/entities/File.ts
Made data field nullable (Buffer | null) and added new optional url field (string | null) as text column with nullable: true.
Database Migrations
platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts, platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
Added TypeORM migrations that add url column to files table and make data column nullable, with reversible up/down flows for both platforms.
API Routes
platforms/esigner/api/src/index.ts, platforms/file-manager/api/src/index.ts
Added POST /api/files/presign and POST /api/files/confirm routes guarded by authGuard, enabling presigned upload and upload confirmation workflows.
Web3 Adapter Mappings
platforms/esigner/api/src/web3adapter/mappings/file.mapping.json, platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
Added "url": "url" field mapping in localToUniversalMap for File entities.
Client File Store
platforms/esigner/client/src/lib/stores/files.ts, platforms/file-manager/client/src/lib/stores/files.ts
Added optional url field to Document/File interface; refactored upload flow from multipart FormData to three-step presigned S3 workflow (presign → upload → confirm) with error handling for non-OK S3 responses.
Client Upload UI
platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte, platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
Updated file preview and download logic to prefer file.url when available, falling back to API-based blob fetching; refactored upload handler to use presigned URL workflow with XMLHttpRequest PUT and progress tracking.
Ontology Schema
services/ontology/schemas/file.json
Added url property (type: ["string", "null"], format: "uri") and updated data description to mark it as legacy.

Sequence Diagram(s)

sequenceDiagram
participant Client
participant Controller
participant S3Service
participant S3 as S3 Storage
participant FileService
participant Database
rect rgba(100, 150, 255, 0.5)
Note over Client,Database: Presign Upload Flow
Client->>Controller: POST /api/files/presign<br/>(filename, mimeType, size)
Controller->>S3Service: generateKey(userId, fileId, filename)
S3Service-->>Controller: key
Controller->>S3Service: generateUploadUrl(key, contentType)
S3Service-->>Controller: presigned uploadUrl
Controller-->>Client: { uploadUrl, key, fileId }
end
rect rgba(150, 200, 100, 0.5)
Note over Client,S3: Direct S3 Upload
Client->>S3: PUT uploadUrl<br/>(file data, Content-Type, x-amz-acl)
S3-->>Client: 200 OK
end
rect rgba(200, 150, 100, 0.5)
Note over Client,Database: Confirm Upload Flow
Client->>Controller: POST /api/files/confirm<br/>(key, fileId, filename, mimeType, size)
Controller->>S3Service: headObject(key)
S3Service->>S3: HEAD object
S3-->>S3Service: { contentLength, etag }
S3Service-->>Controller: metadata with md5Hash
Controller->>S3Service: getPublicUrl(key)
S3Service-->>Controller: public url
Controller->>FileService: createFileWithUrl(..., url)
FileService->>Database: save File record
Database-->>FileService: File entity
FileService-->>Controller: File
Controller-->>Client: { file metadata }
end
rect rgba(100, 200, 200, 0.5)
Note over Client,S3: Download/Preview Flow
alt file.url exists
Client->>Client: use file.url<br/>(direct redirect or open)
else file.url is null
Client->>Controller: GET /api/files/[id]/download
Controller->>FileService: getFileDataStream(fileId)
FileService->>Database: fetch File
Database-->>FileService: File record
FileService-->>Controller: Readable stream
Controller-->>Client: file data
end
end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: per user limit #922: Modifies file upload and quota logic in FileController and FileService, touching the same upload-path handlers affected by presign/confirm endpoints.
  • fix: file manager limits #663: Updates FileController in file-manager API with upload-related handler and flow changes, directly overlapping with this PR's controller modifications.
  • feat:(file-manager) fix download speeds #756: Adds server-side file streaming and ZIP export features to file-manager backend (FileController and FileService), operating on the same file storage and retrieval domain.

Suggested reviewers

  • sosweetham
  • xPathin

Poem

🐰 A presigned dance on S3's stage,
No more uploads tied to server's cage,
Direct to storage, swift and clean,
With URLs stored—the finest scene,
File flows that flow, now cloud-aligned! 🌧️✨

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/object-store-for-files
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review March 16, 2026 11:45
@coodos
coodos merged commit d2efd8b into mainMar 16, 2026
3 of 4 checks passed
@coodos
coodos deleted the feat/object-store-for-files branch March 16, 2026 11:47
@coderabbitaicoderabbitaiBot mentioned this pull request Apr 6, 2026
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@coodos
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat: switch to s3 storage - #926

Merged
coodos merged 7 commits into
mainfrom
feat/object-store-for-files
Mar 16, 2026
Merged

feat: switch to s3 storage#926
coodos merged 7 commits into
mainfrom
feat/object-store-for-files

Conversation

@coodos

@coodoscoodos commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description of change

Change from db to object store

Issue Number

Type of change

  • Update (a change which updates existing functionality)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • New Features
    • Integrated S3-compatible cloud storage for file uploads
    • Implemented presigned upload flow for direct client-to-cloud transfers
    • Files now accessible via direct URLs for improved performance on downloads and previews

@coderabbitai

coderabbitaiBot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 752c3fcb-d800-43ee-8d82-ef7eadc43d95

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce20f5 and 5eb8c47.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (24)
  • platforms/esigner/api/package.json
  • platforms/esigner/api/src/controllers/FileController.ts
  • platforms/esigner/api/src/controllers/WebhookController.ts
  • platforms/esigner/api/src/database/entities/File.ts
  • platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts
  • platforms/esigner/api/src/index.ts
  • platforms/esigner/api/src/services/FileService.ts
  • platforms/esigner/api/src/services/S3Service.ts
  • platforms/esigner/api/src/web3adapter/mappings/file.mapping.json
  • platforms/esigner/client/src/lib/stores/files.ts
  • platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte
  • platforms/file-manager/api/package.json
  • platforms/file-manager/api/src/controllers/FileController.ts
  • platforms/file-manager/api/src/controllers/WebhookController.ts
  • platforms/file-manager/api/src/database/entities/File.ts
  • platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
  • platforms/file-manager/api/src/index.ts
  • platforms/file-manager/api/src/services/FileService.ts
  • platforms/file-manager/api/src/services/S3Service.ts
  • platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
  • platforms/file-manager/client/src/lib/stores/files.ts
  • platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
  • platforms/file-manager/client/src/routes/(protected)/files/[id]/+page.svelte
  • services/ontology/schemas/file.json

📝 Walkthrough

Walkthrough

This PR implements S3-compatible object storage integration for file uploads across both esigner and file-manager platforms. It introduces presigned upload endpoints, a new S3Service class for AWS SDK operations, modifies the File entity to add a url field and make data nullable, updates database schemas via migrations, and refactors client-side upload flows from server-side multipart to direct S3 uploads with presigned URLs.

Changes

Cohort / File(s)Summary
Package Dependencies
platforms/esigner/api/package.json, platforms/file-manager/api/package.json
Added AWS SDK dependencies (@aws-sdk/client-s3, @aws-sdk/s3-request-presigner) and repositioned signature-validator within the dependencies block.
S3 Service Integration
platforms/esigner/api/src/services/S3Service.ts, platforms/file-manager/api/src/services/S3Service.ts
New S3Service class encapsulating AWS SDK v3 operations with lazy initialization, presigned URL generation, bucket/region derivation, and methods for upload URLs, public URLs, object metadata, and streaming (identical implementations across both platforms).
File Controller Upload Endpoints
platforms/esigner/api/src/controllers/FileController.ts, platforms/file-manager/api/src/controllers/FileController.ts
Added presignUpload and confirmUpload methods to handle S3 presigned URL generation and upload confirmation; presignUpload generates uploadUrl and fileId, while confirmUpload validates inputs, fetches S3 object metadata (MD5 hash), and creates file records.
File Service S3 Integration
platforms/esigner/api/src/services/FileService.ts, platforms/file-manager/api/src/services/FileService.ts
Added s3Service instance, createFileWithUrl method to create file records with S3 URLs, and updated getDocumentsWithStatus/getFileMetadataById to include url field in response payloads.
Webhook File Handling
platforms/esigner/api/src/controllers/WebhookController.ts, platforms/file-manager/api/src/controllers/WebhookController.ts
Updated file creation and update paths to handle URL assignment from local.data.url and initialize file data directly with base64 decoding (when present) instead of temporary buffer construction.
File Entity & Schema
platforms/esigner/api/src/database/entities/File.ts, platforms/file-manager/api/src/database/entities/File.ts
Made data field nullable (Buffer | null) and added new optional url field (string | null) as text column with nullable: true.
Database Migrations
platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts, platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
Added TypeORM migrations that add url column to files table and make data column nullable, with reversible up/down flows for both platforms.
API Routes
platforms/esigner/api/src/index.ts, platforms/file-manager/api/src/index.ts
Added POST /api/files/presign and POST /api/files/confirm routes guarded by authGuard, enabling presigned upload and upload confirmation workflows.
Web3 Adapter Mappings
platforms/esigner/api/src/web3adapter/mappings/file.mapping.json, platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
Added "url": "url" field mapping in localToUniversalMap for File entities.
Client File Store
platforms/esigner/client/src/lib/stores/files.ts, platforms/file-manager/client/src/lib/stores/files.ts
Added optional url field to Document/File interface; refactored upload flow from multipart FormData to three-step presigned S3 workflow (presign → upload → confirm) with error handling for non-OK S3 responses.
Client Upload UI
platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte, platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
Updated file preview and download logic to prefer file.url when available, falling back to API-based blob fetching; refactored upload handler to use presigned URL workflow with XMLHttpRequest PUT and progress tracking.
Ontology Schema
services/ontology/schemas/file.json
Added url property (type: ["string", "null"], format: "uri") and updated data description to mark it as legacy.

Sequence Diagram(s)

sequenceDiagram
participant Client
participant Controller
participant S3Service
participant S3 as S3 Storage
participant FileService
participant Database
rect rgba(100, 150, 255, 0.5)
Note over Client,Database: Presign Upload Flow
Client->>Controller: POST /api/files/presign<br/>(filename, mimeType, size)
Controller->>S3Service: generateKey(userId, fileId, filename)
S3Service-->>Controller: key
Controller->>S3Service: generateUploadUrl(key, contentType)
S3Service-->>Controller: presigned uploadUrl
Controller-->>Client: { uploadUrl, key, fileId }
end
rect rgba(150, 200, 100, 0.5)
Note over Client,S3: Direct S3 Upload
Client->>S3: PUT uploadUrl<br/>(file data, Content-Type, x-amz-acl)
S3-->>Client: 200 OK
end
rect rgba(200, 150, 100, 0.5)
Note over Client,Database: Confirm Upload Flow
Client->>Controller: POST /api/files/confirm<br/>(key, fileId, filename, mimeType, size)
Controller->>S3Service: headObject(key)
S3Service->>S3: HEAD object
S3-->>S3Service: { contentLength, etag }
S3Service-->>Controller: metadata with md5Hash
Controller->>S3Service: getPublicUrl(key)
S3Service-->>Controller: public url
Controller->>FileService: createFileWithUrl(..., url)
FileService->>Database: save File record
Database-->>FileService: File entity
FileService-->>Controller: File
Controller-->>Client: { file metadata }
end
rect rgba(100, 200, 200, 0.5)
Note over Client,S3: Download/Preview Flow
alt file.url exists
Client->>Client: use file.url<br/>(direct redirect or open)
else file.url is null
Client->>Controller: GET /api/files/[id]/download
Controller->>FileService: getFileDataStream(fileId)
FileService->>Database: fetch File
Database-->>FileService: File record
FileService-->>Controller: Readable stream
Controller-->>Client: file data
end
end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: per user limit #922: Modifies file upload and quota logic in FileController and FileService, touching the same upload-path handlers affected by presign/confirm endpoints.
  • fix: file manager limits #663: Updates FileController in file-manager API with upload-related handler and flow changes, directly overlapping with this PR's controller modifications.
  • feat:(file-manager) fix download speeds #756: Adds server-side file streaming and ZIP export features to file-manager backend (FileController and FileService), operating on the same file storage and retrieval domain.

Suggested reviewers

  • sosweetham
  • xPathin

Poem

🐰 A presigned dance on S3's stage,
No more uploads tied to server's cage,
Direct to storage, swift and clean,
With URLs stored—the finest scene,
File flows that flow, now cloud-aligned! 🌧️✨

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/object-store-for-files
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review March 16, 2026 11:45
@coodos
coodos merged commit d2efd8b into mainMar 16, 2026
3 of 4 checks passed
@coodos
coodos deleted the feat/object-store-for-files branch March 16, 2026 11:47
@coderabbitaicoderabbitaiBot mentioned this pull request Apr 6, 2026
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@coodos
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: switch to s3 storage - #926

Merged
coodos merged 7 commits into
mainfrom
feat/object-store-for-files
Mar 16, 2026
Merged

feat: switch to s3 storage#926
coodos merged 7 commits into
mainfrom
feat/object-store-for-files

Conversation

@coodos

@coodoscoodos commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description of change

Change from db to object store

Issue Number

Type of change

  • Update (a change which updates existing functionality)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • New Features
    • Integrated S3-compatible cloud storage for file uploads
    • Implemented presigned upload flow for direct client-to-cloud transfers
    • Files now accessible via direct URLs for improved performance on downloads and previews

@coderabbitai

coderabbitaiBot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 752c3fcb-d800-43ee-8d82-ef7eadc43d95

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce20f5 and 5eb8c47.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (24)
  • platforms/esigner/api/package.json
  • platforms/esigner/api/src/controllers/FileController.ts
  • platforms/esigner/api/src/controllers/WebhookController.ts
  • platforms/esigner/api/src/database/entities/File.ts
  • platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts
  • platforms/esigner/api/src/index.ts
  • platforms/esigner/api/src/services/FileService.ts
  • platforms/esigner/api/src/services/S3Service.ts
  • platforms/esigner/api/src/web3adapter/mappings/file.mapping.json
  • platforms/esigner/client/src/lib/stores/files.ts
  • platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte
  • platforms/file-manager/api/package.json
  • platforms/file-manager/api/src/controllers/FileController.ts
  • platforms/file-manager/api/src/controllers/WebhookController.ts
  • platforms/file-manager/api/src/database/entities/File.ts
  • platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
  • platforms/file-manager/api/src/index.ts
  • platforms/file-manager/api/src/services/FileService.ts
  • platforms/file-manager/api/src/services/S3Service.ts
  • platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
  • platforms/file-manager/client/src/lib/stores/files.ts
  • platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
  • platforms/file-manager/client/src/routes/(protected)/files/[id]/+page.svelte
  • services/ontology/schemas/file.json

📝 Walkthrough

Walkthrough

This PR implements S3-compatible object storage integration for file uploads across both esigner and file-manager platforms. It introduces presigned upload endpoints, a new S3Service class for AWS SDK operations, modifies the File entity to add a url field and make data nullable, updates database schemas via migrations, and refactors client-side upload flows from server-side multipart to direct S3 uploads with presigned URLs.

Changes

Cohort / File(s)Summary
Package Dependencies
platforms/esigner/api/package.json, platforms/file-manager/api/package.json
Added AWS SDK dependencies (@aws-sdk/client-s3, @aws-sdk/s3-request-presigner) and repositioned signature-validator within the dependencies block.
S3 Service Integration
platforms/esigner/api/src/services/S3Service.ts, platforms/file-manager/api/src/services/S3Service.ts
New S3Service class encapsulating AWS SDK v3 operations with lazy initialization, presigned URL generation, bucket/region derivation, and methods for upload URLs, public URLs, object metadata, and streaming (identical implementations across both platforms).
File Controller Upload Endpoints
platforms/esigner/api/src/controllers/FileController.ts, platforms/file-manager/api/src/controllers/FileController.ts
Added presignUpload and confirmUpload methods to handle S3 presigned URL generation and upload confirmation; presignUpload generates uploadUrl and fileId, while confirmUpload validates inputs, fetches S3 object metadata (MD5 hash), and creates file records.
File Service S3 Integration
platforms/esigner/api/src/services/FileService.ts, platforms/file-manager/api/src/services/FileService.ts
Added s3Service instance, createFileWithUrl method to create file records with S3 URLs, and updated getDocumentsWithStatus/getFileMetadataById to include url field in response payloads.
Webhook File Handling
platforms/esigner/api/src/controllers/WebhookController.ts, platforms/file-manager/api/src/controllers/WebhookController.ts
Updated file creation and update paths to handle URL assignment from local.data.url and initialize file data directly with base64 decoding (when present) instead of temporary buffer construction.
File Entity & Schema
platforms/esigner/api/src/database/entities/File.ts, platforms/file-manager/api/src/database/entities/File.ts
Made data field nullable (Buffer | null) and added new optional url field (string | null) as text column with nullable: true.
Database Migrations
platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts, platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
Added TypeORM migrations that add url column to files table and make data column nullable, with reversible up/down flows for both platforms.
API Routes
platforms/esigner/api/src/index.ts, platforms/file-manager/api/src/index.ts
Added POST /api/files/presign and POST /api/files/confirm routes guarded by authGuard, enabling presigned upload and upload confirmation workflows.
Web3 Adapter Mappings
platforms/esigner/api/src/web3adapter/mappings/file.mapping.json, platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
Added "url": "url" field mapping in localToUniversalMap for File entities.
Client File Store
platforms/esigner/client/src/lib/stores/files.ts, platforms/file-manager/client/src/lib/stores/files.ts
Added optional url field to Document/File interface; refactored upload flow from multipart FormData to three-step presigned S3 workflow (presign → upload → confirm) with error handling for non-OK S3 responses.
Client Upload UI
platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte, platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
Updated file preview and download logic to prefer file.url when available, falling back to API-based blob fetching; refactored upload handler to use presigned URL workflow with XMLHttpRequest PUT and progress tracking.
Ontology Schema
services/ontology/schemas/file.json
Added url property (type: ["string", "null"], format: "uri") and updated data description to mark it as legacy.

Sequence Diagram(s)

sequenceDiagram
participant Client
participant Controller
participant S3Service
participant S3 as S3 Storage
participant FileService
participant Database
rect rgba(100, 150, 255, 0.5)
Note over Client,Database: Presign Upload Flow
Client->>Controller: POST /api/files/presign<br/>(filename, mimeType, size)
Controller->>S3Service: generateKey(userId, fileId, filename)
S3Service-->>Controller: key
Controller->>S3Service: generateUploadUrl(key, contentType)
S3Service-->>Controller: presigned uploadUrl
Controller-->>Client: { uploadUrl, key, fileId }
end
rect rgba(150, 200, 100, 0.5)
Note over Client,S3: Direct S3 Upload
Client->>S3: PUT uploadUrl<br/>(file data, Content-Type, x-amz-acl)
S3-->>Client: 200 OK
end
rect rgba(200, 150, 100, 0.5)
Note over Client,Database: Confirm Upload Flow
Client->>Controller: POST /api/files/confirm<br/>(key, fileId, filename, mimeType, size)
Controller->>S3Service: headObject(key)
S3Service->>S3: HEAD object
S3-->>S3Service: { contentLength, etag }
S3Service-->>Controller: metadata with md5Hash
Controller->>S3Service: getPublicUrl(key)
S3Service-->>Controller: public url
Controller->>FileService: createFileWithUrl(..., url)
FileService->>Database: save File record
Database-->>FileService: File entity
FileService-->>Controller: File
Controller-->>Client: { file metadata }
end
rect rgba(100, 200, 200, 0.5)
Note over Client,S3: Download/Preview Flow
alt file.url exists
Client->>Client: use file.url<br/>(direct redirect or open)
else file.url is null
Client->>Controller: GET /api/files/[id]/download
Controller->>FileService: getFileDataStream(fileId)
FileService->>Database: fetch File
Database-->>FileService: File record
FileService-->>Controller: Readable stream
Controller-->>Client: file data
end
end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: per user limit #922: Modifies file upload and quota logic in FileController and FileService, touching the same upload-path handlers affected by presign/confirm endpoints.
  • fix: file manager limits #663: Updates FileController in file-manager API with upload-related handler and flow changes, directly overlapping with this PR's controller modifications.
  • feat:(file-manager) fix download speeds #756: Adds server-side file streaming and ZIP export features to file-manager backend (FileController and FileService), operating on the same file storage and retrieval domain.

Suggested reviewers

  • sosweetham
  • xPathin

Poem

🐰 A presigned dance on S3's stage,
No more uploads tied to server's cage,
Direct to storage, swift and clean,
With URLs stored—the finest scene,
File flows that flow, now cloud-aligned! 🌧️✨

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/object-store-for-files
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review March 16, 2026 11:45
@coodos
coodos merged commit d2efd8b into mainMar 16, 2026
3 of 4 checks passed
@coodos
coodos deleted the feat/object-store-for-files branch March 16, 2026 11:47
@coderabbitaicoderabbitaiBot mentioned this pull request Apr 6, 2026
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@coodos
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: switch to s3 storage - #926

Merged
coodos merged 7 commits into
mainfrom
feat/object-store-for-files
Mar 16, 2026
Merged

feat: switch to s3 storage#926
coodos merged 7 commits into
mainfrom
feat/object-store-for-files

Conversation

@coodos

@coodoscoodos commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description of change

Change from db to object store

Issue Number

Type of change

  • Update (a change which updates existing functionality)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • New Features
    • Integrated S3-compatible cloud storage for file uploads
    • Implemented presigned upload flow for direct client-to-cloud transfers
    • Files now accessible via direct URLs for improved performance on downloads and previews

@coderabbitai

coderabbitaiBot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 752c3fcb-d800-43ee-8d82-ef7eadc43d95

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce20f5 and 5eb8c47.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (24)
  • platforms/esigner/api/package.json
  • platforms/esigner/api/src/controllers/FileController.ts
  • platforms/esigner/api/src/controllers/WebhookController.ts
  • platforms/esigner/api/src/database/entities/File.ts
  • platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts
  • platforms/esigner/api/src/index.ts
  • platforms/esigner/api/src/services/FileService.ts
  • platforms/esigner/api/src/services/S3Service.ts
  • platforms/esigner/api/src/web3adapter/mappings/file.mapping.json
  • platforms/esigner/client/src/lib/stores/files.ts
  • platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte
  • platforms/file-manager/api/package.json
  • platforms/file-manager/api/src/controllers/FileController.ts
  • platforms/file-manager/api/src/controllers/WebhookController.ts
  • platforms/file-manager/api/src/database/entities/File.ts
  • platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
  • platforms/file-manager/api/src/index.ts
  • platforms/file-manager/api/src/services/FileService.ts
  • platforms/file-manager/api/src/services/S3Service.ts
  • platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
  • platforms/file-manager/client/src/lib/stores/files.ts
  • platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
  • platforms/file-manager/client/src/routes/(protected)/files/[id]/+page.svelte
  • services/ontology/schemas/file.json

📝 Walkthrough

Walkthrough

This PR implements S3-compatible object storage integration for file uploads across both esigner and file-manager platforms. It introduces presigned upload endpoints, a new S3Service class for AWS SDK operations, modifies the File entity to add a url field and make data nullable, updates database schemas via migrations, and refactors client-side upload flows from server-side multipart to direct S3 uploads with presigned URLs.

Changes

Cohort / File(s)Summary
Package Dependencies
platforms/esigner/api/package.json, platforms/file-manager/api/package.json
Added AWS SDK dependencies (@aws-sdk/client-s3, @aws-sdk/s3-request-presigner) and repositioned signature-validator within the dependencies block.
S3 Service Integration
platforms/esigner/api/src/services/S3Service.ts, platforms/file-manager/api/src/services/S3Service.ts
New S3Service class encapsulating AWS SDK v3 operations with lazy initialization, presigned URL generation, bucket/region derivation, and methods for upload URLs, public URLs, object metadata, and streaming (identical implementations across both platforms).
File Controller Upload Endpoints
platforms/esigner/api/src/controllers/FileController.ts, platforms/file-manager/api/src/controllers/FileController.ts
Added presignUpload and confirmUpload methods to handle S3 presigned URL generation and upload confirmation; presignUpload generates uploadUrl and fileId, while confirmUpload validates inputs, fetches S3 object metadata (MD5 hash), and creates file records.
File Service S3 Integration
platforms/esigner/api/src/services/FileService.ts, platforms/file-manager/api/src/services/FileService.ts
Added s3Service instance, createFileWithUrl method to create file records with S3 URLs, and updated getDocumentsWithStatus/getFileMetadataById to include url field in response payloads.
Webhook File Handling
platforms/esigner/api/src/controllers/WebhookController.ts, platforms/file-manager/api/src/controllers/WebhookController.ts
Updated file creation and update paths to handle URL assignment from local.data.url and initialize file data directly with base64 decoding (when present) instead of temporary buffer construction.
File Entity & Schema
platforms/esigner/api/src/database/entities/File.ts, platforms/file-manager/api/src/database/entities/File.ts
Made data field nullable (Buffer | null) and added new optional url field (string | null) as text column with nullable: true.
Database Migrations
platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts, platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
Added TypeORM migrations that add url column to files table and make data column nullable, with reversible up/down flows for both platforms.
API Routes
platforms/esigner/api/src/index.ts, platforms/file-manager/api/src/index.ts
Added POST /api/files/presign and POST /api/files/confirm routes guarded by authGuard, enabling presigned upload and upload confirmation workflows.
Web3 Adapter Mappings
platforms/esigner/api/src/web3adapter/mappings/file.mapping.json, platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
Added "url": "url" field mapping in localToUniversalMap for File entities.
Client File Store
platforms/esigner/client/src/lib/stores/files.ts, platforms/file-manager/client/src/lib/stores/files.ts
Added optional url field to Document/File interface; refactored upload flow from multipart FormData to three-step presigned S3 workflow (presign → upload → confirm) with error handling for non-OK S3 responses.
Client Upload UI
platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte, platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
Updated file preview and download logic to prefer file.url when available, falling back to API-based blob fetching; refactored upload handler to use presigned URL workflow with XMLHttpRequest PUT and progress tracking.
Ontology Schema
services/ontology/schemas/file.json
Added url property (type: ["string", "null"], format: "uri") and updated data description to mark it as legacy.

Sequence Diagram(s)

sequenceDiagram
participant Client
participant Controller
participant S3Service
participant S3 as S3 Storage
participant FileService
participant Database
rect rgba(100, 150, 255, 0.5)
Note over Client,Database: Presign Upload Flow
Client->>Controller: POST /api/files/presign<br/>(filename, mimeType, size)
Controller->>S3Service: generateKey(userId, fileId, filename)
S3Service-->>Controller: key
Controller->>S3Service: generateUploadUrl(key, contentType)
S3Service-->>Controller: presigned uploadUrl
Controller-->>Client: { uploadUrl, key, fileId }
end
rect rgba(150, 200, 100, 0.5)
Note over Client,S3: Direct S3 Upload
Client->>S3: PUT uploadUrl<br/>(file data, Content-Type, x-amz-acl)
S3-->>Client: 200 OK
end
rect rgba(200, 150, 100, 0.5)
Note over Client,Database: Confirm Upload Flow
Client->>Controller: POST /api/files/confirm<br/>(key, fileId, filename, mimeType, size)
Controller->>S3Service: headObject(key)
S3Service->>S3: HEAD object
S3-->>S3Service: { contentLength, etag }
S3Service-->>Controller: metadata with md5Hash
Controller->>S3Service: getPublicUrl(key)
S3Service-->>Controller: public url
Controller->>FileService: createFileWithUrl(..., url)
FileService->>Database: save File record
Database-->>FileService: File entity
FileService-->>Controller: File
Controller-->>Client: { file metadata }
end
rect rgba(100, 200, 200, 0.5)
Note over Client,S3: Download/Preview Flow
alt file.url exists
Client->>Client: use file.url<br/>(direct redirect or open)
else file.url is null
Client->>Controller: GET /api/files/[id]/download
Controller->>FileService: getFileDataStream(fileId)
FileService->>Database: fetch File
Database-->>FileService: File record
FileService-->>Controller: Readable stream
Controller-->>Client: file data
end
end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: per user limit #922: Modifies file upload and quota logic in FileController and FileService, touching the same upload-path handlers affected by presign/confirm endpoints.
  • fix: file manager limits #663: Updates FileController in file-manager API with upload-related handler and flow changes, directly overlapping with this PR's controller modifications.
  • feat:(file-manager) fix download speeds #756: Adds server-side file streaming and ZIP export features to file-manager backend (FileController and FileService), operating on the same file storage and retrieval domain.

Suggested reviewers

  • sosweetham
  • xPathin

Poem

🐰 A presigned dance on S3's stage,
No more uploads tied to server's cage,
Direct to storage, swift and clean,
With URLs stored—the finest scene,
File flows that flow, now cloud-aligned! 🌧️✨

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/object-store-for-files
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review March 16, 2026 11:45
@coodos
coodos merged commit d2efd8b into mainMar 16, 2026
3 of 4 checks passed
@coodos
coodos deleted the feat/object-store-for-files branch March 16, 2026 11:47
@coderabbitaicoderabbitaiBot mentioned this pull request Apr 6, 2026
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@coodos
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat: switch to s3 storage - #926

Merged
coodos merged 7 commits into
mainfrom
feat/object-store-for-files
Mar 16, 2026
Merged

feat: switch to s3 storage#926
coodos merged 7 commits into
mainfrom
feat/object-store-for-files

Conversation

@coodos

@coodoscoodos commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description of change

Change from db to object store

Issue Number

Type of change

  • Update (a change which updates existing functionality)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • New Features
    • Integrated S3-compatible cloud storage for file uploads
    • Implemented presigned upload flow for direct client-to-cloud transfers
    • Files now accessible via direct URLs for improved performance on downloads and previews

@coderabbitai

coderabbitaiBot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 752c3fcb-d800-43ee-8d82-ef7eadc43d95

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce20f5 and 5eb8c47.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (24)
  • platforms/esigner/api/package.json
  • platforms/esigner/api/src/controllers/FileController.ts
  • platforms/esigner/api/src/controllers/WebhookController.ts
  • platforms/esigner/api/src/database/entities/File.ts
  • platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts
  • platforms/esigner/api/src/index.ts
  • platforms/esigner/api/src/services/FileService.ts
  • platforms/esigner/api/src/services/S3Service.ts
  • platforms/esigner/api/src/web3adapter/mappings/file.mapping.json
  • platforms/esigner/client/src/lib/stores/files.ts
  • platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte
  • platforms/file-manager/api/package.json
  • platforms/file-manager/api/src/controllers/FileController.ts
  • platforms/file-manager/api/src/controllers/WebhookController.ts
  • platforms/file-manager/api/src/database/entities/File.ts
  • platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
  • platforms/file-manager/api/src/index.ts
  • platforms/file-manager/api/src/services/FileService.ts
  • platforms/file-manager/api/src/services/S3Service.ts
  • platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
  • platforms/file-manager/client/src/lib/stores/files.ts
  • platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
  • platforms/file-manager/client/src/routes/(protected)/files/[id]/+page.svelte
  • services/ontology/schemas/file.json

📝 Walkthrough

Walkthrough

This PR implements S3-compatible object storage integration for file uploads across both esigner and file-manager platforms. It introduces presigned upload endpoints, a new S3Service class for AWS SDK operations, modifies the File entity to add a url field and make data nullable, updates database schemas via migrations, and refactors client-side upload flows from server-side multipart to direct S3 uploads with presigned URLs.

Changes

Cohort / File(s)Summary
Package Dependencies
platforms/esigner/api/package.json, platforms/file-manager/api/package.json
Added AWS SDK dependencies (@aws-sdk/client-s3, @aws-sdk/s3-request-presigner) and repositioned signature-validator within the dependencies block.
S3 Service Integration
platforms/esigner/api/src/services/S3Service.ts, platforms/file-manager/api/src/services/S3Service.ts
New S3Service class encapsulating AWS SDK v3 operations with lazy initialization, presigned URL generation, bucket/region derivation, and methods for upload URLs, public URLs, object metadata, and streaming (identical implementations across both platforms).
File Controller Upload Endpoints
platforms/esigner/api/src/controllers/FileController.ts, platforms/file-manager/api/src/controllers/FileController.ts
Added presignUpload and confirmUpload methods to handle S3 presigned URL generation and upload confirmation; presignUpload generates uploadUrl and fileId, while confirmUpload validates inputs, fetches S3 object metadata (MD5 hash), and creates file records.
File Service S3 Integration
platforms/esigner/api/src/services/FileService.ts, platforms/file-manager/api/src/services/FileService.ts
Added s3Service instance, createFileWithUrl method to create file records with S3 URLs, and updated getDocumentsWithStatus/getFileMetadataById to include url field in response payloads.
Webhook File Handling
platforms/esigner/api/src/controllers/WebhookController.ts, platforms/file-manager/api/src/controllers/WebhookController.ts
Updated file creation and update paths to handle URL assignment from local.data.url and initialize file data directly with base64 decoding (when present) instead of temporary buffer construction.
File Entity & Schema
platforms/esigner/api/src/database/entities/File.ts, platforms/file-manager/api/src/database/entities/File.ts
Made data field nullable (Buffer | null) and added new optional url field (string | null) as text column with nullable: true.
Database Migrations
platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts, platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
Added TypeORM migrations that add url column to files table and make data column nullable, with reversible up/down flows for both platforms.
API Routes
platforms/esigner/api/src/index.ts, platforms/file-manager/api/src/index.ts
Added POST /api/files/presign and POST /api/files/confirm routes guarded by authGuard, enabling presigned upload and upload confirmation workflows.
Web3 Adapter Mappings
platforms/esigner/api/src/web3adapter/mappings/file.mapping.json, platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
Added "url": "url" field mapping in localToUniversalMap for File entities.
Client File Store
platforms/esigner/client/src/lib/stores/files.ts, platforms/file-manager/client/src/lib/stores/files.ts
Added optional url field to Document/File interface; refactored upload flow from multipart FormData to three-step presigned S3 workflow (presign → upload → confirm) with error handling for non-OK S3 responses.
Client Upload UI
platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte, platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
Updated file preview and download logic to prefer file.url when available, falling back to API-based blob fetching; refactored upload handler to use presigned URL workflow with XMLHttpRequest PUT and progress tracking.
Ontology Schema
services/ontology/schemas/file.json
Added url property (type: ["string", "null"], format: "uri") and updated data description to mark it as legacy.

Sequence Diagram(s)

sequenceDiagram
participant Client
participant Controller
participant S3Service
participant S3 as S3 Storage
participant FileService
participant Database
rect rgba(100, 150, 255, 0.5)
Note over Client,Database: Presign Upload Flow
Client->>Controller: POST /api/files/presign<br/>(filename, mimeType, size)
Controller->>S3Service: generateKey(userId, fileId, filename)
S3Service-->>Controller: key
Controller->>S3Service: generateUploadUrl(key, contentType)
S3Service-->>Controller: presigned uploadUrl
Controller-->>Client: { uploadUrl, key, fileId }
end
rect rgba(150, 200, 100, 0.5)
Note over Client,S3: Direct S3 Upload
Client->>S3: PUT uploadUrl<br/>(file data, Content-Type, x-amz-acl)
S3-->>Client: 200 OK
end
rect rgba(200, 150, 100, 0.5)
Note over Client,Database: Confirm Upload Flow
Client->>Controller: POST /api/files/confirm<br/>(key, fileId, filename, mimeType, size)
Controller->>S3Service: headObject(key)
S3Service->>S3: HEAD object
S3-->>S3Service: { contentLength, etag }
S3Service-->>Controller: metadata with md5Hash
Controller->>S3Service: getPublicUrl(key)
S3Service-->>Controller: public url
Controller->>FileService: createFileWithUrl(..., url)
FileService->>Database: save File record
Database-->>FileService: File entity
FileService-->>Controller: File
Controller-->>Client: { file metadata }
end
rect rgba(100, 200, 200, 0.5)
Note over Client,S3: Download/Preview Flow
alt file.url exists
Client->>Client: use file.url<br/>(direct redirect or open)
else file.url is null
Client->>Controller: GET /api/files/[id]/download
Controller->>FileService: getFileDataStream(fileId)
FileService->>Database: fetch File
Database-->>FileService: File record
FileService-->>Controller: Readable stream
Controller-->>Client: file data
end
end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: per user limit #922: Modifies file upload and quota logic in FileController and FileService, touching the same upload-path handlers affected by presign/confirm endpoints.
  • fix: file manager limits #663: Updates FileController in file-manager API with upload-related handler and flow changes, directly overlapping with this PR's controller modifications.
  • feat:(file-manager) fix download speeds #756: Adds server-side file streaming and ZIP export features to file-manager backend (FileController and FileService), operating on the same file storage and retrieval domain.

Suggested reviewers

  • sosweetham
  • xPathin

Poem

🐰 A presigned dance on S3's stage,
No more uploads tied to server's cage,
Direct to storage, swift and clean,
With URLs stored—the finest scene,
File flows that flow, now cloud-aligned! 🌧️✨

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/object-store-for-files
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review March 16, 2026 11:45
@coodos
coodos merged commit d2efd8b into mainMar 16, 2026
3 of 4 checks passed
@coodos
coodos deleted the feat/object-store-for-files branch March 16, 2026 11:47
@coderabbitaicoderabbitaiBot mentioned this pull request Apr 6, 2026
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@coodos
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: switch to s3 storage - #926

Merged
coodos merged 7 commits into
mainfrom
feat/object-store-for-files
Mar 16, 2026
Merged

feat: switch to s3 storage#926
coodos merged 7 commits into
mainfrom
feat/object-store-for-files

Conversation

@coodos

@coodoscoodos commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description of change

Change from db to object store

Issue Number

Type of change

  • Update (a change which updates existing functionality)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • New Features
    • Integrated S3-compatible cloud storage for file uploads
    • Implemented presigned upload flow for direct client-to-cloud transfers
    • Files now accessible via direct URLs for improved performance on downloads and previews

@coderabbitai

coderabbitaiBot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 752c3fcb-d800-43ee-8d82-ef7eadc43d95

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce20f5 and 5eb8c47.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (24)
  • platforms/esigner/api/package.json
  • platforms/esigner/api/src/controllers/FileController.ts
  • platforms/esigner/api/src/controllers/WebhookController.ts
  • platforms/esigner/api/src/database/entities/File.ts
  • platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts
  • platforms/esigner/api/src/index.ts
  • platforms/esigner/api/src/services/FileService.ts
  • platforms/esigner/api/src/services/S3Service.ts
  • platforms/esigner/api/src/web3adapter/mappings/file.mapping.json
  • platforms/esigner/client/src/lib/stores/files.ts
  • platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte
  • platforms/file-manager/api/package.json
  • platforms/file-manager/api/src/controllers/FileController.ts
  • platforms/file-manager/api/src/controllers/WebhookController.ts
  • platforms/file-manager/api/src/database/entities/File.ts
  • platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
  • platforms/file-manager/api/src/index.ts
  • platforms/file-manager/api/src/services/FileService.ts
  • platforms/file-manager/api/src/services/S3Service.ts
  • platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
  • platforms/file-manager/client/src/lib/stores/files.ts
  • platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
  • platforms/file-manager/client/src/routes/(protected)/files/[id]/+page.svelte
  • services/ontology/schemas/file.json

📝 Walkthrough

Walkthrough

This PR implements S3-compatible object storage integration for file uploads across both esigner and file-manager platforms. It introduces presigned upload endpoints, a new S3Service class for AWS SDK operations, modifies the File entity to add a url field and make data nullable, updates database schemas via migrations, and refactors client-side upload flows from server-side multipart to direct S3 uploads with presigned URLs.

Changes

Cohort / File(s)Summary
Package Dependencies
platforms/esigner/api/package.json, platforms/file-manager/api/package.json
Added AWS SDK dependencies (@aws-sdk/client-s3, @aws-sdk/s3-request-presigner) and repositioned signature-validator within the dependencies block.
S3 Service Integration
platforms/esigner/api/src/services/S3Service.ts, platforms/file-manager/api/src/services/S3Service.ts
New S3Service class encapsulating AWS SDK v3 operations with lazy initialization, presigned URL generation, bucket/region derivation, and methods for upload URLs, public URLs, object metadata, and streaming (identical implementations across both platforms).
File Controller Upload Endpoints
platforms/esigner/api/src/controllers/FileController.ts, platforms/file-manager/api/src/controllers/FileController.ts
Added presignUpload and confirmUpload methods to handle S3 presigned URL generation and upload confirmation; presignUpload generates uploadUrl and fileId, while confirmUpload validates inputs, fetches S3 object metadata (MD5 hash), and creates file records.
File Service S3 Integration
platforms/esigner/api/src/services/FileService.ts, platforms/file-manager/api/src/services/FileService.ts
Added s3Service instance, createFileWithUrl method to create file records with S3 URLs, and updated getDocumentsWithStatus/getFileMetadataById to include url field in response payloads.
Webhook File Handling
platforms/esigner/api/src/controllers/WebhookController.ts, platforms/file-manager/api/src/controllers/WebhookController.ts
Updated file creation and update paths to handle URL assignment from local.data.url and initialize file data directly with base64 decoding (when present) instead of temporary buffer construction.
File Entity & Schema
platforms/esigner/api/src/database/entities/File.ts, platforms/file-manager/api/src/database/entities/File.ts
Made data field nullable (Buffer | null) and added new optional url field (string | null) as text column with nullable: true.
Database Migrations
platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts, platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
Added TypeORM migrations that add url column to files table and make data column nullable, with reversible up/down flows for both platforms.
API Routes
platforms/esigner/api/src/index.ts, platforms/file-manager/api/src/index.ts
Added POST /api/files/presign and POST /api/files/confirm routes guarded by authGuard, enabling presigned upload and upload confirmation workflows.
Web3 Adapter Mappings
platforms/esigner/api/src/web3adapter/mappings/file.mapping.json, platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
Added "url": "url" field mapping in localToUniversalMap for File entities.
Client File Store
platforms/esigner/client/src/lib/stores/files.ts, platforms/file-manager/client/src/lib/stores/files.ts
Added optional url field to Document/File interface; refactored upload flow from multipart FormData to three-step presigned S3 workflow (presign → upload → confirm) with error handling for non-OK S3 responses.
Client Upload UI
platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte, platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
Updated file preview and download logic to prefer file.url when available, falling back to API-based blob fetching; refactored upload handler to use presigned URL workflow with XMLHttpRequest PUT and progress tracking.
Ontology Schema
services/ontology/schemas/file.json
Added url property (type: ["string", "null"], format: "uri") and updated data description to mark it as legacy.

Sequence Diagram(s)

sequenceDiagram
participant Client
participant Controller
participant S3Service
participant S3 as S3 Storage
participant FileService
participant Database
rect rgba(100, 150, 255, 0.5)
Note over Client,Database: Presign Upload Flow
Client->>Controller: POST /api/files/presign<br/>(filename, mimeType, size)
Controller->>S3Service: generateKey(userId, fileId, filename)
S3Service-->>Controller: key
Controller->>S3Service: generateUploadUrl(key, contentType)
S3Service-->>Controller: presigned uploadUrl
Controller-->>Client: { uploadUrl, key, fileId }
end
rect rgba(150, 200, 100, 0.5)
Note over Client,S3: Direct S3 Upload
Client->>S3: PUT uploadUrl<br/>(file data, Content-Type, x-amz-acl)
S3-->>Client: 200 OK
end
rect rgba(200, 150, 100, 0.5)
Note over Client,Database: Confirm Upload Flow
Client->>Controller: POST /api/files/confirm<br/>(key, fileId, filename, mimeType, size)
Controller->>S3Service: headObject(key)
S3Service->>S3: HEAD object
S3-->>S3Service: { contentLength, etag }
S3Service-->>Controller: metadata with md5Hash
Controller->>S3Service: getPublicUrl(key)
S3Service-->>Controller: public url
Controller->>FileService: createFileWithUrl(..., url)
FileService->>Database: save File record
Database-->>FileService: File entity
FileService-->>Controller: File
Controller-->>Client: { file metadata }
end
rect rgba(100, 200, 200, 0.5)
Note over Client,S3: Download/Preview Flow
alt file.url exists
Client->>Client: use file.url<br/>(direct redirect or open)
else file.url is null
Client->>Controller: GET /api/files/[id]/download
Controller->>FileService: getFileDataStream(fileId)
FileService->>Database: fetch File
Database-->>FileService: File record
FileService-->>Controller: Readable stream
Controller-->>Client: file data
end
end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: per user limit #922: Modifies file upload and quota logic in FileController and FileService, touching the same upload-path handlers affected by presign/confirm endpoints.
  • fix: file manager limits #663: Updates FileController in file-manager API with upload-related handler and flow changes, directly overlapping with this PR's controller modifications.
  • feat:(file-manager) fix download speeds #756: Adds server-side file streaming and ZIP export features to file-manager backend (FileController and FileService), operating on the same file storage and retrieval domain.

Suggested reviewers

  • sosweetham
  • xPathin

Poem

🐰 A presigned dance on S3's stage,
No more uploads tied to server's cage,
Direct to storage, swift and clean,
With URLs stored—the finest scene,
File flows that flow, now cloud-aligned! 🌧️✨

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/object-store-for-files
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review March 16, 2026 11:45
@coodos
coodos merged commit d2efd8b into mainMar 16, 2026
3 of 4 checks passed
@coodos
coodos deleted the feat/object-store-for-files branch March 16, 2026 11:47
@coderabbitaicoderabbitaiBot mentioned this pull request Apr 6, 2026
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@coodos
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: switch to s3 storage - #926

Merged
coodos merged 7 commits into
mainfrom
feat/object-store-for-files
Mar 16, 2026
Merged

feat: switch to s3 storage#926
coodos merged 7 commits into
mainfrom
feat/object-store-for-files

Conversation

@coodos

@coodoscoodos commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description of change

Change from db to object store

Issue Number

Type of change

  • Update (a change which updates existing functionality)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • New Features
    • Integrated S3-compatible cloud storage for file uploads
    • Implemented presigned upload flow for direct client-to-cloud transfers
    • Files now accessible via direct URLs for improved performance on downloads and previews

@coderabbitai

coderabbitaiBot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 752c3fcb-d800-43ee-8d82-ef7eadc43d95

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce20f5 and 5eb8c47.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (24)
  • platforms/esigner/api/package.json
  • platforms/esigner/api/src/controllers/FileController.ts
  • platforms/esigner/api/src/controllers/WebhookController.ts
  • platforms/esigner/api/src/database/entities/File.ts
  • platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts
  • platforms/esigner/api/src/index.ts
  • platforms/esigner/api/src/services/FileService.ts
  • platforms/esigner/api/src/services/S3Service.ts
  • platforms/esigner/api/src/web3adapter/mappings/file.mapping.json
  • platforms/esigner/client/src/lib/stores/files.ts
  • platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte
  • platforms/file-manager/api/package.json
  • platforms/file-manager/api/src/controllers/FileController.ts
  • platforms/file-manager/api/src/controllers/WebhookController.ts
  • platforms/file-manager/api/src/database/entities/File.ts
  • platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
  • platforms/file-manager/api/src/index.ts
  • platforms/file-manager/api/src/services/FileService.ts
  • platforms/file-manager/api/src/services/S3Service.ts
  • platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
  • platforms/file-manager/client/src/lib/stores/files.ts
  • platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
  • platforms/file-manager/client/src/routes/(protected)/files/[id]/+page.svelte
  • services/ontology/schemas/file.json

📝 Walkthrough

Walkthrough

This PR implements S3-compatible object storage integration for file uploads across both esigner and file-manager platforms. It introduces presigned upload endpoints, a new S3Service class for AWS SDK operations, modifies the File entity to add a url field and make data nullable, updates database schemas via migrations, and refactors client-side upload flows from server-side multipart to direct S3 uploads with presigned URLs.

Changes

Cohort / File(s)Summary
Package Dependencies
platforms/esigner/api/package.json, platforms/file-manager/api/package.json
Added AWS SDK dependencies (@aws-sdk/client-s3, @aws-sdk/s3-request-presigner) and repositioned signature-validator within the dependencies block.
S3 Service Integration
platforms/esigner/api/src/services/S3Service.ts, platforms/file-manager/api/src/services/S3Service.ts
New S3Service class encapsulating AWS SDK v3 operations with lazy initialization, presigned URL generation, bucket/region derivation, and methods for upload URLs, public URLs, object metadata, and streaming (identical implementations across both platforms).
File Controller Upload Endpoints
platforms/esigner/api/src/controllers/FileController.ts, platforms/file-manager/api/src/controllers/FileController.ts
Added presignUpload and confirmUpload methods to handle S3 presigned URL generation and upload confirmation; presignUpload generates uploadUrl and fileId, while confirmUpload validates inputs, fetches S3 object metadata (MD5 hash), and creates file records.
File Service S3 Integration
platforms/esigner/api/src/services/FileService.ts, platforms/file-manager/api/src/services/FileService.ts
Added s3Service instance, createFileWithUrl method to create file records with S3 URLs, and updated getDocumentsWithStatus/getFileMetadataById to include url field in response payloads.
Webhook File Handling
platforms/esigner/api/src/controllers/WebhookController.ts, platforms/file-manager/api/src/controllers/WebhookController.ts
Updated file creation and update paths to handle URL assignment from local.data.url and initialize file data directly with base64 decoding (when present) instead of temporary buffer construction.
File Entity & Schema
platforms/esigner/api/src/database/entities/File.ts, platforms/file-manager/api/src/database/entities/File.ts
Made data field nullable (Buffer | null) and added new optional url field (string | null) as text column with nullable: true.
Database Migrations
platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts, platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
Added TypeORM migrations that add url column to files table and make data column nullable, with reversible up/down flows for both platforms.
API Routes
platforms/esigner/api/src/index.ts, platforms/file-manager/api/src/index.ts
Added POST /api/files/presign and POST /api/files/confirm routes guarded by authGuard, enabling presigned upload and upload confirmation workflows.
Web3 Adapter Mappings
platforms/esigner/api/src/web3adapter/mappings/file.mapping.json, platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
Added "url": "url" field mapping in localToUniversalMap for File entities.
Client File Store
platforms/esigner/client/src/lib/stores/files.ts, platforms/file-manager/client/src/lib/stores/files.ts
Added optional url field to Document/File interface; refactored upload flow from multipart FormData to three-step presigned S3 workflow (presign → upload → confirm) with error handling for non-OK S3 responses.
Client Upload UI
platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte, platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
Updated file preview and download logic to prefer file.url when available, falling back to API-based blob fetching; refactored upload handler to use presigned URL workflow with XMLHttpRequest PUT and progress tracking.
Ontology Schema
services/ontology/schemas/file.json
Added url property (type: ["string", "null"], format: "uri") and updated data description to mark it as legacy.

Sequence Diagram(s)

sequenceDiagram
participant Client
participant Controller
participant S3Service
participant S3 as S3 Storage
participant FileService
participant Database
rect rgba(100, 150, 255, 0.5)
Note over Client,Database: Presign Upload Flow
Client->>Controller: POST /api/files/presign<br/>(filename, mimeType, size)
Controller->>S3Service: generateKey(userId, fileId, filename)
S3Service-->>Controller: key
Controller->>S3Service: generateUploadUrl(key, contentType)
S3Service-->>Controller: presigned uploadUrl
Controller-->>Client: { uploadUrl, key, fileId }
end
rect rgba(150, 200, 100, 0.5)
Note over Client,S3: Direct S3 Upload
Client->>S3: PUT uploadUrl<br/>(file data, Content-Type, x-amz-acl)
S3-->>Client: 200 OK
end
rect rgba(200, 150, 100, 0.5)
Note over Client,Database: Confirm Upload Flow
Client->>Controller: POST /api/files/confirm<br/>(key, fileId, filename, mimeType, size)
Controller->>S3Service: headObject(key)
S3Service->>S3: HEAD object
S3-->>S3Service: { contentLength, etag }
S3Service-->>Controller: metadata with md5Hash
Controller->>S3Service: getPublicUrl(key)
S3Service-->>Controller: public url
Controller->>FileService: createFileWithUrl(..., url)
FileService->>Database: save File record
Database-->>FileService: File entity
FileService-->>Controller: File
Controller-->>Client: { file metadata }
end
rect rgba(100, 200, 200, 0.5)
Note over Client,S3: Download/Preview Flow
alt file.url exists
Client->>Client: use file.url<br/>(direct redirect or open)
else file.url is null
Client->>Controller: GET /api/files/[id]/download
Controller->>FileService: getFileDataStream(fileId)
FileService->>Database: fetch File
Database-->>FileService: File record
FileService-->>Controller: Readable stream
Controller-->>Client: file data
end
end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: per user limit #922: Modifies file upload and quota logic in FileController and FileService, touching the same upload-path handlers affected by presign/confirm endpoints.
  • fix: file manager limits #663: Updates FileController in file-manager API with upload-related handler and flow changes, directly overlapping with this PR's controller modifications.
  • feat:(file-manager) fix download speeds #756: Adds server-side file streaming and ZIP export features to file-manager backend (FileController and FileService), operating on the same file storage and retrieval domain.

Suggested reviewers

  • sosweetham
  • xPathin

Poem

🐰 A presigned dance on S3's stage,
No more uploads tied to server's cage,
Direct to storage, swift and clean,
With URLs stored—the finest scene,
File flows that flow, now cloud-aligned! 🌧️✨

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/object-store-for-files
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review March 16, 2026 11:45
@coodos
coodos merged commit d2efd8b into mainMar 16, 2026
3 of 4 checks passed
@coodos
coodos deleted the feat/object-store-for-files branch March 16, 2026 11:47
@coderabbitaicoderabbitaiBot mentioned this pull request Apr 6, 2026
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@coodos
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat: switch to s3 storage - #926

Merged
coodos merged 7 commits into
mainfrom
feat/object-store-for-files
Mar 16, 2026
Merged

feat: switch to s3 storage#926
coodos merged 7 commits into
mainfrom
feat/object-store-for-files

Conversation

@coodos

@coodoscoodos commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Description of change

Change from db to object store

Issue Number

Type of change

  • Update (a change which updates existing functionality)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • New Features
    • Integrated S3-compatible cloud storage for file uploads
    • Implemented presigned upload flow for direct client-to-cloud transfers
    • Files now accessible via direct URLs for improved performance on downloads and previews

@coderabbitai

coderabbitaiBot commented Mar 16, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 752c3fcb-d800-43ee-8d82-ef7eadc43d95

📥 Commits

Reviewing files that changed from the base of the PR and between 7ce20f5 and 5eb8c47.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (24)
  • platforms/esigner/api/package.json
  • platforms/esigner/api/src/controllers/FileController.ts
  • platforms/esigner/api/src/controllers/WebhookController.ts
  • platforms/esigner/api/src/database/entities/File.ts
  • platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts
  • platforms/esigner/api/src/index.ts
  • platforms/esigner/api/src/services/FileService.ts
  • platforms/esigner/api/src/services/S3Service.ts
  • platforms/esigner/api/src/web3adapter/mappings/file.mapping.json
  • platforms/esigner/client/src/lib/stores/files.ts
  • platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte
  • platforms/file-manager/api/package.json
  • platforms/file-manager/api/src/controllers/FileController.ts
  • platforms/file-manager/api/src/controllers/WebhookController.ts
  • platforms/file-manager/api/src/database/entities/File.ts
  • platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
  • platforms/file-manager/api/src/index.ts
  • platforms/file-manager/api/src/services/FileService.ts
  • platforms/file-manager/api/src/services/S3Service.ts
  • platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
  • platforms/file-manager/client/src/lib/stores/files.ts
  • platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
  • platforms/file-manager/client/src/routes/(protected)/files/[id]/+page.svelte
  • services/ontology/schemas/file.json

📝 Walkthrough

Walkthrough

This PR implements S3-compatible object storage integration for file uploads across both esigner and file-manager platforms. It introduces presigned upload endpoints, a new S3Service class for AWS SDK operations, modifies the File entity to add a url field and make data nullable, updates database schemas via migrations, and refactors client-side upload flows from server-side multipart to direct S3 uploads with presigned URLs.

Changes

Cohort / File(s)Summary
Package Dependencies
platforms/esigner/api/package.json, platforms/file-manager/api/package.json
Added AWS SDK dependencies (@aws-sdk/client-s3, @aws-sdk/s3-request-presigner) and repositioned signature-validator within the dependencies block.
S3 Service Integration
platforms/esigner/api/src/services/S3Service.ts, platforms/file-manager/api/src/services/S3Service.ts
New S3Service class encapsulating AWS SDK v3 operations with lazy initialization, presigned URL generation, bucket/region derivation, and methods for upload URLs, public URLs, object metadata, and streaming (identical implementations across both platforms).
File Controller Upload Endpoints
platforms/esigner/api/src/controllers/FileController.ts, platforms/file-manager/api/src/controllers/FileController.ts
Added presignUpload and confirmUpload methods to handle S3 presigned URL generation and upload confirmation; presignUpload generates uploadUrl and fileId, while confirmUpload validates inputs, fetches S3 object metadata (MD5 hash), and creates file records.
File Service S3 Integration
platforms/esigner/api/src/services/FileService.ts, platforms/file-manager/api/src/services/FileService.ts
Added s3Service instance, createFileWithUrl method to create file records with S3 URLs, and updated getDocumentsWithStatus/getFileMetadataById to include url field in response payloads.
Webhook File Handling
platforms/esigner/api/src/controllers/WebhookController.ts, platforms/file-manager/api/src/controllers/WebhookController.ts
Updated file creation and update paths to handle URL assignment from local.data.url and initialize file data directly with base64 decoding (when present) instead of temporary buffer construction.
File Entity & Schema
platforms/esigner/api/src/database/entities/File.ts, platforms/file-manager/api/src/database/entities/File.ts
Made data field nullable (Buffer | null) and added new optional url field (string | null) as text column with nullable: true.
Database Migrations
platforms/esigner/api/src/database/migrations/1773657072411-addurl.ts, platforms/file-manager/api/src/database/migrations/1773657041144-addurl.ts
Added TypeORM migrations that add url column to files table and make data column nullable, with reversible up/down flows for both platforms.
API Routes
platforms/esigner/api/src/index.ts, platforms/file-manager/api/src/index.ts
Added POST /api/files/presign and POST /api/files/confirm routes guarded by authGuard, enabling presigned upload and upload confirmation workflows.
Web3 Adapter Mappings
platforms/esigner/api/src/web3adapter/mappings/file.mapping.json, platforms/file-manager/api/src/web3adapter/mappings/file.mapping.json
Added "url": "url" field mapping in localToUniversalMap for File entities.
Client File Store
platforms/esigner/client/src/lib/stores/files.ts, platforms/file-manager/client/src/lib/stores/files.ts
Added optional url field to Document/File interface; refactored upload flow from multipart FormData to three-step presigned S3 workflow (presign → upload → confirm) with error handling for non-OK S3 responses.
Client Upload UI
platforms/esigner/client/src/routes/(protected)/files/[id]/+page.svelte, platforms/file-manager/client/src/routes/(protected)/files/+page.svelte
Updated file preview and download logic to prefer file.url when available, falling back to API-based blob fetching; refactored upload handler to use presigned URL workflow with XMLHttpRequest PUT and progress tracking.
Ontology Schema
services/ontology/schemas/file.json
Added url property (type: ["string", "null"], format: "uri") and updated data description to mark it as legacy.

Sequence Diagram(s)

sequenceDiagram
participant Client
participant Controller
participant S3Service
participant S3 as S3 Storage
participant FileService
participant Database
rect rgba(100, 150, 255, 0.5)
Note over Client,Database: Presign Upload Flow
Client->>Controller: POST /api/files/presign<br/>(filename, mimeType, size)
Controller->>S3Service: generateKey(userId, fileId, filename)
S3Service-->>Controller: key
Controller->>S3Service: generateUploadUrl(key, contentType)
S3Service-->>Controller: presigned uploadUrl
Controller-->>Client: { uploadUrl, key, fileId }
end
rect rgba(150, 200, 100, 0.5)
Note over Client,S3: Direct S3 Upload
Client->>S3: PUT uploadUrl<br/>(file data, Content-Type, x-amz-acl)
S3-->>Client: 200 OK
end
rect rgba(200, 150, 100, 0.5)
Note over Client,Database: Confirm Upload Flow
Client->>Controller: POST /api/files/confirm<br/>(key, fileId, filename, mimeType, size)
Controller->>S3Service: headObject(key)
S3Service->>S3: HEAD object
S3-->>S3Service: { contentLength, etag }
S3Service-->>Controller: metadata with md5Hash
Controller->>S3Service: getPublicUrl(key)
S3Service-->>Controller: public url
Controller->>FileService: createFileWithUrl(..., url)
FileService->>Database: save File record
Database-->>FileService: File entity
FileService-->>Controller: File
Controller-->>Client: { file metadata }
end
rect rgba(100, 200, 200, 0.5)
Note over Client,S3: Download/Preview Flow
alt file.url exists
Client->>Client: use file.url<br/>(direct redirect or open)
else file.url is null
Client->>Controller: GET /api/files/[id]/download
Controller->>FileService: getFileDataStream(fileId)
FileService->>Database: fetch File
Database-->>FileService: File record
FileService-->>Controller: Readable stream
Controller-->>Client: file data
end
end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • feat: per user limit #922: Modifies file upload and quota logic in FileController and FileService, touching the same upload-path handlers affected by presign/confirm endpoints.
  • fix: file manager limits #663: Updates FileController in file-manager API with upload-related handler and flow changes, directly overlapping with this PR's controller modifications.
  • feat:(file-manager) fix download speeds #756: Adds server-side file streaming and ZIP export features to file-manager backend (FileController and FileService), operating on the same file storage and retrieval domain.

Suggested reviewers

  • sosweetham
  • xPathin

Poem

🐰 A presigned dance on S3's stage,
No more uploads tied to server's cage,
Direct to storage, swift and clean,
With URLs stored—the finest scene,
File flows that flow, now cloud-aligned! 🌧️✨

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/object-store-for-files
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review March 16, 2026 11:45
@coodos
coodos merged commit d2efd8b into mainMar 16, 2026
3 of 4 checks passed
@coodos
coodos deleted the feat/object-store-for-files branch March 16, 2026 11:47
@coderabbitaicoderabbitaiBot mentioned this pull request Apr 6, 2026
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@coodos