Uh oh!
There was an error while loading. Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork 3.8k
fix(files): reserve image layout space so images stop reflowing on load#6299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Uh oh!
There was an error while loading. Please reload this page.
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
ba2bf7a
fix(files): reserve image layout space so images stop reflowing on load
waleedlatif1 1c026d5
fix(files): address review — reserve on stale memo, clear dims on con…
waleedlatif1 770c1f5
fix(files): re-derive image dimensions on content swap instead of cle…
waleedlatif1 17499b1
fix(files): self-heal image dimensions from the browser instead of se…
waleedlatif1 ba8507c
fix(files): clear image dimensions on content swap (completes self-heal)
waleedlatif1 cd1b2be
fix(files): guard dimension writes by content key so a stale PATCH ca…
waleedlatif1 94348f0
chore(files): fix stale route TSDoc and hoist a regex literal (cleanu…
waleedlatif1 497b223
fix(files): reflect the content-version guard outcome in the dimensio…
waleedlatif1 4cabb83
fix(files): reconcile the cache when a dimension write is content-ver…
waleedlatif1 d2b1195
docs(files): align stale dimension docs with the overwrite/self-heal …
waleedlatif1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Diff view
There are no files selected for viewing
93 changes: 93 additions & 0 deletions
93 apps/sim/app/api/workspaces/[id]/files/[fileId]/dimensions/route.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| /** | ||
| * @vitest-environment node | ||
| */ | ||
| import { authMockFns, permissionsMock, permissionsMockFns } from '@sim/testing' | ||
| import { NextRequest } from 'next/server' | ||
| import { beforeEach, describe, expect, it, vi } from 'vitest' | ||
| const { mockUpdateWorkspaceFileDimensions } = vi.hoisted(() => ({ | ||
| mockUpdateWorkspaceFileDimensions: vi.fn(), | ||
| })) | ||
| vi.mock('@/lib/uploads/contexts/workspace/workspace-file-manager', () => ({ | ||
| updateWorkspaceFileDimensions: mockUpdateWorkspaceFileDimensions, | ||
| })) | ||
| vi.mock('@/lib/workspaces/permissions/utils', () => permissionsMock) | ||
| const WS = '7727ef3f-8cf6-4686-b063-2bb006a10785' | ||
| const FILE = 'wf_abc123' | ||
| const KEY = 'workspace/7727ef3f/screenshot.png' | ||
| import { PATCH } from '@/app/api/workspaces/[id]/files/[fileId]/dimensions/route' | ||
| const routeContext = { params: Promise.resolve({ id: WS, fileId: FILE }) } | ||
| function buildRequest(body: unknown): NextRequest { | ||
| return new NextRequest(`http://localhost/api/workspaces/${WS}/files/${FILE}/dimensions`, { | ||
| method: 'PATCH', | ||
| headers: { 'content-type': 'application/json' }, | ||
| body: JSON.stringify(body), | ||
| }) | ||
| } | ||
| describe('PATCH /api/workspaces/[id]/files/[fileId]/dimensions', () => { | ||
| beforeEach(() => { | ||
| vi.clearAllMocks() | ||
| authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } }) | ||
| permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('write') | ||
| mockUpdateWorkspaceFileDimensions.mockResolvedValue(true) | ||
| }) | ||
| it('stores dimensions for a writer, keyed to the content version', async () => { | ||
| const res = await PATCH(buildRequest({ key: KEY, width: 1600, height: 900 }), routeContext) | ||
| expect(res.status).toBe(200) | ||
| expect(await res.json()).toEqual({ success: true }) | ||
| expect(mockUpdateWorkspaceFileDimensions).toHaveBeenCalledWith(WS, FILE, { | ||
| key: KEY, | ||
| width: 1600, | ||
| height: 900, | ||
| }) | ||
| }) | ||
| it('allows an admin', async () => { | ||
| permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('admin') | ||
| const res = await PATCH(buildRequest({ key: KEY, width: 10, height: 20 }), routeContext) | ||
| expect(res.status).toBe(200) | ||
| expect(mockUpdateWorkspaceFileDimensions).toHaveBeenCalledOnce() | ||
| }) | ||
| it('reports success:false when the content-version guard rejects the write (key changed)', async () => { | ||
| mockUpdateWorkspaceFileDimensions.mockResolvedValue(false) | ||
| const res = await PATCH(buildRequest({ key: KEY, width: 10, height: 20 }), routeContext) | ||
| expect(res.status).toBe(200) | ||
| expect(await res.json()).toEqual({ success: false }) | ||
| }) | ||
| it('rejects an unauthenticated caller before touching the DB', async () => { | ||
| authMockFns.mockGetSession.mockResolvedValue(null) | ||
| const res = await PATCH(buildRequest({ key: KEY, width: 10, height: 10 }), routeContext) | ||
| expect(res.status).toBe(401) | ||
| expect(mockUpdateWorkspaceFileDimensions).not.toHaveBeenCalled() | ||
| }) | ||
| it('rejects a read-only member (backfill requires write)', async () => { | ||
| permissionsMockFns.mockGetUserEntityPermissions.mockResolvedValue('read') | ||
| const res = await PATCH(buildRequest({ key: KEY, width: 10, height: 10 }), routeContext) | ||
| expect(res.status).toBe(403) | ||
| expect(mockUpdateWorkspaceFileDimensions).not.toHaveBeenCalled() | ||
| }) | ||
| it('rejects a missing key or non-positive / non-integer dimensions', async () => { | ||
| for (const body of [ | ||
| { width: 10, height: 10 }, // missing key | ||
| { key: KEY, width: 0, height: 10 }, | ||
| { key: KEY, width: 10, height: -5 }, | ||
| { key: KEY, width: 10.5, height: 10 }, | ||
| { key: KEY, width: 10 }, | ||
| ]) { | ||
| const res = await PATCH(buildRequest(body), routeContext) | ||
| expect(res.status).toBe(400) | ||
| } | ||
| expect(mockUpdateWorkspaceFileDimensions).not.toHaveBeenCalled() | ||
| }) | ||
| }) |
57 changes: 57 additions & 0 deletions
57 apps/sim/app/api/workspaces/[id]/files/[fileId]/dimensions/route.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| import { createLogger } from '@sim/logger' | ||
| import { getErrorMessage } from '@sim/utils/errors' | ||
| import { type NextRequest, NextResponse } from 'next/server' | ||
| import { updateWorkspaceFileDimensionsContract } from '@/lib/api/contracts/workspace-files' | ||
| import { parseRequest } from '@/lib/api/server' | ||
| import { getSession } from '@/lib/auth' | ||
| import { withRouteHandler } from '@/lib/core/utils/with-route-handler' | ||
| import { updateWorkspaceFileDimensions } from '@/lib/uploads/contexts/workspace/workspace-file-manager' | ||
| import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' | ||
| const logger = createLogger('WorkspaceFileDimensionsAPI') | ||
| /** | ||
| * PATCH /api/workspaces/[id]/files/[fileId]/dimensions | ||
| * | ||
| * Store an image file's intrinsic pixel dimensions — a pure rendering hint the editor uses to reserve | ||
| * layout space before the image loads. Requires write permission. The write commits whenever the row | ||
| * still holds the measured storage key, overwriting any stale value so a wrong size self-corrects; the | ||
| * client reports only on a real mismatch, so this is not storm-y despite not being a backfill-once no-op. | ||
| */ | ||
| export const PATCH = withRouteHandler( | ||
| async (request: NextRequest, context: { params: Promise<{ id: string; fileId: string }> }) => { | ||
| const session = await getSession() | ||
| if (!session?.user?.id) { | ||
| return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) | ||
| } | ||
| const parsed = await parseRequest(updateWorkspaceFileDimensionsContract, request, context) | ||
| if (!parsed.success) return parsed.response | ||
| const { id: workspaceId, fileId } = parsed.data.params | ||
| const { key, width, height } = parsed.data.body | ||
| const permission = await getUserEntityPermissions(session.user.id, 'workspace', workspaceId) | ||
| if (permission !== 'admin' && permission !== 'write') { | ||
| return NextResponse.json({ error: 'Insufficient permissions' }, { status: 403 }) | ||
| } | ||
| try { | ||
| // `written` is false when the content-version guard rejected the write (the row's storage key no | ||
| // longer matches the key the client measured — the content was replaced since). That is not an | ||
| // error; the client's next measurement, once its file list has the new key, persists correctly. | ||
| const written = await updateWorkspaceFileDimensions(workspaceId, fileId, { | ||
| key, | ||
| width, | ||
| height, | ||
| }) | ||
| return NextResponse.json({ success: written }) | ||
| } catch (error) { | ||
| logger.error('Failed to store workspace file dimensions', { | ||
| workspaceId, | ||
| fileId, | ||
| error: getErrorMessage(error), | ||
| }) | ||
| return NextResponse.json({ error: 'Failed to update dimensions' }, { status: 500 }) | ||
| } | ||
| } | ||
| ) |
11 changes: 8 additions & 3 deletions
11 apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/file-viewer.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
76 changes: 64 additions & 12 deletions
76 ...m/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/image.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,18 @@ | ||
| import { useEffect, useRef, useState } from 'react' | ||
| import { type CSSProperties, useEffect, useMemo, useRef, useState } from 'react' | ||
| import { cn } from '@sim/emcn' | ||
| import { NodeSelection, Plugin } from '@tiptap/pm/state' | ||
| import type { ReactNodeViewProps } from '@tiptap/react' | ||
| import { NodeViewWrapper, ReactNodeViewRenderer } from '@tiptap/react' | ||
| import { useFileContentSource } from '@/hooks/use-file-content-source' | ||
| import { type ImageDimensions, useFileContentSource } from '@/hooks/use-file-content-source' | ||
| import { MarkdownImage } from './image-schema' | ||
| import { normalizeLinkHref } from './markdown-fidelity' | ||
| import { useEditorEditable } from './use-editor-editable' | ||
| const MIN_WIDTH = 64 | ||
| /** A bare pixel count (`"640"`) that needs a `px` suffix, vs. an already-unit'd width (`"50%"`). */ | ||
| const BARE_PIXEL_WIDTH = /^\d+$/ | ||
| /** | ||
| * Drag-to-resize image node view (handle at the bottom-right, revealed on selection). Dragging | ||
| * commits the new pixel width to the `width` attribute, which serializes to `<img width>`. | ||
| @@ -24,6 +27,11 @@ function ResizableImageView({ node, updateAttributes, selected, editor }: ReactN | ||
| const [dragWidth, setDragWidth] = useState<number | null>(null) | ||
| /** Whether the current src failed to load; reset on src change so a retried/edited src can load. */ | ||
| const [failed, setFailed] = useState(false) | ||
| /** | ||
| * Intrinsic dimensions measured from the loaded image — holds the aspect-ratio box for THIS view when | ||
| * the content source has no stored dimensions yet (the first-ever view of an image). Reset on src change. | ||
| */ | ||
| const [measuredDimensions, setMeasuredDimensions] = useState<ImageDimensions | null>(null) | ||
| const attrs = node.attrs as { | ||
| src?: string | ||
| alt?: string | ||
| @@ -33,7 +41,16 @@ function ResizableImageView({ node, updateAttributes, selected, editor }: ReactN | ||
| } | ||
| useEffect(() => () => dragAbortRef.current?.abort(), []) | ||
| useEffect(() => setFailed(false), [attrs.src]) | ||
| // Reset the load-failure flag and this-session measurement when the src changes — adjusted during | ||
| // render (not in an effect) so the previous image's aspect-ratio box never paints for a frame. A `key` | ||
| // remount isn't available here: TipTap owns this node view's instantiation. | ||
| const [prevSrc, setPrevSrc] = useState(attrs.src) | ||
| if (prevSrc !== attrs.src) { | ||
| setPrevSrc(attrs.src) | ||
| setFailed(false) | ||
| setMeasuredDimensions(null) | ||
| } | ||
| const startResize = (event: React.PointerEvent) => { | ||
| event.preventDefault() | ||
| @@ -69,16 +86,34 @@ function ResizableImageView({ node, updateAttributes, selected, editor }: ReactN | ||
| } | ||
| const committedWidth = attrs.width | ||
| ? /^\d+$/.test(attrs.width) | ||
| ? BARE_PIXEL_WIDTH.test(attrs.width) | ||
| ? `${attrs.width}px` | ||
| : attrs.width | ||
| : undefined | ||
| const widthStyle = | ||
| // Stored intrinsic dimensions reserve the box on the very first render. Memoized on the src (not the | ||
| // live drag width) so a resize drag never re-scans the file list. Falls back to what we measured on | ||
| // load this session for a first-ever view the metadata hasn't caught up on. | ||
| const storedDimensions = useMemo( | ||
| () => source.getImageDimensions?.(attrs.src) ?? null, | ||
| [source, attrs.src] | ||
| ) | ||
waleedlatif1 marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| // The browser's post-load measurement is authoritative — EXIF-corrected, and correct even when the | ||
| // stored value is stale (e.g. left over after the file's content was replaced) — so it wins once | ||
| // available; stored metadata only reserves the box pre-load. Equal in the common case, so no shift. | ||
| const intrinsicDimensions = measuredDimensions ?? storedDimensions | ||
| const displayWidth = | ||
| dragWidth !== null | ||
| ? { width: `${dragWidth}px` } | ||
| : committedWidth | ||
| ? { width: committedWidth } | ||
| : undefined | ||
| ? `${dragWidth}px` | ||
| : (committedWidth ?? (intrinsicDimensions ? `${intrinsicDimensions.width}px` : undefined)) | ||
| // width + aspect-ratio (with `max-w-full`/`h-auto` from the class list) reserves a responsive box the | ||
| // image can't reflow into, per the CLS-avoidance pattern for known-ratio responsive images. React drops | ||
| // the undefined keys, so an unmeasured image simply gets no reservation (its prior behavior). | ||
| const imageStyle: CSSProperties = { | ||
| width: displayWidth, | ||
| aspectRatio: intrinsicDimensions | ||
| ? `${intrinsicDimensions.width} / ${intrinsicDimensions.height}` | ||
| : undefined, | ||
| } | ||
| // Sanitize the linked-image target before rendering the anchor — a parsed markdown href is | ||
| // untrusted and could be `javascript:`/`data:`; an unsafe value drops the link (image only). | ||
| @@ -99,11 +134,28 @@ function ResizableImageView({ node, updateAttributes, selected, editor }: ReactN | ||
| // the resize button sits outside this element, so it keeps its own pointer behavior.) | ||
| draggable={editable} | ||
| data-drag-handle={editable ? '' : undefined} | ||
| style={widthStyle} | ||
| style={imageStyle} | ||
| onError={() => setFailed(true)} | ||
| onLoad={() => setFailed(false)} | ||
| onLoad={(event) => { | ||
| setFailed(false) | ||
| const { naturalWidth, naturalHeight } = event.currentTarget | ||
| if (naturalWidth <= 0 || naturalHeight <= 0) return | ||
| // The browser's measurement is authoritative. Reserve from it and persist whenever the stored | ||
| // metadata is absent or disagrees (EXIF-rotated, or stale after a content swap), so a wrong value | ||
| // self-corrects instead of sticking. Compare the memoized `storedDimensions` the render uses, NOT | ||
| // a fresh cache read — the memo is non-reactive, and this keeps the guard consistent with render. | ||
| if ( | ||
| storedDimensions && | ||
| storedDimensions.width === naturalWidth && | ||
| storedDimensions.height === naturalHeight | ||
| ) { | ||
| return | ||
| } | ||
| setMeasuredDimensions({ width: naturalWidth, height: naturalHeight }) | ||
| source.reportImageDimensions?.(attrs.src, { width: naturalWidth, height: naturalHeight }) | ||
| }} | ||
| className={cn( | ||
| 'block max-w-full rounded-lg border border-[var(--border)]', | ||
| 'block h-auto max-w-full rounded-lg border border-[var(--border)]', | ||
| editable && 'cursor-grab', | ||
| failed && | ||
| 'min-h-[72px] min-w-[140px] bg-[var(--surface-5)] p-3 text-[var(--text-muted)] text-caption' | ||
45 changes: 45 additions & 0 deletions
45 apps/sim/hooks/queries/utils/find-workspace-file-by-src.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| /** | ||
| * @vitest-environment node | ||
| */ | ||
| import { describe, expect, it } from 'vitest' | ||
| import type { WorkspaceFileRecord } from '@/lib/uploads/contexts/workspace' | ||
| import { findWorkspaceFileBySrc } from '@/hooks/queries/utils/find-workspace-file-by-src' | ||
| function record(over: Partial<WorkspaceFileRecord>): WorkspaceFileRecord { | ||
| return { id: 'wf_x', key: 'workspace/ws1/x.png', ...over } as WorkspaceFileRecord | ||
| } | ||
| const records = [ | ||
| record({ id: 'wf_a', key: 'workspace/ws1/a.png' }), | ||
| record({ id: 'wf_b', key: 'workspace/ws1/b.png' }), | ||
| ] | ||
| const serveUrl = (key: string) => `/api/files/serve/${encodeURIComponent(key)}?context=workspace` | ||
| describe('findWorkspaceFileBySrc', () => { | ||
| it('matches a serve URL by storage key', () => { | ||
| expect(findWorkspaceFileBySrc(records, serveUrl('workspace/ws1/b.png'))?.id).toBe('wf_b') | ||
| }) | ||
| it('matches a /api/files/view/<id> URL by file id', () => { | ||
| expect(findWorkspaceFileBySrc(records, '/api/files/view/wf_a')?.id).toBe('wf_a') | ||
| }) | ||
| it('matches a /workspace/<ws>/files/<id> URL by file id', () => { | ||
| expect(findWorkspaceFileBySrc(records, '/workspace/ws1/files/wf_b')?.id).toBe('wf_b') | ||
| }) | ||
| it('returns undefined for a serve URL whose key is not in the list', () => { | ||
| expect(findWorkspaceFileBySrc(records, serveUrl('workspace/ws1/missing.png'))).toBeUndefined() | ||
| }) | ||
| it('returns undefined for external, data:, and undefined srcs', () => { | ||
| expect(findWorkspaceFileBySrc(records, 'https://example.com/x.png')).toBeUndefined() | ||
| expect(findWorkspaceFileBySrc(records, 'data:image/png;base64,AAAA')).toBeUndefined() | ||
| expect(findWorkspaceFileBySrc(records, undefined)).toBeUndefined() | ||
| }) | ||
| it('returns undefined when the file list has not loaded yet', () => { | ||
| expect(findWorkspaceFileBySrc(undefined, '/api/files/view/wf_a')).toBeUndefined() | ||
| }) | ||
| }) |
19 changes: 19 additions & 0 deletions
19 apps/sim/hooks/queries/utils/find-workspace-file-by-src.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| import type { WorkspaceFileRecord } from '@/lib/uploads/contexts/workspace' | ||
| import { extractEmbeddedFileRef } from '@/lib/uploads/utils/embedded-image-ref' | ||
| /** | ||
| * Resolve the workspace file record an embedded image `src` points at, matching the persisted serve-URL | ||
| * shape by storage key or file id. Returns `undefined` for external / `data:` / unrecognized srcs, and | ||
| * when the file list isn't loaded — callers then fall back to on-load measurement rather than reserving | ||
| * from metadata. | ||
| */ | ||
| export function findWorkspaceFileBySrc( | ||
| records: WorkspaceFileRecord[] | undefined, | ||
| src: string | undefined | ||
| ): WorkspaceFileRecord | undefined { | ||
| const ref = src ? extractEmbeddedFileRef(src) : null | ||
| if (!ref || !records) return undefined | ||
| return 'key' in ref | ||
| ? records.find((record) => record.key === ref.key) | ||
| : records.find((record) => record.id === ref.fileId) | ||
| } |
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.