v0.5.3 by ajslater · Pull Request #10 · ajslater/codex

ajslater · 2020-08-28T17:18:53Z

Fixed login bug introduces in v0.5.3 (thanks hubcaps)
Fixed filtering bug introduced in v0.5.2
Versioned API
Toast popup for admins indicating libraries being scanned.
Periodic frontend refresh during long scans.
Codex version displayed in browser footer

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile

commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API

commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works

commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login

check to see if the queue is empty too for scan_notify

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir * ensure all STATIC_DIRS exist for whitenoise * new black. update poetry, npm & lint * 0.5.4 version bump * only use the build type static files in dev mode. always should've been like this. * collect static needs dev marker to actually collect the dev static dirs

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir * ensure all STATIC_DIRS exist for whitenoise * new black. update poetry, npm & lint * 0.5.4 version bump * only use the build type static files in dev mode. always should've been like this. * collect static needs dev marker to actually collect the dev static dirs * revert whitenoise * codex v0.5.5 * fix logging error * borders on browse-cards * fix border interference with unread flag * v0.5.6

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir * ensure all STATIC_DIRS exist for whitenoise * new black. update poetry, npm & lint * 0.5.4 version bump * only use the build type static files in dev mode. always should've been like this. * collect static needs dev marker to actually collect the dev static dirs * revert whitenoise * codex v0.5.5 * fix logging error * borders on browse-cards * fix border interference with unread flag * v0.5.6 * comicbox 1.3 * allow null credit roles * V0.5.7 (#35) * v0.5.7 news and version * remove transparency from child count * center pages better in reader. still has a two page gap. * scroll to the top of the page when the route changes * more explicit read/unread menu. Add download comic to menu * fix previous comic always start of series bug * lint migrations * npm update * logfiles * more items to news * lint * reorg setup-dev.sh * Docker caching for wheel builder image * remove scanlocks on startup. fix library change by not disabling element * Squashed commit of the following: commit e1994b7d8f76d1aff43de8fddd927e3ac6686502 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:37:37 2020 -0700 bum news commit f230dd6590b2c0e404f5820dca76810207c74a74 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:33:23 2020 -0700 remove inconsistent document.title setting commit 129323516a24bb1959f1e5cee244af56151b273f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:29:20 2020 -0700 restore child counts and bookmarks commit 21f25ec7b15822b5ee51c2c97b4a700d584306fb Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:42:49 2020 -0700 modular frontend comic name code. disambiguate volumes more commit 3f8c7a4fedf931b81da9196147f6556632194fbf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:46 2020 -0700 volume parsing on the back end commit 557ace75ccb3522302a1b402466cb19d75a38b22 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:19 2020 -0700 fix issue type in browser commit 4d2331d240f0b78c48e2cc365dc4d41f9155ff7d Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:55:02 2020 -0700 remove extra loggins commit 566e353f78ce81f502b9ceac103f0f41b300b91c Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:52:40 2020 -0700 move ensure cover path to serializer so we only hit the db once commit adfeda4d5d3a91e355ae55e6182411b2cb7f73bf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:15:12 2020 -0700 pass series_name, volume_name, issue as fields. no more janky parsing commit 361e0ad253356ea7d5733bb1ca54e2e80173c9e4 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:48:43 2020 -0700 rename containerList commit 830454081e9163039dbd7893a91b099b7f094380 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:45:53 2020 -0700 fix up route not starting from 0 commit d17e0de419e9dd0c715d46f71e856c77a619a845 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:34:47 2020 -0700 pagination union with header name annotated instead of python computed commit b1aac346d372dd5a41397c53428de0303733958a Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:06:44 2020 -0700 npm update commit 69414ec4e09c97c7a6c6c7a7da3406b71da7f178 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:24 2020 -0700 update poetry commit e34d4af0c79d86f7ca2bc072be9ee5b8965ae54f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:04 2020 -0700 clear cache on startup commit 34a96e36638d98be77ed5de3681c7011b4d2b6a2 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:00:38 2020 -0700 fix last_scan create library bug commit fef826cd24f2cd0315113a45f2c8eee2dcbf9667 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:38:20 2020 -0700 pagination proof of concept. breaks with folder view commit 01eefe612e6e63556e6f58545e1fbf9846689f85 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:34:46 2020 -0700 lint * friendly /me message * Squashed commit of the following: commit b73eab2312d1648dc24203ec7bca712d820daeb2 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:15:54 2020 -0700 remove unused var commit 12d990586f00f9b6c65eade9495de3ed65aea1fe Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:11:25 2020 -0700 watch for user changes in the browse app, not the auth app for browse stuff. commit 9ba8575c0e4f4c011fde28ee2ec37d206063700f Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:10:34 2020 -0700 consolidate route generation in browse store and add page * reform browse console logging * Squashed commit of the following: commit d72c5eda5744f68d442e91f942bf7e8077271976 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:34:00 2020 -0700 refactor browse card to be a single card object. * oder the common browse fields * fix cover on metadata page * move progress bar to browse card for better positioning. * more tests for comic model * bump codex version oops * v0.5.8 * poetry update * more helpful 404 page * V0.5.9 (#50) * poetry update * more helpful 404 page * update npm * fix logging bug on null role import * Squashed commit of the following: commit 0980b6e2ce4d8a7b5a55722427febb3d6d35b05e Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 18:09:12 2020 -0700 fix folder sorts. reorganize annotation code. eliminate x_order_key in favor of order_value commit 4797fc411aa6d43d43a66ff98a90641c6c8338e4 Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 15:39:43 2020 -0700 first draft of folder sort fixes. needs cleanup * Squashed commit of the following: commit 36329ac2572bd95d55fe2e6b79156a0355eea42b Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 19:25:15 2020 -0700 missing cover support * poetry update * version bump * comicbox 0.1.4 * lint * Squashed commit of the following: commit 9f22f50e97ecc743270e26217012d8d5097016dc Author: AJ Slater <aj@slater.net> Date: Tue Sep 8 13:53:41 2020 -0700 use base images when building runnable docker * V0.5.10 (#53) * fix filtering bug * bump version * clean up comments * fix reader settings * mostly fix reader image layout * fix reader to browser uninitialized error * bump news * update poetry * npm update * simpler image labels * V0.5.11 (#55) * simpler image labels * simply image tagging even more. publish to pypi with ci * ignore node_modules of course * use Min instead of Max for sorting containers. Added benefit is covers match sort values now * flex layout to the rescue for adaptive browse tile row heights. * fix bookmarks not working for sessions * update poetry & npm * bump version to 0.5.11 & news * fix mobile safari bug with stretched images * Squashed commit of the following: commit 8573d66e625a99288cc52092aad33c14845bb631 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 13:05:53 2020 -0700 fix the whitenoise prefixing for dev mode commit 46700920e7da71e0a1779eb0a701ac321dad1d71 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 12:33:35 2020 -0700 works in prod * abuse WHITENOISE_ROOT for covers so I can turn off WHITENOISE_USE_FINDERS in production * poorly spelled variable prevented setting global reader settings * bump news * another news bump * admin failed imports list * Squashed commit of the following: commit 3a3e206df83ee9ad103b2c7600a2ae1b3fb305f8 Author: AJ Slater <aj@slater.net> Date: Mon Sep 14 17:42:25 2020 -0700 scan notify snack * bump version and news * lint and cleanup prints * Squashed commit of the following: commit ee19a333161c174b387a4b2b525d04ec1d549b91 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 14:38:45 2020 -0700 latest version * Squashed commit of the following: commit 6001d4a338bf9dc86238b0da565e76a891d3de02 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:55:53 2020 -0700 remove the init_admin_flags from migrations commit 38556ea04959e02522c08288c152f61ea18b7a3a Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:47:38 2020 -0700 minor cleanups and comments commit 3e40d41c52b188353b0d04e96c7e35dec5315d95 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:31:46 2020 -0700 bump news commit 1321826d522c32cb62a13abb1ea3df8173f8cf3d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 18:00:20 2020 -0700 add dev config dir to prettierignore commit 784e165efc57f6f97092e8d17236a0bde40b226d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 17:52:09 2020 -0700 disable session users first try * change failed import comment * v0.5.13 * crap i'm sure now * Squashed commit of the following: commit be04fac73fe3fdcd2ee600aac8106125f45bff7c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:56:48 2020 -0700 lint commit f93023cf01b2e2f8fe0e83fdcd0c051ed1a8428e Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:55:47 2020 -0700 bump news commit 2637d430f90a382145218d691e64e28939c4f024 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:53:27 2020 -0700 poetry and npm update commit a7d8af755fea91b202e018ce63d084dc22229daa Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:49:01 2020 -0700 add force to comic cover creation commit c6b11e3ea7866c980bc0e4b9b0772d39f81a3052 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:25:20 2020 -0700 fix unable to delete libs commit af224ffc3b34b172ecc2ae84c848fcf1d64bf5c0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 17:08:07 2020 -0700 rehide edit button and fix issueRow layout on mobile commit 17bee0af42a14d1d53285f6ef8f4c583888a36c7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:50:27 2020 -0700 minor cleanup of checkbox commit c7f235af7abd00914c874b2fb65ebd1be545e1a4 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:33:27 2020 -0700 remove comment cruft commit 073b6f08e39100aa71dc33eee6969abb473164fc Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:32:00 2020 -0700 bump version & news commit 0c8bdfb5f216df4353533d9371437caa65dfb7f7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:29:14 2020 -0700 dict for websocket messages commit 7a8e1e9af9bd57a93b08d9f587435ed4f47f525b Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 15:51:41 2020 -0700 minor cleanups commit d1ad0ef4159aade2793c424be693f4dd3d36f943 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 14:01:29 2020 -0700 remove cruft commit df1d69ef2d28fd88049032d8ee7c6d21163708f6 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:56:06 2020 -0700 minor cleanups and formalize timing for metadata commit 0e68de8a3b85bb05657b83f7d2871f02094696a0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:48:51 2020 -0700 two minute timeout! commit 650099da1be7510c25fb0c1ed562fdeb56a31e2c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:15:34 2020 -0700 cleanups for metadata commit 5c06d6c8d43f3400d92bfa9062851951b0ea9981 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:30:56 2020 -0700 progress meter for metadata commit 7775cf15f56f347136817bb1bf33f00091b13a05 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:29:43 2020 -0700 fix no credits bug. fix pk lookup syntax commit 70688d27a01bc2eb75f8a7cce007ec2d92fbeddb Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:53 2020 -0700 lint commit 72b53b462cb7d5feebbe4fea6159fc3f9588fbc3 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:47 2020 -0700 move comic-name into components commit f300f21097bb0d194ce1359eaf37b25c7f23ce20 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:45 2020 -0700 show less metadata if not in editMode commit 7731a45e630108eae0a254d49fd009a6293e8247 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:10 2020 -0700 first minort cleanups commit 30399ee10b665e39c30cef5a5332b2f31c8459f9 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:54 2020 -0700 freeze the comic metadata it won't change ... yet? commit ded8672c4c0f437febcbbb0ee393cd7b056d9216 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:20 2020 -0700 fix null issueChoices commit 5ecab8eb652941ff208c31cd5e1b7734a40da73f Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:02 2020 -0700 share initialItem across autocomplete and combobox commit 02a4e2f71c1662d6874741d025e4e10ac315f6e6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 16:21:59 2020 -0700 placeholder for metdata commit c4a4c37dee9f243626f8137dc6d5c35266dc6dbd Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:45 2020 -0700 Squashed commit of the following: commit 3637b056f2d2d947658767f11a5f29b8c664e817 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:24 2020 -0700 fix date display and all autocompletes commit 02d14266742e64620b17787049d8d6f0594258ea Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:35:32 2020 -0700 align issues better commit 7f8efe405e883b686a10cdf8d84e0e4edd41f1fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:49 2020 -0700 fix language and country display commit 0dc983f501c421bbd223e5348abdf096337f67f4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:25 2020 -0700 wipe comic data before lookup commit fa7236f6a74e061ecb475672c72dff4de19134e0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:29:42 2020 -0700 fix pycountry lookup bug commit 9a6bb655e51d3f9416b5bfc3bfb2feb6ab3b72b4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:49:28 2020 -0700 optimize getting only the aggregate filter for metadata commit 5129983d1f589cba0acad9edbe921203645c4db4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:37:38 2020 -0700 compress all metdata consts into the QUERY_SERIALIZER map commit c7e35a64fd491fedbee6ba4682975a5afb54df2b Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:14:02 2020 -0700 move frontend api files to versioned directory commit 49389216c3fb60718aceb45f34fe942e46cb45fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:08:34 2020 -0700 rename variables. clean up cruft. comments commit 39788fc1fcce0c8ce131da1be03be2284766d386 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:53:31 2020 -0700 lint and comments commit 3baa071e140c6f46bab63bed10bbfab13e546e5a Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:40 2020 -0700 lint commit 49565bc5c50c7b058e76837f8ab382db3caa63c8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:36 2020 -0700 lint ci scripts as well commit e1cac1604f0ae5e7f83fd94e9125c96c4b3104a6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:45:29 2020 -0700 add a progress bar to metdata commit d0e2412ae18b9a5f75f64dde1ab770e1968f1927 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:38:21 2020 -0700 move top book cover decorations into book-cover so they display on the metadata page commit 07426ce9fdacab8c980deaebac2e913f8c3c1eba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:55:20 2020 -0700 simpler tag icon is better commit f8e8ed39b725accffa57ba7f4978515e16a6fc4e Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:54:16 2020 -0700 Tag multiple icon commit a9a79e656cf20f141f44eb3e1b8c7b542ea27d6b Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:47:20 2020 -0700 remove log commit ee14d91ebfa3c96597003d50da2bed96351665cc Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:44:58 2020 -0700 change metadata icon commit 9b76a9bc7ca5a445d17bf505823241428588063f Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:31:11 2020 -0700 format issue decimals commit a49061082610264d7aaed1d38d1bf24f0baaddf6 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:16:02 2020 -0700 fix issues to handle years commit 08bd7639ebca0d68cf4bff985725681be72c7dba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 19:39:48 2020 -0700 credits working commit 268df3cd136972c7bc28aa7d0ea1761f4a2d24e9 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 18:20:51 2020 -0700 working even better. credits, issue display and folder test remain commit a5d6d88244fb1536c840789857c7624bb8cd2c4d Author: AJ Slater <aj@slater.net> Date: Mon Sep 21 19:31:49 2020 -0700 kind of working but kludgy backend solution commit 05c67e90052cc69010c20bd9d2c39d0e21745032 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 16:54:50 2020 -0700 move children and page count into base class commit c0f337bcde505729380caa10a6ea665fa2d5aa38 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 15:14:13 2020 -0700 move bookmark aggregation up into metadataclass for real. remove unused fit_to & two_pages commit 7c3ab1eb99e433b1181c4a3ff319329f1bd0a287 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 14:08:31 2020 -0700 move bookmark annotations to browse_metadata_base

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir * ensure all STATIC_DIRS exist for whitenoise * new black. update poetry, npm & lint * 0.5.4 version bump * only use the build type static files in dev mode. always should've been like this. * collect static needs dev marker to actually collect the dev static dirs * revert whitenoise * codex v0.5.5 * fix logging error * borders on browse-cards * fix border interference with unread flag * v0.5.6 * comicbox 1.3 * allow null credit roles * V0.5.7 (#35) * v0.5.7 news and version * remove transparency from child count * center pages better in reader. still has a two page gap. * scroll to the top of the page when the route changes * more explicit read/unread menu. Add download comic to menu * fix previous comic always start of series bug * lint migrations * npm update * logfiles * more items to news * lint * reorg setup-dev.sh * Docker caching for wheel builder image * remove scanlocks on startup. fix library change by not disabling element * Squashed commit of the following: commit e1994b7d8f76d1aff43de8fddd927e3ac6686502 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:37:37 2020 -0700 bum news commit f230dd6590b2c0e404f5820dca76810207c74a74 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:33:23 2020 -0700 remove inconsistent document.title setting commit 129323516a24bb1959f1e5cee244af56151b273f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:29:20 2020 -0700 restore child counts and bookmarks commit 21f25ec7b15822b5ee51c2c97b4a700d584306fb Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:42:49 2020 -0700 modular frontend comic name code. disambiguate volumes more commit 3f8c7a4fedf931b81da9196147f6556632194fbf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:46 2020 -0700 volume parsing on the back end commit 557ace75ccb3522302a1b402466cb19d75a38b22 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:19 2020 -0700 fix issue type in browser commit 4d2331d240f0b78c48e2cc365dc4d41f9155ff7d Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:55:02 2020 -0700 remove extra loggins commit 566e353f78ce81f502b9ceac103f0f41b300b91c Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:52:40 2020 -0700 move ensure cover path to serializer so we only hit the db once commit adfeda4d5d3a91e355ae55e6182411b2cb7f73bf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:15:12 2020 -0700 pass series_name, volume_name, issue as fields. no more janky parsing commit 361e0ad253356ea7d5733bb1ca54e2e80173c9e4 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:48:43 2020 -0700 rename containerList commit 830454081e9163039dbd7893a91b099b7f094380 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:45:53 2020 -0700 fix up route not starting from 0 commit d17e0de419e9dd0c715d46f71e856c77a619a845 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:34:47 2020 -0700 pagination union with header name annotated instead of python computed commit b1aac346d372dd5a41397c53428de0303733958a Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:06:44 2020 -0700 npm update commit 69414ec4e09c97c7a6c6c7a7da3406b71da7f178 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:24 2020 -0700 update poetry commit e34d4af0c79d86f7ca2bc072be9ee5b8965ae54f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:04 2020 -0700 clear cache on startup commit 34a96e36638d98be77ed5de3681c7011b4d2b6a2 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:00:38 2020 -0700 fix last_scan create library bug commit fef826cd24f2cd0315113a45f2c8eee2dcbf9667 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:38:20 2020 -0700 pagination proof of concept. breaks with folder view commit 01eefe612e6e63556e6f58545e1fbf9846689f85 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:34:46 2020 -0700 lint * friendly /me message * Squashed commit of the following: commit b73eab2312d1648dc24203ec7bca712d820daeb2 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:15:54 2020 -0700 remove unused var commit 12d990586f00f9b6c65eade9495de3ed65aea1fe Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:11:25 2020 -0700 watch for user changes in the browse app, not the auth app for browse stuff. commit 9ba8575c0e4f4c011fde28ee2ec37d206063700f Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:10:34 2020 -0700 consolidate route generation in browse store and add page * reform browse console logging * Squashed commit of the following: commit d72c5eda5744f68d442e91f942bf7e8077271976 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:34:00 2020 -0700 refactor browse card to be a single card object. * oder the common browse fields * fix cover on metadata page * move progress bar to browse card for better positioning. * more tests for comic model * bump codex version oops * v0.5.8 * poetry update * more helpful 404 page * V0.5.9 (#50) * poetry update * more helpful 404 page * update npm * fix logging bug on null role import * Squashed commit of the following: commit 0980b6e2ce4d8a7b5a55722427febb3d6d35b05e Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 18:09:12 2020 -0700 fix folder sorts. reorganize annotation code. eliminate x_order_key in favor of order_value commit 4797fc411aa6d43d43a66ff98a90641c6c8338e4 Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 15:39:43 2020 -0700 first draft of folder sort fixes. needs cleanup * Squashed commit of the following: commit 36329ac2572bd95d55fe2e6b79156a0355eea42b Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 19:25:15 2020 -0700 missing cover support * poetry update * version bump * comicbox 0.1.4 * lint * Squashed commit of the following: commit 9f22f50e97ecc743270e26217012d8d5097016dc Author: AJ Slater <aj@slater.net> Date: Tue Sep 8 13:53:41 2020 -0700 use base images when building runnable docker * V0.5.10 (#53) * fix filtering bug * bump version * clean up comments * fix reader settings * mostly fix reader image layout * fix reader to browser uninitialized error * bump news * update poetry * npm update * simpler image labels * V0.5.11 (#55) * simpler image labels * simply image tagging even more. publish to pypi with ci * ignore node_modules of course * use Min instead of Max for sorting containers. Added benefit is covers match sort values now * flex layout to the rescue for adaptive browse tile row heights. * fix bookmarks not working for sessions * update poetry & npm * bump version to 0.5.11 & news * fix mobile safari bug with stretched images * Squashed commit of the following: commit 8573d66e625a99288cc52092aad33c14845bb631 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 13:05:53 2020 -0700 fix the whitenoise prefixing for dev mode commit 46700920e7da71e0a1779eb0a701ac321dad1d71 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 12:33:35 2020 -0700 works in prod * abuse WHITENOISE_ROOT for covers so I can turn off WHITENOISE_USE_FINDERS in production * poorly spelled variable prevented setting global reader settings * bump news * another news bump * admin failed imports list * Squashed commit of the following: commit 3a3e206df83ee9ad103b2c7600a2ae1b3fb305f8 Author: AJ Slater <aj@slater.net> Date: Mon Sep 14 17:42:25 2020 -0700 scan notify snack * bump version and news * lint and cleanup prints * Squashed commit of the following: commit ee19a333161c174b387a4b2b525d04ec1d549b91 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 14:38:45 2020 -0700 latest version * Squashed commit of the following: commit 6001d4a338bf9dc86238b0da565e76a891d3de02 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:55:53 2020 -0700 remove the init_admin_flags from migrations commit 38556ea04959e02522c08288c152f61ea18b7a3a Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:47:38 2020 -0700 minor cleanups and comments commit 3e40d41c52b188353b0d04e96c7e35dec5315d95 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:31:46 2020 -0700 bump news commit 1321826d522c32cb62a13abb1ea3df8173f8cf3d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 18:00:20 2020 -0700 add dev config dir to prettierignore commit 784e165efc57f6f97092e8d17236a0bde40b226d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 17:52:09 2020 -0700 disable session users first try * change failed import comment * v0.5.13 * crap i'm sure now * Squashed commit of the following: commit be04fac73fe3fdcd2ee600aac8106125f45bff7c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:56:48 2020 -0700 lint commit f93023cf01b2e2f8fe0e83fdcd0c051ed1a8428e Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:55:47 2020 -0700 bump news commit 2637d430f90a382145218d691e64e28939c4f024 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:53:27 2020 -0700 poetry and npm update commit a7d8af755fea91b202e018ce63d084dc22229daa Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:49:01 2020 -0700 add force to comic cover creation commit c6b11e3ea7866c980bc0e4b9b0772d39f81a3052 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:25:20 2020 -0700 fix unable to delete libs commit af224ffc3b34b172ecc2ae84c848fcf1d64bf5c0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 17:08:07 2020 -0700 rehide edit button and fix issueRow layout on mobile commit 17bee0af42a14d1d53285f6ef8f4c583888a36c7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:50:27 2020 -0700 minor cleanup of checkbox commit c7f235af7abd00914c874b2fb65ebd1be545e1a4 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:33:27 2020 -0700 remove comment cruft commit 073b6f08e39100aa71dc33eee6969abb473164fc Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:32:00 2020 -0700 bump version & news commit 0c8bdfb5f216df4353533d9371437caa65dfb7f7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:29:14 2020 -0700 dict for websocket messages commit 7a8e1e9af9bd57a93b08d9f587435ed4f47f525b Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 15:51:41 2020 -0700 minor cleanups commit d1ad0ef4159aade2793c424be693f4dd3d36f943 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 14:01:29 2020 -0700 remove cruft commit df1d69ef2d28fd88049032d8ee7c6d21163708f6 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:56:06 2020 -0700 minor cleanups and formalize timing for metadata commit 0e68de8a3b85bb05657b83f7d2871f02094696a0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:48:51 2020 -0700 two minute timeout! commit 650099da1be7510c25fb0c1ed562fdeb56a31e2c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:15:34 2020 -0700 cleanups for metadata commit 5c06d6c8d43f3400d92bfa9062851951b0ea9981 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:30:56 2020 -0700 progress meter for metadata commit 7775cf15f56f347136817bb1bf33f00091b13a05 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:29:43 2020 -0700 fix no credits bug. fix pk lookup syntax commit 70688d27a01bc2eb75f8a7cce007ec2d92fbeddb Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:53 2020 -0700 lint commit 72b53b462cb7d5feebbe4fea6159fc3f9588fbc3 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:47 2020 -0700 move comic-name into components commit f300f21097bb0d194ce1359eaf37b25c7f23ce20 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:45 2020 -0700 show less metadata if not in editMode commit 7731a45e630108eae0a254d49fd009a6293e8247 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:10 2020 -0700 first minort cleanups commit 30399ee10b665e39c30cef5a5332b2f31c8459f9 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:54 2020 -0700 freeze the comic metadata it won't change ... yet? commit ded8672c4c0f437febcbbb0ee393cd7b056d9216 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:20 2020 -0700 fix null issueChoices commit 5ecab8eb652941ff208c31cd5e1b7734a40da73f Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:02 2020 -0700 share initialItem across autocomplete and combobox commit 02a4e2f71c1662d6874741d025e4e10ac315f6e6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 16:21:59 2020 -0700 placeholder for metdata commit c4a4c37dee9f243626f8137dc6d5c35266dc6dbd Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:45 2020 -0700 Squashed commit of the following: commit 3637b056f2d2d947658767f11a5f29b8c664e817 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:24 2020 -0700 fix date display and all autocompletes commit 02d14266742e64620b17787049d8d6f0594258ea Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:35:32 2020 -0700 align issues better commit 7f8efe405e883b686a10cdf8d84e0e4edd41f1fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:49 2020 -0700 fix language and country display commit 0dc983f501c421bbd223e5348abdf096337f67f4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:25 2020 -0700 wipe comic data before lookup commit fa7236f6a74e061ecb475672c72dff4de19134e0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:29:42 2020 -0700 fix pycountry lookup bug commit 9a6bb655e51d3f9416b5bfc3bfb2feb6ab3b72b4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:49:28 2020 -0700 optimize getting only the aggregate filter for metadata commit 5129983d1f589cba0acad9edbe921203645c4db4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:37:38 2020 -0700 compress all metdata consts into the QUERY_SERIALIZER map commit c7e35a64fd491fedbee6ba4682975a5afb54df2b Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:14:02 2020 -0700 move frontend api files to versioned directory commit 49389216c3fb60718aceb45f34fe942e46cb45fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:08:34 2020 -0700 rename variables. clean up cruft. comments commit 39788fc1fcce0c8ce131da1be03be2284766d386 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:53:31 2020 -0700 lint and comments commit 3baa071e140c6f46bab63bed10bbfab13e546e5a Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:40 2020 -0700 lint commit 49565bc5c50c7b058e76837f8ab382db3caa63c8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:36 2020 -0700 lint ci scripts as well commit e1cac1604f0ae5e7f83fd94e9125c96c4b3104a6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:45:29 2020 -0700 add a progress bar to metdata commit d0e2412ae18b9a5f75f64dde1ab770e1968f1927 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:38:21 2020 -0700 move top book cover decorations into book-cover so they display on the metadata page commit 07426ce9fdacab8c980deaebac2e913f8c3c1eba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:55:20 2020 -0700 simpler tag icon is better commit f8e8ed39b725accffa57ba7f4978515e16a6fc4e Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:54:16 2020 -0700 Tag multiple icon commit a9a79e656cf20f141f44eb3e1b8c7b542ea27d6b Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:47:20 2020 -0700 remove log commit ee14d91ebfa3c96597003d50da2bed96351665cc Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:44:58 2020 -0700 change metadata icon commit 9b76a9bc7ca5a445d17bf505823241428588063f Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:31:11 2020 -0700 format issue decimals commit a49061082610264d7aaed1d38d1bf24f0baaddf6 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:16:02 2020 -0700 fix issues to handle years commit 08bd7639ebca0d68cf4bff985725681be72c7dba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 19:39:48 2020 -0700 credits working commit 268df3cd136972c7bc28aa7d0ea1761f4a2d24e9 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 18:20:51 2020 -0700 working even better. credits, issue display and folder test remain commit a5d6d88244fb1536c840789857c7624bb8cd2c4d Author: AJ Slater <aj@slater.net> Date: Mon Sep 21 19:31:49 2020 -0700 kind of working but kludgy backend solution commit 05c67e90052cc69010c20bd9d2c39d0e21745032 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 16:54:50 2020 -0700 move children and page count into base class commit c0f337bcde505729380caa10a6ea665fa2d5aa38 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 15:14:13 2020 -0700 move bookmark aggregation up into metadataclass for real. remove unused fit_to & two_pages commit 7c3ab1eb99e433b1181c4a3ff319329f1bd0a287 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 14:08:31 2020 -0700 move bookmark annotations to browse_metadata_base * speed up metadata by 50% * Squashed commit of the following: commit 97c87f28bbd4285efb89051de6c3844f1948c80f Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:09:01 2020 -0700 lint and comments. remove cruft commit 4d10bd16f07ad2e3c83ccb963f5e90978a61b214 Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:02:02 2020 -0700 major cleanup commit 8a23d7aa13702b0d75e4d1576e8ae75b03329af8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 20:15:59 2020 -0700 partial cleanups commit 27b66e26aa0d302364d1576c52140a0ea060b222 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:56:06 2020 -0700 isort commit 1c320e2187dabda7b3cc6bc55423902add58b3d0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:54:37 2020 -0700 fix related field counts and cleanup a bit of cruft. more cleanup to do. commit bd045eb0692e9232c51dd33053801d407c415039 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:59:09 2020 -0700 it motherfucking works commit 752d1773671196a4bbb3472490f35d394df806d7 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:48:58 2020 -0700 working agg all. now just do pks commit af6b59b69824836f8485892d83ad7156d055372e Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:56:57 2020 -0700 coment. lint commit f678f8f35d9724c9eca514e85c8372d6c534621a Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:34:21 2020 -0700 clean up metadata container aggregate a bit commit 49ca05eedf6f9a6ade3682272c35d7e0733203c9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:14:01 2020 -0700 simplify a map commit fc2a6d4d0834ccc056474414b6d15d61bbffe7df Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 11:59:21 2020 -0700 major cleanup of metadata view commit 0a90f5c5c75ec2a60f00334dcddc8d5eb95bf641 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:28 2020 -0700 more explicit nginx docs commit fe54d13f985d796b95593efb9f8e6c1ad539d879 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:18 2020 -0700 clean up metadata serializers commit 0e182239e8525ee33a8b095b6b406cd567e7acf9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:04 2020 -0700 remove a little cruft commit f8972d6b19c0f9c829d605c6ddb6fd229cef2466 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 19:37:45 2020 -0700 ugly but working bufix for too much data commit 0f4d7d70c4c8b44a8e629bbee217df742b76de08 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 16:38:35 2020 -0700 new metadata api works on frontend. backend bug gives too much info for groups commit 4241cacf42e8b4cf27a08c657f672694be1bde4a Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 00:20:50 2020 -0700 lang serialiezer returns value if text lookup fails commit c9a6465fd913185c2df340c082d1e0985b6fa189 Author: AJ Slater <aj@slater.net> Date: Mon Sep 28 18:18:10 2020 -0700 doubled speed of metadata * bump version * Squashed commit of the following: commit 88404c565b4cc3fbad899ca1bccd366b43b199b5 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 14:54:40 2020 -0700 remove TEST variable. always a bad idea commit a3bef259dc6c72b0a8bb885d0353d7a8bad1b81f Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:58:02 2020 -0700 comments commit 265784ce9fdf5fe2429fda3fa98f271c2275667b Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:54:39 2020 -0700 optimize select_related for metadata just a little more commit a488427a15ab8d0efb4a742308a0696d313492dc Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:45:15 2020 -0700 pop fields before serialization. should be faster commit 07df6a610a7683cf16551c4e644207d10be96f27 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:39:42 2020 -0700 clean up serializers and choices. get choices from src too. commit 5d59c5a95dcc6194f76d45c578efdd529a895f90 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:19:19 2020 -0700 fix improper null items on undefined metadata fields commit 905aa375760b374f3e5c8ccdc29f074cf2e3e754 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:18:59 2020 -0700 fix metadata folder view commit d6017dd76297f448735198e1a3a1c9f111865a2e Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 23:15:54 2020 -0700 reading choices and messages from the webpack json works now commit 87246c285ac93674eff1ba55be651a05e967102b Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:14:37 2020 -0700 move item xformer into api commit 102b291511cc3e6c34d7efe2f4e7839d5637766c Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:04:05 2020 -0700 fix null items commit b7212c3e3a47987d1b20ba604aba408555b32f70 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:44:22 2020 -0700 sorted items commit 5c8cef6cd7eb85c246f3099b35a71f2b87555e51 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:27:26 2020 -0700 componentize browser-settings-menu commit d3fd0f57ea0d5500389e97e1c071896abac3791a Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:17:06 2020 -0700 componentize browser settings dialog commit 8510179f3be465df0d7848751e62a699902dd458 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:05:52 2020 -0700 working filter menu. componentize top toolbar filters. * massive rename of containers and browse for consistency * docstyle linting * lint * remove mypy and vulture. i don't use them. * Squashed commit of the following: commit 936a6c8a202b09db2c8d6bedc8ce1d8add3e3f5e Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:13:22 2020 -0700 move secret key and logging init management into modules. websocket server doesn't configure logging right. commit 328bb8288bbee74b88c0e0754bc13abdb7ea114d Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:35:23 2020 -0700 less picky about passwords commit 8a953e20676ac5ca7e5c226b8792e075dbae1d11 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:17:14 2020 -0700 hypercorn 0.11.0 includes asgi signals and a big logging change. fix the logging with a custom formatter and a flag to logconfig_dict commit e3eae3400a2c07cab11997b74a71397e70f9d3e2 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:20 2020 -0700 upgrade babel/core commit 16034b60a7fb41fe47d67a8c02972231094f9946 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:04 2020 -0700 remove comment cruft commit 23885db544421469e46d0d42a75844c73b082fa5 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:13:31 2020 -0700 dev run script alter python path for local dir commit 80b92b45921dadc9d6a3811f7e92df5bd9377505 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:46 2020 -0700 correct logging __name__ and remove print cruft commit 2a6d347dcc71e39a88b2e7c515c8b2fcae07b0de Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:24 2020 -0700 document where i got the timezones middleware commit 59778ffa39967f7b6e42a5bf4fb4c52c8fbe5ce4 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:04 2020 -0700 debug lifespan logging commit b3ae98d90fbf3e9f1bf4b5eb17fb467c63f21f54 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:11:35 2020 -0700 rename procs & threads for logging commit 77f064e89aaa639c97b1bf7aeb4748a2fefe04d4 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 21:55:52 2020 -0700 broken asgi signals handling? * Squashed commit of the following: commit 4eca7486fa16467cfee0e38f4d6560c8b8487d12 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:19:12 2020 -0700 replace old DEV flag with DEBUG remove redundancy * lint * stop pytest from testing migrations * oops broke flake8 * Squashed commit of the following: commit efa7750ba35e3e32ba005a01cf53aab949c6993c Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:27:34 2020 -0700 componentize browser-main commit 3e9caadb2ffa7e7b679ab813d4c5acb4a7f366cb Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:06:01 2020 -0700 oops forgot to commit settings file for module change commit ca5168afd78bf05bdc7572120c746beb2e93ef22 Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:05:31 2020 -0700 settings module commit 54af69ae2562fdb7b101ef57d5ebbf9dd8088482 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 20:25:01 2020 -0700 componentize browser * lint * componentize reader * remove todo suggestion * more metadata fields * update news * fix restarts * its closer to 5 seconds in practice * update poetry and npm packages

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir * ensure all STATIC_DIRS exist for whitenoise * new black. update poetry, npm & lint * 0.5.4 version bump * only use the build type static files in dev mode. always should've been like this. * collect static needs dev marker to actually collect the dev static dirs * revert whitenoise * codex v0.5.5 * fix logging error * borders on browse-cards * fix border interference with unread flag * v0.5.6 * comicbox 1.3 * allow null credit roles * V0.5.7 (#35) * v0.5.7 news and version * remove transparency from child count * center pages better in reader. still has a two page gap. * scroll to the top of the page when the route changes * more explicit read/unread menu. Add download comic to menu * fix previous comic always start of series bug * lint migrations * npm update * logfiles * more items to news * lint * reorg setup-dev.sh * Docker caching for wheel builder image * remove scanlocks on startup. fix library change by not disabling element * Squashed commit of the following: commit e1994b7d8f76d1aff43de8fddd927e3ac6686502 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:37:37 2020 -0700 bum news commit f230dd6590b2c0e404f5820dca76810207c74a74 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:33:23 2020 -0700 remove inconsistent document.title setting commit 129323516a24bb1959f1e5cee244af56151b273f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:29:20 2020 -0700 restore child counts and bookmarks commit 21f25ec7b15822b5ee51c2c97b4a700d584306fb Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:42:49 2020 -0700 modular frontend comic name code. disambiguate volumes more commit 3f8c7a4fedf931b81da9196147f6556632194fbf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:46 2020 -0700 volume parsing on the back end commit 557ace75ccb3522302a1b402466cb19d75a38b22 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:19 2020 -0700 fix issue type in browser commit 4d2331d240f0b78c48e2cc365dc4d41f9155ff7d Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:55:02 2020 -0700 remove extra loggins commit 566e353f78ce81f502b9ceac103f0f41b300b91c Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:52:40 2020 -0700 move ensure cover path to serializer so we only hit the db once commit adfeda4d5d3a91e355ae55e6182411b2cb7f73bf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:15:12 2020 -0700 pass series_name, volume_name, issue as fields. no more janky parsing commit 361e0ad253356ea7d5733bb1ca54e2e80173c9e4 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:48:43 2020 -0700 rename containerList commit 830454081e9163039dbd7893a91b099b7f094380 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:45:53 2020 -0700 fix up route not starting from 0 commit d17e0de419e9dd0c715d46f71e856c77a619a845 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:34:47 2020 -0700 pagination union with header name annotated instead of python computed commit b1aac346d372dd5a41397c53428de0303733958a Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:06:44 2020 -0700 npm update commit 69414ec4e09c97c7a6c6c7a7da3406b71da7f178 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:24 2020 -0700 update poetry commit e34d4af0c79d86f7ca2bc072be9ee5b8965ae54f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:04 2020 -0700 clear cache on startup commit 34a96e36638d98be77ed5de3681c7011b4d2b6a2 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:00:38 2020 -0700 fix last_scan create library bug commit fef826cd24f2cd0315113a45f2c8eee2dcbf9667 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:38:20 2020 -0700 pagination proof of concept. breaks with folder view commit 01eefe612e6e63556e6f58545e1fbf9846689f85 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:34:46 2020 -0700 lint * friendly /me message * Squashed commit of the following: commit b73eab2312d1648dc24203ec7bca712d820daeb2 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:15:54 2020 -0700 remove unused var commit 12d990586f00f9b6c65eade9495de3ed65aea1fe Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:11:25 2020 -0700 watch for user changes in the browse app, not the auth app for browse stuff. commit 9ba8575c0e4f4c011fde28ee2ec37d206063700f Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:10:34 2020 -0700 consolidate route generation in browse store and add page * reform browse console logging * Squashed commit of the following: commit d72c5eda5744f68d442e91f942bf7e8077271976 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:34:00 2020 -0700 refactor browse card to be a single card object. * oder the common browse fields * fix cover on metadata page * move progress bar to browse card for better positioning. * more tests for comic model * bump codex version oops * v0.5.8 * poetry update * more helpful 404 page * V0.5.9 (#50) * poetry update * more helpful 404 page * update npm * fix logging bug on null role import * Squashed commit of the following: commit 0980b6e2ce4d8a7b5a55722427febb3d6d35b05e Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 18:09:12 2020 -0700 fix folder sorts. reorganize annotation code. eliminate x_order_key in favor of order_value commit 4797fc411aa6d43d43a66ff98a90641c6c8338e4 Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 15:39:43 2020 -0700 first draft of folder sort fixes. needs cleanup * Squashed commit of the following: commit 36329ac2572bd95d55fe2e6b79156a0355eea42b Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 19:25:15 2020 -0700 missing cover support * poetry update * version bump * comicbox 0.1.4 * lint * Squashed commit of the following: commit 9f22f50e97ecc743270e26217012d8d5097016dc Author: AJ Slater <aj@slater.net> Date: Tue Sep 8 13:53:41 2020 -0700 use base images when building runnable docker * V0.5.10 (#53) * fix filtering bug * bump version * clean up comments * fix reader settings * mostly fix reader image layout * fix reader to browser uninitialized error * bump news * update poetry * npm update * simpler image labels * V0.5.11 (#55) * simpler image labels * simply image tagging even more. publish to pypi with ci * ignore node_modules of course * use Min instead of Max for sorting containers. Added benefit is covers match sort values now * flex layout to the rescue for adaptive browse tile row heights. * fix bookmarks not working for sessions * update poetry & npm * bump version to 0.5.11 & news * fix mobile safari bug with stretched images * Squashed commit of the following: commit 8573d66e625a99288cc52092aad33c14845bb631 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 13:05:53 2020 -0700 fix the whitenoise prefixing for dev mode commit 46700920e7da71e0a1779eb0a701ac321dad1d71 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 12:33:35 2020 -0700 works in prod * abuse WHITENOISE_ROOT for covers so I can turn off WHITENOISE_USE_FINDERS in production * poorly spelled variable prevented setting global reader settings * bump news * another news bump * admin failed imports list * Squashed commit of the following: commit 3a3e206df83ee9ad103b2c7600a2ae1b3fb305f8 Author: AJ Slater <aj@slater.net> Date: Mon Sep 14 17:42:25 2020 -0700 scan notify snack * bump version and news * lint and cleanup prints * Squashed commit of the following: commit ee19a333161c174b387a4b2b525d04ec1d549b91 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 14:38:45 2020 -0700 latest version * Squashed commit of the following: commit 6001d4a338bf9dc86238b0da565e76a891d3de02 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:55:53 2020 -0700 remove the init_admin_flags from migrations commit 38556ea04959e02522c08288c152f61ea18b7a3a Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:47:38 2020 -0700 minor cleanups and comments commit 3e40d41c52b188353b0d04e96c7e35dec5315d95 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:31:46 2020 -0700 bump news commit 1321826d522c32cb62a13abb1ea3df8173f8cf3d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 18:00:20 2020 -0700 add dev config dir to prettierignore commit 784e165efc57f6f97092e8d17236a0bde40b226d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 17:52:09 2020 -0700 disable session users first try * change failed import comment * v0.5.13 * crap i'm sure now * Squashed commit of the following: commit be04fac73fe3fdcd2ee600aac8106125f45bff7c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:56:48 2020 -0700 lint commit f93023cf01b2e2f8fe0e83fdcd0c051ed1a8428e Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:55:47 2020 -0700 bump news commit 2637d430f90a382145218d691e64e28939c4f024 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:53:27 2020 -0700 poetry and npm update commit a7d8af755fea91b202e018ce63d084dc22229daa Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:49:01 2020 -0700 add force to comic cover creation commit c6b11e3ea7866c980bc0e4b9b0772d39f81a3052 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:25:20 2020 -0700 fix unable to delete libs commit af224ffc3b34b172ecc2ae84c848fcf1d64bf5c0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 17:08:07 2020 -0700 rehide edit button and fix issueRow layout on mobile commit 17bee0af42a14d1d53285f6ef8f4c583888a36c7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:50:27 2020 -0700 minor cleanup of checkbox commit c7f235af7abd00914c874b2fb65ebd1be545e1a4 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:33:27 2020 -0700 remove comment cruft commit 073b6f08e39100aa71dc33eee6969abb473164fc Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:32:00 2020 -0700 bump version & news commit 0c8bdfb5f216df4353533d9371437caa65dfb7f7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:29:14 2020 -0700 dict for websocket messages commit 7a8e1e9af9bd57a93b08d9f587435ed4f47f525b Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 15:51:41 2020 -0700 minor cleanups commit d1ad0ef4159aade2793c424be693f4dd3d36f943 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 14:01:29 2020 -0700 remove cruft commit df1d69ef2d28fd88049032d8ee7c6d21163708f6 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:56:06 2020 -0700 minor cleanups and formalize timing for metadata commit 0e68de8a3b85bb05657b83f7d2871f02094696a0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:48:51 2020 -0700 two minute timeout! commit 650099da1be7510c25fb0c1ed562fdeb56a31e2c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:15:34 2020 -0700 cleanups for metadata commit 5c06d6c8d43f3400d92bfa9062851951b0ea9981 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:30:56 2020 -0700 progress meter for metadata commit 7775cf15f56f347136817bb1bf33f00091b13a05 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:29:43 2020 -0700 fix no credits bug. fix pk lookup syntax commit 70688d27a01bc2eb75f8a7cce007ec2d92fbeddb Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:53 2020 -0700 lint commit 72b53b462cb7d5feebbe4fea6159fc3f9588fbc3 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:47 2020 -0700 move comic-name into components commit f300f21097bb0d194ce1359eaf37b25c7f23ce20 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:45 2020 -0700 show less metadata if not in editMode commit 7731a45e630108eae0a254d49fd009a6293e8247 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:10 2020 -0700 first minort cleanups commit 30399ee10b665e39c30cef5a5332b2f31c8459f9 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:54 2020 -0700 freeze the comic metadata it won't change ... yet? commit ded8672c4c0f437febcbbb0ee393cd7b056d9216 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:20 2020 -0700 fix null issueChoices commit 5ecab8eb652941ff208c31cd5e1b7734a40da73f Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:02 2020 -0700 share initialItem across autocomplete and combobox commit 02a4e2f71c1662d6874741d025e4e10ac315f6e6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 16:21:59 2020 -0700 placeholder for metdata commit c4a4c37dee9f243626f8137dc6d5c35266dc6dbd Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:45 2020 -0700 Squashed commit of the following: commit 3637b056f2d2d947658767f11a5f29b8c664e817 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:24 2020 -0700 fix date display and all autocompletes commit 02d14266742e64620b17787049d8d6f0594258ea Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:35:32 2020 -0700 align issues better commit 7f8efe405e883b686a10cdf8d84e0e4edd41f1fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:49 2020 -0700 fix language and country display commit 0dc983f501c421bbd223e5348abdf096337f67f4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:25 2020 -0700 wipe comic data before lookup commit fa7236f6a74e061ecb475672c72dff4de19134e0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:29:42 2020 -0700 fix pycountry lookup bug commit 9a6bb655e51d3f9416b5bfc3bfb2feb6ab3b72b4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:49:28 2020 -0700 optimize getting only the aggregate filter for metadata commit 5129983d1f589cba0acad9edbe921203645c4db4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:37:38 2020 -0700 compress all metdata consts into the QUERY_SERIALIZER map commit c7e35a64fd491fedbee6ba4682975a5afb54df2b Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:14:02 2020 -0700 move frontend api files to versioned directory commit 49389216c3fb60718aceb45f34fe942e46cb45fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:08:34 2020 -0700 rename variables. clean up cruft. comments commit 39788fc1fcce0c8ce131da1be03be2284766d386 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:53:31 2020 -0700 lint and comments commit 3baa071e140c6f46bab63bed10bbfab13e546e5a Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:40 2020 -0700 lint commit 49565bc5c50c7b058e76837f8ab382db3caa63c8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:36 2020 -0700 lint ci scripts as well commit e1cac1604f0ae5e7f83fd94e9125c96c4b3104a6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:45:29 2020 -0700 add a progress bar to metdata commit d0e2412ae18b9a5f75f64dde1ab770e1968f1927 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:38:21 2020 -0700 move top book cover decorations into book-cover so they display on the metadata page commit 07426ce9fdacab8c980deaebac2e913f8c3c1eba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:55:20 2020 -0700 simpler tag icon is better commit f8e8ed39b725accffa57ba7f4978515e16a6fc4e Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:54:16 2020 -0700 Tag multiple icon commit a9a79e656cf20f141f44eb3e1b8c7b542ea27d6b Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:47:20 2020 -0700 remove log commit ee14d91ebfa3c96597003d50da2bed96351665cc Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:44:58 2020 -0700 change metadata icon commit 9b76a9bc7ca5a445d17bf505823241428588063f Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:31:11 2020 -0700 format issue decimals commit a49061082610264d7aaed1d38d1bf24f0baaddf6 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:16:02 2020 -0700 fix issues to handle years commit 08bd7639ebca0d68cf4bff985725681be72c7dba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 19:39:48 2020 -0700 credits working commit 268df3cd136972c7bc28aa7d0ea1761f4a2d24e9 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 18:20:51 2020 -0700 working even better. credits, issue display and folder test remain commit a5d6d88244fb1536c840789857c7624bb8cd2c4d Author: AJ Slater <aj@slater.net> Date: Mon Sep 21 19:31:49 2020 -0700 kind of working but kludgy backend solution commit 05c67e90052cc69010c20bd9d2c39d0e21745032 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 16:54:50 2020 -0700 move children and page count into base class commit c0f337bcde505729380caa10a6ea665fa2d5aa38 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 15:14:13 2020 -0700 move bookmark aggregation up into metadataclass for real. remove unused fit_to & two_pages commit 7c3ab1eb99e433b1181c4a3ff319329f1bd0a287 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 14:08:31 2020 -0700 move bookmark annotations to browse_metadata_base * speed up metadata by 50% * Squashed commit of the following: commit 97c87f28bbd4285efb89051de6c3844f1948c80f Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:09:01 2020 -0700 lint and comments. remove cruft commit 4d10bd16f07ad2e3c83ccb963f5e90978a61b214 Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:02:02 2020 -0700 major cleanup commit 8a23d7aa13702b0d75e4d1576e8ae75b03329af8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 20:15:59 2020 -0700 partial cleanups commit 27b66e26aa0d302364d1576c52140a0ea060b222 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:56:06 2020 -0700 isort commit 1c320e2187dabda7b3cc6bc55423902add58b3d0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:54:37 2020 -0700 fix related field counts and cleanup a bit of cruft. more cleanup to do. commit bd045eb0692e9232c51dd33053801d407c415039 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:59:09 2020 -0700 it motherfucking works commit 752d1773671196a4bbb3472490f35d394df806d7 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:48:58 2020 -0700 working agg all. now just do pks commit af6b59b69824836f8485892d83ad7156d055372e Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:56:57 2020 -0700 coment. lint commit f678f8f35d9724c9eca514e85c8372d6c534621a Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:34:21 2020 -0700 clean up metadata container aggregate a bit commit 49ca05eedf6f9a6ade3682272c35d7e0733203c9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:14:01 2020 -0700 simplify a map commit fc2a6d4d0834ccc056474414b6d15d61bbffe7df Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 11:59:21 2020 -0700 major cleanup of metadata view commit 0a90f5c5c75ec2a60f00334dcddc8d5eb95bf641 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:28 2020 -0700 more explicit nginx docs commit fe54d13f985d796b95593efb9f8e6c1ad539d879 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:18 2020 -0700 clean up metadata serializers commit 0e182239e8525ee33a8b095b6b406cd567e7acf9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:04 2020 -0700 remove a little cruft commit f8972d6b19c0f9c829d605c6ddb6fd229cef2466 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 19:37:45 2020 -0700 ugly but working bufix for too much data commit 0f4d7d70c4c8b44a8e629bbee217df742b76de08 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 16:38:35 2020 -0700 new metadata api works on frontend. backend bug gives too much info for groups commit 4241cacf42e8b4cf27a08c657f672694be1bde4a Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 00:20:50 2020 -0700 lang serialiezer returns value if text lookup fails commit c9a6465fd913185c2df340c082d1e0985b6fa189 Author: AJ Slater <aj@slater.net> Date: Mon Sep 28 18:18:10 2020 -0700 doubled speed of metadata * bump version * Squashed commit of the following: commit 88404c565b4cc3fbad899ca1bccd366b43b199b5 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 14:54:40 2020 -0700 remove TEST variable. always a bad idea commit a3bef259dc6c72b0a8bb885d0353d7a8bad1b81f Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:58:02 2020 -0700 comments commit 265784ce9fdf5fe2429fda3fa98f271c2275667b Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:54:39 2020 -0700 optimize select_related for metadata just a little more commit a488427a15ab8d0efb4a742308a0696d313492dc Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:45:15 2020 -0700 pop fields before serialization. should be faster commit 07df6a610a7683cf16551c4e644207d10be96f27 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:39:42 2020 -0700 clean up serializers and choices. get choices from src too. commit 5d59c5a95dcc6194f76d45c578efdd529a895f90 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:19:19 2020 -0700 fix improper null items on undefined metadata fields commit 905aa375760b374f3e5c8ccdc29f074cf2e3e754 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:18:59 2020 -0700 fix metadata folder view commit d6017dd76297f448735198e1a3a1c9f111865a2e Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 23:15:54 2020 -0700 reading choices and messages from the webpack json works now commit 87246c285ac93674eff1ba55be651a05e967102b Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:14:37 2020 -0700 move item xformer into api commit 102b291511cc3e6c34d7efe2f4e7839d5637766c Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:04:05 2020 -0700 fix null items commit b7212c3e3a47987d1b20ba604aba408555b32f70 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:44:22 2020 -0700 sorted items commit 5c8cef6cd7eb85c246f3099b35a71f2b87555e51 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:27:26 2020 -0700 componentize browser-settings-menu commit d3fd0f57ea0d5500389e97e1c071896abac3791a Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:17:06 2020 -0700 componentize browser settings dialog commit 8510179f3be465df0d7848751e62a699902dd458 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:05:52 2020 -0700 working filter menu. componentize top toolbar filters. * massive rename of containers and browse for consistency * docstyle linting * lint * remove mypy and vulture. i don't use them. * Squashed commit of the following: commit 936a6c8a202b09db2c8d6bedc8ce1d8add3e3f5e Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:13:22 2020 -0700 move secret key and logging init management into modules. websocket server doesn't configure logging right. commit 328bb8288bbee74b88c0e0754bc13abdb7ea114d Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:35:23 2020 -0700 less picky about passwords commit 8a953e20676ac5ca7e5c226b8792e075dbae1d11 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:17:14 2020 -0700 hypercorn 0.11.0 includes asgi signals and a big logging change. fix the logging with a custom formatter and a flag to logconfig_dict commit e3eae3400a2c07cab11997b74a71397e70f9d3e2 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:20 2020 -0700 upgrade babel/core commit 16034b60a7fb41fe47d67a8c02972231094f9946 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:04 2020 -0700 remove comment cruft commit 23885db544421469e46d0d42a75844c73b082fa5 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:13:31 2020 -0700 dev run script alter python path for local dir commit 80b92b45921dadc9d6a3811f7e92df5bd9377505 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:46 2020 -0700 correct logging __name__ and remove print cruft commit 2a6d347dcc71e39a88b2e7c515c8b2fcae07b0de Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:24 2020 -0700 document where i got the timezones middleware commit 59778ffa39967f7b6e42a5bf4fb4c52c8fbe5ce4 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:04 2020 -0700 debug lifespan logging commit b3ae98d90fbf3e9f1bf4b5eb17fb467c63f21f54 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:11:35 2020 -0700 rename procs & threads for logging commit 77f064e89aaa639c97b1bf7aeb4748a2fefe04d4 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 21:55:52 2020 -0700 broken asgi signals handling? * Squashed commit of the following: commit 4eca7486fa16467cfee0e38f4d6560c8b8487d12 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:19:12 2020 -0700 replace old DEV flag with DEBUG remove redundancy * lint * stop pytest from testing migrations * oops broke flake8 * Squashed commit of the following: commit efa7750ba35e3e32ba005a01cf53aab949c6993c Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:27:34 2020 -0700 componentize browser-main commit 3e9caadb2ffa7e7b679ab813d4c5acb4a7f366cb Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:06:01 2020 -0700 oops forgot to commit settings file for module change commit ca5168afd78bf05bdc7572120c746beb2e93ef22 Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:05:31 2020 -0700 settings module commit 54af69ae2562fdb7b101ef57d5ebbf9dd8088482 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 20:25:01 2020 -0700 componentize browser * lint * componentize reader * remove todo suggestion * more metadata fields * update news * fix restarts * its closer to 5 seconds in practice * update poetry and npm packages * change order of stop and start daemons a bit * fix finding webpack files. Use regex instead of glob * bump news and version

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir * ensure all STATIC_DIRS exist for whitenoise * new black. update poetry, npm & lint * 0.5.4 version bump * only use the build type static files in dev mode. always should've been like this. * collect static needs dev marker to actually collect the dev static dirs * revert whitenoise * codex v0.5.5 * fix logging error * borders on browse-cards * fix border interference with unread flag * v0.5.6 * comicbox 1.3 * allow null credit roles * V0.5.7 (#35) * v0.5.7 news and version * remove transparency from child count * center pages better in reader. still has a two page gap. * scroll to the top of the page when the route changes * more explicit read/unread menu. Add download comic to menu * fix previous comic always start of series bug * lint migrations * npm update * logfiles * more items to news * lint * reorg setup-dev.sh * Docker caching for wheel builder image * remove scanlocks on startup. fix library change by not disabling element * Squashed commit of the following: commit e1994b7d8f76d1aff43de8fddd927e3ac6686502 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:37:37 2020 -0700 bum news commit f230dd6590b2c0e404f5820dca76810207c74a74 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:33:23 2020 -0700 remove inconsistent document.title setting commit 129323516a24bb1959f1e5cee244af56151b273f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:29:20 2020 -0700 restore child counts and bookmarks commit 21f25ec7b15822b5ee51c2c97b4a700d584306fb Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:42:49 2020 -0700 modular frontend comic name code. disambiguate volumes more commit 3f8c7a4fedf931b81da9196147f6556632194fbf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:46 2020 -0700 volume parsing on the back end commit 557ace75ccb3522302a1b402466cb19d75a38b22 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:19 2020 -0700 fix issue type in browser commit 4d2331d240f0b78c48e2cc365dc4d41f9155ff7d Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:55:02 2020 -0700 remove extra loggins commit 566e353f78ce81f502b9ceac103f0f41b300b91c Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:52:40 2020 -0700 move ensure cover path to serializer so we only hit the db once commit adfeda4d5d3a91e355ae55e6182411b2cb7f73bf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:15:12 2020 -0700 pass series_name, volume_name, issue as fields. no more janky parsing commit 361e0ad253356ea7d5733bb1ca54e2e80173c9e4 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:48:43 2020 -0700 rename containerList commit 830454081e9163039dbd7893a91b099b7f094380 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:45:53 2020 -0700 fix up route not starting from 0 commit d17e0de419e9dd0c715d46f71e856c77a619a845 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:34:47 2020 -0700 pagination union with header name annotated instead of python computed commit b1aac346d372dd5a41397c53428de0303733958a Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:06:44 2020 -0700 npm update commit 69414ec4e09c97c7a6c6c7a7da3406b71da7f178 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:24 2020 -0700 update poetry commit e34d4af0c79d86f7ca2bc072be9ee5b8965ae54f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:04 2020 -0700 clear cache on startup commit 34a96e36638d98be77ed5de3681c7011b4d2b6a2 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:00:38 2020 -0700 fix last_scan create library bug commit fef826cd24f2cd0315113a45f2c8eee2dcbf9667 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:38:20 2020 -0700 pagination proof of concept. breaks with folder view commit 01eefe612e6e63556e6f58545e1fbf9846689f85 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:34:46 2020 -0700 lint * friendly /me message * Squashed commit of the following: commit b73eab2312d1648dc24203ec7bca712d820daeb2 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:15:54 2020 -0700 remove unused var commit 12d990586f00f9b6c65eade9495de3ed65aea1fe Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:11:25 2020 -0700 watch for user changes in the browse app, not the auth app for browse stuff. commit 9ba8575c0e4f4c011fde28ee2ec37d206063700f Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:10:34 2020 -0700 consolidate route generation in browse store and add page * reform browse console logging * Squashed commit of the following: commit d72c5eda5744f68d442e91f942bf7e8077271976 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:34:00 2020 -0700 refactor browse card to be a single card object. * oder the common browse fields * fix cover on metadata page * move progress bar to browse card for better positioning. * more tests for comic model * bump codex version oops * v0.5.8 * poetry update * more helpful 404 page * V0.5.9 (#50) * poetry update * more helpful 404 page * update npm * fix logging bug on null role import * Squashed commit of the following: commit 0980b6e2ce4d8a7b5a55722427febb3d6d35b05e Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 18:09:12 2020 -0700 fix folder sorts. reorganize annotation code. eliminate x_order_key in favor of order_value commit 4797fc411aa6d43d43a66ff98a90641c6c8338e4 Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 15:39:43 2020 -0700 first draft of folder sort fixes. needs cleanup * Squashed commit of the following: commit 36329ac2572bd95d55fe2e6b79156a0355eea42b Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 19:25:15 2020 -0700 missing cover support * poetry update * version bump * comicbox 0.1.4 * lint * Squashed commit of the following: commit 9f22f50e97ecc743270e26217012d8d5097016dc Author: AJ Slater <aj@slater.net> Date: Tue Sep 8 13:53:41 2020 -0700 use base images when building runnable docker * V0.5.10 (#53) * fix filtering bug * bump version * clean up comments * fix reader settings * mostly fix reader image layout * fix reader to browser uninitialized error * bump news * update poetry * npm update * simpler image labels * V0.5.11 (#55) * simpler image labels * simply image tagging even more. publish to pypi with ci * ignore node_modules of course * use Min instead of Max for sorting containers. Added benefit is covers match sort values now * flex layout to the rescue for adaptive browse tile row heights. * fix bookmarks not working for sessions * update poetry & npm * bump version to 0.5.11 & news * fix mobile safari bug with stretched images * Squashed commit of the following: commit 8573d66e625a99288cc52092aad33c14845bb631 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 13:05:53 2020 -0700 fix the whitenoise prefixing for dev mode commit 46700920e7da71e0a1779eb0a701ac321dad1d71 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 12:33:35 2020 -0700 works in prod * abuse WHITENOISE_ROOT for covers so I can turn off WHITENOISE_USE_FINDERS in production * poorly spelled variable prevented setting global reader settings * bump news * another news bump * admin failed imports list * Squashed commit of the following: commit 3a3e206df83ee9ad103b2c7600a2ae1b3fb305f8 Author: AJ Slater <aj@slater.net> Date: Mon Sep 14 17:42:25 2020 -0700 scan notify snack * bump version and news * lint and cleanup prints * Squashed commit of the following: commit ee19a333161c174b387a4b2b525d04ec1d549b91 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 14:38:45 2020 -0700 latest version * Squashed commit of the following: commit 6001d4a338bf9dc86238b0da565e76a891d3de02 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:55:53 2020 -0700 remove the init_admin_flags from migrations commit 38556ea04959e02522c08288c152f61ea18b7a3a Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:47:38 2020 -0700 minor cleanups and comments commit 3e40d41c52b188353b0d04e96c7e35dec5315d95 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:31:46 2020 -0700 bump news commit 1321826d522c32cb62a13abb1ea3df8173f8cf3d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 18:00:20 2020 -0700 add dev config dir to prettierignore commit 784e165efc57f6f97092e8d17236a0bde40b226d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 17:52:09 2020 -0700 disable session users first try * change failed import comment * v0.5.13 * crap i'm sure now * Squashed commit of the following: commit be04fac73fe3fdcd2ee600aac8106125f45bff7c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:56:48 2020 -0700 lint commit f93023cf01b2e2f8fe0e83fdcd0c051ed1a8428e Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:55:47 2020 -0700 bump news commit 2637d430f90a382145218d691e64e28939c4f024 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:53:27 2020 -0700 poetry and npm update commit a7d8af755fea91b202e018ce63d084dc22229daa Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:49:01 2020 -0700 add force to comic cover creation commit c6b11e3ea7866c980bc0e4b9b0772d39f81a3052 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:25:20 2020 -0700 fix unable to delete libs commit af224ffc3b34b172ecc2ae84c848fcf1d64bf5c0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 17:08:07 2020 -0700 rehide edit button and fix issueRow layout on mobile commit 17bee0af42a14d1d53285f6ef8f4c583888a36c7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:50:27 2020 -0700 minor cleanup of checkbox commit c7f235af7abd00914c874b2fb65ebd1be545e1a4 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:33:27 2020 -0700 remove comment cruft commit 073b6f08e39100aa71dc33eee6969abb473164fc Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:32:00 2020 -0700 bump version & news commit 0c8bdfb5f216df4353533d9371437caa65dfb7f7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:29:14 2020 -0700 dict for websocket messages commit 7a8e1e9af9bd57a93b08d9f587435ed4f47f525b Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 15:51:41 2020 -0700 minor cleanups commit d1ad0ef4159aade2793c424be693f4dd3d36f943 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 14:01:29 2020 -0700 remove cruft commit df1d69ef2d28fd88049032d8ee7c6d21163708f6 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:56:06 2020 -0700 minor cleanups and formalize timing for metadata commit 0e68de8a3b85bb05657b83f7d2871f02094696a0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:48:51 2020 -0700 two minute timeout! commit 650099da1be7510c25fb0c1ed562fdeb56a31e2c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:15:34 2020 -0700 cleanups for metadata commit 5c06d6c8d43f3400d92bfa9062851951b0ea9981 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:30:56 2020 -0700 progress meter for metadata commit 7775cf15f56f347136817bb1bf33f00091b13a05 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:29:43 2020 -0700 fix no credits bug. fix pk lookup syntax commit 70688d27a01bc2eb75f8a7cce007ec2d92fbeddb Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:53 2020 -0700 lint commit 72b53b462cb7d5feebbe4fea6159fc3f9588fbc3 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:47 2020 -0700 move comic-name into components commit f300f21097bb0d194ce1359eaf37b25c7f23ce20 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:45 2020 -0700 show less metadata if not in editMode commit 7731a45e630108eae0a254d49fd009a6293e8247 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:10 2020 -0700 first minort cleanups commit 30399ee10b665e39c30cef5a5332b2f31c8459f9 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:54 2020 -0700 freeze the comic metadata it won't change ... yet? commit ded8672c4c0f437febcbbb0ee393cd7b056d9216 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:20 2020 -0700 fix null issueChoices commit 5ecab8eb652941ff208c31cd5e1b7734a40da73f Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:02 2020 -0700 share initialItem across autocomplete and combobox commit 02a4e2f71c1662d6874741d025e4e10ac315f6e6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 16:21:59 2020 -0700 placeholder for metdata commit c4a4c37dee9f243626f8137dc6d5c35266dc6dbd Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:45 2020 -0700 Squashed commit of the following: commit 3637b056f2d2d947658767f11a5f29b8c664e817 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:24 2020 -0700 fix date display and all autocompletes commit 02d14266742e64620b17787049d8d6f0594258ea Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:35:32 2020 -0700 align issues better commit 7f8efe405e883b686a10cdf8d84e0e4edd41f1fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:49 2020 -0700 fix language and country display commit 0dc983f501c421bbd223e5348abdf096337f67f4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:25 2020 -0700 wipe comic data before lookup commit fa7236f6a74e061ecb475672c72dff4de19134e0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:29:42 2020 -0700 fix pycountry lookup bug commit 9a6bb655e51d3f9416b5bfc3bfb2feb6ab3b72b4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:49:28 2020 -0700 optimize getting only the aggregate filter for metadata commit 5129983d1f589cba0acad9edbe921203645c4db4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:37:38 2020 -0700 compress all metdata consts into the QUERY_SERIALIZER map commit c7e35a64fd491fedbee6ba4682975a5afb54df2b Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:14:02 2020 -0700 move frontend api files to versioned directory commit 49389216c3fb60718aceb45f34fe942e46cb45fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:08:34 2020 -0700 rename variables. clean up cruft. comments commit 39788fc1fcce0c8ce131da1be03be2284766d386 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:53:31 2020 -0700 lint and comments commit 3baa071e140c6f46bab63bed10bbfab13e546e5a Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:40 2020 -0700 lint commit 49565bc5c50c7b058e76837f8ab382db3caa63c8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:36 2020 -0700 lint ci scripts as well commit e1cac1604f0ae5e7f83fd94e9125c96c4b3104a6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:45:29 2020 -0700 add a progress bar to metdata commit d0e2412ae18b9a5f75f64dde1ab770e1968f1927 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:38:21 2020 -0700 move top book cover decorations into book-cover so they display on the metadata page commit 07426ce9fdacab8c980deaebac2e913f8c3c1eba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:55:20 2020 -0700 simpler tag icon is better commit f8e8ed39b725accffa57ba7f4978515e16a6fc4e Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:54:16 2020 -0700 Tag multiple icon commit a9a79e656cf20f141f44eb3e1b8c7b542ea27d6b Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:47:20 2020 -0700 remove log commit ee14d91ebfa3c96597003d50da2bed96351665cc Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:44:58 2020 -0700 change metadata icon commit 9b76a9bc7ca5a445d17bf505823241428588063f Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:31:11 2020 -0700 format issue decimals commit a49061082610264d7aaed1d38d1bf24f0baaddf6 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:16:02 2020 -0700 fix issues to handle years commit 08bd7639ebca0d68cf4bff985725681be72c7dba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 19:39:48 2020 -0700 credits working commit 268df3cd136972c7bc28aa7d0ea1761f4a2d24e9 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 18:20:51 2020 -0700 working even better. credits, issue display and folder test remain commit a5d6d88244fb1536c840789857c7624bb8cd2c4d Author: AJ Slater <aj@slater.net> Date: Mon Sep 21 19:31:49 2020 -0700 kind of working but kludgy backend solution commit 05c67e90052cc69010c20bd9d2c39d0e21745032 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 16:54:50 2020 -0700 move children and page count into base class commit c0f337bcde505729380caa10a6ea665fa2d5aa38 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 15:14:13 2020 -0700 move bookmark aggregation up into metadataclass for real. remove unused fit_to & two_pages commit 7c3ab1eb99e433b1181c4a3ff319329f1bd0a287 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 14:08:31 2020 -0700 move bookmark annotations to browse_metadata_base * speed up metadata by 50% * Squashed commit of the following: commit 97c87f28bbd4285efb89051de6c3844f1948c80f Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:09:01 2020 -0700 lint and comments. remove cruft commit 4d10bd16f07ad2e3c83ccb963f5e90978a61b214 Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:02:02 2020 -0700 major cleanup commit 8a23d7aa13702b0d75e4d1576e8ae75b03329af8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 20:15:59 2020 -0700 partial cleanups commit 27b66e26aa0d302364d1576c52140a0ea060b222 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:56:06 2020 -0700 isort commit 1c320e2187dabda7b3cc6bc55423902add58b3d0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:54:37 2020 -0700 fix related field counts and cleanup a bit of cruft. more cleanup to do. commit bd045eb0692e9232c51dd33053801d407c415039 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:59:09 2020 -0700 it motherfucking works commit 752d1773671196a4bbb3472490f35d394df806d7 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:48:58 2020 -0700 working agg all. now just do pks commit af6b59b69824836f8485892d83ad7156d055372e Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:56:57 2020 -0700 coment. lint commit f678f8f35d9724c9eca514e85c8372d6c534621a Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:34:21 2020 -0700 clean up metadata container aggregate a bit commit 49ca05eedf6f9a6ade3682272c35d7e0733203c9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:14:01 2020 -0700 simplify a map commit fc2a6d4d0834ccc056474414b6d15d61bbffe7df Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 11:59:21 2020 -0700 major cleanup of metadata view commit 0a90f5c5c75ec2a60f00334dcddc8d5eb95bf641 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:28 2020 -0700 more explicit nginx docs commit fe54d13f985d796b95593efb9f8e6c1ad539d879 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:18 2020 -0700 clean up metadata serializers commit 0e182239e8525ee33a8b095b6b406cd567e7acf9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:04 2020 -0700 remove a little cruft commit f8972d6b19c0f9c829d605c6ddb6fd229cef2466 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 19:37:45 2020 -0700 ugly but working bufix for too much data commit 0f4d7d70c4c8b44a8e629bbee217df742b76de08 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 16:38:35 2020 -0700 new metadata api works on frontend. backend bug gives too much info for groups commit 4241cacf42e8b4cf27a08c657f672694be1bde4a Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 00:20:50 2020 -0700 lang serialiezer returns value if text lookup fails commit c9a6465fd913185c2df340c082d1e0985b6fa189 Author: AJ Slater <aj@slater.net> Date: Mon Sep 28 18:18:10 2020 -0700 doubled speed of metadata * bump version * Squashed commit of the following: commit 88404c565b4cc3fbad899ca1bccd366b43b199b5 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 14:54:40 2020 -0700 remove TEST variable. always a bad idea commit a3bef259dc6c72b0a8bb885d0353d7a8bad1b81f Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:58:02 2020 -0700 comments commit 265784ce9fdf5fe2429fda3fa98f271c2275667b Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:54:39 2020 -0700 optimize select_related for metadata just a little more commit a488427a15ab8d0efb4a742308a0696d313492dc Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:45:15 2020 -0700 pop fields before serialization. should be faster commit 07df6a610a7683cf16551c4e644207d10be96f27 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:39:42 2020 -0700 clean up serializers and choices. get choices from src too. commit 5d59c5a95dcc6194f76d45c578efdd529a895f90 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:19:19 2020 -0700 fix improper null items on undefined metadata fields commit 905aa375760b374f3e5c8ccdc29f074cf2e3e754 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:18:59 2020 -0700 fix metadata folder view commit d6017dd76297f448735198e1a3a1c9f111865a2e Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 23:15:54 2020 -0700 reading choices and messages from the webpack json works now commit 87246c285ac93674eff1ba55be651a05e967102b Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:14:37 2020 -0700 move item xformer into api commit 102b291511cc3e6c34d7efe2f4e7839d5637766c Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:04:05 2020 -0700 fix null items commit b7212c3e3a47987d1b20ba604aba408555b32f70 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:44:22 2020 -0700 sorted items commit 5c8cef6cd7eb85c246f3099b35a71f2b87555e51 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:27:26 2020 -0700 componentize browser-settings-menu commit d3fd0f57ea0d5500389e97e1c071896abac3791a Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:17:06 2020 -0700 componentize browser settings dialog commit 8510179f3be465df0d7848751e62a699902dd458 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:05:52 2020 -0700 working filter menu. componentize top toolbar filters. * massive rename of containers and browse for consistency * docstyle linting * lint * remove mypy and vulture. i don't use them. * Squashed commit of the following: commit 936a6c8a202b09db2c8d6bedc8ce1d8add3e3f5e Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:13:22 2020 -0700 move secret key and logging init management into modules. websocket server doesn't configure logging right. commit 328bb8288bbee74b88c0e0754bc13abdb7ea114d Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:35:23 2020 -0700 less picky about passwords commit 8a953e20676ac5ca7e5c226b8792e075dbae1d11 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:17:14 2020 -0700 hypercorn 0.11.0 includes asgi signals and a big logging change. fix the logging with a custom formatter and a flag to logconfig_dict commit e3eae3400a2c07cab11997b74a71397e70f9d3e2 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:20 2020 -0700 upgrade babel/core commit 16034b60a7fb41fe47d67a8c02972231094f9946 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:04 2020 -0700 remove comment cruft commit 23885db544421469e46d0d42a75844c73b082fa5 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:13:31 2020 -0700 dev run script alter python path for local dir commit 80b92b45921dadc9d6a3811f7e92df5bd9377505 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:46 2020 -0700 correct logging __name__ and remove print cruft commit 2a6d347dcc71e39a88b2e7c515c8b2fcae07b0de Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:24 2020 -0700 document where i got the timezones middleware commit 59778ffa39967f7b6e42a5bf4fb4c52c8fbe5ce4 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:04 2020 -0700 debug lifespan logging commit b3ae98d90fbf3e9f1bf4b5eb17fb467c63f21f54 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:11:35 2020 -0700 rename procs & threads for logging commit 77f064e89aaa639c97b1bf7aeb4748a2fefe04d4 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 21:55:52 2020 -0700 broken asgi signals handling? * Squashed commit of the following: commit 4eca7486fa16467cfee0e38f4d6560c8b8487d12 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:19:12 2020 -0700 replace old DEV flag with DEBUG remove redundancy * lint * stop pytest from testing migrations * oops broke flake8 * Squashed commit of the following: commit efa7750ba35e3e32ba005a01cf53aab949c6993c Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:27:34 2020 -0700 componentize browser-main commit 3e9caadb2ffa7e7b679ab813d4c5acb4a7f366cb Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:06:01 2020 -0700 oops forgot to commit settings file for module change commit ca5168afd78bf05bdc7572120c746beb2e93ef22 Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:05:31 2020 -0700 settings module commit 54af69ae2562fdb7b101ef57d5ebbf9dd8088482 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 20:25:01 2020 -0700 componentize browser * lint * componentize reader * remove todo suggestion * more metadata fields * update news * fix restarts * its closer to 5 seconds in practice * update poetry and npm packages * change order of stop and start daemons a bit * fix finding webpack files. Use regex instead of glob * bump news and version * fix root_path bug. v0.5.17

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir * ensure all STATIC_DIRS exist for whitenoise * new black. update poetry, npm & lint * 0.5.4 version bump * only use the build type static files in dev mode. always should've been like this. * collect static needs dev marker to actually collect the dev static dirs * revert whitenoise * codex v0.5.5 * fix logging error * borders on browse-cards * fix border interference with unread flag * v0.5.6 * comicbox 1.3 * allow null credit roles * V0.5.7 (#35) * v0.5.7 news and version * remove transparency from child count * center pages better in reader. still has a two page gap. * scroll to the top of the page when the route changes * more explicit read/unread menu. Add download comic to menu * fix previous comic always start of series bug * lint migrations * npm update * logfiles * more items to news * lint * reorg setup-dev.sh * Docker caching for wheel builder image * remove scanlocks on startup. fix library change by not disabling element * Squashed commit of the following: commit e1994b7d8f76d1aff43de8fddd927e3ac6686502 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:37:37 2020 -0700 bum news commit f230dd6590b2c0e404f5820dca76810207c74a74 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:33:23 2020 -0700 remove inconsistent document.title setting commit 129323516a24bb1959f1e5cee244af56151b273f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:29:20 2020 -0700 restore child counts and bookmarks commit 21f25ec7b15822b5ee51c2c97b4a700d584306fb Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:42:49 2020 -0700 modular frontend comic name code. disambiguate volumes more commit 3f8c7a4fedf931b81da9196147f6556632194fbf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:46 2020 -0700 volume parsing on the back end commit 557ace75ccb3522302a1b402466cb19d75a38b22 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:19 2020 -0700 fix issue type in browser commit 4d2331d240f0b78c48e2cc365dc4d41f9155ff7d Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:55:02 2020 -0700 remove extra loggins commit 566e353f78ce81f502b9ceac103f0f41b300b91c Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:52:40 2020 -0700 move ensure cover path to serializer so we only hit the db once commit adfeda4d5d3a91e355ae55e6182411b2cb7f73bf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:15:12 2020 -0700 pass series_name, volume_name, issue as fields. no more janky parsing commit 361e0ad253356ea7d5733bb1ca54e2e80173c9e4 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:48:43 2020 -0700 rename containerList commit 830454081e9163039dbd7893a91b099b7f094380 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:45:53 2020 -0700 fix up route not starting from 0 commit d17e0de419e9dd0c715d46f71e856c77a619a845 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:34:47 2020 -0700 pagination union with header name annotated instead of python computed commit b1aac346d372dd5a41397c53428de0303733958a Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:06:44 2020 -0700 npm update commit 69414ec4e09c97c7a6c6c7a7da3406b71da7f178 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:24 2020 -0700 update poetry commit e34d4af0c79d86f7ca2bc072be9ee5b8965ae54f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:04 2020 -0700 clear cache on startup commit 34a96e36638d98be77ed5de3681c7011b4d2b6a2 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:00:38 2020 -0700 fix last_scan create library bug commit fef826cd24f2cd0315113a45f2c8eee2dcbf9667 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:38:20 2020 -0700 pagination proof of concept. breaks with folder view commit 01eefe612e6e63556e6f58545e1fbf9846689f85 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:34:46 2020 -0700 lint * friendly /me message * Squashed commit of the following: commit b73eab2312d1648dc24203ec7bca712d820daeb2 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:15:54 2020 -0700 remove unused var commit 12d990586f00f9b6c65eade9495de3ed65aea1fe Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:11:25 2020 -0700 watch for user changes in the browse app, not the auth app for browse stuff. commit 9ba8575c0e4f4c011fde28ee2ec37d206063700f Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:10:34 2020 -0700 consolidate route generation in browse store and add page * reform browse console logging * Squashed commit of the following: commit d72c5eda5744f68d442e91f942bf7e8077271976 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:34:00 2020 -0700 refactor browse card to be a single card object. * oder the common browse fields * fix cover on metadata page * move progress bar to browse card for better positioning. * more tests for comic model * bump codex version oops * v0.5.8 * poetry update * more helpful 404 page * V0.5.9 (#50) * poetry update * more helpful 404 page * update npm * fix logging bug on null role import * Squashed commit of the following: commit 0980b6e2ce4d8a7b5a55722427febb3d6d35b05e Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 18:09:12 2020 -0700 fix folder sorts. reorganize annotation code. eliminate x_order_key in favor of order_value commit 4797fc411aa6d43d43a66ff98a90641c6c8338e4 Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 15:39:43 2020 -0700 first draft of folder sort fixes. needs cleanup * Squashed commit of the following: commit 36329ac2572bd95d55fe2e6b79156a0355eea42b Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 19:25:15 2020 -0700 missing cover support * poetry update * version bump * comicbox 0.1.4 * lint * Squashed commit of the following: commit 9f22f50e97ecc743270e26217012d8d5097016dc Author: AJ Slater <aj@slater.net> Date: Tue Sep 8 13:53:41 2020 -0700 use base images when building runnable docker * V0.5.10 (#53) * fix filtering bug * bump version * clean up comments * fix reader settings * mostly fix reader image layout * fix reader to browser uninitialized error * bump news * update poetry * npm update * simpler image labels * V0.5.11 (#55) * simpler image labels * simply image tagging even more. publish to pypi with ci * ignore node_modules of course * use Min instead of Max for sorting containers. Added benefit is covers match sort values now * flex layout to the rescue for adaptive browse tile row heights. * fix bookmarks not working for sessions * update poetry & npm * bump version to 0.5.11 & news * fix mobile safari bug with stretched images * Squashed commit of the following: commit 8573d66e625a99288cc52092aad33c14845bb631 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 13:05:53 2020 -0700 fix the whitenoise prefixing for dev mode commit 46700920e7da71e0a1779eb0a701ac321dad1d71 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 12:33:35 2020 -0700 works in prod * abuse WHITENOISE_ROOT for covers so I can turn off WHITENOISE_USE_FINDERS in production * poorly spelled variable prevented setting global reader settings * bump news * another news bump * admin failed imports list * Squashed commit of the following: commit 3a3e206df83ee9ad103b2c7600a2ae1b3fb305f8 Author: AJ Slater <aj@slater.net> Date: Mon Sep 14 17:42:25 2020 -0700 scan notify snack * bump version and news * lint and cleanup prints * Squashed commit of the following: commit ee19a333161c174b387a4b2b525d04ec1d549b91 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 14:38:45 2020 -0700 latest version * Squashed commit of the following: commit 6001d4a338bf9dc86238b0da565e76a891d3de02 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:55:53 2020 -0700 remove the init_admin_flags from migrations commit 38556ea04959e02522c08288c152f61ea18b7a3a Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:47:38 2020 -0700 minor cleanups and comments commit 3e40d41c52b188353b0d04e96c7e35dec5315d95 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:31:46 2020 -0700 bump news commit 1321826d522c32cb62a13abb1ea3df8173f8cf3d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 18:00:20 2020 -0700 add dev config dir to prettierignore commit 784e165efc57f6f97092e8d17236a0bde40b226d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 17:52:09 2020 -0700 disable session users first try * change failed import comment * v0.5.13 * crap i'm sure now * Squashed commit of the following: commit be04fac73fe3fdcd2ee600aac8106125f45bff7c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:56:48 2020 -0700 lint commit f93023cf01b2e2f8fe0e83fdcd0c051ed1a8428e Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:55:47 2020 -0700 bump news commit 2637d430f90a382145218d691e64e28939c4f024 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:53:27 2020 -0700 poetry and npm update commit a7d8af755fea91b202e018ce63d084dc22229daa Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:49:01 2020 -0700 add force to comic cover creation commit c6b11e3ea7866c980bc0e4b9b0772d39f81a3052 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:25:20 2020 -0700 fix unable to delete libs commit af224ffc3b34b172ecc2ae84c848fcf1d64bf5c0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 17:08:07 2020 -0700 rehide edit button and fix issueRow layout on mobile commit 17bee0af42a14d1d53285f6ef8f4c583888a36c7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:50:27 2020 -0700 minor cleanup of checkbox commit c7f235af7abd00914c874b2fb65ebd1be545e1a4 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:33:27 2020 -0700 remove comment cruft commit 073b6f08e39100aa71dc33eee6969abb473164fc Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:32:00 2020 -0700 bump version & news commit 0c8bdfb5f216df4353533d9371437caa65dfb7f7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:29:14 2020 -0700 dict for websocket messages commit 7a8e1e9af9bd57a93b08d9f587435ed4f47f525b Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 15:51:41 2020 -0700 minor cleanups commit d1ad0ef4159aade2793c424be693f4dd3d36f943 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 14:01:29 2020 -0700 remove cruft commit df1d69ef2d28fd88049032d8ee7c6d21163708f6 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:56:06 2020 -0700 minor cleanups and formalize timing for metadata commit 0e68de8a3b85bb05657b83f7d2871f02094696a0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:48:51 2020 -0700 two minute timeout! commit 650099da1be7510c25fb0c1ed562fdeb56a31e2c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:15:34 2020 -0700 cleanups for metadata commit 5c06d6c8d43f3400d92bfa9062851951b0ea9981 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:30:56 2020 -0700 progress meter for metadata commit 7775cf15f56f347136817bb1bf33f00091b13a05 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:29:43 2020 -0700 fix no credits bug. fix pk lookup syntax commit 70688d27a01bc2eb75f8a7cce007ec2d92fbeddb Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:53 2020 -0700 lint commit 72b53b462cb7d5feebbe4fea6159fc3f9588fbc3 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:47 2020 -0700 move comic-name into components commit f300f21097bb0d194ce1359eaf37b25c7f23ce20 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:45 2020 -0700 show less metadata if not in editMode commit 7731a45e630108eae0a254d49fd009a6293e8247 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:10 2020 -0700 first minort cleanups commit 30399ee10b665e39c30cef5a5332b2f31c8459f9 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:54 2020 -0700 freeze the comic metadata it won't change ... yet? commit ded8672c4c0f437febcbbb0ee393cd7b056d9216 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:20 2020 -0700 fix null issueChoices commit 5ecab8eb652941ff208c31cd5e1b7734a40da73f Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:02 2020 -0700 share initialItem across autocomplete and combobox commit 02a4e2f71c1662d6874741d025e4e10ac315f6e6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 16:21:59 2020 -0700 placeholder for metdata commit c4a4c37dee9f243626f8137dc6d5c35266dc6dbd Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:45 2020 -0700 Squashed commit of the following: commit 3637b056f2d2d947658767f11a5f29b8c664e817 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:24 2020 -0700 fix date display and all autocompletes commit 02d14266742e64620b17787049d8d6f0594258ea Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:35:32 2020 -0700 align issues better commit 7f8efe405e883b686a10cdf8d84e0e4edd41f1fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:49 2020 -0700 fix language and country display commit 0dc983f501c421bbd223e5348abdf096337f67f4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:25 2020 -0700 wipe comic data before lookup commit fa7236f6a74e061ecb475672c72dff4de19134e0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:29:42 2020 -0700 fix pycountry lookup bug commit 9a6bb655e51d3f9416b5bfc3bfb2feb6ab3b72b4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:49:28 2020 -0700 optimize getting only the aggregate filter for metadata commit 5129983d1f589cba0acad9edbe921203645c4db4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:37:38 2020 -0700 compress all metdata consts into the QUERY_SERIALIZER map commit c7e35a64fd491fedbee6ba4682975a5afb54df2b Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:14:02 2020 -0700 move frontend api files to versioned directory commit 49389216c3fb60718aceb45f34fe942e46cb45fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:08:34 2020 -0700 rename variables. clean up cruft. comments commit 39788fc1fcce0c8ce131da1be03be2284766d386 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:53:31 2020 -0700 lint and comments commit 3baa071e140c6f46bab63bed10bbfab13e546e5a Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:40 2020 -0700 lint commit 49565bc5c50c7b058e76837f8ab382db3caa63c8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:36 2020 -0700 lint ci scripts as well commit e1cac1604f0ae5e7f83fd94e9125c96c4b3104a6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:45:29 2020 -0700 add a progress bar to metdata commit d0e2412ae18b9a5f75f64dde1ab770e1968f1927 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:38:21 2020 -0700 move top book cover decorations into book-cover so they display on the metadata page commit 07426ce9fdacab8c980deaebac2e913f8c3c1eba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:55:20 2020 -0700 simpler tag icon is better commit f8e8ed39b725accffa57ba7f4978515e16a6fc4e Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:54:16 2020 -0700 Tag multiple icon commit a9a79e656cf20f141f44eb3e1b8c7b542ea27d6b Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:47:20 2020 -0700 remove log commit ee14d91ebfa3c96597003d50da2bed96351665cc Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:44:58 2020 -0700 change metadata icon commit 9b76a9bc7ca5a445d17bf505823241428588063f Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:31:11 2020 -0700 format issue decimals commit a49061082610264d7aaed1d38d1bf24f0baaddf6 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:16:02 2020 -0700 fix issues to handle years commit 08bd7639ebca0d68cf4bff985725681be72c7dba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 19:39:48 2020 -0700 credits working commit 268df3cd136972c7bc28aa7d0ea1761f4a2d24e9 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 18:20:51 2020 -0700 working even better. credits, issue display and folder test remain commit a5d6d88244fb1536c840789857c7624bb8cd2c4d Author: AJ Slater <aj@slater.net> Date: Mon Sep 21 19:31:49 2020 -0700 kind of working but kludgy backend solution commit 05c67e90052cc69010c20bd9d2c39d0e21745032 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 16:54:50 2020 -0700 move children and page count into base class commit c0f337bcde505729380caa10a6ea665fa2d5aa38 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 15:14:13 2020 -0700 move bookmark aggregation up into metadataclass for real. remove unused fit_to & two_pages commit 7c3ab1eb99e433b1181c4a3ff319329f1bd0a287 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 14:08:31 2020 -0700 move bookmark annotations to browse_metadata_base * speed up metadata by 50% * Squashed commit of the following: commit 97c87f28bbd4285efb89051de6c3844f1948c80f Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:09:01 2020 -0700 lint and comments. remove cruft commit 4d10bd16f07ad2e3c83ccb963f5e90978a61b214 Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:02:02 2020 -0700 major cleanup commit 8a23d7aa13702b0d75e4d1576e8ae75b03329af8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 20:15:59 2020 -0700 partial cleanups commit 27b66e26aa0d302364d1576c52140a0ea060b222 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:56:06 2020 -0700 isort commit 1c320e2187dabda7b3cc6bc55423902add58b3d0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:54:37 2020 -0700 fix related field counts and cleanup a bit of cruft. more cleanup to do. commit bd045eb0692e9232c51dd33053801d407c415039 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:59:09 2020 -0700 it motherfucking works commit 752d1773671196a4bbb3472490f35d394df806d7 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:48:58 2020 -0700 working agg all. now just do pks commit af6b59b69824836f8485892d83ad7156d055372e Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:56:57 2020 -0700 coment. lint commit f678f8f35d9724c9eca514e85c8372d6c534621a Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:34:21 2020 -0700 clean up metadata container aggregate a bit commit 49ca05eedf6f9a6ade3682272c35d7e0733203c9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:14:01 2020 -0700 simplify a map commit fc2a6d4d0834ccc056474414b6d15d61bbffe7df Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 11:59:21 2020 -0700 major cleanup of metadata view commit 0a90f5c5c75ec2a60f00334dcddc8d5eb95bf641 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:28 2020 -0700 more explicit nginx docs commit fe54d13f985d796b95593efb9f8e6c1ad539d879 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:18 2020 -0700 clean up metadata serializers commit 0e182239e8525ee33a8b095b6b406cd567e7acf9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:04 2020 -0700 remove a little cruft commit f8972d6b19c0f9c829d605c6ddb6fd229cef2466 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 19:37:45 2020 -0700 ugly but working bufix for too much data commit 0f4d7d70c4c8b44a8e629bbee217df742b76de08 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 16:38:35 2020 -0700 new metadata api works on frontend. backend bug gives too much info for groups commit 4241cacf42e8b4cf27a08c657f672694be1bde4a Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 00:20:50 2020 -0700 lang serialiezer returns value if text lookup fails commit c9a6465fd913185c2df340c082d1e0985b6fa189 Author: AJ Slater <aj@slater.net> Date: Mon Sep 28 18:18:10 2020 -0700 doubled speed of metadata * bump version * Squashed commit of the following: commit 88404c565b4cc3fbad899ca1bccd366b43b199b5 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 14:54:40 2020 -0700 remove TEST variable. always a bad idea commit a3bef259dc6c72b0a8bb885d0353d7a8bad1b81f Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:58:02 2020 -0700 comments commit 265784ce9fdf5fe2429fda3fa98f271c2275667b Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:54:39 2020 -0700 optimize select_related for metadata just a little more commit a488427a15ab8d0efb4a742308a0696d313492dc Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:45:15 2020 -0700 pop fields before serialization. should be faster commit 07df6a610a7683cf16551c4e644207d10be96f27 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:39:42 2020 -0700 clean up serializers and choices. get choices from src too. commit 5d59c5a95dcc6194f76d45c578efdd529a895f90 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:19:19 2020 -0700 fix improper null items on undefined metadata fields commit 905aa375760b374f3e5c8ccdc29f074cf2e3e754 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:18:59 2020 -0700 fix metadata folder view commit d6017dd76297f448735198e1a3a1c9f111865a2e Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 23:15:54 2020 -0700 reading choices and messages from the webpack json works now commit 87246c285ac93674eff1ba55be651a05e967102b Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:14:37 2020 -0700 move item xformer into api commit 102b291511cc3e6c34d7efe2f4e7839d5637766c Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:04:05 2020 -0700 fix null items commit b7212c3e3a47987d1b20ba604aba408555b32f70 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:44:22 2020 -0700 sorted items commit 5c8cef6cd7eb85c246f3099b35a71f2b87555e51 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:27:26 2020 -0700 componentize browser-settings-menu commit d3fd0f57ea0d5500389e97e1c071896abac3791a Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:17:06 2020 -0700 componentize browser settings dialog commit 8510179f3be465df0d7848751e62a699902dd458 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:05:52 2020 -0700 working filter menu. componentize top toolbar filters. * massive rename of containers and browse for consistency * docstyle linting * lint * remove mypy and vulture. i don't use them. * Squashed commit of the following: commit 936a6c8a202b09db2c8d6bedc8ce1d8add3e3f5e Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:13:22 2020 -0700 move secret key and logging init management into modules. websocket server doesn't configure logging right. commit 328bb8288bbee74b88c0e0754bc13abdb7ea114d Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:35:23 2020 -0700 less picky about passwords commit 8a953e20676ac5ca7e5c226b8792e075dbae1d11 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:17:14 2020 -0700 hypercorn 0.11.0 includes asgi signals and a big logging change. fix the logging with a custom formatter and a flag to logconfig_dict commit e3eae3400a2c07cab11997b74a71397e70f9d3e2 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:20 2020 -0700 upgrade babel/core commit 16034b60a7fb41fe47d67a8c02972231094f9946 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:04 2020 -0700 remove comment cruft commit 23885db544421469e46d0d42a75844c73b082fa5 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:13:31 2020 -0700 dev run script alter python path for local dir commit 80b92b45921dadc9d6a3811f7e92df5bd9377505 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:46 2020 -0700 correct logging __name__ and remove print cruft commit 2a6d347dcc71e39a88b2e7c515c8b2fcae07b0de Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:24 2020 -0700 document where i got the timezones middleware commit 59778ffa39967f7b6e42a5bf4fb4c52c8fbe5ce4 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:04 2020 -0700 debug lifespan logging commit b3ae98d90fbf3e9f1bf4b5eb17fb467c63f21f54 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:11:35 2020 -0700 rename procs & threads for logging commit 77f064e89aaa639c97b1bf7aeb4748a2fefe04d4 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 21:55:52 2020 -0700 broken asgi signals handling? * Squashed commit of the following: commit 4eca7486fa16467cfee0e38f4d6560c8b8487d12 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:19:12 2020 -0700 replace old DEV flag with DEBUG remove redundancy * lint * stop pytest from testing migrations * oops broke flake8 * Squashed commit of the following: commit efa7750ba35e3e32ba005a01cf53aab949c6993c Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:27:34 2020 -0700 componentize browser-main commit 3e9caadb2ffa7e7b679ab813d4c5acb4a7f366cb Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:06:01 2020 -0700 oops forgot to commit settings file for module change commit ca5168afd78bf05bdc7572120c746beb2e93ef22 Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:05:31 2020 -0700 settings module commit 54af69ae2562fdb7b101ef57d5ebbf9dd8088482 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 20:25:01 2020 -0700 componentize browser * lint * componentize reader * remove todo suggestion * more metadata fields * update news * fix restarts * its closer to 5 seconds in practice * update poetry and npm packages * change order of stop and start daemons a bit * fix finding webpack files. Use regex instead of glob * bump news and version * fix root_path bug. v0.5.17 * renaming some variables fixes settings changing * bump version and news * reform progress dialog with offspring timing * no touch stuff actually used * always return * Move sockets to vue-native-sockets. Componentize snackbars and move them to app level * hypercorn bugfix doesn't squash logging anymore. remove workaround. * new poetry install option * lint * update npm * remove an extra futile conenction attempt * add a publish check * lint * fix websocket dumb test * move includes to set.has * log the path as well as the reason to stdout for log watchers * bump news * Squashed commit of the following: commit 926deb8f3214f1c0a3c818904492df0182e42ded Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:39:25 2020 -0700 remove orjson parser. needs glibc commit b98a47f8847b7ea8ac3f50beb7c3045e42cae3d1 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:38:59 2020 -0700 bump Next Up commit 03248e99c0aeb686d4059ceb57f149974ef255a7 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:38:01 2020 -0700 alphabetize deps commit a1f96f39963c3e9bc15325aa09cf68a641c43771 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:37:46 2020 -0700 lint commit b1de6c0228d68e1b73d7c034bad68d73de8c619b Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:36:51 2020 -0700 fix hypercorn config path commit 27390428900926886cfeccbde97e7149c04486c7 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 19:31:30 2020 -0700 alphabetize pyproject commit edfd191eb7429487acb8ea345cc10ade34dd0df9 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 16:56:11 2020 -0700 bumpt version to 0.6.0 commit 91c52925ada6bbebd19599d7ae04394f92c46eee Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:34:20 2020 -0700 just do next to nothing for asgi test commit fe6fd4ce49db31639ea47bbb02b17b111b1b7a5e Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:33:51 2020 -0700 fix deprecated log method commit 37ac8132c8c97d74a52c9fa0525b9822204bde41 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:29:13 2020 -0700 fix pytest settings and enhance asgi test commit 364fef6d675eb0cd3e43aed96a238c1474281798 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:19:50 2020 -0700 poetry update, npm update commit 52ece20751a726f78566fe3ef6085e95a1b529ac Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:17:48 2020 -0700 centralize issue nameing and handle decimals and 1/2 better commit 33434f2d0aa43e0577ec63d8fefdeea08d9b2dad Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:50:42 2020 -0700 fix overcaching of reader settings commit 9363e3ae218ab2c12042c79d7be70e61b76627d3 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:33:31 2020 -0700 lint commit e40f710e383ee003f2941467172bc7f7337bca77 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:33:01 2020 -0700 fix padding on dialogs commit f96df2da458a1895441dd9ecd54f4c66f7b84b97 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:26:34 2020 -0700 fix reader image centering. move bookmark serializers into own file commit 63610d21bba02614aba6ba236224d23862bab6d4 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:06:50 2020 -0700 componentize reader settings dialog commit 2c209bef96781281ddf59b3c7a33785008b7b7e5 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:51:40 2020 -0700 clear form choices more often commit 18ea7033b98aedbf099b2abd410a6f394420e897 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:49:14 2020 -0700 move the vuetify null code into choices json commit 8bd0d2d3836915725e9802c30110366cb07fe2c8 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:31:20 2020 -0700 show browser progress for some user interactions, not others commit a2e3fd015894f26f123f9287d92445c738ffefbb Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:30:47 2020 -0700 fix mark read menu for folders and cleanups commit 785fdbf100fc1389d5b027977a94df20dee3ccdb Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:22:59 2020 -0700 fix manage.py for new settings location commit 77a0d05edc258306cab4e89b6712fa50f65b6cdd Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:04:41 2020 -0700 remove comments commit 6deae30b7effa5fad7116e18b11fe503a54b98ba Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:03:58 2020 -0700 clean up serializers with custom fields commit c1c3ab51c4935e84f38021e7a9292945954e9b7e Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 12:43:48 2020 -0700 move special django setup stuff into own file commit e07ea7a552316b92cff452dc6031d7c4f7979ccc Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 12:34:15 2020 -0700 move settings into module commit 1226bb969e03038db677e5702bbcec0f19ebd177 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 12:15:59 2020 -0700 shrink filter controls more. especailly on mobile commit c6fc22f8263b65366e26bbddb3befcb806b51c25 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 12:03:30 2020 -0700 recheck notify after failed_states notification commit b6cd4b5531ddef8867b8a11e9b37229f0d55dc5e Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 20:50:25 2020 -0700 update npm commit cf192c0c26dd7dedb072d85e4f6cb45de7533307 Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 20:42:45 2020 -0700 comicbox 0.1.5 and update poetry commit 81f17d68a7bf552647b915182c563f4014cabe9a Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 20:42:20 2020 -0700 remoe dry run nonsense commit 99fef62c685c70f7b7c3cf2c457f71bcedc6565e Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 19:57:26 2020 -0700 lint NEWS commit d642581bc62b3514f109b1c18fe4de560cb990a0 Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 19:56:30 2020 -0700 news to md commit 6348b028259b23f2653974f91ec2f2c677a27c09 Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 19:52:14 2020 -0700 unbreak storyArc and seriesgroup filter endpoints commit 87feba07c27f472cd8023311d3738f14cdf6542a Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 19:07:14 2020 -0700 consistent naming of Creators commit ce905132a40e6cc2ca4ee64404a85c0a886e57ca Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 17:34:41 2020 -0700 fix null boolean on book cover commit be0ff1fb7f197cb10c6c88a37e24dc610ad8d58f Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 17:29:04 2020 -0700 reader slider pagination. fixes for reader nav & thumb-label on the inside of the slider button. commit 7617f6e666708d7ca7a54927d7d532b3a14a8212 Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 14:07:17 2020 -0700 bump NEWS commit 8e48e15177f175be1acdcd31e672baea797f9cad Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 14:05:58 2020 -0700 version footer moves to setting dialog commit 515a594e6d015769f91f0522739e0604b054709b Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 14:47:46 2020 -0700 fix settings menu closing commit 5ff9da5d607a061829b953b85a8b667ba12878b2 Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 14:21:08 2020 -0700 change sort icon and add labels on focus to the toolbar commit 06c235f6e1e57afcb7e05a4cfd69251b4b06befa Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 13:03:38 2020 -0700 remove useless sorts. fix reverse sorts commit b3f21bce7a6ce73cdab2f8060678c39117520bad Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 13:03:12 2020 -0700 change warning to debug commit 7641e9b626230bb3b1ce63942559876c8f1f3ffe Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 00:43:15 2020 -0700 format times for browser card order-by. fix sort_by menu length. clean up eslint cruft commit 40f1e6f54378e961e12cb9a79ab3931b63e5335c Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 00:42:21 2020 -0700 fix crash with get_aggregate_field on new sort_by name for Comic commit 415ee902908122d8a0238351dba00a5ebbab283c Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 20:11:44 2020 -0700 lint commit 2a987eef30717f87c374e830af358187c326ed90 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 18:49:23 2020 -0700 first draft of fitlers and sorts commit 21333947890622b82cad6208009913a32aefbb76 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 18:04:49 2020 -0700 rename credit person to creators commit b6738f855d269fefe0b2652b3318ba6004b2359a Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 17:53:19 2020 -0700 fix filtering on none for charfields commit 2d9b9452bf9d2091b4a44c264093f6b7cd4b83d0 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 17:27:46 2020 -0700 add credit_person, teams. fix bug with setBrowseChoices calling twice commit 50fd303026258ae3ce1edf0b23f10ef42fb9cc7f Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 14:53:30 2020 -0700 for choices use aggregate filters commit fe966722fd79eb07a5507ce3699089150289cf18 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 14:07:10 2020 -0700 working filters commit 8c4a0fb8a53d64bdd1e4ea3f7e5baaa07acec908 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 13:12:40 2020 -0700 new filters like pycountry and credit person display but do not filter yet commit 6e717061a6c5f6b6852d5c4a016fcd5efee8c95b Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 12:09:05 2020 -0700 simple new filters commit 9fc372c6be7e984170e81daec208c18448134cfe Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 11:50:20 2020 -0700 add year filter and condense source of filters to one object commit e90a65e77169956dfda212e7f221e06ad472c810 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 11:46:50 2020 -0700 extra protection for empty filter commit 4846d50b6e4cb73535fbfe47f8914e3603b70025 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 11:43:07 2020 -0700 replace ujson with orjson commit d84d725b6310bd101488e505e754748b7d3a24c0 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 00:17:07 2020 -0700 move default hypercorn toml location. simplify get_root_path function: commit 3f807a9791dd3bf2fb32213b8b25db14c2b21460 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 23:46:13 2020 -0700 modify news commit f91864b71794a641350a37c9f8f68e022c803933 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:42:34 2020 -0700 bump news commit 07dc108e40b0518c0f43ab9db1a2f77f3884858f Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:40:25 2020 -0700 faster json parsing commit 26137ac75f0c009f5ecc12b67d9cfb1c8f5006f5 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:38:18 2020 -0700 fix too visible uproute when there is no route commit 316fefef6340c6a87ae2adefb994f40d3641aa00 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:38:04 2020 -0700 fix nav to r with positive pk commit 8795021b14cf97ab6c1f958747b9b0b371062db7 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:00:13 2020 -0700 api v2 basically working * lint * lint * fix notify on start bug. minor cleanups. * fix redundant navigation. debug logs for browser opened & getpage. don't get on create, wait for opentosee * rename pageNumber to page for consistency with new api * change action names to be declarative * poetry & npm updates * fix my one frontend unit test * oops forgot to remove dry run thing * fix collectstatic for new settings * librarian exception logging just in case * error exit on collect static just incase * remove personal details from git * codex 0.6.1 * lint * Squashed commit of the following: commit a1eb9d1630950052fab714efcdb8a0c04f082b63 Author: AJ Slater <aj@slater.net> Date: Wed Oct 14 18:45:36 2020 -0700 bump version commit a7bfecd8cdaadf6415aaa9dbc1a35cd27c15aef1 Author: AJ Slater <aj@slater.net> Date: Wed Oct 14 18:43:31 2020 -0700 fix warning messages and add an init message commit 76a69aaf868621574d30437dc2146a1cfe61f9cb Author: AJ Slater <aj@slater.net> Date: Wed Oct 14 18:32:40 2020 -0700 lint commit b71759260cab7ae44a29bf740935ffc3421242ed Author: AJ Slater <aj@slater.net> Date: Wed Oct 14 18:31:09 2020 -0700 restore dev-flag * speling * debug logging * turn librariand into a class * faulthander * turn debug on for development * refactor hypercorn config parsing * rename some hypercorn settings methods * collapse DEV into DEBUG again. add LOGLEVEL * bump news * bump version * remove cruft * remove faulthandler * update npm * fix reader bug that only displayed first page * v0.6.4 * docker node stuff moved to codex-builder repo * switch to groovy builder * update poetry * get parent pid instead of passing it in * update runnable image base * adjust height of menues for mobile * easier prefetch managment with vue-meta * update npm * refactor procs & threads * refactor of some websocket ipc stuff * seems fixed * librarian cleanup * move all startup and shutdown into lifespan app * move librarain startup/shutdown back into class * troubleshooting section * poetry update and use cimg:3.9 for circleci * use newer compose version format * name poetry step * poetry update * poetry & npm updates * Multi arch ci (#72) * update poetry & npm * fix websocket test with pytest.asyncio * fix ignores for .venv * update npm * lint * multi-arch first attempt * lint * lint * add eslint to lint & fix * fix eslint call to use local version * lint * better eslint i think * fix build-dist script * fix build multiarch script * dark admin pages * more django-dark fixes * minor notification optimization * npm update, v0.6.6 * bump eslint-plugin-simple-import-sort * remove old rule name * revert simple-import-sort upgrade. * move extrastyle to base * README formatting fix & update roadmap * update roadmap * add anchors to README, more thanks and clairity to many sections * don't duplicate language * make links like github does * build for amd64 with the test script * fix admin template loading. how did this ever work? * 0.6.7 the darkening

* Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * v0.5.3 (#10) * v0.5.2 (#9) * Alpha dev ci (#3) Ready for alpha release. Test artifacts and ci process looking good. * fix tagged version name * Squashed commit of the following: commit 7bf35c37269a801656141a57a6d59aaa22759675 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:55:12 2020 -0700 use browse loaded for brwose placeholder commit 25001f4158904dc5e3297822cc4d16f773745638 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:24:40 2020 -0700 plural libraries commit 87237403632fe2f7ccef9300b3f7c265b0dfbe25 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:20:38 2020 -0700 just nuke all that libraries exist stuff commit 723e31eb988b433d22e8d4362e9a01c06f2a1a70 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 23:19:13 2020 -0700 auth dialog enter submits form. autofocus on username commit ce74887c2ced4e704bcf1cf77087ff6618e011b7 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:45:08 2020 -0700 update package.json commit 7546fa58bb080ffec0486e781e526b35b9a4df7e Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:43:50 2020 -0700 update poetry commit 985bd5e61fe1b8a7dcfc1583b3c634bbbf292791 Author: AJ Slater <aj@slater.net> Date: Tue Aug 25 22:32:27 2020 -0700 let enter submit logins * Squashed commit of the following: commit 688165d01c797b723387d057dac7d004461c258e Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:28:20 2020 -0700 0.5.1 minor bugfixes commit 2487d151e0a687f5bac3aa152acb066d556beadf Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:18:31 2020 -0700 lint commit 31432a3c4f6e3463ef71a3d4b606713cabdaf5cb Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 01:07:50 2020 -0700 ignore webpack-stats.json for prettierignore and sort it * oops forgot to bump docker-env version * V0.5.2 (#8) * v0.5.1 with correct docker tag. (#7) * oops forgot to bump docker-env version * tweak NEWS to trigger ci deploy * update README roadmap to include OPDS * admin flag documentation * fix bad multiprocessing method for windows * Squashed commit of the following: commit c12e685bd192fa724bce8bd956f7981a0df571dc Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:01:22 2020 -0700 cleanup filtering commit 134b1d0e25f2018979f957f764f01546fe797726 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:54 2020 -0700 move get_folders_filter into choices only commit cad82011ac6012c26b1f933adf70aafb8b64b143 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:47:31 2020 -0700 fix filter menu close commit 61567ace2f997ccafd6b9b7030bde77e3c55fbd3 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:31 2020 -0700 reorg and optimize some browse_base view stuff commit db9fc9b9ae606e53c9c4e364c010ef4df0e7dcb0 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:15:10 2020 -0700 lint commit 3bf26fd1d8e374eec862a671446775c1773750f6 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 18:14:29 2020 -0700 poetry extras doesn't work like i thought it did. move that stuff into development deps commit 84b42d11aa33f8a0559a5763e25e78e4ecd47110 Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 16:59:24 2020 -0700 lazy filter choices * Squashed commit of the following: commit bcb685cac1ee10a78838b8404c5411f63c53ba2d Author: AJ Slater <aj@slater.net> Date: Wed Aug 26 19:16:19 2020 -0700 return of librariesExist * bump v0.5.2 version numbers * declare PKG_VERSION in Dockerfile * whitenoise 5.2.0 fixes the AUTOREFRESH BUG * Squashed commit of the following: commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API * remove print * fix filter error with aggregates. fucking confusing. * make the no comics found label a little less garish * Squashed commit of the following: commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works * master version bumping script * only flood control libaryChanged message * broadcast a message when we scan libraries * scan notify api endpoint * frontend for scan notify snackbar * lint * move isAdmin to auth getters * use queue for scan-notify * Squashed commit of the following: commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login * update NEWS * fix sed for bsd * fix tests on docker where whitenoise complains about missing static dir * ensure all STATIC_DIRS exist for whitenoise * new black. update poetry, npm & lint * 0.5.4 version bump * only use the build type static files in dev mode. always should've been like this. * collect static needs dev marker to actually collect the dev static dirs * revert whitenoise * codex v0.5.5 * fix logging error * borders on browse-cards * fix border interference with unread flag * v0.5.6 * comicbox 1.3 * allow null credit roles * V0.5.7 (#35) * v0.5.7 news and version * remove transparency from child count * center pages better in reader. still has a two page gap. * scroll to the top of the page when the route changes * more explicit read/unread menu. Add download comic to menu * fix previous comic always start of series bug * lint migrations * npm update * logfiles * more items to news * lint * reorg setup-dev.sh * Docker caching for wheel builder image * remove scanlocks on startup. fix library change by not disabling element * Squashed commit of the following: commit e1994b7d8f76d1aff43de8fddd927e3ac6686502 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:37:37 2020 -0700 bum news commit f230dd6590b2c0e404f5820dca76810207c74a74 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:33:23 2020 -0700 remove inconsistent document.title setting commit 129323516a24bb1959f1e5cee244af56151b273f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 15:29:20 2020 -0700 restore child counts and bookmarks commit 21f25ec7b15822b5ee51c2c97b4a700d584306fb Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:42:49 2020 -0700 modular frontend comic name code. disambiguate volumes more commit 3f8c7a4fedf931b81da9196147f6556632194fbf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:46 2020 -0700 volume parsing on the back end commit 557ace75ccb3522302a1b402466cb19d75a38b22 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 14:21:19 2020 -0700 fix issue type in browser commit 4d2331d240f0b78c48e2cc365dc4d41f9155ff7d Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:55:02 2020 -0700 remove extra loggins commit 566e353f78ce81f502b9ceac103f0f41b300b91c Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:52:40 2020 -0700 move ensure cover path to serializer so we only hit the db once commit adfeda4d5d3a91e355ae55e6182411b2cb7f73bf Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 13:15:12 2020 -0700 pass series_name, volume_name, issue as fields. no more janky parsing commit 361e0ad253356ea7d5733bb1ca54e2e80173c9e4 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:48:43 2020 -0700 rename containerList commit 830454081e9163039dbd7893a91b099b7f094380 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:45:53 2020 -0700 fix up route not starting from 0 commit d17e0de419e9dd0c715d46f71e856c77a619a845 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:34:47 2020 -0700 pagination union with header name annotated instead of python computed commit b1aac346d372dd5a41397c53428de0303733958a Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:06:44 2020 -0700 npm update commit 69414ec4e09c97c7a6c6c7a7da3406b71da7f178 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:24 2020 -0700 update poetry commit e34d4af0c79d86f7ca2bc072be9ee5b8965ae54f Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:02:04 2020 -0700 clear cache on startup commit 34a96e36638d98be77ed5de3681c7011b4d2b6a2 Author: AJ Slater <aj@slater.net> Date: Thu Sep 3 12:00:38 2020 -0700 fix last_scan create library bug commit fef826cd24f2cd0315113a45f2c8eee2dcbf9667 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:38:20 2020 -0700 pagination proof of concept. breaks with folder view commit 01eefe612e6e63556e6f58545e1fbf9846689f85 Author: AJ Slater <aj@slater.net> Date: Tue Sep 1 16:34:46 2020 -0700 lint * friendly /me message * Squashed commit of the following: commit b73eab2312d1648dc24203ec7bca712d820daeb2 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:15:54 2020 -0700 remove unused var commit 12d990586f00f9b6c65eade9495de3ed65aea1fe Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:11:25 2020 -0700 watch for user changes in the browse app, not the auth app for browse stuff. commit 9ba8575c0e4f4c011fde28ee2ec37d206063700f Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:10:34 2020 -0700 consolidate route generation in browse store and add page * reform browse console logging * Squashed commit of the following: commit d72c5eda5744f68d442e91f942bf7e8077271976 Author: AJ Slater <aj@slater.net> Date: Fri Sep 4 13:34:00 2020 -0700 refactor browse card to be a single card object. * oder the common browse fields * fix cover on metadata page * move progress bar to browse card for better positioning. * more tests for comic model * bump codex version oops * v0.5.8 * poetry update * more helpful 404 page * V0.5.9 (#50) * poetry update * more helpful 404 page * update npm * fix logging bug on null role import * Squashed commit of the following: commit 0980b6e2ce4d8a7b5a55722427febb3d6d35b05e Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 18:09:12 2020 -0700 fix folder sorts. reorganize annotation code. eliminate x_order_key in favor of order_value commit 4797fc411aa6d43d43a66ff98a90641c6c8338e4 Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 15:39:43 2020 -0700 first draft of folder sort fixes. needs cleanup * Squashed commit of the following: commit 36329ac2572bd95d55fe2e6b79156a0355eea42b Author: AJ Slater <aj@slater.net> Date: Sat Sep 5 19:25:15 2020 -0700 missing cover support * poetry update * version bump * comicbox 0.1.4 * lint * Squashed commit of the following: commit 9f22f50e97ecc743270e26217012d8d5097016dc Author: AJ Slater <aj@slater.net> Date: Tue Sep 8 13:53:41 2020 -0700 use base images when building runnable docker * V0.5.10 (#53) * fix filtering bug * bump version * clean up comments * fix reader settings * mostly fix reader image layout * fix reader to browser uninitialized error * bump news * update poetry * npm update * simpler image labels * V0.5.11 (#55) * simpler image labels * simply image tagging even more. publish to pypi with ci * ignore node_modules of course * use Min instead of Max for sorting containers. Added benefit is covers match sort values now * flex layout to the rescue for adaptive browse tile row heights. * fix bookmarks not working for sessions * update poetry & npm * bump version to 0.5.11 & news * fix mobile safari bug with stretched images * Squashed commit of the following: commit 8573d66e625a99288cc52092aad33c14845bb631 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 13:05:53 2020 -0700 fix the whitenoise prefixing for dev mode commit 46700920e7da71e0a1779eb0a701ac321dad1d71 Author: AJ Slater <aj@slater.net> Date: Fri Sep 11 12:33:35 2020 -0700 works in prod * abuse WHITENOISE_ROOT for covers so I can turn off WHITENOISE_USE_FINDERS in production * poorly spelled variable prevented setting global reader settings * bump news * another news bump * admin failed imports list * Squashed commit of the following: commit 3a3e206df83ee9ad103b2c7600a2ae1b3fb305f8 Author: AJ Slater <aj@slater.net> Date: Mon Sep 14 17:42:25 2020 -0700 scan notify snack * bump version and news * lint and cleanup prints * Squashed commit of the following: commit ee19a333161c174b387a4b2b525d04ec1d549b91 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 14:38:45 2020 -0700 latest version * Squashed commit of the following: commit 6001d4a338bf9dc86238b0da565e76a891d3de02 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:55:53 2020 -0700 remove the init_admin_flags from migrations commit 38556ea04959e02522c08288c152f61ea18b7a3a Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:47:38 2020 -0700 minor cleanups and comments commit 3e40d41c52b188353b0d04e96c7e35dec5315d95 Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 19:31:46 2020 -0700 bump news commit 1321826d522c32cb62a13abb1ea3df8173f8cf3d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 18:00:20 2020 -0700 add dev config dir to prettierignore commit 784e165efc57f6f97092e8d17236a0bde40b226d Author: AJ Slater <aj@slater.net> Date: Tue Sep 15 17:52:09 2020 -0700 disable session users first try * change failed import comment * v0.5.13 * crap i'm sure now * Squashed commit of the following: commit be04fac73fe3fdcd2ee600aac8106125f45bff7c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:56:48 2020 -0700 lint commit f93023cf01b2e2f8fe0e83fdcd0c051ed1a8428e Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:55:47 2020 -0700 bump news commit 2637d430f90a382145218d691e64e28939c4f024 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:53:27 2020 -0700 poetry and npm update commit a7d8af755fea91b202e018ce63d084dc22229daa Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:49:01 2020 -0700 add force to comic cover creation commit c6b11e3ea7866c980bc0e4b9b0772d39f81a3052 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 19:25:20 2020 -0700 fix unable to delete libs commit af224ffc3b34b172ecc2ae84c848fcf1d64bf5c0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 17:08:07 2020 -0700 rehide edit button and fix issueRow layout on mobile commit 17bee0af42a14d1d53285f6ef8f4c583888a36c7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:50:27 2020 -0700 minor cleanup of checkbox commit c7f235af7abd00914c874b2fb65ebd1be545e1a4 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:33:27 2020 -0700 remove comment cruft commit 073b6f08e39100aa71dc33eee6969abb473164fc Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:32:00 2020 -0700 bump version & news commit 0c8bdfb5f216df4353533d9371437caa65dfb7f7 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 16:29:14 2020 -0700 dict for websocket messages commit 7a8e1e9af9bd57a93b08d9f587435ed4f47f525b Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 15:51:41 2020 -0700 minor cleanups commit d1ad0ef4159aade2793c424be693f4dd3d36f943 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 14:01:29 2020 -0700 remove cruft commit df1d69ef2d28fd88049032d8ee7c6d21163708f6 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:56:06 2020 -0700 minor cleanups and formalize timing for metadata commit 0e68de8a3b85bb05657b83f7d2871f02094696a0 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:48:51 2020 -0700 two minute timeout! commit 650099da1be7510c25fb0c1ed562fdeb56a31e2c Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 13:15:34 2020 -0700 cleanups for metadata commit 5c06d6c8d43f3400d92bfa9062851951b0ea9981 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:30:56 2020 -0700 progress meter for metadata commit 7775cf15f56f347136817bb1bf33f00091b13a05 Author: AJ Slater <aj@slater.net> Date: Fri Sep 25 02:29:43 2020 -0700 fix no credits bug. fix pk lookup syntax commit 70688d27a01bc2eb75f8a7cce007ec2d92fbeddb Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:53 2020 -0700 lint commit 72b53b462cb7d5feebbe4fea6159fc3f9588fbc3 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:17:47 2020 -0700 move comic-name into components commit f300f21097bb0d194ce1359eaf37b25c7f23ce20 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:45 2020 -0700 show less metadata if not in editMode commit 7731a45e630108eae0a254d49fd009a6293e8247 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 22:15:10 2020 -0700 first minort cleanups commit 30399ee10b665e39c30cef5a5332b2f31c8459f9 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:54 2020 -0700 freeze the comic metadata it won't change ... yet? commit ded8672c4c0f437febcbbb0ee393cd7b056d9216 Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:20 2020 -0700 fix null issueChoices commit 5ecab8eb652941ff208c31cd5e1b7734a40da73f Author: AJ Slater <aj@slater.net> Date: Thu Sep 24 18:44:02 2020 -0700 share initialItem across autocomplete and combobox commit 02a4e2f71c1662d6874741d025e4e10ac315f6e6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 16:21:59 2020 -0700 placeholder for metdata commit c4a4c37dee9f243626f8137dc6d5c35266dc6dbd Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:45 2020 -0700 Squashed commit of the following: commit 3637b056f2d2d947658767f11a5f29b8c664e817 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:41:24 2020 -0700 fix date display and all autocompletes commit 02d14266742e64620b17787049d8d6f0594258ea Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:35:32 2020 -0700 align issues better commit 7f8efe405e883b686a10cdf8d84e0e4edd41f1fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:49 2020 -0700 fix language and country display commit 0dc983f501c421bbd223e5348abdf096337f67f4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:30:25 2020 -0700 wipe comic data before lookup commit fa7236f6a74e061ecb475672c72dff4de19134e0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 15:29:42 2020 -0700 fix pycountry lookup bug commit 9a6bb655e51d3f9416b5bfc3bfb2feb6ab3b72b4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:49:28 2020 -0700 optimize getting only the aggregate filter for metadata commit 5129983d1f589cba0acad9edbe921203645c4db4 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:37:38 2020 -0700 compress all metdata consts into the QUERY_SERIALIZER map commit c7e35a64fd491fedbee6ba4682975a5afb54df2b Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:14:02 2020 -0700 move frontend api files to versioned directory commit 49389216c3fb60718aceb45f34fe942e46cb45fc Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 13:08:34 2020 -0700 rename variables. clean up cruft. comments commit 39788fc1fcce0c8ce131da1be03be2284766d386 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:53:31 2020 -0700 lint and comments commit 3baa071e140c6f46bab63bed10bbfab13e546e5a Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:40 2020 -0700 lint commit 49565bc5c50c7b058e76837f8ab382db3caa63c8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:48:36 2020 -0700 lint ci scripts as well commit e1cac1604f0ae5e7f83fd94e9125c96c4b3104a6 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:45:29 2020 -0700 add a progress bar to metdata commit d0e2412ae18b9a5f75f64dde1ab770e1968f1927 Author: AJ Slater <aj@slater.net> Date: Wed Sep 23 12:38:21 2020 -0700 move top book cover decorations into book-cover so they display on the metadata page commit 07426ce9fdacab8c980deaebac2e913f8c3c1eba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:55:20 2020 -0700 simpler tag icon is better commit f8e8ed39b725accffa57ba7f4978515e16a6fc4e Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:54:16 2020 -0700 Tag multiple icon commit a9a79e656cf20f141f44eb3e1b8c7b542ea27d6b Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:47:20 2020 -0700 remove log commit ee14d91ebfa3c96597003d50da2bed96351665cc Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:44:58 2020 -0700 change metadata icon commit 9b76a9bc7ca5a445d17bf505823241428588063f Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:31:11 2020 -0700 format issue decimals commit a49061082610264d7aaed1d38d1bf24f0baaddf6 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 20:16:02 2020 -0700 fix issues to handle years commit 08bd7639ebca0d68cf4bff985725681be72c7dba Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 19:39:48 2020 -0700 credits working commit 268df3cd136972c7bc28aa7d0ea1761f4a2d24e9 Author: AJ Slater <aj@slater.net> Date: Tue Sep 22 18:20:51 2020 -0700 working even better. credits, issue display and folder test remain commit a5d6d88244fb1536c840789857c7624bb8cd2c4d Author: AJ Slater <aj@slater.net> Date: Mon Sep 21 19:31:49 2020 -0700 kind of working but kludgy backend solution commit 05c67e90052cc69010c20bd9d2c39d0e21745032 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 16:54:50 2020 -0700 move children and page count into base class commit c0f337bcde505729380caa10a6ea665fa2d5aa38 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 15:14:13 2020 -0700 move bookmark aggregation up into metadataclass for real. remove unused fit_to & two_pages commit 7c3ab1eb99e433b1181c4a3ff319329f1bd0a287 Author: AJ Slater <aj@slater.net> Date: Fri Sep 18 14:08:31 2020 -0700 move bookmark annotations to browse_metadata_base * speed up metadata by 50% * Squashed commit of the following: commit 97c87f28bbd4285efb89051de6c3844f1948c80f Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:09:01 2020 -0700 lint and comments. remove cruft commit 4d10bd16f07ad2e3c83ccb963f5e90978a61b214 Author: AJ Slater <aj@slater.net> Date: Thu Oct 1 15:02:02 2020 -0700 major cleanup commit 8a23d7aa13702b0d75e4d1576e8ae75b03329af8 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 20:15:59 2020 -0700 partial cleanups commit 27b66e26aa0d302364d1576c52140a0ea060b222 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:56:06 2020 -0700 isort commit 1c320e2187dabda7b3cc6bc55423902add58b3d0 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 19:54:37 2020 -0700 fix related field counts and cleanup a bit of cruft. more cleanup to do. commit bd045eb0692e9232c51dd33053801d407c415039 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:59:09 2020 -0700 it motherfucking works commit 752d1773671196a4bbb3472490f35d394df806d7 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 18:48:58 2020 -0700 working agg all. now just do pks commit af6b59b69824836f8485892d83ad7156d055372e Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:56:57 2020 -0700 coment. lint commit f678f8f35d9724c9eca514e85c8372d6c534621a Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:34:21 2020 -0700 clean up metadata container aggregate a bit commit 49ca05eedf6f9a6ade3682272c35d7e0733203c9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 12:14:01 2020 -0700 simplify a map commit fc2a6d4d0834ccc056474414b6d15d61bbffe7df Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 11:59:21 2020 -0700 major cleanup of metadata view commit 0a90f5c5c75ec2a60f00334dcddc8d5eb95bf641 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:28 2020 -0700 more explicit nginx docs commit fe54d13f985d796b95593efb9f8e6c1ad539d879 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:18 2020 -0700 clean up metadata serializers commit 0e182239e8525ee33a8b095b6b406cd567e7acf9 Author: AJ Slater <aj@slater.net> Date: Wed Sep 30 09:04:04 2020 -0700 remove a little cruft commit f8972d6b19c0f9c829d605c6ddb6fd229cef2466 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 19:37:45 2020 -0700 ugly but working bufix for too much data commit 0f4d7d70c4c8b44a8e629bbee217df742b76de08 Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 16:38:35 2020 -0700 new metadata api works on frontend. backend bug gives too much info for groups commit 4241cacf42e8b4cf27a08c657f672694be1bde4a Author: AJ Slater <aj@slater.net> Date: Tue Sep 29 00:20:50 2020 -0700 lang serialiezer returns value if text lookup fails commit c9a6465fd913185c2df340c082d1e0985b6fa189 Author: AJ Slater <aj@slater.net> Date: Mon Sep 28 18:18:10 2020 -0700 doubled speed of metadata * bump version * Squashed commit of the following: commit 88404c565b4cc3fbad899ca1bccd366b43b199b5 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 14:54:40 2020 -0700 remove TEST variable. always a bad idea commit a3bef259dc6c72b0a8bb885d0353d7a8bad1b81f Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:58:02 2020 -0700 comments commit 265784ce9fdf5fe2429fda3fa98f271c2275667b Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:54:39 2020 -0700 optimize select_related for metadata just a little more commit a488427a15ab8d0efb4a742308a0696d313492dc Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:45:15 2020 -0700 pop fields before serialization. should be faster commit 07df6a610a7683cf16551c4e644207d10be96f27 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 13:39:42 2020 -0700 clean up serializers and choices. get choices from src too. commit 5d59c5a95dcc6194f76d45c578efdd529a895f90 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:19:19 2020 -0700 fix improper null items on undefined metadata fields commit 905aa375760b374f3e5c8ccdc29f074cf2e3e754 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 00:18:59 2020 -0700 fix metadata folder view commit d6017dd76297f448735198e1a3a1c9f111865a2e Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 23:15:54 2020 -0700 reading choices and messages from the webpack json works now commit 87246c285ac93674eff1ba55be651a05e967102b Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:14:37 2020 -0700 move item xformer into api commit 102b291511cc3e6c34d7efe2f4e7839d5637766c Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 14:04:05 2020 -0700 fix null items commit b7212c3e3a47987d1b20ba604aba408555b32f70 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:44:22 2020 -0700 sorted items commit 5c8cef6cd7eb85c246f3099b35a71f2b87555e51 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:27:26 2020 -0700 componentize browser-settings-menu commit d3fd0f57ea0d5500389e97e1c071896abac3791a Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:17:06 2020 -0700 componentize browser settings dialog commit 8510179f3be465df0d7848751e62a699902dd458 Author: AJ Slater <aj@slater.net> Date: Fri Oct 2 13:05:52 2020 -0700 working filter menu. componentize top toolbar filters. * massive rename of containers and browse for consistency * docstyle linting * lint * remove mypy and vulture. i don't use them. * Squashed commit of the following: commit 936a6c8a202b09db2c8d6bedc8ce1d8add3e3f5e Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:13:22 2020 -0700 move secret key and logging init management into modules. websocket server doesn't configure logging right. commit 328bb8288bbee74b88c0e0754bc13abdb7ea114d Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:35:23 2020 -0700 less picky about passwords commit 8a953e20676ac5ca7e5c226b8792e075dbae1d11 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:17:14 2020 -0700 hypercorn 0.11.0 includes asgi signals and a big logging change. fix the logging with a custom formatter and a flag to logconfig_dict commit e3eae3400a2c07cab11997b74a71397e70f9d3e2 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:20 2020 -0700 upgrade babel/core commit 16034b60a7fb41fe47d67a8c02972231094f9946 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:14:04 2020 -0700 remove comment cruft commit 23885db544421469e46d0d42a75844c73b082fa5 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:13:31 2020 -0700 dev run script alter python path for local dir commit 80b92b45921dadc9d6a3811f7e92df5bd9377505 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:46 2020 -0700 correct logging __name__ and remove print cruft commit 2a6d347dcc71e39a88b2e7c515c8b2fcae07b0de Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:24 2020 -0700 document where i got the timezones middleware commit 59778ffa39967f7b6e42a5bf4fb4c52c8fbe5ce4 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:12:04 2020 -0700 debug lifespan logging commit b3ae98d90fbf3e9f1bf4b5eb17fb467c63f21f54 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 17:11:35 2020 -0700 rename procs & threads for logging commit 77f064e89aaa639c97b1bf7aeb4748a2fefe04d4 Author: AJ Slater <aj@slater.net> Date: Sat Oct 3 21:55:52 2020 -0700 broken asgi signals handling? * Squashed commit of the following: commit 4eca7486fa16467cfee0e38f4d6560c8b8487d12 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 18:19:12 2020 -0700 replace old DEV flag with DEBUG remove redundancy * lint * stop pytest from testing migrations * oops broke flake8 * Squashed commit of the following: commit efa7750ba35e3e32ba005a01cf53aab949c6993c Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:27:34 2020 -0700 componentize browser-main commit 3e9caadb2ffa7e7b679ab813d4c5acb4a7f366cb Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:06:01 2020 -0700 oops forgot to commit settings file for module change commit ca5168afd78bf05bdc7572120c746beb2e93ef22 Author: AJ Slater <aj@slater.net> Date: Mon Oct 5 12:05:31 2020 -0700 settings module commit 54af69ae2562fdb7b101ef57d5ebbf9dd8088482 Author: AJ Slater <aj@slater.net> Date: Sun Oct 4 20:25:01 2020 -0700 componentize browser * lint * componentize reader * remove todo suggestion * more metadata fields * update news * fix restarts * its closer to 5 seconds in practice * update poetry and npm packages * change order of stop and start daemons a bit * fix finding webpack files. Use regex instead of glob * bump news and version * fix root_path bug. v0.5.17 * renaming some variables fixes settings changing * bump version and news * reform progress dialog with offspring timing * no touch stuff actually used * always return * Move sockets to vue-native-sockets. Componentize snackbars and move them to app level * hypercorn bugfix doesn't squash logging anymore. remove workaround. * new poetry install option * lint * update npm * remove an extra futile conenction attempt * add a publish check * lint * fix websocket dumb test * move includes to set.has * log the path as well as the reason to stdout for log watchers * bump news * Squashed commit of the following: commit 926deb8f3214f1c0a3c818904492df0182e42ded Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:39:25 2020 -0700 remove orjson parser. needs glibc commit b98a47f8847b7ea8ac3f50beb7c3045e42cae3d1 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:38:59 2020 -0700 bump Next Up commit 03248e99c0aeb686d4059ceb57f149974ef255a7 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:38:01 2020 -0700 alphabetize deps commit a1f96f39963c3e9bc15325aa09cf68a641c43771 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:37:46 2020 -0700 lint commit b1de6c0228d68e1b73d7c034bad68d73de8c619b Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 20:36:51 2020 -0700 fix hypercorn config path commit 27390428900926886cfeccbde97e7149c04486c7 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 19:31:30 2020 -0700 alphabetize pyproject commit edfd191eb7429487acb8ea345cc10ade34dd0df9 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 16:56:11 2020 -0700 bumpt version to 0.6.0 commit 91c52925ada6bbebd19599d7ae04394f92c46eee Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:34:20 2020 -0700 just do next to nothing for asgi test commit fe6fd4ce49db31639ea47bbb02b17b111b1b7a5e Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:33:51 2020 -0700 fix deprecated log method commit 37ac8132c8c97d74a52c9fa0525b9822204bde41 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:29:13 2020 -0700 fix pytest settings and enhance asgi test commit 364fef6d675eb0cd3e43aed96a238c1474281798 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:19:50 2020 -0700 poetry update, npm update commit 52ece20751a726f78566fe3ef6085e95a1b529ac Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 15:17:48 2020 -0700 centralize issue nameing and handle decimals and 1/2 better commit 33434f2d0aa43e0577ec63d8fefdeea08d9b2dad Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:50:42 2020 -0700 fix overcaching of reader settings commit 9363e3ae218ab2c12042c79d7be70e61b76627d3 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:33:31 2020 -0700 lint commit e40f710e383ee003f2941467172bc7f7337bca77 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:33:01 2020 -0700 fix padding on dialogs commit f96df2da458a1895441dd9ecd54f4c66f7b84b97 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:26:34 2020 -0700 fix reader image centering. move bookmark serializers into own file commit 63610d21bba02614aba6ba236224d23862bab6d4 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 14:06:50 2020 -0700 componentize reader settings dialog commit 2c209bef96781281ddf59b3c7a33785008b7b7e5 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:51:40 2020 -0700 clear form choices more often commit 18ea7033b98aedbf099b2abd410a6f394420e897 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:49:14 2020 -0700 move the vuetify null code into choices json commit 8bd0d2d3836915725e9802c30110366cb07fe2c8 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:31:20 2020 -0700 show browser progress for some user interactions, not others commit a2e3fd015894f26f123f9287d92445c738ffefbb Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:30:47 2020 -0700 fix mark read menu for folders and cleanups commit 785fdbf100fc1389d5b027977a94df20dee3ccdb Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:22:59 2020 -0700 fix manage.py for new settings location commit 77a0d05edc258306cab4e89b6712fa50f65b6cdd Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:04:41 2020 -0700 remove comments commit 6deae30b7effa5fad7116e18b11fe503a54b98ba Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 13:03:58 2020 -0700 clean up serializers with custom fields commit c1c3ab51c4935e84f38021e7a9292945954e9b7e Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 12:43:48 2020 -0700 move special django setup stuff into own file commit e07ea7a552316b92cff452dc6031d7c4f7979ccc Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 12:34:15 2020 -0700 move settings into module commit 1226bb969e03038db677e5702bbcec0f19ebd177 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 12:15:59 2020 -0700 shrink filter controls more. especailly on mobile commit c6fc22f8263b65366e26bbddb3befcb806b51c25 Author: AJ Slater <aj@slater.net> Date: Tue Oct 13 12:03:30 2020 -0700 recheck notify after failed_states notification commit b6cd4b5531ddef8867b8a11e9b37229f0d55dc5e Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 20:50:25 2020 -0700 update npm commit cf192c0c26dd7dedb072d85e4f6cb45de7533307 Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 20:42:45 2020 -0700 comicbox 0.1.5 and update poetry commit 81f17d68a7bf552647b915182c563f4014cabe9a Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 20:42:20 2020 -0700 remoe dry run nonsense commit 99fef62c685c70f7b7c3cf2c457f71bcedc6565e Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 19:57:26 2020 -0700 lint NEWS commit d642581bc62b3514f109b1c18fe4de560cb990a0 Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 19:56:30 2020 -0700 news to md commit 6348b028259b23f2653974f91ec2f2c677a27c09 Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 19:52:14 2020 -0700 unbreak storyArc and seriesgroup filter endpoints commit 87feba07c27f472cd8023311d3738f14cdf6542a Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 19:07:14 2020 -0700 consistent naming of Creators commit ce905132a40e6cc2ca4ee64404a85c0a886e57ca Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 17:34:41 2020 -0700 fix null boolean on book cover commit be0ff1fb7f197cb10c6c88a37e24dc610ad8d58f Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 17:29:04 2020 -0700 reader slider pagination. fixes for reader nav & thumb-label on the inside of the slider button. commit 7617f6e666708d7ca7a54927d7d532b3a14a8212 Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 14:07:17 2020 -0700 bump NEWS commit 8e48e15177f175be1acdcd31e672baea797f9cad Author: AJ Slater <aj@slater.net> Date: Mon Oct 12 14:05:58 2020 -0700 version footer moves to setting dialog commit 515a594e6d015769f91f0522739e0604b054709b Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 14:47:46 2020 -0700 fix settings menu closing commit 5ff9da5d607a061829b953b85a8b667ba12878b2 Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 14:21:08 2020 -0700 change sort icon and add labels on focus to the toolbar commit 06c235f6e1e57afcb7e05a4cfd69251b4b06befa Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 13:03:38 2020 -0700 remove useless sorts. fix reverse sorts commit b3f21bce7a6ce73cdab2f8060678c39117520bad Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 13:03:12 2020 -0700 change warning to debug commit 7641e9b626230bb3b1ce63942559876c8f1f3ffe Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 00:43:15 2020 -0700 format times for browser card order-by. fix sort_by menu length. clean up eslint cruft commit 40f1e6f54378e961e12cb9a79ab3931b63e5335c Author: AJ Slater <aj@slater.net> Date: Sat Oct 10 00:42:21 2020 -0700 fix crash with get_aggregate_field on new sort_by name for Comic commit 415ee902908122d8a0238351dba00a5ebbab283c Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 20:11:44 2020 -0700 lint commit 2a987eef30717f87c374e830af358187c326ed90 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 18:49:23 2020 -0700 first draft of fitlers and sorts commit 21333947890622b82cad6208009913a32aefbb76 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 18:04:49 2020 -0700 rename credit person to creators commit b6738f855d269fefe0b2652b3318ba6004b2359a Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 17:53:19 2020 -0700 fix filtering on none for charfields commit 2d9b9452bf9d2091b4a44c264093f6b7cd4b83d0 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 17:27:46 2020 -0700 add credit_person, teams. fix bug with setBrowseChoices calling twice commit 50fd303026258ae3ce1edf0b23f10ef42fb9cc7f Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 14:53:30 2020 -0700 for choices use aggregate filters commit fe966722fd79eb07a5507ce3699089150289cf18 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 14:07:10 2020 -0700 working filters commit 8c4a0fb8a53d64bdd1e4ea3f7e5baaa07acec908 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 13:12:40 2020 -0700 new filters like pycountry and credit person display but do not filter yet commit 6e717061a6c5f6b6852d5c4a016fcd5efee8c95b Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 12:09:05 2020 -0700 simple new filters commit 9fc372c6be7e984170e81daec208c18448134cfe Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 11:50:20 2020 -0700 add year filter and condense source of filters to one object commit e90a65e77169956dfda212e7f221e06ad472c810 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 11:46:50 2020 -0700 extra protection for empty filter commit 4846d50b6e4cb73535fbfe47f8914e3603b70025 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 11:43:07 2020 -0700 replace ujson with orjson commit d84d725b6310bd101488e505e754748b7d3a24c0 Author: AJ Slater <aj@slater.net> Date: Fri Oct 9 00:17:07 2020 -0700 move default hypercorn toml location. simplify get_root_path function: commit 3f807a9791dd3bf2fb32213b8b25db14c2b21460 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 23:46:13 2020 -0700 modify news commit f91864b71794a641350a37c9f8f68e022c803933 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:42:34 2020 -0700 bump news commit 07dc108e40b0518c0f43ab9db1a2f77f3884858f Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:40:25 2020 -0700 faster json parsing commit 26137ac75f0c009f5ecc12b67d9cfb1c8f5006f5 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:38:18 2020 -0700 fix too visible uproute when there is no route commit 316fefef6340c6a87ae2adefb994f40d3641aa00 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:38:04 2020 -0700 fix nav to r with positive pk commit 8795021b14cf97ab6c1f958747b9b0b371062db7 Author: AJ Slater <aj@slater.net> Date: Thu Oct 8 18:00:13 2020 -0700 api v2 basically working * lint * lint * fix notify on start bug. minor cleanups. * fix redundant navigation. debug logs for browser opened & getpage. don't get on create, wait for opentosee * rename pageNumber to page for consistency with new api * change action names to be declarative * poetry & npm updates * fix my one frontend unit test * oops forgot to remove dry run thing * fix collectstatic for new settings * librarian exception logging just in case * error exit on collect static just incase * remove personal details from git * codex 0.6.1 * lint * Squashed commit of the following: commit a1eb9d1630950052fab714efcdb8a0c04f082b63 Author: AJ Slater <aj@slater.net> Date: Wed Oct 14 18:45:36 2020 -0700 bump version commit a7bfecd8cdaadf6415aaa9dbc1a35cd27c15aef1 Author: AJ Slater <aj@slater.net> Date: Wed Oct 14 18:43:31 2020 -0700 fix warning messages and add an init message commit 76a69aaf868621574d30437dc2146a1cfe61f9cb Author: AJ Slater <aj@slater.net> Date: Wed Oct 14 18:32:40 2020 -0700 lint commit b71759260cab7ae44a29bf740935ffc3421242ed Author: AJ Slater <aj@slater.net> Date: Wed Oct 14 18:31:09 2020 -0700 restore dev-flag * speling * debug logging * turn librariand into a class * faulthander * turn debug on for development * refactor hypercorn config parsing * rename some hypercorn settings methods * collapse DEV into DEBUG again. add LOGLEVEL * bump news * bump version * remove cruft * remove faulthandler * update npm * fix reader bug that only displayed first page * v0.6.4 * docker node stuff moved to codex-builder repo * switch to groovy builder * update poetry * get parent pid instead of passing it in * update runnable image base * adjust height of menues for mobile * easier prefetch managment with vue-meta * update npm * refactor procs & threads * refactor of some websocket ipc stuff * seems fixed * librarian cleanup * move all startup and shutdown into lifespan app * move librarain startup/shutdown back into class * troubleshooting section * poetry update and use cimg:3.9 for circleci * use newer compose version format * name poetry step * poetry update * poetry & npm updates * Multi arch ci (#72) * update poetry & npm * fix websocket test with pytest.asyncio * fix ignores for .venv * update npm * lint * multi-arch first attempt * lint * lint * add eslint to lint & fix * fix eslint call to use local version * lint * better eslint i think * fix build-dist script * fix build multiarch script * dark admin pages * more django-dark fixes * minor notification optimization * npm update, v0.6.6 * bump eslint-plugin-simple-import-sort * remove old rule name * revert simple-import-sort upgrade. * move extrastyle to base * README formatting fix & update roadmap * update roadmap * add anchors to README, more thanks and clairity to many sections * don't duplicate language * make links like github does * build for amd64 with the test script * fix admin template loading. how did this ever work? * 0.6.7 the darkening * update deps and lint * fix jest config * fix docker deps to include armhf versions * lint fixes and remove version from docker-compose.yaml * ci testing script * update NEWS.md * type checker cleanups * update comicbox to 1.6 * Vacuum & Rebuild DB (#85) * fix comment * vacuum cron * removed dep on regex package * switch to rc0 version * oops forgot the actual vacuum code * lint * rebuild db facility * fix language in README * modification to NEWS * fix build-dist for macos * env for 0.6.8rc0 version * update npm & poetry * update jest plugin * lint * switch codex to using simpleques * use much simpler EnvironmentPlugin for passing in package version * advance eslint version * upgrade babel eslint parser and webpack to v5 * update sass-loader * lint * oops sass-loader 11+ needs vuecli v5 * new browser title more explicit * fix titles for folders * fix loading browserChoices for python * rc1 uses comicbox 0.1.7 which doesn't use the regex package * restore all platforms to env * 0.6.8rc2 uses python 3.9, 3.10 is incompatible with hypercorn * change defaults to series instead of volume * toolbar with subheading for group type * update package.json * update deps * update axios * bump version to rc3 * catch a rare navigate to same place error * django 3.2 migration for auto ids * ignore personal TODO list * lint * fix version number * Squashed commit of the following: commit 4b50bb83599d7084a3d96264b49d363d7b3d5114 Author: AJ Slater <aj@slater.net> Date: Tue Oct 12 17:32:34 2021 -0700 Better docs for autofield commit b9ffd1e7300333e7c2dd1d15ea95cbcf535bccb3 Author: AJ Slater <aj@slater.net> Date: Tue Oct 12 16:51:07 2021 -0700 Autofield instead of bigautofield * 0.6.8rc4 * add swiping back to reader * jsconfig for vueter * update proxy tester nginx config to use port 80 * update deps * Squashed commit of the following: kb help commit a2b14296ec6b542ce81e962eaf33a1214ce2f549 Author: AJ Slater <aj@slater.net> Date: Wed Oct 13 14:37:47 2021 -0700 remove cruft commit 2117b1b5c3dd82b77b8bf79c86f096288931d649 Author: AJ Slater <aj@slater.net> Date: Wed Oct 13 14:37:08 2021 -0700 fix 2 hotkey commit 8624c28d1b9d7a6914ee4c051c2362cc1277d625 Author: AJ Slater <aj@slater.net> Date: Wed Oct 13 14:20:09 2021 -0700 mostly working * 0.6.8rc5 * v0.6.8 * better fix for sed in build dist

Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

…#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

* title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the parent queryset's annotations, select_related, prefetch_related, and only(*fields) so the 1-3 returned rows have everything the serializer needs. `arc_count = len(pks)` replaces the prior separate count query. Tier 2 #4 — per-book settings/bookmark batching ================================================ `_append_with_settings` fired `SettingsReader.objects.filter(comic=book).first()` and `Bookmark.objects.filter(comic=book).first()` per book — 2-6 queries for 1-3 books. New `_batch_settings_and_bookmarks` helper fires two `filter(comic_id__in=window_pks)` calls partitioned by comic_id in Python. Always 2 queries regardless of book count. Headline numbers ================ reader_open_large_arc cold: 28 → 25 queries / 89 → 52 ms (-42%) reader_open cold: 26 → 25 queries / 49 → 51 ms (noise) (other 5 flows stable) The 855-issue series exercises the worst case for the prior iteration. The wall-time win scales with arc size; smaller arcs show ±0 noise. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * use bunx for ttab * caps for close book button again * Reader views perf: Stage 2 — route caching + settings name-lookup batching (#602) Closes Phase D from `tasks/reader-views-perf/99-summary.md`. Tier 1 #3 — `cache_page(READER_TIMEOUT) + vary_on_cookie` on c/<pk> ================================================================== New `READER_TIMEOUT = 60` constant in `codex/urls/const.py` (mirrors `OPDS_TIMEOUT` shape and TTL trade-off). `vary_on_cookie` scopes the cache key per session so per-user responses (bookmark, settings, ACL-filtered arcs) don't leak across users. reader_open warm: 24 → 0 queries / 36 → 1.9 ms (-94%) reader_open_large_arc warm: 24 → 0 queries / 35 → 1.9 ms (-95%) Cold pass unchanged — view runs the full pipeline on cache miss. Real-world reader traffic is dominated by tab refreshes / mobile-app re-foreground patterns where the same `c/<pk>` URL gets hit in quick succession; those collapse to the warm path now. Cross-user isolation verified manually: User A and User B see different payloads on the same URL (762 B vs 739 B reflecting different ACL); each user's warm response matches their own cold; `Vary: Accept, Cookie, origin` on every response. Tier 2 #7 — Fold settings name-lookup into comic prefetch ========================================================== Settings GET pre-fetches the comic to resolve scope FK values. Added `select_related` for the related models keyed off a new `_COMIC_FK_TO_RELATED` map; the comic prefetch now joins `codex_series` / `codex_folder` and the new `_resolve_scope_name` helper reads display names off the joined row instead of firing a separate `Model.objects.filter(pk).values_list("name")` query per scope. Story-arc scope (`a`) takes its `scope_pk` from a `?story_arc_pk=` query param so the comic prefetch can't help; falls back to the targeted lookup when requested. settings_multiscope (?scopes=g,s,c) cold: 8 → 7 queries Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 3 — Comicbox archive cache for page endpoint (#603) Closes the highest-impact remaining item in the reader plan (Tier 1 #1 in tasks/reader-views-perf/99-summary.md). Process-wide LRU of open Comicbox archives keyed by file path. Codex runs Granian in single-worker embed mode (see run.py) so one in-process cache is shared by every request thread without multi-worker dilution. Telemetry-driven sizing (1208 unique installations from the chronicle backup): - p50 install peaks at 2 concurrent sessions, p95 at 22. - 83% Linux x86_64 + Docker, 11% Linux ARM (RPi/ARM NAS). - Many installs run on 1-2 GB RAM NAS boxes — conservative defaults essential. Configuration (env vars; conservative defaults suit constrained NAS deployments): - CODEX_READER_ARCHIVE_CACHE_SIZE (default 4) - CODEX_READER_ARCHIVE_CACHE_TTL (default 30 s) - CODEX_READER_ARCHIVE_CACHE_DISABLE (default false) Memory budget per cached entry: ~1-3 MB for CBZ, 5-30 MB for PDF. 4 entries × 5 MB = 20 MB worst-case. FD budget: 4 + Granian baseline (codex already raises RLIMIT_NOFILE to 8192). Per-archive `threading.Lock` serializes extraction because zipfile / rarfile / PDF backends aren't documented as thread-safe for concurrent read() calls on the same instance. The cache structure itself uses a separate short-held lock so unrelated archives proceed in parallel. Honest accounting in stage3.md: - On dev hardware (NVMe + warm OS cache) the cache is invisible in the harness — Comicbox open is ~0.7 ms vs ~13 ms Django middleware overhead. - On the deployment shapes the telemetry shows dominate (HDD- backed NAS, ARM SBCs, USB-3 external storage) the open cost is 5-50 ms+ and the cache wins meaningfully. Architecture is sound; the win is theoretical without slow-hardware test infrastructure. - cacheBook (whole-book prefetch, opt-in, default off): per-archive lock serializes the burst. Acceptable trade-off for correctness; future Stage could pool 2-4 Comicboxes per path to allow parallel extraction at multiplied memory cost. Functional verification: byte-identical responses to pre-cache implementation across multiple page indexes (compared via CODEX_READER_ARCHIVE_CACHE_DISABLE=1). Harness re-run shows no regression. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 4 — Phase F cleanups bundled (#604) Closes Phase F from `tasks/reader-views-perf/99-summary.md` — three implementations + one item closed as superseded. Tier 3 #8 — `Max("updated_at")` in `_get_story_arcs` ==================================================== The prior code used `JsonGroupArray("updated_at")` to gather ISO-string timestamps then parsed them per-row in Python via `strptime`. Replaced with `Max("updated_at")` which returns a typed `datetime` via the field's `from_db_value` hook — eliminates the per-row parse and the Python-side fold over EPOCH_START. Drops now-unused imports (datetime, EPOCH_START, _UPDATED_ATS_DATE_FORMAT_STR). Tier 4 #12 — De-dup `_get_bookmark_auth_filter` ================================================ `ReaderSettingsBaseView` had its own copy inlined from the former BookmarkAuthMixin. Added BookmarkAuthMixin to the base classes and dropped the local copy. Settings lookup now calls the inherited `get_bookmark_auth_filter()`. Code-health only — no behaviour change. Tier 4 #15 — Page-endpoint ACL decision cache ============================================= New `PageAclCache` (in `_archive_cache.py`) keyed on `(auth_key, comic_pk) → (path, file_type)` with a 60 s TTL. ReaderPageView's new `_resolve_path_and_type` helper checks the cache first; misses fall through to the existing `Comic.objects.filter(acl_filter).get(pk=pk)` path and populate the cache. Configuration knobs (env vars, mirroring archive cache shape): - CODEX_READER_PAGE_ACL_CACHE_SIZE (default 64) - CODEX_READER_PAGE_ACL_CACHE_TTL (default 60 s) - CODEX_READER_PAGE_ACL_CACHE_DISABLE (default false) Sequential reads of the same comic skip the ACL filter SQL after the first hit. For a 200-page read: page 1 cold (9 queries / 15ms), pages 2-200 warm (2 queries / 3.6ms each). page_first / middle / no_bookmark warm: 8 → 2 queries / 10 → 3.6 ms (-64%) Cold pass intentionally unchanged (first hit pays full ACL pipeline). Tier 3 #11 — Superseded ======================= Stage 1's rewrite already addresses the underlying concern: `get_book_collection` materializes only 1-3 fully-annotated rows (via `values_list("pk")` + `pk__in` window). Slimming the annotation pyramid for prev/next would shave microseconds off 1-2 extra rows. Closed as won't-fix. Harness update ============== `_capture` now also clears `archive_cache` + `page_acl_cache` between flows so the cold-pass measurement is a true cold-cache reading rather than a warm-up-loop carryover. Without this fix, Stage 4's warm wins were leaking into the cold-pass numbers. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * minor lint fixes * reduce comic create complexity * ignore tasks for eslint * Cover cleanup: rename, logo recolor, per-type placeholders, 404/202 (#605) * Cover cleanup: rename missing-cover → comic - img/missing-cover.svg → img/comic.svg - codex/static_src/img/missing-cover.svg → comic.svg - codex/static_src/img/missing-cover-165.webp → comic-165.webp Plus the build script and the runtime constant: - bin/icons_transform.py — ICONS dict key + the two special-case branches that referenced "missing-cover". - codex/views/const.py — MISSING_COVER_FN. The webp keeps the -165 width suffix so the existing ``f"{name}-{width}"`` filename template in icons_transform.py applies without modification. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Cover cleanup: recolor comic.svg logo to grey The snail-eye logo was embedded via <image xlink:href="logo.svg">, which doesn't cascade fills into the embedded SVG — so the existing fill:#808080 attempt on the <image> wrapper was a no-op and the logo rendered in its own orange gradient against the grey book. Inline the logo paths via a nested <svg> at the same position (81,149 / 180×144 / preserveAspectRatio=none) and recolor to fill:#808080 so the logo matches the book line art. Drop the black background circle so the result is pure grey line art. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Cover cleanup: per-type placeholders on web cards book-cover.vue's <v-img> had no lazy-src, so cards rendered blank between mount and the cover endpoint resolving. Add a per-type SVG placeholder via lazy-src so the right shape (publisher / series / volume / folder / story-arc / comic) shows immediately. The dispatch lives in api/v3/browser.js as getPlaceholderSrc(group) to mirror MISSING_COVER_NAME_MAP in views/const.py — single source of truth across both Vue consumers. Unknown groups fall back to comic.svg; root "r" is omitted intentionally since root never appears as a card group. Drive-by: metadata-cover.vue's forceGenericCover branch hardcoded volume.svg regardless of group. Switch it to the same dispatch so the metadata pane shows the right placeholder. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Cover cleanup: 404/202 empty bodies on missing covers The cover endpoint used to serve the comic-165.webp placeholder body with a 200 (or 202 + Retry-After while pending). With the web client now showing a per-type SVG via lazy-src and OPDS clients having their own default rendering, the placeholder body is dead weight. Switch _missing_cover_response to return an empty body — 404 by default, 202 for the queued case. The web polling loop already treats !== 202 as "stop polling, render whatever loaded"; the empty 404 body lets <v-img> keep displaying its lazy-src SVG. OPDS clients see 404 and fall back to whatever they normally render. Cache-Control: no-store moves into the helper so it covers both 404 and 202 — defensive against reverse-proxy 404 caching delaying a cover from appearing once the cover thread writes the real bytes. The MISSING_COVER_FN / MISSING_COVER_PATH constants in views/const.py and the comic-165.webp asset stay in place — keep them dormant until OPDS-client testing confirms the 404 path works across the ecosystem. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Cover cleanup: capture plan in tasks/ Track the four-commit cover cleanup design (rename, logo recolor, per-type placeholders, 404/202 status-code split) for future reference and the OPDS-client testing follow-up. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * use prettier for remarkignore * Cover cleanup: render missing covers unblurred (#606) When the cover endpoint returned 404, <v-img> kept showing the blurred lazy-src — but 404 means "no cover ever," not "still loading," so the loading blur was misleading. Track a `missing` flag set by the polling loop on 404 and switch the v-img to render the placeholder SVG directly as `src` (no lazy-src) so it appears unblurred. Loading and 202-pending paths keep the blurred lazy-src as before, since those are genuine loading states. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Admin views perf: stats cache, query reshaping, write-path cleanups (#610) * Admin views perf: capture plan in tasks/ Surface map of codex/views/admin/, ranked findings, suggested ordering, methodology, risks. Single-file plan because the surface is small (755 LOC, 11 modules) — mirrors the structure of the reader-views-perf and opds-views-perf plans without the per-file split. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin views perf: cache stats response (TTL=60s) The /admin/stats endpoint runs ~30 SELECT COUNT(*) round trips per call (config: 4, groups: 7, metadata: 19), plus a Session decode loop and Python aggregation over every SettingsBrowser / SettingsReader row. Stats tab loads and the nightly telemeter both hit it. Cache the assembled CodexStats payload in django.core.cache keyed by a sha256 digest of the requested params. AdminLibraryViewSet and AdminGroupViewSet already call cache.clear() on writes, so library / group changes evict these keys alongside cachalot's ORM cache. Other writes (User CRUD, FailedImport churn) just wait for the 60s TTL — fine for a stats tab. The api key is added after cache retrieval so a rotation reflects immediately; the deepcopy on hit keeps callers from mutating the cached dict via _add_api_key. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin views perf: SQL-aggregate user settings, sample sessions Two compounding hot spots in CodexStats._get_session_stats: 1. Decoded every Session row (HMAC + JSON parse per row) just to count how many lacked _auth_user_id. With session history this dominated /admin/stats cold time. 2. Iterated every SettingsBrowser and SettingsReader row as a full ORM instance and bucketed three fields each in Python. The SettingsReader table can be huge — one row per (user, scope, client) — so a heavy reader's instance hit thousands of rows just to populate three histograms. Replace with: - Sample-based anon-session estimate. Decode at most 100 sessions, count anonymous in the sample, scale by the total. Telemetry is approximate by nature; the ratio is what matters. - Per-field SQL aggregation via .values(field).annotate(Count("pk")). Three columns × two models = six index-friendly GROUP BY queries instead of O(N+M) Python rows. Also flatten _USER_STATS to a typed tuple-of-tuples so basedpyright can narrow the model class without # ignore comments. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin views perf: correlated subquery counts on Library list The AdminLibraryViewSet annotated comic_count and failed_count with Count(..., distinct=True), which joined Comic and FailedImport into the Library query at the same time. DIS…

* update deps * v1.10.1 fix opds v2 links * fix news * fix gha trigger gate logic * fix gha yaml bug * fix gha build and deploy from firing on develop * fix admin update user view & serializer to allow user updates. * fix gha yaml syntax error' * fix runing build on develop branch * fix gha branch gate logic again * skip tests on deploy if identical files. change discord colors & messages. move ci contaiiner to compose * fix polling all libraries when no ids submitted in task * fix poll all library button * fix opds v2 manifest series link * force browser reset on start page of opds v2 * fix dev csp for vite hmr * working server folder picker with enter activates menu and waits to render so the menu isn't lost in the center of the screen * clean up server folder code to be smaller and remove cruft * bump news * attempt to make granian auto reload not hold on to file handles by adjusting template settings in debug mode * Squashed commit of the following: commit c76660006840abb36aa37d7355d5c7e242babebf Merge: b2b5a011a be94a0ae0 Author: AJ Slater <aj@slater.net> Date: Sun Apr 5 15:49:11 2026 -0700 Merge branch 'develop' into select-multiple commit b2b5a011ab207a7307505b092877f789e1a66ab3 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 22:15:54 2026 -0700 fix card menu hover highlighting commit 5a98fe23d89be5e11fa46ada927100ccd3e08cda Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 22:13:37 2026 -0700 new design for select many mode. no settings drawer involvement. reconfigure cards commit 82c612e3f77eec90b02465174b49fc8552871686 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 01:50:11 2026 -0700 add github config to source include. remove dockerhub config commit b02bd450f4598300d78433a2d749741872d14894 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 01:42:32 2026 -0700 update deps commit d7bbc875e7936d808259c9ba726127fcc3af905e Author: AJ Slater <aj@slater.net> Date: Fri Apr 3 23:56:12 2026 -0700 select many toolbar commit 246d5ecda59aaabb83e41db3ad32ec552804b9ae Author: AJ Slater <aj@slater.net> Date: Fri Apr 3 22:23:24 2026 -0700 select many feature first attempt * bump news * efficiency and bugfixes for gha yaml * fix container name for gha * fix test report steps running if tests don't run * use eslint_d for frontend lint * make sure to build the ci compose locally * fix too exuberant volumes mounting for gha compose * remove working dir entry i'm not sure it's good * add debug line to gha workflow * remove debug line * fix test make order * stop docker compose when done in gha * i don't think i need DJGNO_SETTINGS_MODULE th ci thing * fix down call to own bring down ci task * fix docker compose down" * fix test results upload * fix test step name * fix gha permissions * bump news * update deps * browser set_params method saves last route and saves params to settings * bump version to 1.10.4 * modify gha discord notification * use happy-dom for frontend tests * docker tag script better than the one in ci/ * update deps * far saner param initialization for browsers and opds start pages * update dpes & bump alpha version * v1.10.5a0 (#551) * bump news * cirlcle ci no longer handles pre-release * remove alpha scripts for circleci" * fix lint-ci * fix pre-relase gha * fix default last route on start pages * bump version * fix default params * alpha2 * regular version 1.10.5 * remove develop circleci builds * adjust variable name * bump version to 6 alpha 0 * fix default params for feed_views in opds 2 * debug logging for django crash * v1.10.6a0 (#553) * bump news * ignore ruff qa for debug build * ignore shellcheck" * lint * minor refactors of opds v2 feed * regular v1.10.6 version * fix docker-tag-latest cscript * ignore gh token file * try to log more request errors * reconfigure logging to hopefully be more verbose about request errors in production * update deps and bump to alpha version * bump news (#555) * v1.10.7 * bump news * update devenv * update devevn and deps * fix pm script * move django-check to test category * workflow build frontend and collect static for prodcution build. fix test upload * bump version and news 1.10.8 * fix dev-module script * fix news * explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-thro…

* update deps * v1.10.1 fix opds v2 links * fix news * fix gha trigger gate logic * fix gha yaml bug * fix gha build and deploy from firing on develop * fix admin update user view & serializer to allow user updates. * fix gha yaml syntax error' * fix runing build on develop branch * fix gha branch gate logic again * skip tests on deploy if identical files. change discord colors & messages. move ci contaiiner to compose * fix polling all libraries when no ids submitted in task * fix poll all library button * fix opds v2 manifest series link * force browser reset on start page of opds v2 * fix dev csp for vite hmr * working server folder picker with enter activates menu and waits to render so the menu isn't lost in the center of the screen * clean up server folder code to be smaller and remove cruft * bump news * attempt to make granian auto reload not hold on to file handles by adjusting template settings in debug mode * Squashed commit of the following: commit c76660006840abb36aa37d7355d5c7e242babebf Merge: b2b5a011a be94a0ae0 Author: AJ Slater <aj@slater.net> Date: Sun Apr 5 15:49:11 2026 -0700 Merge branch 'develop' into select-multiple commit b2b5a011ab207a7307505b092877f789e1a66ab3 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 22:15:54 2026 -0700 fix card menu hover highlighting commit 5a98fe23d89be5e11fa46ada927100ccd3e08cda Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 22:13:37 2026 -0700 new design for select many mode. no settings drawer involvement. reconfigure cards commit 82c612e3f77eec90b02465174b49fc8552871686 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 01:50:11 2026 -0700 add github config to source include. remove dockerhub config commit b02bd450f4598300d78433a2d749741872d14894 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 01:42:32 2026 -0700 update deps commit d7bbc875e7936d808259c9ba726127fcc3af905e Author: AJ Slater <aj@slater.net> Date: Fri Apr 3 23:56:12 2026 -0700 select many toolbar commit 246d5ecda59aaabb83e41db3ad32ec552804b9ae Author: AJ Slater <aj@slater.net> Date: Fri Apr 3 22:23:24 2026 -0700 select many feature first attempt * bump news * efficiency and bugfixes for gha yaml * fix container name for gha * fix test report steps running if tests don't run * use eslint_d for frontend lint * make sure to build the ci compose locally * fix too exuberant volumes mounting for gha compose * remove working dir entry i'm not sure it's good * add debug line to gha workflow * remove debug line * fix test make order * stop docker compose when done in gha * i don't think i need DJGNO_SETTINGS_MODULE th ci thing * fix down call to own bring down ci task * fix docker compose down" * fix test results upload * fix test step name * fix gha permissions * bump news * update deps * browser set_params method saves last route and saves params to settings * bump version to 1.10.4 * modify gha discord notification * use happy-dom for frontend tests * docker tag script better than the one in ci/ * update deps * far saner param initialization for browsers and opds start pages * update dpes & bump alpha version * v1.10.5a0 (#551) * bump news * cirlcle ci no longer handles pre-release * remove alpha scripts for circleci" * fix lint-ci * fix pre-relase gha * fix default last route on start pages * bump version * fix default params * alpha2 * regular version 1.10.5 * remove develop circleci builds * adjust variable name * bump version to 6 alpha 0 * fix default params for feed_views in opds 2 * debug logging for django crash * v1.10.6a0 (#553) * bump news * ignore ruff qa for debug build * ignore shellcheck" * lint * minor refactors of opds v2 feed * regular v1.10.6 version * fix docker-tag-latest cscript * ignore gh token file * try to log more request errors * reconfigure logging to hopefully be more verbose about request errors in production * update deps and bump to alpha version * bump news (#555) * v1.10.7 * bump news * update devenv * update devevn and deps * fix pm script * move django-check to test category * workflow build frontend and collect static for prodcution build. fix test upload * bump version and news 1.10.8 * fix dev-module script * fix news * explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotsp…

* clean up server folder code to be smaller and remove cruft * bump news * attempt to make granian auto reload not hold on to file handles by adjusting template settings in debug mode * Squashed commit of the following: commit c76660006840abb36aa37d7355d5c7e242babebf Merge: b2b5a011a be94a0ae0 Author: AJ Slater <aj@slater.net> Date: Sun Apr 5 15:49:11 2026 -0700 Merge branch 'develop' into select-multiple commit b2b5a011ab207a7307505b092877f789e1a66ab3 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 22:15:54 2026 -0700 fix card menu hover highlighting commit 5a98fe23d89be5e11fa46ada927100ccd3e08cda Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 22:13:37 2026 -0700 new design for select many mode. no settings drawer involvement. reconfigure cards commit 82c612e3f77eec90b02465174b49fc8552871686 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 01:50:11 2026 -0700 add github config to source include. remove dockerhub config commit b02bd450f4598300d78433a2d749741872d14894 Author: AJ Slater <aj@slater.net> Date: Sat Apr 4 01:42:32 2026 -0700 update deps commit d7bbc875e7936d808259c9ba726127fcc3af905e Author: AJ Slater <aj@slater.net> Date: Fri Apr 3 23:56:12 2026 -0700 select many toolbar commit 246d5ecda59aaabb83e41db3ad32ec552804b9ae Author: AJ Slater <aj@slater.net> Date: Fri Apr 3 22:23:24 2026 -0700 select many feature first attempt * bump news * efficiency and bugfixes for gha yaml * fix container name for gha * fix test report steps running if tests don't run * use eslint_d for frontend lint * make sure to build the ci compose locally * fix too exuberant volumes mounting for gha compose * remove working dir entry i'm not sure it's good * add debug line to gha workflow * remove debug line * fix test make order * stop docker compose when done in gha * i don't think i need DJGNO_SETTINGS_MODULE th ci thing * fix down call to own bring down ci task * fix docker compose down" * fix test results upload * fix test step name * fix gha permissions * bump news * update deps * browser set_params method saves last route and saves params to settings * bump version to 1.10.4 * modify gha discord notification * use happy-dom for frontend tests * docker tag script better than the one in ci/ * update deps * far saner param initialization for browsers and opds start pages * update dpes & bump alpha version * v1.10.5a0 (#551) * bump news * cirlcle ci no longer handles pre-release * remove alpha scripts for circleci" * fix lint-ci * fix pre-relase gha * fix default last route on start pages * bump version * fix default params * alpha2 * regular version 1.10.5 * remove develop circleci builds * adjust variable name * bump version to 6 alpha 0 * fix default params for feed_views in opds 2 * debug logging for django crash * v1.10.6a0 (#553) * bump news * ignore ruff qa for debug build * ignore shellcheck" * lint * minor refactors of opds v2 feed * regular v1.10.6 version * fix docker-tag-latest cscript * ignore gh token file * try to log more request errors * reconfigure logging to hopefully be more verbose about request errors in production * update deps and bump to alpha version * bump news (#555) * v1.10.7 * bump news * update devenv * update devevn and deps * fix pm script * move django-check to test category * workflow build frontend and collect static for prodcution build. fix test upload * bump version and news 1.10.8 * fix dev-module script * fix news * explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no impleme…

* add debug line to gha workflow * remove debug line * fix test make order * stop docker compose when done in gha * i don't think i need DJGNO_SETTINGS_MODULE th ci thing * fix down call to own bring down ci task * fix docker compose down" * fix test results upload * fix test step name * fix gha permissions * bump news * update deps * browser set_params method saves last route and saves params to settings * bump version to 1.10.4 * modify gha discord notification * use happy-dom for frontend tests * docker tag script better than the one in ci/ * update deps * far saner param initialization for browsers and opds start pages * update dpes & bump alpha version * v1.10.5a0 (#551) * bump news * cirlcle ci no longer handles pre-release * remove alpha scripts for circleci" * fix lint-ci * fix pre-relase gha * fix default last route on start pages * bump version * fix default params * alpha2 * regular version 1.10.5 * remove develop circleci builds * adjust variable name * bump version to 6 alpha 0 * fix default params for feed_views in opds 2 * debug logging for django crash * v1.10.6a0 (#553) * bump news * ignore ruff qa for debug build * ignore shellcheck" * lint * minor refactors of opds v2 feed * regular v1.10.6 version * fix docker-tag-latest cscript * ignore gh token file * try to log more request errors * reconfigure logging to hopefully be more verbose about request errors in production * update deps and bump to alpha version * bump news (#555) * v1.10.7 * bump news * update devenv * update devevn and deps * fix pm script * move django-check to test category * workflow build frontend and collect static for prodcution build. fix test upload * bump version and news 1.10.8 * fix dev-module script * fix news * explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the in…

* i don't think i need DJGNO_SETTINGS_MODULE th ci thing * fix down call to own bring down ci task * fix docker compose down" * fix test results upload * fix test step name * fix gha permissions * bump news * update deps * browser set_params method saves last route and saves params to settings * bump version to 1.10.4 * modify gha discord notification * use happy-dom for frontend tests * docker tag script better than the one in ci/ * update deps * far saner param initialization for browsers and opds start pages * update dpes & bump alpha version * v1.10.5a0 (#551) * bump news * cirlcle ci no longer handles pre-release * remove alpha scripts for circleci" * fix lint-ci * fix pre-relase gha * fix default last route on start pages * bump version * fix default params * alpha2 * regular version 1.10.5 * remove develop circleci builds * adjust variable name * bump version to 6 alpha 0 * fix default params for feed_views in opds 2 * debug logging for django crash * v1.10.6a0 (#553) * bump news * ignore ruff qa for debug build * ignore shellcheck" * lint * minor refactors of opds v2 feed * regular v1.10.6 version * fix docker-tag-latest cscript * ignore gh token file * try to log more request errors * reconfigure logging to hopefully be more verbose about request errors in production * update deps and bump to alpha version * bump news (#555) * v1.10.7 * bump news * update devenv * update devevn and deps * fix pm script * move django-check to test category * workflow build frontend and collect static for prodcution build. fix test upload * bump version and news 1.10.8 * fix dev-module script * fix news * explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on …

* bump news * update deps * browser set_params method saves last route and saves params to settings * bump version to 1.10.4 * modify gha discord notification * use happy-dom for frontend tests * docker tag script better than the one in ci/ * update deps * far saner param initialization for browsers and opds start pages * update dpes & bump alpha version * v1.10.5a0 (#551) * bump news * cirlcle ci no longer handles pre-release * remove alpha scripts for circleci" * fix lint-ci * fix pre-relase gha * fix default last route on start pages * bump version * fix default params * alpha2 * regular version 1.10.5 * remove develop circleci builds * adjust variable name * bump version to 6 alpha 0 * fix default params for feed_views in opds 2 * debug logging for django crash * v1.10.6a0 (#553) * bump news * ignore ruff qa for debug build * ignore shellcheck" * lint * minor refactors of opds v2 feed * regular v1.10.6 version * fix docker-tag-latest cscript * ignore gh token file * try to log more request errors * reconfigure logging to hopefully be more verbose about request errors in production * update deps and bump to alpha version * bump news (#555) * v1.10.7 * bump news * update devenv * update devevn and deps * fix pm script * move django-check to test category * workflow build frontend and collect static for prodcution build. fix test upload * bump version and news 1.10.8 * fix dev-module script * fix news * explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop red…

* docker tag script better than the one in ci/ * update deps * far saner param initialization for browsers and opds start pages * update dpes & bump alpha version * v1.10.5a0 (#551) * bump news * cirlcle ci no longer handles pre-release * remove alpha scripts for circleci" * fix lint-ci * fix pre-relase gha * fix default last route on start pages * bump version * fix default params * alpha2 * regular version 1.10.5 * remove develop circleci builds * adjust variable name * bump version to 6 alpha 0 * fix default params for feed_views in opds 2 * debug logging for django crash * v1.10.6a0 (#553) * bump news * ignore ruff qa for debug build * ignore shellcheck" * lint * minor refactors of opds v2 feed * regular v1.10.6 version * fix docker-tag-latest cscript * ignore gh token file * try to log more request errors * reconfigure logging to hopefully be more verbose about request errors in production * update deps and bump to alpha version * bump news (#555) * v1.10.7 * bump news * update devenv * update devevn and deps * fix pm script * move django-check to test category * workflow build frontend and collect static for prodcution build. fix test upload * bump version and news 1.10.8 * fix dev-module script * fix news * explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open …

* update devevn and deps * fix pm script * move django-check to test category * workflow build frontend and collect static for prodcution build. fix test upload * bump version and news 1.10.8 * fix dev-module script * fix news * explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset bu…

* explain news * fix opds clear search setting * fix clear search button * use a registry cache instead of gha cache for the dist-builder * gha use more env vars for image names. retain python dist for 2 days. * new quick deploy gha script. update deps & devenv. * silence watchfiles 5 second timeout debug message * consolidate null values const * make scope private * update devenv * update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the pa…

* update deps. migrate to unhead v3 * update deps * fix creating reader global settings * fix caching * rename codex build-dist to codex-ci * fix image name. make gha steps depend on each other more. * fix gha syntax errors * names for gha steps * use ghcr.io for python-debian base * update deps * format dockerfile * picopt treestamps * fix custom covers not importing. v1.10.11 * fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the parent queryset's annotations, select_related, prefetch_related, and only(*fields) so the 1-3 returned rows have everything the serializer needs. `arc_count = len(pks)` replaces the prior separate count query. Tier 2 #4 — per-book settings/bookmark batching ================================================ `_append_with_settings` fired `SettingsReader.objects.filter(comic=book).first()` a…

* fix custom covers count in admin view * bump news for custom cover count fix * update deps * codex identification in server tag and opds generator tag * update deps * force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the parent queryset's annotations, select_related, prefetch_related, and only(*fields) so the 1-3 returned rows have everything the serializer needs. `arc_count = len(pks)` replaces the prior separate count query. Tier 2 #4 — per-book settings/bookmark batching ================================================ `_append_with_settings` fired `SettingsReader.objects.filter(comic=book).first()` and `Bookmark.objects.filter(comic=book).first()` per book — 2-6 queries for 1-3 books. New `_batch_settings_and_bookmarks` helper fires two `filter(comic_id__in=window_pks)` calls partitioned by comic_id in Python. Always 2 queries regardless of book count. Headline numbers ================ reader_open_large_arc cold: 28 → 25 queries / 89 → 52 ms (-42%) reader_open co…

* force no entries on opds start page * common opds start page mixin. emtpy group objects on start page * update deps. typechecking. * api change q to search * standardize search param as 'search' instead of 'q' or other variations * remove errant icecream * clear settings on backend * Squashed commit of the following: Fix clear settings null bug, add global settings clear button - clearComicSettings was setting book.settings to null, causing Object.entries() to throw TypeError downstream in getBookSettings - Add null guard in getBookSettings as defense-in-depth - Add null guard in isClearDisabled computed - Add clearGlobalSettings action and clear button to Default Settings panel - Compare against READER_DEFAULTS to determine if global clear is disabled * simplify settings class hierarchy * rename select-many store to browser-select-many * switch to bun. updated devenv * add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the parent queryset's annotations, select_related, prefetch_related, and only(*fields) so the 1-3 returned rows have everything the serializer needs. `arc_count = len(pks)` replaces the prior separate count query. Tier 2 #4 — per-book settings/bookmark batching ================================================ `_append_with_settings` fired `SettingsReader.objects.filter(comic=book).first()` and `Bookmark.objects.filter(comic=book).first()` per book — 2-6 queries for 1-3 books. New `_batch_settings_and_bookmarks` helper fires two `filter(comic_id__in=window_pks)` calls partitioned by comic_id in Python. Always 2 queries regardless of book count. Headline numbers ================ reader_open_large_arc cold: 28 → 25 queries / 89 → 52 ms (-42%) reader_open cold: 26 → 25 queries / 49 → 51 ms (noise) (other 5 flows stable) The 855-issue series exercises the worst case for the prior iteration. The wall-time win scales wit…

* add a claude md * use frozenattrdict to speed up configuration * fix rename of browserSelectMany store * auth token help * fix sort-ignores to make deterministic across shells with different locales * fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the parent queryset's annotations, select_related, prefetch_related, and only(*fields) so the 1-3 returned rows have everything the serializer needs. `arc_count = len(pks)` replaces the prior separate count query. Tier 2 #4 — per-book settings/bookmark batching ================================================ `_append_with_settings` fired `SettingsReader.objects.filter(comic=book).first()` and `Bookmark.objects.filter(comic=book).first()` per book — 2-6 queries for 1-3 books. New `_batch_settings_and_bookmarks` helper fires two `filter(comic_id__in=window_pks)` calls partitioned by comic_id in Python. Always 2 queries regardless of book count. Headline numbers ================ reader_open_large_arc cold: 28 → 25 queries / 89 → 52 ms (-42%) reader_open cold: 26 → 25 queries / 49 → 51 ms (noise) (other 5 flows stable) The 855-issue series exercises the worst case for the prior iteration. The wall-time win scales with arc size; smaller arcs show ±0 noise. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * use bunx for ttab * caps for close book button again * Reader views perf: Stage 2 — route caching + settings name-lookup batching (#602) Closes Phase D from `tasks/reader-views-perf/99-summary.md`. Tier 1 #3 — `cache_page(READER_TIMEOUT) + vary_on_cookie` on c/<pk> ================================================================== New `READER_TIMEOUT = 60` constant in `codex/urls/const.py` (mirrors `OPDS_TIMEOUT` shape and TTL trade-off). `vary_on_cookie` scopes the cache key per session so per-user responses (bookmark, settings, ACL-filtered arcs) don't leak across users. reader_open warm: 24 → 0 queries / 36 → 1.9 ms (-94%) reader_open_large_arc warm: 24 → 0 queries / 35 → 1.9 ms (-95%) Cold pass unchanged — view runs the full pipeline on cache miss. Real-world reader tra…

* fix crash on settings not being raw * another gaurd for getMetadta() * remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the parent queryset's annotations, select_related, prefetch_related, and only(*fields) so the 1-3 returned rows have everything the serializer needs. `arc_count = len(pks)` replaces the prior separate count query. Tier 2 #4 — per-book settings/bookmark batching ================================================ `_append_with_settings` fired `SettingsReader.objects.filter(comic=book).first()` and `Bookmark.objects.filter(comic=book).first()` per book — 2-6 queries for 1-3 books. New `_batch_settings_and_bookmarks` helper fires two `filter(comic_id__in=window_pks)` calls partitioned by comic_id in Python. Always 2 queries regardless of book count. Headline numbers ================ reader_open_large_arc cold: 28 → 25 queries / 89 → 52 ms (-42%) reader_open cold: 26 → 25 queries / 49 → 51 ms (noise) (other 5 flows stable) The 855-issue series exercises the worst case for the prior iteration. The wall-time win scales with arc size; smaller arcs show ±0 noise. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * use bunx for ttab * caps for close book button again * Reader views perf: Stage 2 — route caching + settings name-lookup batching (#602) Closes Phase D from `tasks/reader-views-perf/99-summary.md`. Tier 1 #3 — `cache_page(READER_TIMEOUT) + vary_on_cookie` on c/<pk> ================================================================== New `READER_TIMEOUT = 60` constant in `codex/urls/const.py` (mirrors `OPDS_TIMEOUT` shape and TTL trade-off). `vary_on_cookie` scopes the cache key per session so per-user responses (bookmark, settings, ACL-filtered arcs) don't leak across users. reader_open warm: 24 → 0 queries / 36 → 1.9 ms (-94%) reader_open_large_arc warm: 24 → 0 queries / 35 → 1.9 ms (-95%) Cold pass unchanged — view runs the full pipeline on cache miss. Real-world reader traffic is dominated by tab refreshes / mobile-app re-foreground patterns where the same `c/<pk>` URL gets hit in quick succession; those collapse to the warm path now. Cross-user isolation verified manually:…

* remove keys from unhead meta headers * fix unhead description for admin tabs" * fix overzealous lazy importer * fix lazyImportEnabled variable in metadata-activator * fix metadata activator from bad cherry pick * fix errant quote * fix typechecking * update devenv & deps * bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the parent queryset's annotations, select_related, prefetch_related, and only(*fields) so the 1-3 returned rows have everything the serializer needs. `arc_count = len(pks)` replaces the prior separate count query. Tier 2 #4 — per-book settings/bookmark batching ================================================ `_append_with_settings` fired `SettingsReader.objects.filter(comic=book).first()` and `Bookmark.objects.filter(comic=book).first()` per book — 2-6 queries for 1-3 books. New `_batch_settings_and_bookmarks` helper fires two `filter(comic_id__in=window_pks)` calls partitioned by comic_id in Python. Always 2 queries regardless of book count. Headline numbers ================ reader_open_large_arc cold: 28 → 25 queries / 89 → 52 ms (-42%) reader_open cold: 26 → 25 queries / 49 → 51 ms (noise) (other 5 flows stable) The 855-issue series exercises the worst case for the prior iteration. The wall-time win scales with arc size; smaller arcs show ±0 noise. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * use bunx for ttab * caps for close book button again * Reader views perf: Stage 2 — route caching + settings name-lookup batching (#602) Closes Phase D from `tasks/reader-views-perf/99-summary.md`. Tier 1 #3 — `cache_page(READER_TIMEOUT) + vary_on_cookie` on c/<pk> ================================================================== New `READER_TIMEOUT = 60` constant in `codex/urls/const.py` (mirrors `OPDS_TIMEOUT` shape and TTL trade-off). `vary_on_cookie` scopes the cache key per session so per-user responses (bookmark, settings, ACL-filtered arcs) don't leak across users. reader_open warm: 24 → 0 queries / 36 → 1.9 ms (-94%) reader_open_large_arc warm: 24 → 0 queries / 35 → 1.9 ms (-95%) Cold pass unchanged — view runs the full pipeline on cache miss. Real-world reader traffic is dominated by tab refreshes / mobile-app re-foreground patterns where the same `c/<pk>` URL gets hit in quick succession; those collapse to the warm path now. Cross-user isolation verified manually: User A and User B see different payloads on the same URL (762 B vs 739 B…

* bump news * fix import bug linking folders * fix possible batching crashes. adjust import variables for throughput * batch comic updates * move INTERNAL_IPS setting to general django area * fix typechecking issue * update deps * bump version to v1.10.12 * fix redirect on OPDS alternate view with metadata * minor change to browser empty page for better first time experience * allow browsing to comics with any top group in opds * bring project up to speed with bun and no package-lock.json * fix browser paginator * bump news for browser paginaor fix * fix search combobox clearing * uppercase book close button * version v1.10.13. update deps * fix pdfs not displaying * fix csp for pdfs * fix OPDS FK constraint failure when session row is missing (#607) iOS Panels (and other Basic-Auth OPDS clients) intermittently hit sqlite3.IntegrityError: FOREIGN KEY constraint failed when settings or bookmarks were saved. Two interacting bugs caused it: - Janitor cleanup_sessions used `if not session.get_decoded():` to detect "corrupt" sessions. get_decoded() returns {} for both real decode failures and legitimate anonymous sessions with no stored data — exactly what Basic-Auth OPDS clients produce. The nightly task was wiping valid session rows. Replaced with a direct signing.loads() call so only genuine signature/decode failures are flagged. - _ensure_session_key returned the cookie's session_key without verifying the row still exists. With cached_db the session loads from cache without rechecking, so a stale cookie key would slip through and cause an FK violation when used as SettingsBrowser / SettingsReader.session_id. Now we verify existence and flush+save to cycle the key when the row is gone. Either fix alone closes the user-visible error; both together also stop the underlying churn that created the bad state. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * extend session-key validation to bookmark + reader-settings paths (#608) The same stale-session_key FK-violation pattern existed in two more places that also write rows whose session FK can be stale: - BookmarkAuthMixin.get_bookmark_auth_filter — feeds session_id into Bookmark.objects.bulk_create / bulk_update. - ReaderSettingsBaseView._get_bookmark_auth_filter — feeds session_id into SettingsReader.objects.create. Both used the old `if not session.session_key: save()` pattern that trusts the cookie. Hoist the validated _ensure_session_key helper from SettingsBaseView up to AuthMixin so every auth-aware view shares one implementation, and switch both call sites to it. BookmarkAuthMixin now extends AuthMixin to inherit the helper. BookmarkFilterMixin is unchanged — it's read-only (filter Q only) and a missing session correctly returns no rows. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * format * bump news * Stats: fix user_registered_count / auth_group_count always-zero bug (#611) The /admin/stats endpoint's user_registered_count and auth_group_count fields have been silently returning 0 since at least Sep 2024. The Stats tab in the admin UI shows 0 registered users even on installs with multiple accounts. Root cause: _add_config tried to rename the per-model count keys produced by _get_model_counts: config["user_registered_count"] = config.pop("users_count", 0) config["auth_group_count"] = config.pop("groups_count", 0) But _get_model_counts builds keys via ``snakecase(model.__name__) + "_count"``. For Django's ``django.contrib.auth.models.User`` / ``Group`` that produces ``user_count`` and ``group_count`` (singular). The pop()s with the plural names never matched, so the default ``0`` won every time — and the actual ``user_count`` / ``group_count`` keys were left orphaned in the dict, then dropped by the StatsConfigSerializer which only declares ``user_registered_count`` / ``auth_group_count``. Fix: pop the right source keys. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * defaults for dockerfile ARGs * format' * Frontend correctness: 10 bug fixes from sub-plan 01 (#647) * Reader page: fix load-progress spinner that never appeared Two stacked bugs in the same setTimeout: 1. Non-arrow callback lost ``this``. The function ran with the timer's context, not the component's, so ``this.loaded`` and the write below were both no-ops. 2. The write targeted ``this.loading``, which has never been a data field on this component. The template binds the spinner to ``showProgress`` (line 15: ``v-if="showProgress && !loaded"``). So even if the arrow had been there from the start, the spinner still wouldn't have rendered — both bugs had to land at once. Net: ``LoadingPage`` has been dead code for slow image loads. Switch to an arrow function and write ``showProgress`` instead of ``loading``. Stash the timer ID so ``beforeUnmount`` can clear it; a fast page swap mid-delay would otherwise fire the write on a torn-down component. Implements B1 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: fix arc-mtime fallback that itself 500'd ``loadMtimes`` builds an arcs list of ``{ group, pks }`` from ``this.arcs``; if the dict is empty the function previously fell back to ``arcs.push({ r: "0" })``. The comment noted that "No arcs is a 500 from the mtime api" — the fallback was added to dodge that 500 — but the wrong-shape fallback also produced a 500 because the API expects ``group``/``pks`` keys, not ``r``. Use the canonical shape so the fallback actually works. Implements B2 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * iOS PWA download: pass the object URL to revokeObjectURL, not the Blob ``URL.createObjectURL(blob)`` returns a ``blob:...`` URL string; ``URL.revokeObjectURL`` must receive that same string to free the mapping. The previous code passed ``response.data`` (the Blob itself), which silently no-op'd and leaked one object URL per download. On iOS PWAs this matters more than elsewhere because the leak accumulates across the user's session and can't be reclaimed short of reloading the app. Implements B6 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser API: stop mutating caller's settings in getGroupDownloadURL ``getGroupDownloadURL`` did ``delete settings.show`` on the caller's object before building the URL. Side-effect: any caller that re-used the settings dict after the download-URL build saw its ``show`` key silently vanish. This was probably fine when the function was first written but it's a footgun now that settings flow through a Pinia store. Destructure-and-spread to drop ``show`` without touching the input. Implements B8 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Auth store: make logout awaitable + clear state unconditionally Two changes to the logout action: 1. ``async`` so callers can ``await``. The current call site (``auth-menu.vue``) fires and forgets, but a future UX pass that wants to disable the button while logout is in flight needs the promise. 2. Clear ``this.user`` in ``finally`` rather than only on success. The user clicked "log out" — UI should reflect the logged-out state immediately, regardless of whether the server-side logout endpoint succeeded. Server-side cookies that survive the network failure will get cleaned up by the next 401. Implements B7 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser filter menu: stop double-rendering each filter row ``<v-list>`` was passed both ``:items="vuetifyItems"`` AND a default-slot ``v-for`` over the same list. Vuetify renders the items prop into ``v-list-item`` children directly, so every row was being built twice — once by the prop, once by the manual ``v-for``. Visible to users on filter menus with large choice lists (genres, characters, etc.); each row appeared duplicated and the DOM cost doubled. Drop the prop. Keep the ``v-for`` because it carries the custom ``#append`` slot for ``metronName`` rendering. ``:model-value`` / ``@update:selected`` still drive selection state via each list- item's ``:value`` prop. Implements B10 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Admin job-tab: remove API-fetching click handler from expanded panel The expanded status panel had ``@click="loadAllStatuses"`` on its container div. Any click inside the panel — including clicks on child elements that bubbled — refetched the entire status map. Probably copy-pasted as a "refresh on click" gesture, but it fired far too often: a user inspecting a long status list would trigger N API calls just from glancing around. The status data is pushed through the websocket already (socket.js fans librarian notifications into the admin store), so the panel is up-to-date without a manual refresh. Implements B11 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader store: handle bookmark-write errors instead of silently rejecting ``setRoutesAndBookmarkPage`` awaited ``_setBookmarkPage`` but didn't catch its errors. On a network blip the promise rejected, the bookmark didn't persist, and the failure became an unhandled rejection in the browser console — not visible to the user, not retried, just lost. Wrap in try/catch. The local page state stays where it is (the user is reading forward; the bookmark catches up on the next write), but the failure is logged so debugging surfaces. A proper user-visible toast + retry path is broader UX work tracked in the plan. Implements B3 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: clear progress timer on unmount ``updateProgress`` chains itself via ``setTimeout`` until the metadata loads or progress reaches 100. The timer ID was never stashed, so closing the dialog mid-animation left the chain running — each tick fired on a torn-down component, writing ``this.progress`` and re-scheduling against now-null refs. Stash the timer ID and clear it in ``beforeUnmount`` so the chain stops cleanly when the dialog goes away. Implements B12 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Reader pager: key dynamic component on identity to force remount ``<component :is="...">`` without ``:key`` lets Vue reuse the existing instance across an ``is`` change when the components share enough surface (props, name). For the reader's vertical/horizontal pager swap that's wrong: scroll listeners attached by the previous mode persist, the new mode's ``mounted`` runs against stale internal state, and any abort/teardown logic in ``beforeUnmount`` never fires. Add ``:key="component.name"`` so the swap is a true unmount + remount — old listeners go away, new mode starts clean. Implements B13 of tasks/frontend-perf/01-correctness-bugs.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Metadata dialog: lint cleanup for the B12 fixup Vue option-order rule: ``beforeUnmount`` belongs above ``methods``. Block-comment style required for the multi-line explanation in ``updateProgress``. Both surfaced when running eslint on the prior commit; pure cleanup, no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * news for cherry pick * OPDS auth: send WWW-Authenticate + opds-authentication content-type on 401 (#652) Panels and other strict OPDS clients require the WWW-Authenticate header (per RFC 7235) to trigger the auth prompt, and the spec calls for application/opds-authentication+json on the auth document. The exception handler was already converting 403 to 401 with the auth doc body, but was bypassing DRF's natural 401 response and dropping both the header and the proper content type, which Panels read as a forbidden state. Also fix a stray `from re import DEBUG` in the auth view that always forced the absolute-URL path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix typing inheritence with drf perms * bump version to v1.10.14 * update picopt treestamps * fix typing import error * fix vulture ignorelist * readd the mistakenly removed vulture_ignorelist * v1.10.15 fix show order bug. update deps * fix nightly job manual expansion * fix news version number * V1.11 performance (#714) * title for age rating tagged * rename unrestricted to adult for clarity * remove circleci code * update to new devenv scripts in frontend * fix sort-ignores to make deterministic across shells with different locales * batch comic updates * fix possible batching crashes. adjust import variables for throughput * bump news for v1.11.0 * Add features to readme * add saved view feature * Add browser-views perf measurement harness (Stage 0) (#574) Stand up the measurement infrastructure that gates the rest of the browser-views performance work: - Add django-silk 5.5 to the dev group; install + route it to a separate ``silky`` SQLite DB via a new ``SilkRouter`` so profiler traces never touch the live DB. - Wire SilkyMiddleware below ServeStaticMiddleware so it only wraps the API stack, not static-file responses. - Expose /silk/ under the existing DEBUG URL block. - Add ``tests/perf/run_baseline.py``: hits the live dev DB via ``django.test.Client``, runs three cold/warm flow pairs (root browse, filtered search, series metadata), and writes a JSON baseline artifact. Cachalot + page-cache are invalidated before each cold pass so the numbers reflect actual DB work. - Add ``make perf-baseline`` target. - Commit the per-view analysis docs and initial baseline capture under ``tasks/browser-views-perf/`` for reference during the follow-on cleanup stages. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 1 - startup + annotation + filter micros (#575) Bundle of small, surgical wins observed on the slimlib baseline: - Cache `libraries_exist` with a 60s TTL in Django's default cache; invalidate on Library save/delete via lazy-imported signal handlers. Drops a redundant `Library.objects.filter(...).exists()` from every browser response. - Short-circuit the four `_save_browser_*` code paths in `codex/views/settings.py` when nothing actually changed. Avoids gratuitous `.save()` calls that flush cachalot on no-op PATCH writes. - Dedupe the `add_group_by(qs)` call in `codex/views/browser/browser.py`: group once in `_get_common_queryset` (no-op for Comic), drop the second call in `_get_group_queryset`. - Memoize `get_max_bookmark_updated_at_aggregate` per `(model, agg_func, default)` on the view instance — the three callers (group_mtime, order, bookmark) now share one Aggregate. - Move the `bmua_is_max` flag off the per-row `Value` annotation and read it from `self.context["view"]` in the browser serializer. - `@lru_cache(maxsize=256)` on `_preparse_search_query`: extract to a pure module-level helper keyed on `(text, path_allowed)`; returns frozen tokens. - `@lru_cache(maxsize=512)` on `get_field_query`: cache the Lark-parsed Q tree, `copy.deepcopy` on return because `_hoist_filters` mutates `child.negated` downstream. - Stash the `BaseDatabaseOperations(None)` singleton as `_DB_OPS` in `search/field/expression.py`; `prep_for_like_query` doesn't use the connection. - Pre-filter the field loop in `filters/field.py` to keys actually set in the request — saves ~20 no-op calls per browser. - Delete `search/field/optimize.py` (`like_qs_to_regex_q` and friends were already unused — grep confirms only self-references). Slimlib cold baseline (3 flows, stage1.json vs Stage 0 baseline.json): - flow_a_root_browse: 21→18 SQL, 189.6→182.3 ms - flow_b_filtered_search: 21→17 SQL, 187.4→178.0 ms - flow_c_series_metadata: 34→31 SQL, 251.1→226.9 ms Warm paths unchanged (0 SQL, ~2 ms). Full tests + ruff pass. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix type warnings * fix lint error * format * Run silk migrations on silky DB at startup (#576) Stage 0 wired a second SQLite DB (silky) for django-silk profiling traces and a DATABASE_ROUTERS entry that routes silk app models there. But ensure_db_schema only invoked `call_command("migrate")` without a database arg, which only migrates the default DB. The router then blocks silk migrations from running on default — so the silky DB was never populated, and the first request through SilkyMiddleware failed with "no such table: silk_request". Mirror the pattern from tests/perf/run_baseline.py: after the default migrate, also run `migrate silk --database=silky` when the silky DB is configured. Guarded on `"silky" in connections.databases` so production (DEBUG=False, no silky DB) is a no-op. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 2 — triple COUNT + page mtime cache (#577) ## PR 2a — Eliminate triple COUNT on the paginate path Each browse request ran three COUNT queries per section (groups & books): 1. The outer grouped COUNT in `_get_common_queryset`. 2. Paginator's internal COUNT triggered by `paginator.page()`. 3. An explicit `.count()` on the paginated slice. The outer COUNT is needed (sizing the paginator). The other two are redundant — the page row count is bounded by `per_page` and derivable from `end_index - start_index + 1`. - Shadow `Paginator.count` (a `@cached_property`) with the pre-computed total to skip Paginator's internal COUNT. - Derive page row count arithmetically from `Page.start_index()` / `end_index()`. - Pass `book_count` through `paginate()` alongside `group_count`; drop `book_qs.count()` on the opds2 path. - `_paginate_section` returns `(qs, count)` directly. Short-circuits on `total_count == 0` (avoids Paginator instantiation on empty sections) and preserves the EmptyPage warning branch. ## PR 2b — Short-TTL page mtime cache `BrowserView._get_page_mtime()` calls `get_group_mtime(page_mtime=True)` on every browse request. The query is a filtered Max aggregate that cachalot caches — but any write to Comic / Bookmark invalidates it, so bookmark-heavy usage forces recomputation. Cold-path silk traces show this aggregate at ~26ms on flow_a — the second-slowest query in the request. Add a 5s TTL cache layer gated on page_mtime=True. Key includes user, model, group, pks, page, and a hash of filter-affecting params (filters, search, q, order_by, order_reverse). The polling MtimeView path (no page_mtime) is unaffected, so frontend change-detection stays live. ## Measurements tests/perf/run_baseline.py on the slimlib DB. Cold = full cache invalidation; warm = cachalot populated. | flow | stage1 cold | stage2 cold | |-------------------------|------------------------|------------------------| | flow_a root browse | 18 queries / 182.3 ms | 16 queries / 135.6 ms | | flow_b filtered search | 17 queries / 178.0 ms | 15 queries / 130.3 ms | | flow_c series metadata | 31 queries / 226.9 ms | 31 queries / 229.9 ms | flow_a / flow_b: -2 queries, ~26% cold wall-time reduction. flow_c unaffected (metadata doesn't traverse paginate). PR 2b's benefit is a dogpile guard after cachalot invalidation — doesn't show in the harness (cold = both caches empty; warm = cachalot wins first). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix browser paginator * typecheck browser paginate * Browser views perf: Stage 3 — cover fan-out collapse (#578) * Stage 3 — cover fan-out: per-pk endpoint + pre-resolved cover_pk Before stage 3, every cover card triggered a full BrowserAnnotateOrderView pipeline to pick the representative comic pk. A default page does ~100 of these in parallel. This PR collapses that into one correlated Subquery on the browse response plus a thin per-pk cover endpoint. Components - Option A: annotate_order_aggregates(for_cover=True) drops the JsonGroupArray + page_count aggregates from the cover path — unused for picking a pk. - Option B.1: BrowserView group cards get cover_pk / cover_custom_pk via correlated Subquery that replicates CoverView.get_group_filter exactly (direct fk match when dynamic_covers/Volume/Folder; sort_name fuzzy match otherwise, correlated on _GROUP_BY columns so the same comic set as `ids` is picked — no JSON parsing, no peer aggregate). - New endpoints /api/v3/c/<pk>/cover.webp and /api/v3/cc/<pk>/cover.webp serve already-resolved pks with a cheap single-row ACL probe and the existing CoverPathMixin / CoverCreateThread pipeline. - Frontend getCoverSrc prefers the new per-pk URL when cover_pk / cover_custom_pk is on the card; falls back to the old group+pks URL otherwise, so OPDS and search-active browses keep working. - FTS skip: cover_pk annotation is skipped when params["search"] is set. MATCH inside a correlated subquery re-scans the FTS5 index per outer row (~900ms on a 100-group page). The old URL path still applies the search filter per cover — same behavior, parallelized over HTTP. Perf (slimlib dev DB, Flow D = browse + every card's cover): Flow Before cold After cold Delta A root browse 156.9ms / 15 q 180.5ms / 15 q +24ms / +0 q B search 148.3ms / 16 q 132.4ms / 16 q -16ms / +0 q C metadata 161.9ms / 31 q 165.0ms / 31 q +3ms / +0 q D browse+covers 2311ms / 1216q 1161ms / 815 q -50% / -33% Flow A takes a mild regression for the correlated cover subquery, but Flow D — the realistic user wall-clock — drops by half and 400 SQL queries. Subsequent cover fetches drop from ~90ms each (full pipeline) to ~5ms (disk read + 1 ACL probe). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Stage 3 follow-ups: custom_cover URL, FTS pre-materialization, OPDS thin covers Address review feedback on the cover fan-out collapse: - Rename /api/v3/cc/<pk>/cover.webp → /api/v3/custom_cover/<pk>/cover.webp. Descriptive over terse; matches the view name. - FTS pre-materialization replaces the FTS-skip fallback. A correlated MATCH re-scans FTS5 per outer row (~900ms on a 100-group page); the old path worked around this by skipping cover_pk annotation on search and sending every cover through the legacy pipeline once. We now pre-select the FTS match set as a non-correlated sub-SELECT in the outer cover subquery — SQLite materializes it once and each correlated cover row lookup becomes an indexed pk filter. Cover_pk is annotated on search responses, the thin endpoint handles each cover. - OPDS now emits thin-endpoint cover URLs (v1 and v2): - New OPDSComicCoverByPkView / OPDSCustomCoverByPkView wrap the browser thin views with OPDSAuthMixin's Basic Auth. - New opds:bin:cover_by_pk and opds:bin:custom_cover_by_pk URL names. - v1 _cover_link picks: cover_custom_pk → custom thin; group=='c' → own pk thin; cover_pk → thin; else legacy group+pks fallback. - v2 _thumb always uses the thin endpoint (publications are Comic rows so obj.pk IS the cover pk). - OPDS inherits annotate_cover() via BrowserView._get_group_and_books, so group rows already carry cover_pk / cover_custom_pk. - tests/perf/run_baseline.py gains a flow_e (search + covers) to measure the search path end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix search combobox clearing * format * Stage 3 follow-ups: folder covers, endpoint rename, offline cover pipeline (#579) - Fix folder cover_pk picking comics not actually in a folder's subtree by using the recursive ``folders`` M2M (same relation the browse filter uses) for the per-card cover subquery instead of the direct ``parent_folder`` FK. - Delete the legacy thick cover endpoint. Rename cover_by_pk.py to cover.py and drop the "_by_pk" suffix from view class names (CoverView, CustomCoverView, OPDSCoverView, OPDSCustomCoverView). Update URL configs, OPDS wrappers, and the frontend client. - Move all cover generation off the HTTP path. Add CoverCreateTask and enqueue it from the importer after bulk_create so new comics get thumbnails pre-warmed offline. When a cached thumb is missing the per-pk endpoint enqueues the task and responds 202 Accepted with Retry-After plus the missing-cover placeholder instead of synthesizing the WebP inline under a worker thread. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update devenv sort ignores * Force cover refresh after 202 so the placeholder doesn't stick (#580) Browsers don't honor Retry-After on <img> elements and happily cache the placeholder served with the 202 response, so even after the cover thread finishes writing the real thumb the img src keeps rendering the stale placeholder bytes. - Backend sends Cache-Control: no-store on the 202 placeholder so the response isn't cached at that URL. - BookCover now probes the cover URL with fetch() on mount and, if it sees 202, waits Retry-After and bumps a reactive `retry` counter that becomes a cache-busting query param on coverSrc. v-img re-fetches with the new URL and gets the real cover once the cover thread is done. Retries are capped at 5 and aborted on unmount via AbortController. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Harden cover endpoint against concurrent read/write races (#581) Under a full cold-cache page load, 32 HTTP workers hitting the cover endpoint at the same time as the cover thread was writing thumbs returned 500s for a handful of covers. The endpoint used a three-step exists()/stat()/read_bytes() sequence that could race against an in-flight write, and save_cover_to_cache wrote directly to the target path so readers could observe a truncated file mid-write. - save_cover_to_cache now stages to a ``{name}.{pid}.tmp`` sibling and renames with Path.replace, so a read either sees the pre-existing state or the fully written file — never a partial one. Stale temps are unlinked on failure. - _get_cover_response collapses to a single read_bytes() that catches FileNotFoundError and logs OSError, then falls through to the 202 path. No race window between stat and read. - LIBRARIAN_QUEUE.put and the outer get() methods are wrapped in try/except that log and return the placeholder, so a surprise exception is never user-visible as a 500 and always leaves a log line. - cleanup_orphan_covers runs a dedicated ``*.tmp`` sweep over both cover roots before the orphan scan, so stale temps from crashed writers don't accumulate. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix custom cover generation * relegate watch for changes restart to a en env variable settings * await and finish watch task * update devenv * Fix metadata cover fetching pk=0 after Stage 3 cover fan-out (#583) Stage 3's follow-up commit (2ff08524) dropped the legacy `group+pks` fallback from the frontend `getCoverSrc()` helper, leaving it to only build per-pk URLs from `coverPk` / `coverCustomPk`. The browser card pipeline annotates those fields via `BrowserAnnotateCoverView`, but the metadata endpoint was never wired into that annotation, and `metadata-cover.vue` never passed the cover pks through to `BookCover`. As a result every metadata dialog tried to load `/api/v3/c/0/cover.webp` — pk 0 — and fell back to the missing-cover placeholder. Wire the annotation + serializer + component end-to-end: - `MetadataAnnotateView` inherits from `BrowserAnnotateCoverView` so `annotate_cover()` is available on the metadata queryset. - `MetadataView.get_object()` calls `annotate_cover(qs)` after `annotate_card_aggregates(qs)`, mirroring `browser.py`'s ordering. - `MetadataSerializer` exposes `cover_pk` and `cover_custom_pk` as `SerializerMethodField`s with the same fallback-to-`obj.pk` semantics that `BrowserCardSerializer` uses — so Comic metadata (`group=c`) works without annotation, falling back to its own pk. - `metadata-cover.vue` forwards `md.coverPk` / `md.coverCustomPk` to `BookCover`. Verified on /api/v3/{p,i,s,v,c,f}/*/metadata: `coverPk` now resolves to a real comic pk on every group, and the comic path falls back to its own pk. No SQL query delta — `annotate_cover` is a correlated Subquery that inlines into the outer SELECT. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hints (#582) * Browser views perf: Stage 4 — metadata annotation batch + FK/M2M hydration hints Fold the three `_intersection_annotate()` passes (value fields, conflict shadow fields, related-count fields) into a single batched call that collapses N distinct-count probes into one `aggregate()` and one `values_list()`, with unique synthetic keys so annotation groups don't collide. Extend `FK_QUERY_OPTIMIZERS` so intersecting FK hydration (`AgeRating`, `Character`, `Team`) carries `select_related` + `.only()` hints for the nested fields the metadata serializer actually reads (`AgeRating.metron`, `Character.identifier.url`). Without this, each nested access fired a follow-up query per instance. For the M2M intersection path, short-circuit empty-intersection fields with `Model.objects.none()` so we skip the optimizer setup (and, for the Comic self-reference path, pointless prefetch dispatches on already empty results). Add `flow_c2_comic_metadata` to the perf baseline harness so the comic-detail metadata path (the FK + M2M heavy flow) is tracked alongside the series flow. Measured: flow_c_series_metadata cold SQL drops from 31 → 28 (-3). flow_c2_comic_metadata baseline captured at 47 queries for future stages. Other flows unchanged within noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * WATCH for changes variable depends on debug --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Fix folder + FTS crash and preserve rank-ordered cover selection (#584) * Fix folder browse + FTS crash on search_score ordering Browsing folders (or any group) with an FTS search and ``orderBy=search_score`` blew up with:: sqlite3.OperationalError: no such column: codex_comicfts.rank Root cause: Stage 3's cover fan-out collapse builds a correlated Comic subquery per group card to pick ``cover_pk``. The follow-up (#579) replaced the correlated ``comicfts__match`` filter with a non-correlated ``pk__in fts_sq`` pre-materialization — so the Comic subquery no longer joins ``codex_comicfts``. But ``annotate_order_aggregates`` still annotated ``search_score=ComicFTSRank()`` inside the subquery, and ``add_order_by`` issued ``ORDER BY "codex_comicfts"."rank" * -1``, which SQLite can't resolve from the subquery's FROM list. - ``_annotate_search_scores`` now takes ``for_cover`` and skips the annotation when set. ``annotate_order_aggregates`` threads the flag through (matching the existing ``for_cover`` pipeline-trims). - ``_cover_comic_subquery`` passes ``order_key="sort_name"`` to ``add_order_by`` whenever the user's order is ``search_score`` — the cover's tie-break isn't user-visible, and ``sort_name`` is a real indexed column on Comic. Verified: /api/v3/{r,f,a,p}/0/1?q=iron+man&orderBy=search_score all return 200 with a real ``coverPk`` (not 0). Lint + 20-test pytest suite pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Preserve FTS rank ordering in cover subquery The previous fix short-circuited to ``sort_name`` when ``order_by`` was ``search_score`` so a group card's cover could be picked without referencing ``codex_comicfts.rank``. That fixed the crash but meant the cover chosen for each card was the alphabetically-first matching comic, not the top-ranked FTS match — a UX regression on searched pages. Restore rank-ordered cover selection by: * Applying ``fts_q`` (``comicfts__match=...``) directly in the cover subquery so ``codex_comicfts`` is joined and ``rank`` is populated, while keeping the ``pk__in`` pre-materialization for cheap filtering. * Teaching ``ComicFTSRank`` to resolve the query-local alias for ``codex_comicfts`` at compile time. Its literal template worked for the top-level browse query but emitted an unresolvable column ref in nested subqueries where Django aliases the join as ``V4``/``U1``. * Skipping ``.group_by("id")`` in the cover-subquery search_score annotation. The custom force-group-by compiler emits a literal ``"codex_comic"."id"`` that breaks under nested aliasing, and the cover subquery is already ``.distinct() … LIMIT 1`` so dedup is redundant there. Verified against the dev DB: for each publisher card on a ``batman`` search, the annotated ``coverPk`` now matches the top-ranked comic returned by a direct ``ComicFTSRank`` ordering within that publisher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update for claude rules * Browser views perf: Stage 5a — server-side cache_page on cover endpoints (#585) Wrap /api/v3/c/<pk>/cover.webp, /api/v3/custom_cover/<pk>/cover.webp, and their OPDS counterparts in cache_page(COVER_MAX_AGE). Compose with cache_control + vary_on_cookie (API) or vary_on_headers("Cookie", "Authorization") (OPDS, which accepts Basic + Bearer + Session auth) so the Vary header is set before cache_page stores the response — the cache key is keyed per auth identity, no cross-user leakage. Also raise Django FileBasedCache MAX_ENTRIES from the default 300 to 10000. Cachalot query results + cache_page entries (browser + cover) exceed 300 during a single browse-with-covers pageload, triggering the 2/3 random cull that silently evicts just-populated cover entries before the next request can read them. Without this, Flow D warm only dropped to 743 (~50% cover-cache hit rate); with it, Flow D warm drops to 0. Perf impact (stage5a-after.json vs. stage4-after.json): Flow D — browse + 100 covers warm 802 → 0 queries Flow E — search + 46 covers warm 368 → 232 queries Flow E's residual 232 queries are 29 covers that return 202 Accepted (cover not yet generated; response has Cache-Control: no-store, which cache_page correctly skips). The 17 covers that returned 200 were 0 queries each. In production, 202s resolve within seconds as the cover thread catches up, so steady-state warm on Flow E is also 0. Cold query counts and Flow A/B/C are unchanged. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Raise RLIMIT_NOFILE on startup to avoid macOS 256 FD cap (#586) Cold browser sessions loading a full 100-card grid intermittently crash the dev server with `OSError: [Errno 24] Too many open files`. Diagnosis: - macOS ships a 256 soft cap for RLIMIT_NOFILE (`ulimit -Sn 256`). - Each Django request thread keeps a sticky SQLite connection (`CONN_MAX_AGE=600`). - SQLite WAL mode opens 3 FDs per connection (main + `-wal` + `-shm`). - The thin cover endpoint dispatches to a `sync_to_async` threadpool, so a burst of 100 cold cover requests easily spawns ~100 threads → ~300 SQLite FDs, blowing past the 256 cap before page reads even open. Bumping the soft limit toward the hard cap (or 8192, whichever is lower) at process start is non-invasive and matches what production deployments typically achieve via shell `ulimit`. No-op on Linux (already high) and on platforms without the `resource` module. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps, format json * Browser views perf: Stage 5b — annotation gating + m2m-aware distinct (#587) Three surgical correctness/perf changes that surface the right SQL for the right target. The Stage 5a work (cover endpoint cache_page) collapsed Flow D warm to 0; 5b cleans up cold-path waste the harness flows don't directly exercise. 5.2 — Skip ``updated_ats`` JsonGroupArray outside browser/metadata ``obj.updated_ats`` is consumed only in ``BrowserAggregateSerializerMixin.get_mtime`` (browser + metadata serializers). OPDS computes its own mtime from the bookmark aggregate; cover/download paths never read it. Skip the DISTINCT scalar aggregate for those targets. 5.3 — m2m-aware .distinct() in ``BrowserFilterView.get_filtered_queryset`` New ``comic_filter_uses_m2m`` cached property. Comic queries skip ``.distinct()`` unless a real m2m or m2m-through join is present (story_arc browse, folder browse on cover/choices/bookmark/download TARGETs, or any m2m field filter). Non-Comic queries still always ``.distinct()`` because the ACL alone traverses ``comic__`` (one-to-many). 5.5 — Tie ``search_score`` ``group_by("id")`` to the same flag ``annotate/order.py:_annotate_search_scores`` only emits the GROUP BY when fan-out actually exists. Cover path stays gated by ``for_cover``. 5.4 — Skipped intentionally ``codex/views/auth.py`` already caches the three scalar inputs to ``get_acl_filter`` per request. The remaining cost is composing two Q objects from those scalars — microseconds, not queries. Wrapping a cached_property keyed on ``(model, user_id)`` would be cosmetic. 20 standalone cases of ``comic_filter_uses_m2m`` pass (every default-target group, every m2m-folder TARGET, every BROWSER_FILTER_KEY by category, mixed filters, ``pks=(0,)`` early-out). stage5b-after.json: existing perf flows preserved (query counts identical; wall times within run-to-run noise). The wins are on cold paths the harness doesn't exercise — browsing a Series's comics, default-TARGET folder browse, OPDS feeds — which a follow-up should add. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5c — batched choices_available (#588) Replaces the per-field probe loop in BrowserChoicesAvailableView with a single batched EXISTS annotate. FK fields keep "any non-null exists" semantics; m2m fields decompose into (has_rel, has_null) booleans plus a lazy distinct-count probe for the rare has_rel ∧ ¬has_null corner. The natural EXISTS(SELECT DISTINCT rel ... LIMIT 1 OFFSET 1) form is broken on SQLite — EXISTS short-circuits on the first row from the underlying join, before DISTINCT collapses or OFFSET skips — so the m2m path uses two cheap booleans + a Python-side cap-at-2 distinct probe. Perf: flow_f_choices_available cold drops 34 → 11 queries (−68%) and 121 → 53 ms (−56%) on the dev DB. Other flows are noise-level unchanged. tests/perf/run_baseline.py picks up three new flows (choices_available, m2m field, FK field) so the changed code path is visible in the artifact. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * run frontend with bun * fix syntax error in filter sub menu * Browser views perf: Stage 5d — has_metadata boolean cast + series-browse perf flow (#589) Cleanup bundle (5.7) shrinks against post-5c code: - #26: switch ``has_metadata`` annotation from ``F("metadata_mtime")`` to ``ExpressionWrapper(Q(metadata_mtime__isnull=False), BooleanField())`` in both ``codex/views/browser/annotate/card.py`` and ``codex/views/reader/books.py``. Matches the consumer serializer's ``BooleanField`` and trims the SELECT projection to one byte per row. - Harness: add ``flow_a2_series_browse`` so the harness covers the Comic-queryset / no-m2m-filter path that Stage 5b's distinct + group_by skip wins on. Headline measurement: 6 cold / 4 warm queries, ~13 ms cold. Items #18 (already absorbed in 5c), #25 (parent-aware reduction already in place; trimming further would change ORDER BY semantics), #30 (already coalesced inside ``BookmarkUpdateMixin.update_bookmarks``), and #31 (``zipstream-ng`` already streams via ``FileResponse``) were re-investigated and skipped with reasons documented in §10 of ``05-replan.md``. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff (#590) Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: 99-summary status column + OPDS perf plan (#591) * Browser views perf: Stage 5.9 — FTS demote-joins regression guard + 5e handoff Locks in the conditional ``codex_comicfts`` demote inside ``BrowserFilterView.force_inner_joins`` so future perf work that strips the FTS table from the demote set fails loudly instead of re-introducing ``OperationalError: unable to use function MATCH in the requested context`` on every FTS-enabled browse. Tests in ``tests/test_search_fts.py``: * ``test_left_joined_fts_match_raises_operational_error`` — canonical failure mode. Uses ``Query.promote_joins`` (the documented inverse of ``demote_joins``) to flip the auto-promoted FTS join back to LEFT OUTER, then proves SQLite refuses the MATCH. * ``test_force_inner_joins_demotes_comicfts_when_fts_mode_true`` — positive contract. ``Comic.objects.values("comicfts__pk")`` joins the FTS table LEFT OUTER without a non-null filter (so Django's optimizer leaves it alone), and ``force_inner_joins(fts_mode=True)`` flips it to INNER. * ``test_force_inner_joins_skips_comicfts_when_fts_mode_false`` — negative contract. Same carrier, ``fts_mode=False`` leaves the join LEFT OUTER. * ``test_force_inner_joins_unblocks_match_on_left_joined_query`` — end-to-end repair. The promoted-LEFT-OUTER carrier funneled through ``force_inner_joins(fts_mode=True)`` returns the matching comic. Also lands the Stage 5e handoff doc that scopes the deferred R3 serializer audit (``stage5e-handoff-serializer-audit.md``) and updates ``05-replan.md`` §5 / §10 to reflect Stage 5.9 landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Browser views perf: backlog status column on 99-summary Stage 5 final exit criterion. Adds a Status column to every tier table in tasks/browser-views-perf/99-summary.md so the backlog reads as a closed ledger rather than an open plan. Markers: - Tier 1 (4): all four landed (Stage 2, 3, 4, 5a) - Tier 2 (6): all six landed (Stage 1 / 5b) - Tier 3 (7): four landed (Stage 2, 5b, 5c), one skipped (#16 subsumed by GroupACLMixin per-request scalar caches), three open (#11, #12 absorbed by Stage 4 hints; #15 not on a hot Flow A-H path) - Tier 4 (14): seven landed (Stage 1 / 5c / 5d), four re-investigated in Stage 5d and confirmed already done (#25, #27, #30, #31), three open (#22, #24, #29) - Tier 5 (3): R1 ✅ Stage 5.9 (FTS demote regression test), R2 ❌ skipped (current code is correct), R3 ⏭️ deferred via the stage5e-handoff-serializer-audit.md brief Pure documentation update — no code or test changes. Closes the last unchecked exit criterion on the Stage 5 plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: initial planning pass Mirrors the structure of tasks/browser-views-perf/ for the OPDS surface. No code changes — produces a ranked backlog so the same per-stage rigor can be applied to OPDS once browser-views perf wraps. Files: - 00-meta-plan.md methodology, scope, sub-plan layout, exit criteria - 01-routes-and-cache.md OPDS_TIMEOUT=0 disables cache_page on every feed - 02-feed-pipeline.md v1 + v2 main feeds, BrowserView reuse, preview reruns - 03-entry-serialization-v1.md v1 entries, lazy_metadata Comicbox open, M2M fan-out - 04-publications-v2.md is_allowed static-method bypass of admin_flags cache - 05-manifest.md credit fan-out 11→1, story_arcs N+1, subjects 7→1 - 06-progression-binary-aux.md PUT conflict pre-check + dead expr; binary inheritance - 99-summary.md 5-tier ranked backlog, phasing A–F, cross-cutting guidance Top findings (in landing order): 1. OPDS_TIMEOUT = 0 in codex/urls/const.py — every feed wraps cache_page(OPDS_TIMEOUT) and gets nothing. Single-line config flip is the highest-leverage win and gates Phase A. 2. Manifest credit fan-out (v2/manifest.py:194-199) — 11 separate Credit.objects.filter queries because _MD_CREDIT_MAP is iterated. Collapsible to one query + Python partition. 3. is_allowed static method (v2/feed/publications.py:35-56) bypasses the request-cached admin_flags MappingProxyType; called per link spec on start-page render. 4. Manifest M2M subjects — 7 queries via get_m2m_objects loop. UNION or prefetch collapses. 5. get_publications_preview — full BrowserView pipeline rerun per preview link spec on start page. 6. lazy_metadata() Comicbox open in v1 stream-link path — synchronous file I/O on the request thread for partially-imported books. 7. Progression PUT conflict pre-check — two queries per PUT, foldable into one conditional UPDATE. 8. Story arcs N+1 in manifest — .only("story_arc", "number") defers FK; per-row StoryArc.objects.get. Replace with select_related or .values(). Plan only. No source files touched. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * ignore tasks for prettier * OPDS views perf: Stage 0 — baseline harness + Phase B low-risk wins (#592) Closes Phase A (R1 OPDS_TIMEOUT=0 rationale, R2 perf harness) and four of five Phase B items from `tasks/opds-views-perf/99-summary.md`: - #3 Fix story-arc N+1 in `_publication_belongs_to_story_arcs` — swap `.only("story_arc", "number")` for `.select_related("story_arc").only("number", "story_arc__name")` so `story_arc.name` access doesn't fire one query per row. - #6 Convert `OPDS2PublicationBaseView.is_allowed` from `@staticmethod` to instance method reading `self.admin_flags.get("folder_view")`, and the parallel `OPDS1FacetsView._facet_group` anti-pattern (`AdminFlag.objects.get` inside a per-facet loop) — both now use the request-cached MappingProxyType from `SearchFilterView`. - #13 Extract `_obj_ts(obj)` helper for the `floor(datetime.timestamp(obj.updated_at))` expression repeated at six sites across `v2/feed/publications.py` and `v2/manifest.py`. - #15 Remove dead expression `max(position - 1, 0)` (no assignment) at `v2/progression.py:226`. Phase B #12 (`_update_feed_modified` rescan) deferred — it overlaps with the preview-pipeline pass in Phase D. The harness lives at `tests/perf/run_opds_baseline.py` with eleven flows mirroring `tests/perf/run_baseline.py` shape. Cold + warm captures via django-silk; cold pass invalidates cachalot + django_cache. Captures `baseline.json` (pre-edits) and `stage0-after.json` (post-edits) alongside the harness. See `tasks/opds-views-perf/stage0.md` for the full writeup including the harness reproducibility note. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 1 — manifest credit + subject batching (#593) Closes Phase C from `tasks/opds-views-perf/99-summary.md`. Two manifest items land: - Tier 1 #2 — `_publication_credits` collapses 11 per-role `Credit.objects.filter` calls into a single query with `select_related("person", "role")`. Eliminates the 11-query loop AND the lazy `credit.person` FK fan-out triggered by `_add_tag_link` (7 queries on the dev DB's busiest comic). The role-set partition runs in Python. - Tier 2 #5 — `_publication_subject` collapses 7 per-model M2M queries into a single `UNION ALL` over `(pk, name, _kind)` tuples. Reconstructs `SimpleNamespace` rows so `_add_tag_link` and the downstream `OPDS2SubjectSerializer` continue to work. Drive-by: drop the now-unused `get_credits` helper from `codex/views/opds/metadata.py` (no remaining callers). `v2_manifest`: 47 → 24 cold queries (-23, ~49%), 113 → 87 ms cold. Captured `stage1-before.json` + `stage1-after.json` alongside `stage1.md` for the writeup including the surfaced (but not fixed here) `peniciller` typo in `OPDS2PublicationMetadataSerializer`. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * update deps * speling * OPDS views perf: Stage 2 — re-enable route caching (#595) Closes Tier 1 #1 from `tasks/opds-views-perf/99-summary.md` — the highest-impact item in the entire plan. - `OPDS_TIMEOUT` flips from 0 (cache_page no-op) to 60 s. Long enough to amortize a full feed pipeline run across a tab refresh / reader-app re-fetch, short enough that bookmark-position changes show up before the next poll. The disable rationale is reconstructed in stage0.md § R1. - New `codex/urls/opds/__init__.py:opds_cached` helper composes `cache_page(OPDS_TIMEOUT)` with `vary_on_headers("Cookie", "Authorization")` so the cache key scopes per-user / per-auth-scheme. Mirrors the binary cover-route shape at `codex/urls/opds/binary.py`. Applied uniformly across v1.py, v2.py, and the no-trailing-slash `/opds/v2.0` start in root.py (which previously bypassed `cache_page` entirely). - Progression route (`v2/<group>/<pk>/position`) explicitly NOT wrapped — a PUT mutates the bookmark, and a GET within the cache window would return stale position. Multi-device sync is the worst-case (device A PUTs page 100, device B GETs within 60 s and resumes at the wrong page). The ~9-query / 14 ms cold cost is small compared to the freshness cost. Warm-pass measurements collapse to 0 queries / ~1.5–2.7 ms across every cacheable route (the cache returns the response without entering the view layer): v2_manifest 62 → 2.4 ms warm v2_start 59 → 1.8 ms warm v2_root_browse 28 → 2.4 ms warm v1_root_browse 40 → 1.7 ms warm v1_series_acquisition 24 → 2.7 ms warm Cold-pass numbers unchanged — the view runs the full pipeline on a cache miss. Real-world OPDS traffic is dominated by warm hits. Cross-user isolation verified manually: User A and User B (different ACL) hit the same URL and receive different payloads (16 223 B vs 15 217 B); each user's warm response matches their own cold response; `Vary: Accept, Cookie, Authorization, origin` confirmed on every response. Captured `stage2-before.json` + `stage2-after.json` alongside `stage2.md` for the writeup. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * disable discord notification steps in gha * OPDS views perf: Stage 3 — preview pipeline cache sharing + book queryset joins (#596) Closes Tier 1 #4 (preview-pipeline re-runs) and the related #17 / sub-plan 02 #3 (`select_related` shortfall on the OPDS book queryset). - `OPDS2FeedLinksView.get_book_qs` overrides the parent `BrowserView.get_book_qs` to add `select_related("volume", "language")`. The base method joins `series` only, with an explicit comment that "OPDS doesn't need volume" — but `Comic.get_title(volume=True)` reads `obj.volume.name` / `obj.volume.number_to` per publication, and `_publication_metadata` reads `obj.language.name`. Without these joins, every publication iteration fired one lazy `Volume.objects.get` and one lazy `Language.objects.get`. v1 already does the same join in `v1/facets.py:64`; this brings v2 to parity. - `_get_publications_preview_feed_view` shares `_admin_flags` and `_cached_visible_library_pks` with the parent view. Both are request-scoped (depend on user, not on params/kwargs), so it's safe to skip the per-preview re-fetch. Cuts the visible-library ACL lookup from 1-per-preview to 1-per-request. `v2_start`: 53 → 29 cold queries (-24, ~45% reduction). Pre-fix breakdown: 15 codex_volume (N+1) + 12 codex_library (4 per preview × 3) dominated. Post-fix: 0 codex_volume + 3 codex_library. Out-of-scope hotspots still visible in the trace (documented in stage3.md): per-preview age-rating-with-metadata join (4 queries, needs the bigger UNION-batch rewrite) and the 9 codex_comic filter / annotation queries (preserved as the legitimate per-preview pipeline cost). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * OPDS views perf: Stage 4 — v1 acquisition M2M batching + progression PUT conflict detection (#597) Closes Phase E from `tasks/opds-views-perf/99-summary.md`. #7 — Per-page batching of v1 entry M2M fan-out ============================================== Three new helpers in `codex/views/opds/metadata.py`: - `get_credit_people_by_comic` — one query for all credits across all comics on the page; partitions by comic_id in Python. - `get_m2m_objects_by_comic` — UNION ALL across the 7 `OPDS_M2M_MODELS` tables, partitioned by `(comic_id, kind)`. `OPDS1EntryData` gains `authors_by_pk` / `contributors_by_pk` / `category_groups_by_pk` optional dicts. `OPDS1FeedView._get_entries_section` populates them when `metadata=True` and `key=="books"`. Per-entry properties read from the dicts when present, fall back to the legacy single-comic helpers otherwise (so facet entries / single-comic feeds still work). Result: 9 queries per entry × N entries collapses to 3 queries per page. On the harness's "All Batman" series with `?opdsMetadata=1` (106 comics), `v1_acquisition_with_metadata` drops from 817 cold queries / 1585 ms to **20 cold queries / 154 ms** (~40× / ~10×) — verified in a controlled full-feed-state run. #8 — Progression PUT atomic conditional UPDATE ============================================== Two changes: 1. `OPDS2ProgressionSerializer.modified` flips from `read_only=True` to `required=False`. Previously the field was silently dropped from PUT validated_data, making the conflict pre-check at `view.py:207-217` unreachable (zero progression-related queries fired on PUT today). 2. `OPDS2ProgressionView.put` replaces the dead pre-check (which was `_get_bookmark_query() + qs.first()` — never executed) with a single atomic conditional UPDATE keyed on `updated_at__lte=new_modified`. If the UPDATE matches a row, write succeeds in one query. If 0 rows match AND a bookmark exists, the DB has a fresher row → 409. If 0 rows match AND no bookmark exists, fall through to the existing async `update_bookmark` path (first-time write). Behavior change: clients that previously sent stale `modified` and got silent 200s now correctly receive 409 per the OPDS v2 progression spec. Functional verification: - PUT no `modified` (no bookmark) → 200 (liberal accept, async create) - PUT with stale `modified` → 409 (atomic conflict detection) - PUT with fresh `modified` → 200 (atomic UPDATE) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * bump news for progression bugfix * OPDS views perf: Stage 5 — Tier 3-4 cleanups (#598) Closes Phase F from `tasks/opds-views-perf/99-summary.md`. Three items land; two are intentionally skipped after audit. #16 — `select_related("parent_folder")` on manifest queryset `OPDS2ManifestView.get_object` adds the join; eliminates the lazy `Folder.objects.get` per request when folder_view is on. v2_manifest cold drops 23 → 22 queries. #11 — Memoize filters JSON via `self.params["filters"]` `_subtitle_filters` previously re-parsed `request.GET["filters"]` inline (urllib.parse.unquote + json.loads). The same JSON is already parsed by BrowserSettingsFilterInputSerializer; reading from `self.params` skips the third parse. Sub-ms per request, but on a hot client refresh cumulative. #10 — Resolve `opds:bin:page` URL once for `_publication_reading_order` Replace per-page `self.href()` (which fires `reverse()` each call) with a single sentinel-page resolution + `str.format` substitution. Saves N-1 `reverse()` calls per manifest hit. Invisible on the harness's 1-page comic_pk=10785; visible on high-page-count PDFs. #12 (audited won't-fix): the rescan is functionally necessary — preview-group mtimes aren't covered by `_get_group_and_books`'s mtime; removing the rescan would lose preview mtime tracking. #18 (audited won't-fix): Stage 4 already provides the cleaner SimpleNamespace pattern; legacy `_add_url_to_obj` fires only on cold fallback paths where the cachalot reuse risk doesn't apply (materialized list, not a queryset reused for caching). After Stage 5 the OPDS perf project has reached the point where remaining open items either need production telemetry (R3, #19), medium-risk correctness verification (#9), or are negligible wins versus framework cost (#14). Recommending pause until production data points to a specific path. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf plan: methodology + 3 sub-plans + ranked backlog (#599) Audit of `codex/views/reader/` (~850 LOC across 6 source files) mirroring the OPDS / browser-views perf plan structure. Identifies 15 ranked items + 5 research questions across three view families: - Reader view chain (`c/<pk>` GET) — params/arcs/books/reader.py - Reader settings (`c/settings`, `c/<pk>/settings`) — settings.py - Reader page binary (`c/<pk>/<page>/page.jpg`) — page.py Top three findings: 1. Comicbox archive open on every page request (sub-plan 03 #1) — 200-page comic = 200 archive opens per read-through. The biggest single hotspot; needs an LRU process-cache of open archives. 2. `get_book_collection` materializes the entire arc to find prev/curr/next (sub-plan 01 #1) — 100-issue series = up to 100 rows materialized per reader open. Two targeted LIMIT-1 queries collapse to 3 rows independent of arc size. 3. No server-side caching on the reader / page / settings routes — every request re-runs the full pipeline. Suggested landing order matches the OPDS / browser shape: Phase A measure (build a perf harness mirroring `tests/perf/run_opds_baseline.py`), Phase B low-risk per-request wins, Phase C `get_book_collection` rewrite, Phase D route caching, Phase E page-endpoint optimization (blocked on production traffic data), Phase F cleanups. Plans written but no implementation yet — Phase A is the next step. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins (#600) Closes Phase A (R1: harness, R2: baseline) and Phase B (#5, #9, #10, #13, #14) of `tasks/reader-views-perf/99-summary.md`. Phase A — measure first ======================= Built `tests/perf/run_reader_baseline.py` (mirrors `run_opds_baseline.py`). Seven flows covering the three reader view families: - reader_open + reader_open_large_arc — `c/<pk>` GET - settings_global + settings_multiscope — `c/<pk>/settings` GET - page_first + page_middle + page_no_bookmark — `c/<pk>/<page>/page.jpg` Cold-then-warm methodology via django-silk. Captured `stage0-before.json` and `stage0-after.json` artifacts. Phase B — low-risk wins ======================= #5 — `_get_field_names` hoists settings + admin-flag reads. - Hoisted the per-iteration `get_from_settings("show", browser=True)` call outside the loop. Was 2 SettingsBrowser queries per call; now 1. - The `AdminFlag.objects.get(folder_view)` lookup is now a local `@cached_property`. The reader chain doesn't inherit SearchFilterView (the OPDS / browser `self.admin_flags` source); a local cached_property is the smallest equivalent. Plan misjudgment corrected — see stage0.md. #9 — `Model.objects.get_or_create` over the manual two-query pattern in `_get_global_settings` and `_get_or_create_scoped_settings`. Atomic + race-aware. #10 — `get_reader_default_params` decorated with `@classmethod` + `@cache`. Pure model metadata; doesn't change at runtime. #13 — `_set_selected_arc` pre-builds `frozenset(requested_arc_ids)` outside the inner loop. Drive-by correctness fix: the original loop variable leaked the last iterated value as `arc_ids` even on no-match, so the fallback `if not arc_ids:` only triggered on empty `all_arc_ids`. Iteration now uses a separate `candidate` variable so the fallback fires correctly on no-match. #14 — Drop redundant `.distinct()` on the page-endpoint queryset. The next call is `.get(pk=…)` LIMIT 1, which collapses any ACL JOIN duplicates anyway. Headline numbers ================ reader_open cold: 27 → 26 queries reader_open_large_arc cold: 29 → 28 queries (others stable; cleanups have no measurable per-request impact) The 1-query saving matches the SettingsBrowser hoist. Big wins remain in Phase C (`get_book_collection` rewrite), Phase D (route caching), and Phase E (Comicbox archive cache). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * Reader views perf: Stage 1 — get_book_collection rewrite + per-book batching (#601) Closes Phase C from `tasks/reader-views-perf/99-summary.md`. Tier 1 #2 — get_book_collection rewrite ======================================= The prior implementation iterated the entire arc-ordered queryset in Python to find prev/curr/next, materializing every row's full annotation pyramid (sort_name aliases, arc_index, has_metadata, volume_number_to, …). On the 855-issue Batman series with the user mid-arc that's ~427 fully-materialized Comic instances per reader open, plus a separate `comics.count()` query. The rewrite uses the same arc-ordered queryset but materializes only the pk integer column via `values_list("pk", flat=True)`, finds the current pk's index in Python, and re-fetches the 1-3 prev/curr/next rows with `comics.filter(pk__in=window_pks)`. The filter chain inherits the parent queryset's annotations, select_related, prefetch_related, and only(*fields) so the 1-3 returned rows have everything the serializer needs. `arc_count = len(pks)` replaces the prior separate count query. Tier 2 #4 — per-book settings/bookmark batching ================================================ `_append_with_settings` fired `SettingsReader.objects.filter(comic=book).first()` and `Bookmark.objects.filter(comic=book).first()` per book — 2-6 queries for 1-3 books. New `_batch_settings_and_bookmarks` helper fires two `filter(comic_id__in=window_pks)` calls partitioned by comic_id in Python. Always 2 queries regardless of book count. Headline numbers ================ reader_open_large_arc cold: 28 → 25 queries / 89 → 52 ms (-42%) reader_open cold: 26 → 25 queries / 49 → 51 ms (noise) (other 5 flows stable) The 855-issue series exercises the worst case for the prior iteration. The wall-time win scales with arc size; smaller arcs show ±0 noise. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * use bunx for ttab * caps for close book button again * Reader views perf: Stage 2 — route caching + settings name-lookup batching (#602) Closes Phase D from `tasks/reader-views-perf/99-summary.md`. Tier 1 #3 — `cache_page(READER_TIMEOUT) + vary_on_cookie` on c/<pk> ================================================================== New `READER_TIMEOUT = 60` constant in `codex/urls/const.py` (mirrors `OPDS_TIMEOUT` shape and TTL trade-off). `vary_on_cookie` scopes the cache key per session so per-user responses (bookmark, settings, ACL-filtered arcs) don't leak across users. reader_open warm: 24 → 0 queries / 36 → 1.9 ms (-94%) reader_open_large_arc warm: 24 → 0 queries / 35 → 1.9 ms (-95%) Cold pass unchanged — view runs the full pipeline on cache miss. Real-world reader traffic is dominated by tab refreshes / mobile-app re-foreground patterns where the same `c/<pk>` URL gets hit in quick succession; those collapse to the warm path now. Cross-user isolation verified manually: User A and User B see different payloads on the same URL (762 B vs 739 B reflecting different ACL); each user's warm response matches their own cold; `Vary: Accept, Cookie, origin` on every response. Tier 2 #7 — Fold settings name-lookup into comic prefetch ========================================================== Settings GET pre-fetches the comi…

ComicInfo AlternateSeries / AlternateNumber already imported into the Reprint model, but there was no way to order by the alternate numbering and no way to read an alternate series through in order. Alternate series tags make durable reading lists — they survive a re-import, rename or re-tag — which is what issue #360 asks for. Add an "Alternate Number" sort. Reprint.issue is a string, so "#10" sorted before "#2"; split it into issue_number / issue_suffix columns mirroring Comic's, derived in presave and backfilled by migration. The sort scopes to the alternate series picked by the reprints filter via a FilteredRelation, the way story_arc_number scopes to the browsed arc. Comics carrying no alternate number fall back to their own issue number so a mixed listing stays readable rather than collapsing to NULL. Surface the existing "Alternate Series" sort in cover view too, and sort it through a Coalesce onto the real series sort_name so untagged comics interleave instead of clumping under an empty aggregate. Let the reader follow an alternate series as a reading order. An alternate series is identified by (series_name, volume_number, language) — the Reprint unique key minus the issue — and the arc handle is the whole group's pks, since each issue is its own Reprint row and a per-comic handle would change from book to book. Reprint rows join the TimestampUpdater re-stamp flow and the mtime probe accepts them, so an open reader still notices a re-import. Also fixes a latent error: an M2M primary sort outside table view (cover cards, OPDS) emitted an ORDER BY alias that was only annotated in table mode. Unreachable through the UI before, but surfacing the Alternate Series sort in cover view makes it a normal path. Claude-Session: https://claude.ai/code/session_01CrB4hZ7HtGE9YjV5EkXeQ3 Co-authored-by: Claude <noreply@anthropic.com>

ajslater added 20 commits August 26, 2020 20:07

Merge branch 'release' into develop

4fb8c80

whitenoise 5.2.0 fixes the AUTOREFRESH BUG

0fb8721

Squashed commit of the following:

cfb1c20

commit 52ed30323697d07feef08f57533626df9e84f042 Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 11:43:49 2020 -0700 versioned API

remove print

80a3a2a

fix filter error with aggregates. fucking confusing.

57577cf

make the no comics found label a little less garish

d9bd37c

Squashed commit of the following:

7b0abad

commit cee372d8dfda3c862a9d83883041edc45a013c7a Author: AJ Slater <aj@slater.net> Date: Thu Aug 27 12:48:46 2020 -0700 displaying package version works

master version bumping script

f2ce1f3

only flood control libaryChanged message

b4c389b

broadcast a message when we scan libraries

eab03ff

scan notify api endpoint

827ec90

frontend for scan notify snackbar

d7d19c9

lint

ab87e3b

move isAdmin to auth getters

41e88fc

use queue for scan-notify

5f4b306

Squashed commit of the following:

12aa210

commit 54aed271624a0d93a9f8cd23475151c27dd96710 Author: AJ Slater <aj@slater.net> Date: Fri Aug 28 09:56:33 2020 -0700 fix login

update NEWS

d9c641f

Merge branch 'scan-notify' into v0.5.3

5536343

check to see if the queue is empty too for scan_notify

fix sed for bsd

4757e63

ajslater merged commit 5b5e6db into develop Aug 28, 2020

ajslater deleted the v0.5.3 branch August 28, 2020 17:19

BRTPOB mentioned this pull request Aug 30, 2020

Dictionary update sequence element #0 has length 3; 2 is required exception #27

Closed

ajslater mentioned this pull request Apr 26, 2026

OPDS views perf: Stage 5 — Tier 3-4 cleanups #598

Merged

5 tasks

ajslater mentioned this pull request Apr 26, 2026

Reader views perf: Stage 0 — baseline harness + Phase B low-risk wins #600

Merged

4 tasks

ajslater mentioned this pull request Apr 26, 2026

Reader views perf: Stage 4 — Phase F cleanups bundled #604

Merged

5 tasks

ajslater mentioned this pull request Aug 30, 2026

Merge the Alternate Number sort into the Alternate Series sort #832

Merged

Navigation Menu

v0.5.3 - #10

v0.5.3#10
ajslater merged 20 commits into
developfrom
v0.5.3

ajslater commented Aug 28, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ajslater commented Aug 28, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant