Uh oh!
There was an error while loading. Please reload this page.
Fix configured encoding fallback and add Windows-1250 option (2) - #673
Fix configured encoding fallback and add Windows-1250 option (2)#673Pr0metheus2 wants to merge 2 commits into
Conversation
Hirogen
commented
Jul 28, 2026
First and foremost thank you for participating and trying to help with improving LogExpert. I'm happy for every help regardless how big or small the changes are. I know from personal experience that it can take quite some time to understand the codebase and even though I refactored a lot of code in the last few months and tried to make it more readable and testable it has still a long way to go. This is also the reason I try to be precise with the review, please by all means don't take it personally, I just want to be thorough. I'm quite picky with changes in LogfileReader.cs and PositionAwareReaderxxx.cs because those are the core of the program and changes there can have quite the consequences (I learned that the hard way) The Review:
Summary
What really is a problem:
|
Hirogen
commented
Jul 28, 2026
I made changes to the PluginHashTool, please rebase your branch with those changes, then the release build should pass |
The `Verify Plugin Hashes (fork PRs)` step in build_dotnet.yml regenerates PluginHashGenerator.Generated.cs and runs `git diff --exit-code` on it. That gate could never pass, for two independent reasons. First, the generator embedded `DateTime.UtcNow` in the file, so every regeneration produced a diff even when no hash had changed. Second, the SDK queries git and feeds the commit sha into every assembly by two routes: appended to InformationalVersion, and into the SourceLink map in the portable PDB, whose checksum is embedded in the DLL. So the hashes you commit describe the parent commit's binaries and go stale the instant they land - committing them changes HEAD, which changes the binaries again. Only first-party assemblies moved; the NuGet-supplied DLLs alongside them never did. That made the gate unwinnable rather than merely awkward: PR LogExperts#673 burned two attempts re-pasting hashes, and every push produced a different set again. On Development the same churn was absorbed by the sibling auto-commit step, which is why `chore: update plugin hashes [skip ci]` landed after nearly every merge. It also meant the committed table always described a different build than the one it shipped in, so Release plugin verification never matched its own binaries. Drop the timestamp, stop the SDK's source-control queries, and regenerate. Verified: two clean Release builds at different commits now produce identical plugin hashes, and `git diff --exit-code` on the generated file is clean.
## Before - BOM-less log files could bypass Preferences.DefaultEncoding in the memory-mapped reader and fall directly back to Encoding.Default. - Legacy configured encodings such as Windows-1250/Windows-1252 could be rejected if a log was opened before the Settings dialog, because the code-page provider had not yet been registered. - Encoding precedence allowed an explicit/persisted value to override a file BOM. - Windows-1250 was not selectable in Preferences. ### After - Encoding resolution follows the intended order: BOM → persisted/explicit encoding → Preferences.DefaultEncoding → Encoding.Default - Both stream and memory-mapped readers apply the configured default consistently. - Legacy Windows code pages are available when reading the preference, regardless of whether Settings was opened first. - Windows-1250 is available in the Preferences encoding dropdown. This ensures a new BOM-less log uses the selected application default and saves that resolved encoding into its newly created .lxp file.
ca4346f to
7a250fcComparePr0metheus2
commented
Jul 28, 2026
Seems I cannot make proper fix to fullfil all checks/requests. Sorry, I give up. |
Hirogen
commented
Jul 28, 2026
dont give up, i created a #677 pr, this is based on your findings, the mmr is actually not reachable and was dead code because it only was reachable if IsMultiFile is false, but this is always true, this is another bug, but will be fixed in one of the next pr's, your Idea with the Encodings leads to the EncodingRegistry that lives in core and now can be used by everybody and does not need to be used in Program.cs |
Hirogen
commented
Jul 28, 2026
thx for all the suggestion and input, #677 is all you :) |
Pr0metheus2
commented
Jul 29, 2026
Thanks for making this fix alive! |
…irst .NET does not ship the legacy Windows code pages; Encoding.GetEncoding throws for them until CodePagesEncodingProvider is registered. Registration lived only in the SettingsDialog constructor, and every site that resolves an encoding *name* swallows the ArgumentException and falls back to Encoding.Default. So a user who picked Windows-1252 in Preferences had that choice silently discarded on every restart in which they did not reopen Preferences. Same for a code page persisted per file in a .lxp, and for the settings JSON. Add EncodingRegistry (LogExpert.Core/Helpers) and route all four resolve sites through it: FileOperationService.FillDefaultEncodingFromSettings, EncodingJsonConverter.ReadJson, PersisterXML.ReadEncoding and the Preferences dropdown. Every method registers the provider before it resolves, so correctness does not depend on one entry point having run first — Program.cs is untouched. Registration uses Lazy with ExecutionAndPublication because files load under Task.Run: the flag must not be observable before registration has completed, or a concurrent first resolve hits the original bug. Also add Windows-1250 to the Preferences encoding dropdown, and extract SettingsDialog.GetAvailableEncodings so the offered set is assertable without building the dialog. DetermineEncoding is deliberately unchanged. The precedence chain (explicit encoding, then BOM, then Preferences default, then machine default) is the intended behaviour; it gains regression tests, not edits. PositionAwareReader is not touched. Delete the memory-mapped read path ---------------------------------- MemoryMappedFileReader and LineOffsetIndex were unreachable. LogfileReader sets IsMultiFile = multiFile || fileNames.Length == 1, so a single file makes IsMultiFile true and the `if (!IsMultiFile && ...)` guard never fires; the multi-file ctor passes multiFile: true. _mmfReader was always null, so BuildIndex, ExtendIndex, GetLine and the fast path in GetLogLineMemoryInternal never ran. Deleting them changes no reachable behaviour. It also could not simply be switched on. MemoryMappedFile.CreateFromFile opens with FileShare.Read, so while a view is mapped the process producing the log cannot append — a test appending to a mapped file fails with IOException even when the writer requests FileShare.ReadWrite. A read path that locks out the writer is incompatible with tail mode, so fixing IsMultiFile would not have produced a working memory-mapped reader but a regression. Same rationale as ADR 0006: the reader folder holds only code something can reach. IsMultiFile itself is left exactly as it is; changing it moves every single-file open onto a different name-resolution path and belongs in its own change. Tests: the four resolve sites are pinned at the site, not just on the helper (FileOperationServiceTests, EncodingJsonConverterTests, PersisterXmlEncodingTests, SettingsDialogEncodingListTests), plus the full DetermineEncoding precedence chain in LogfileReaderEncodingTests. Supersedes LogExperts#673 and LogExperts#671.
ClosesLogExperts#688. GB2312 (code page 936) is now offered both as the Preferences default encoding and as a row in View > Encoding, so a simplified-Chinese log can be read without relying on the machine default. The issue also reported "two utf-8 encoding". The encoding menu built its row labels from Encoding.<X>.HeaderName in the constructor, after the localized resource labels had been applied. Encoding.Default is UTF-8 on .NET, so the "ANSI" row rendered as "utf-8" next to the "UTF8" row, which rendered as "utf-8" as well - two rows applying the same encoding under the same name. The ANSI row is dropped (the Preferences combo lost the same duplicate in LogExperts#673, for the same reason) and the HeaderName overwrite with it, so the remaining rows keep their translated labels. Row check state is now matched by code page rather than by runtime type and equality. That ordering had a defect of its own: clicking UTF8 applies UTF-8 without a BOM, which compares equal to Encoding.Default, so the checkmark landed on the ANSI row instead. Pinned by tests at every layer the choice passes through: the offered list, the menu check state, the settings JSON and .lxp round trips, the reader stack (GB2312 is the first offered encoding that is neither single-byte nor Unicode, so the direct reader's byte-position tracking is asserted against the system reader on mixed ASCII/Chinese lines).
Attempt nr.2 (from #671)
I am trying to rewrite changes from previous pull request #671. When tested these changes, the Logexpert was able to correctly save encoding stored in app setting into .lxp file of newly opened non-BOM file (encoded in Windows-1252).
When UTF-8 BOM file was opened (also first time), correct utf-8 encoding was saved regardless app setting, which is requested behaviour.
I am not saying this fix is perfect, but I was able to get final result to requirement, at least I think so :). This project is so complex and its hard to look/search everywhere, at least for me. If you are convinced this fix is not correct, please do it your way. I am tust trying to make this app better.