Uh oh!
There was an error while loading. Please reload this page.
Handle non-existent root directory in PhysicalFilesWatcher - #126411
Conversation
When PhysicalFileProvider is constructed with a root directory that does not yet exist, PhysicalFilesWatcher now defers FileSystemWatcher activation until the root appears. A PendingCreationWatcher monitors the nearest existing ancestor using a non-recursive FileSystemWatcher and cascades through intermediate directory levels as they are created. Once the root directory exists, the main recursive FileSystemWatcher is enabled and any already-existing watched entries are reported. Callers always receive normal FSW-backed change tokens — no re-registration is needed when the root directory appears later. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tagging subscribers to this area: @dotnet/area-extensions-filesystem |
There was a problem hiding this comment.
Pull request overview
This PR updates the Microsoft.Extensions.FileProviders.Physical watcher stack (and related configuration/file-globbing components) to handle roots that don’t exist yet by deferring FileSystemWatcher activation until the directory appears, avoiding broad/incorrect ancestor watching and improving correctness for configuration reload-on-change scenarios.
Changes:
- Add deferred root-creation monitoring to
PhysicalFilesWatcher(including a cascading non-recursive watcher for intermediate directories) and report already-existing entries once the root appears. - Relax
PhysicalFileProviderto allow construction with missing roots; update configuration source resolution to use the file’s immediate parent directory even when missing. - Extend polling and globbing helpers/tests to correctly handle directory paths and missing-directory lifecycles.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/libraries/Microsoft.Extensions.FileSystemGlobbing/src/Abstractions/DirectoryInfoWrapper.cs | Refresh DirectoryInfo before checking Exists to avoid stale existence state. |
| src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PhysicalFilesWatcher.cs | Defer FSW enablement until root exists via PendingCreationWatcher, add gap-coverage scan, adjust filtering and error handling. |
| src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PollingFileChangeToken.cs | Treat watched path as file or directory (directory fallback) so polling detects directory changes. |
| src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PhysicalFileProvider.cs | Allow missing root at construction; centralize separator trimming via PathUtils.PathSeparators; adjust watcher creation. |
| src/libraries/Microsoft.Extensions.FileProviders.Physical/src/Internal/PathUtils.cs | Expose PathSeparators for reuse and make trailing-slash normalization accept both separators. |
| src/libraries/Microsoft.Extensions.FileProviders.Physical/src/Resources/Strings.resx | Add new resource strings for missing-root and invalid FSW-path diagnostics. |
| src/libraries/Microsoft.Extensions.Configuration.FileExtensions/src/FileConfigurationSource.cs | Resolve provider to the file’s immediate directory (even if missing) and reduce Path to the filename. |
| src/libraries/Microsoft.Extensions.FileProviders.Physical/tests/PhysicalFilesWatcherTests.cs | Add/adjust tests covering missing roots, root recreation, directory tokens, exclusion behavior, and polling variants. |
| src/libraries/Microsoft.Extensions.FileProviders.Physical/tests/PhysicalFileProviderTests.cs | Add test ensuring non-existent roots are accepted; update related watcher tests for robustness. |
| src/libraries/Microsoft.Extensions.Configuration.FileExtensions/tests/Microsoft.Extensions.Configuration.FileExtensions.Tests.csproj | Include common TempDirectory helper for new test coverage. |
| src/libraries/Microsoft.Extensions.Configuration.FileExtensions/tests/FileConfigurationProviderTest.cs | Integration test verifying token fires when file appears under previously-missing directory. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
> [!NOTE] > This PR was AI/Copilot-generated. Fixes#126787 ## Problem The `ResolveFileProvider_WithMissingParentDirectory_WatchTokenFiresWhenFileCreated` test introduced in #126411 is flaky on the `net481-windows-Release-x86` CI leg, causing it to time out and block CI. ## Root cause `FileSystemWatcher` on .NET Framework drops directory creation events under I/O contention. The `PendingCreationWatcher` in `PhysicalFilesWatcher` relies on FSW to detect when a missing root directory is created. When the FSW event is lost, the change token never fires and the test hangs. This was verified by: 1. **Reproducing locally** — running the test in parallel (4-8 concurrent `dotnet test` processes) reproduces the hang on net481 x86 (~3% failure rate). 2. **Isolating to raw FSW** — a minimal test using only `FileSystemWatcher` (no `PendingCreationWatcher`) also fails under the same conditions on .NET Framework. 3. **Confirming .NET is unaffected** — 504 runs on net11.0 x86 with 8 concurrent threads produced 0 failures. All 7 CI failures in the issue report are on the same `net481-windows-Release-x86-NET481_Release-Windows.10.Amd64.Client.Open` configuration. ## Changes - **Skip on .NET Framework**: Changed `[Fact]` to `[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsNotNetFramework))]` since the test depends on FSW reliability that .NET Framework cannot guarantee under load. - **Improved timeout**: Replaced `CancellationTokenSource`-based timeout with `Task.WaitAsync(TimeSpan)` (via `TaskTimeoutExtensions` polyfill) and added `TaskCreationOptions.RunContinuationsAsynchronously` — matching the pattern used by all similar tests in `PhysicalFilesWatcherTests`. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Handle non-existent root directory in PhysicalFilesWatcher
Fixes#116713
Fixes#107700
Problem
When
PhysicalFileProvideris constructed with a root directory that does not yet exist (e.g., a configuration file path whose parent directory hasn't been created),Watch()fails becauseFileSystemWatchercannot watch a non-existent directory. This commonly occurs withAddJsonFilewhen the config file's parent directory is missing at startup.FileConfigurationSourcehad its own solution: watching the closest existing directory instead. The problem with this is that it commonly watched too much of the system (possibly even all of it), causing significant performance issues.Solution
PhysicalFilesWatchernow gracefully handles a missing root directory by deferringFileSystemWatcheractivation until the root appears. APendingCreationWatchermonitors the nearest existing ancestor directory using a non-recursiveFileSystemWatcherand cascades through intermediate directory levels as they are created. Once the root directory exists, the main recursiveFileSystemWatcheris enabled and any already-existing watched entries are reported.Callers always receive normal FSW-backed change tokens — no re-registration is needed when the root directory appears later.
FileConfigurationSourcethen usesPhysicalFilesWatcheron a directory that may not exist, which is now handled well.Changes
PhysicalFilesWatcher.csPendingCreationWatcherinner class that watches for a non-existent directory to be created.TryEnableFileSystemWatcherdefers toEnsureRootCreationWatcherwhen_rootdoesn't exist, with a callback to retry once it appears.ReportExistingWatchedEntriesfires tokens for entries created before the FSW was active. The constructor normalizes_rootto always have a trailing separator and validates FSW path relationship.OnFileSystemEntryChangenow usesDirectoryInfofor directory paths so exclusion filters work correctly, and guards against events outside_root.OnErrornow also notifies wildcard tokens.PhysicalFileProvider.csDirectoryNotFoundExceptionfor a missing root. UpdatedWatchdoc comments to include directories. Removed duplicate_pathSeparatorsfield in favor ofPathUtils.PathSeparators.FileConfigurationSource.csResolveFileProvidercreates thePhysicalFileProviderwith the file's immediate parent directory (even if missing), relying on the watcher to handle the non-existent case.PollingFileChangeToken.csGetLastWriteTimeUtcnow falls back to checkingDirectoryInfowhenFileInfo.Existsis false, so polling correctly detects directory changes.DirectoryInfois created lazily.PathUtils.csPathSeparatorsmadeinternalfor reuse across files.PhysicalFilesWatcherTests.csFileConfigurationProviderTest.csPhysicalFileProviderTests.csTokenFiredForGlobbingPatternsPointingToSubDirectorypath length issue on .NET Framework.