Uh oh!
There was an error while loading. Please reload this page.
.NET: [BREAKING] Add file_access_read_lines and move the line-numbering contract onto AgentFileStore - #7671
Conversation
There was a problem hiding this comment.
Pull request overview
Adds line-range reading and aligns .NET grep results with line-editing semantics.
Changes:
- Adds
file_access_read_lines. - Preserves line terminators across grep/read/edit workflows.
- Updates approvals, documentation, and tests.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
InMemoryAgentFileStoreTests.cs | Tests updated grep semantics. |
FileEditorTests.cs | Tests splitting and slicing. |
FileAccessProviderTests.cs | Tests the new tool and approvals. |
HarnessAgentTests.cs | Verifies tool exposure. |
InMemoryAgentFileStore.cs | Aligns grep line handling. |
FileSystemAgentFileStore.cs | Aligns filesystem grep behavior. |
FileSearchMatch.cs | Documents verbatim lines. |
FileEditor.cs | Adds shared splitting and slicing. |
FileAccessProviderOptions.cs | Documents read-only tool behavior. |
FileAccessProvider.cs | Implements file_access_read_lines. |
Harness_Step03_DataProcessing/README.md | Updates security guidance. |
Claw_Step02_WorkingWithData/README.md | Updates security guidance. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…as the schema does Addresses both review comments on microsoft#7671. TrimTrailingNewline removed only "\n", so grep matched against text such as "match\r" on CRLF and lone-CR lines and an end-anchored pattern like "match$" failed even though the line's text was exactly "match". Renamed to TrimLineTerminator and it now strips "\r\n", "\n", or a lone "\r". The file_access_read_lines description and the SliceLines failure messages referred to end_line/start_line, but the generated schema exposes the arguments as endLine/startLine, so the model could be prompted to emit an invalid argument name. Both now use the schema's names. (new_line is left as-is: FileLineEdit sets it explicitly via JsonPropertyName.) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s_grep Ports the fix for the same defect found by review on the .NET side (microsoft#7671). _search_file_content removed only the trailing "\n" before matching, so on a CRLF file the pattern was applied to text such as "beta match\r" and an end-anchored pattern like "match$" failed even though the line's text is exactly "beta match". The terminator is not part of the line's text, so it is stripped in full now. The per-line offset had to move with it: it advanced by len(scanned) + 1, which was only correct while scanned still carried the "\r". It now advances by len(line), whose terminator is already included, keeping the snippet anchored at the match. Also drops a stale claim in _split_lines_keepends' docstring, which still said it reproduced _search_file_content's content.split("\n") — that dependency now runs the other way round. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.
Suppressed comments (2)
dotnet/src/Microsoft.Agents.AI/Harness/FileStore/FileSystemAgentFileStore.cs:209
- The new newline, line-number, and snippet-offset behavior is tested only against
InMemoryAgentFileStore.FileSystemAgentFileStorehas its own copied search loop and an existing comprehensive search test suite, so a store-specific regression here would pass. Add equivalent CRLF, lone-CR, trailing-newline, anchored-pattern, and snippet-offset coverage for this implementation.
// Lines keep their terminators, so these line numbers address the same lines that
// replace_lines edits and each reported line can be reused as a literal new_line.
List<string> lines = FileEditor.SplitLinesKeepEnds(fileContent);
dotnet/src/Microsoft.Agents.AI/Harness/FileAccess/FileAccessProvider.cs:326
- The advertised line-number parity is not guaranteed for custom
AgentFileStoreimplementations.file_access_grepdelegates to the publicAgentFileStore.SearchAsync, whose contract does not define splitting or terminator retention, while this method andreplace_linessplit independently through an internal-only helper. An existing custom store can therefore return a grep number that reads or edits a different line. Define the required semantics on the public store contract and make a shared implementation available (or centralize line matching above the store) before promising parity.
[Description("Read part of a file by 1-based inclusive line number; omit endLine to read to the end of the file, and an endLine past the last line is clamped. Line numbers match file_access_grep and file_access_replace_lines. Each line is prefixed with its number and a tab; everything after that tab is verbatim, including the line's own terminator, so it can be reused as a file_access_replace_lines new_line.")]
private async Task<string> ReadLinesAsync(string fileName, int startLine, int? endLine = null, CancellationToken cancellationToken = default)
…s_grep Ports the fix for the same defect found by review on the .NET side (microsoft#7671). _search_file_content removed only the trailing "\n" before matching, so on a CRLF file the pattern was applied to text such as "beta match\r" and an end-anchored pattern like "match$" failed even though the line's text is exactly "beta match". The terminator is not part of the line's text, so it is stripped in full now. The per-line offset had to move with it: it advanced by len(scanned) + 1, which was only correct while scanned still carried the "\r". It now advances by len(line), whose terminator is already included, keeping the snippet anchored at the match. Also drops a stale claim in _split_lines_keepends' docstring, which still said it reproduced _search_file_content's content.split("\n") — that dependency now runs the other way round. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
file_access_grep runs through AgentFileStore.search, whose contract says nothing about how content is split or whether terminators survive, while read_lines and replace_lines split through the module-private _split_lines_keepends. A custom store can therefore report a line number that addresses a different line than the two editing tools do — the wrong-line edit this branch exists to prevent, moved to custom stores. The claim was written as unconditional in four places, so _split_lines_keepends, _slice_lines, FileSearchMatch.line and AGENTS.md now say where it holds and where it does not. AGENTS.md also still described matching as stripping only the trailing "\n" and anchoring "as before", which stopped being true in 7aa29c6. Corrected to the whole terminator, in the same wording as the PR description. The read_lines tool docstring is left unhedged on purpose: it is prompt text, and teaching the model to doubt the line numbers would send it back to whole-file reads, which is the cost this branch exists to remove. Found while reviewing the .NET port (microsoft#7671), where Copilot raised the same gap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…wn refusal Mirrors the Python changes in microsoft#7669, which came out of @westey-m's review there. The tool descriptions never said how lines are counted. A model that reads a whole file with file_access_read and then edits by number has to count them itself, and nothing told it the rule. read, read_lines, replace_lines and grep now state it, as do the two memory tools that take or report line numbers. The rule is deliberately not the one Python states, and the wording must not be copied between the two SDKs. Here a lone \r terminates a line and content ending in a terminator has no trailing empty line; in Python neither is true. Taken from the cases FileEditorTests already pins rather than from reading the splitter. SearchAlignment's refusal is shared by both providers and names read_lines, which FileMemoryProvider does not register -- there is no file_memory_read_lines at all. ThrowIfMisalignedAsync now takes the message from the caller, defaulting to the existing wording, and the memory provider passes one naming file_memory_read. CreateToolsAsync in the memory tests was typed to InMemoryAgentFileStore, which is sealed, so a skewed double could not be passed to it. Widened to AgentFileStore. The new test was confirmed to fail against the file-access wording. Removing the call argument instead only breaks the build, which shows the constant has one consumer but not that the assertion discriminates. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mo3qRizH73Gtpviqf5BjP7
Uh oh!
There was an error while loading. Please reload this page.
…ovenance From Copilot review 4985186414. BaseSearchResults marked a list as numbered by the base SearchAsync, and IsTrusted skipped verification on the strength of the type alone. Everything in the payload is mutable -- the list itself, FileSearchResult.FileName and MatchingLines, FileSearchMatch.LineNumber -- so an override could await base.SearchAsync, renumber the results in place, and return the very same instance. The marker survived, verification was skipped, and the wrong-line edit it exists to prevent was reachable again. This is not only a hostile-store concern. A store that prepends a header in ReadAsync and "corrects" the base's numbers to compensate is the same shape as the Python defect fixed in microsoft#7669, and it would have kept full trust here. The marker now snapshots the file names and line numbers it was constructed with, and IsUnmodified re-checks them before trust is granted. The tag says who built the list; the snapshot says the numbers in it are still theirs. Comparison is exact rather than hashed, because a collision would mean false trust, which is the one direction this must not fail in. A modified list drops to verification rather than failing outright, matching the conservative posture elsewhere. FileSearchMatch.Line is deliberately excluded. A custom store may report the text differently -- that is why the check matches by pattern rather than by string -- so covering it would reject stores doing something the contract permits. The type's own documentation claimed an override that post-processes the base results loses the tag. That was true only of post-processing into a new list, and is corrected here. Python needs no equivalent: _numbers_are_trusted keys on the store type and the identity of the base search function, neither of which a store can mutate. This gap existed only because .NET routes around GetType().GetMethod for trim-safety. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mo3qRizH73Gtpviqf5BjP7
Anton Sokolovskyi (antsok)
commented
Aug 20, 2026
PR is ready for review by maintainers |
westey (westey-m)
commented
Aug 20, 2026
Thanks for the updates Anton Sokolovskyi (@antsok). I will not be able to do another pass until Tuesday, but will pick it up again then. |
…s_grep Ports the fix for the same defect found by review on the .NET side (microsoft#7671). _search_file_content removed only the trailing "\n" before matching, so on a CRLF file the pattern was applied to text such as "beta match\r" and an end-anchored pattern like "match$" failed even though the line's text is exactly "beta match". The terminator is not part of the line's text, so it is stripped in full now. The per-line offset had to move with it: it advanced by len(scanned) + 1, which was only correct while scanned still carried the "\r". It now advances by len(line), whose terminator is already included, keeping the snippet anchored at the match. Also drops a stale claim in _split_lines_keepends' docstring, which still said it reproduced _search_file_content's content.split("\n") — that dependency now runs the other way round. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
file_access_grep runs through AgentFileStore.search, whose contract says nothing about how content is split or whether terminators survive, while read_lines and replace_lines split through the module-private _split_lines_keepends. A custom store can therefore report a line number that addresses a different line than the two editing tools do — the wrong-line edit this branch exists to prevent, moved to custom stores. The claim was written as unconditional in four places, so _split_lines_keepends, _slice_lines, FileSearchMatch.line and AGENTS.md now say where it holds and where it does not. AGENTS.md also still described matching as stripping only the trailing "\n" and anchoring "as before", which stopped being true in 7aa29c6. Corrected to the whole terminator, in the same wording as the PR description. The read_lines tool docstring is left unhedged on purpose: it is prompt text, and teaching the model to doubt the line numbers would send it back to whole-file reads, which is the cost this branch exists to remove. Found while reviewing the .NET port (microsoft#7671), where Copilot raised the same gap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s_grep Ports the fix for the same defect found by review on the .NET side (microsoft#7671). _search_file_content removed only the trailing "\n" before matching, so on a CRLF file the pattern was applied to text such as "beta match\r" and an end-anchored pattern like "match$" failed even though the line's text is exactly "beta match". The terminator is not part of the line's text, so it is stripped in full now. The per-line offset had to move with it: it advanced by len(scanned) + 1, which was only correct while scanned still carried the "\r". It now advances by len(line), whose terminator is already included, keeping the snippet anchored at the match. Also drops a stale claim in _split_lines_keepends' docstring, which still said it reproduced _search_file_content's content.split("\n") — that dependency now runs the other way round. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
file_access_grep runs through AgentFileStore.search, whose contract says nothing about how content is split or whether terminators survive, while read_lines and replace_lines split through the module-private _split_lines_keepends. A custom store can therefore report a line number that addresses a different line than the two editing tools do — the wrong-line edit this branch exists to prevent, moved to custom stores. The claim was written as unconditional in four places, so _split_lines_keepends, _slice_lines, FileSearchMatch.line and AGENTS.md now say where it holds and where it does not. AGENTS.md also still described matching as stripping only the trailing "\n" and anchoring "as before", which stopped being true in 7aa29c6. Corrected to the whole terminator, in the same wording as the PR description. The read_lines tool docstring is left unhedged on purpose: it is prompt text, and teaching the model to doubt the line numbers would send it back to whole-file reads, which is the cost this branch exists to remove. Found while reviewing the .NET port (microsoft#7671), where Copilot raised the same gap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| /// <param name="cancellationToken">A token to cancel the operation.</param> | ||
| /// <returns>The file content or a not-found message.</returns> | ||
| [Description("Read the content of a file by name. Returns the file content or a message indicating the file was not found.")] | ||
| [Description("Read the content of a file by name. Returns the file content or a message indicating the file was not found. To edit by line number afterwards, count lines terminated by \\n, \\r\\n, or a lone \\r; each line keeps its own terminator, and content ending in a terminator has no extra empty line after it.")] |
There was a problem hiding this comment.
For the few edits to the existing tools descriptions, we should include that the line numbers are 1-based.
| /// <summary> | ||
| /// Throws when <paramref name="results"/> cannot be trusted to address the editor's lines. | ||
| /// </summary> | ||
| internal static async Task ThrowIfMisalignedAsync( |
There was a problem hiding this comment.
I'm not convinced about running this at runtime as it adds overhead for all users to try and cater for implementations that are not per spec.
We should instead just document the expectation around line numbers in the base AgentFileStore, on the abstract/virtual methods. If this is not followed, it is a bug, and custom implementors need to still do testing as well, to ensure that their implementations work correctly.
There was a problem hiding this comment.
The check does not run for users of the shipped stores. IsTrusted short-circuits on ReportsAlignedLineNumbers (SearchAlignment.cs:120-128), and both shipped stores override it to true.
In principle, I do not disagree with you: a store that violates the numbering contract has a bug, and its author should be testing for it. The reason I built detection anyway is the silent failure mode. A skewed line number does not surface as an exception or a wrong search result - grep looks fine, and the damage lands later when replace_lines edits a line the user never saw, in a file the model was told it had located correctly. That is silent and destructive rather than loud and diagnosable, which is why I went for detection at the point where the numbers cross from one component to another.
That said, this is your call westey (@westey-m), and I will take it either way. If you definitely want it gone, please say so, and I will remove SearchAlignment, the trust gate, MisalignedMessage and their tests, and move the expectation onto the abstract and virtual members of AgentFileStore as documented behaviour, with the hazard stated plainly so an implementer knows what breaks if they get it wrong.
| /// Gets or sets the matching line, verbatim. | ||
| /// </summary> | ||
| /// <remarks> | ||
| /// For the <see cref="AgentFileStore"/> implementations in this package, the line keeps its own |
There was a problem hiding this comment.
We shouldn't phrase these comments to only apply to local implementations, but rather as expectations.
E.g. instead of saying implementations in this package we should be saying Implementers should include terminators for each line...
Motivation & Context
Two defects, one of which only became visible while fixing the other.
The reading gap (#7571). The harness file tools are line-precise when editing —
file_access_replace_linestakes 1-based line numbers andfile_access_grepreports them — but all-or-nothing when reading. There is no way to see the lines around a match without reading the whole file, so agents either re-read entire files or edit by a line number they never looked at.Grep and the editor did not agree on what a line is. The stores split on
'\n'and stripped'\r';FileEditorsplit on'\n','\r\n'and a lone'\r'and kept terminators. That is not a cosmetic difference:'\r'terminators, grep's line 1 covers only part of what the model was shown, so editing by that number silently changes the wrong text.NewLineis written verbatim, feeding a grepped line back intoreplace_linesjoined it to the next line.The same hole, one level down — and this is new scope since the first version of this PR. Unifying the splitter fixes the two stores in this package, but
AgentFileStore.SearchAsyncwasabstractwith no numbering contract at all, whileread_linesandreplace_linesre-count fromReadAsync. So a custom store could make "line 5" mean two different things, andreplace_lineswould edit the wrong line — in range, reporting success.FileMemoryProviderhad the identical hole.The previous revision of this PR documented that as a caveat and called it "a maintainer call". @moonbox3 made that call on the Python PR #7669: the rule should live on the contract, not in a doc comment. This PR now mirrors #7669 so both SDKs carry the same contract.
Description & Review Guide
What are the major changes?
The reading gap:
FileEditor.SplitLinesKeepEndsbecomesinternaland is the single definition of a line.FileSearchMatch.Lineis reported verbatim, terminator included.file_access_read_linestool, rendering<n>\t<line>with everything after the tab verbatim — so a row is already a validreplace_linesnew_line.The contract, now on the base class:
4.
SplitLines()publishes the split everyline_numberaddresses. Per-SDK by design: it need not match Python, only be consistent here, because a line number never crosses runtimes.5.
ScanContent()is the numbering primitive. Both shipped stores report through it, which also removes the scan loop that was duplicated between them.6.
FindMatchingFilesAsync()is a new hook for narrowing to the files worth reading. The regex goes down as a hint with superset semantics — over-returning is harmless because the base re-scans, under-returning loses matches. A backend with a native index overrides it and narrows server-side. Its default is built fromListChildrenAsync, so a store implementing nothing beyond the mandatory members gets aligned numbers for free.7.
SearchAsyncis no longer abstract. It reads and numbers candidates itself, and re-applies the glob and the non-recursive rule, since the hook may over-return.8. Overriding
SearchAsyncstays first-class — a backend that can do the whole job natively should — but then it owns numbering, and both providers verify it: each reported line is re-matched against the line the editor would touch, and the whole call is refused on a mismatch. It compares by pattern, not by string, because a custom store is not bound to report the line verbatim and comparing text would reject correct stores.9. Two opt-outs: a store sets
ReportsAlignedLineNumbers, or a provider takesDisableSearchAlignmentCheck. The store flag is narrower and preferred. Both are promises rather than hints, and a test pins that hazard on purpose.10.
FileLineEdit.ExpectedLine— when supplied, the edit is refused unless the target line still says what the caller saw. Catches splitter drift, a stale line number, and the file changing between read and write.Added during review, after the items above:
file_access_readand then edits by line number: it has to count the lines itself, and nothing told it the rule.read,read_lines,replace_lines,grepand the two memory tools now say that lines are terminated by\r\n,\nor a lone\r, that each line keeps its terminator, and that content ending in a terminator has no extra empty line after it. Taken from the casesFileEditorTestsalready pins rather than from reading the splitter.FileMemoryProviderhas its own misalignment message. The shared one told the model to useread_lines, which that provider does not register — there is nofile_memory_read_lines.SearchAlignment.ThrowIfMisalignedAsyncnow takes the wording from the caller.RegexMatchTimeoutExceptionreturned from the whole check, so one unevaluable line handed every later result to the model as though it had been verified; it now skips just that match. And only the upper bound of a reported line number was validated, so a0produced a raw index error instead of the misalignment message.await base.SearchAsync, renumber them in place, and return the same tagged instance — keeping trust while handing the model numbers the base never produced.BaseSearchResultsnow snapshots what it was built with and re-checks it. See focus item 3.What is the impact of these changes?
Breaking, in these ways:
FileSearchMatch.Lineincludes terminatorsgrepoutput'\r'content and on a trailing newlinematch$now matches on a CRLF line where it could not before, and a pattern targeting a literal'\r'no longer matches the one such a line ends withSearchAsyncis no longerabstractoverridestill works)SplitLineshas itsgrepresults refused rather than silently appliedThe whole surface is
[Experimental("MAAI001")]. ApiCompat does not flag any of it —FileEditoris internal andLinekeeps its type — so a passing Release build is not evidence of compatibility. Removingabstractwhile keeping the membervirtualdoes pass Package Validation; verified by building the package in Release withIsReleased=true.Two further changes are not API breaks but do change what the model is told: every line-addressing tool description gains the counting rule, and the
expected_linemismatch message no longer echoes the line it found — raised in review as a read oracle where write tools are auto-approved while read tools are not, and applied in 30115b0.Cost. Verification is exactly one extra
ReadAsyncper matched file, and zero for a store that declares alignment or a provider that opts out. Measured end to end onfile_access_grep, 12 files / 3 matching:The narrowing hook is not a speed-up and is not sold as one — it is "same speed, now safe". A selectivity sweep against a store doing the whole job in its own
SearchAsync:SearchAsyncPer-method timings across local disk, in-memory, Azure Blob and Redis show no method-level effect; happy to attach the full table if useful.
file_access_read_linesjoins the read-only tool set, so it is exposed underDisableWriteToolsand covered byReadOnlyToolsAutoApprovalRule— the auto-approval docs and sample security notes are updated accordingly.What do you want reviewers to focus on?
Whether the contract belongs on
AgentFileStoreat all — that is the substantive question, and it is a maintainer call being made here rather than assumed.The superset semantics of
FindMatchingFilesAsync. Over-returning is harmless, under-returning silently loses matches. Whether that is documented clearly enough for someone implementing it against a native index.How "did the base number these?" is tracked — and why it differs from the Python half. The base tags the returned list (
BaseSearchResults), andSearchAlignment.IsTrustedaccepts that tag only when the list still carries the numbers the base put in it. Detecting an override viaGetType().GetMethod(...)would trip IL2075 and is not trim-safe under--warnaserror; a per-instance flag would be worse, because a store that defers tobase.SearchAsynconly sometimes would buy permanent trust for the results it numbers itself. A test pins that, and it fails against the flag version.Review caught that the tag alone proves provenance, not integrity: the list and every
FileSearchResultin it are mutable, so an override couldawait base.SearchAsync, renumber in place, and return the same instance with the tag intact.BaseSearchResultsnow snapshots the file names and line numbers at construction and re-checks them inIsUnmodified(). Comparison is exact rather than hashed, since a collision would mean false trust;FileSearchMatch.Lineis excluded on purpose, because a store may legitimately report the text differently — which is why the check matches by pattern and not by string.Tagging is viable here only because both shipped stores are
sealed, so nothing can inherit theirSearchAsync. Python cannot rely on that: Python: [BREAKING] Add file_access_read_lines and move the line-numbering contract onto AgentFileStore #7669 keys trust on the concrete store types plus a separate check for the inherited basesearch, because a Python subclass can inherit a shipped store'ssearchwhile overridingread. Worth comparing the two if you review both PRs — the mechanisms are deliberately different, not accidentally divergent.The per-line snippet offset arithmetic in both stores now that terminators are part of each line.
FileSystemAgentFileStoreTestspreviously asserted noLinevalues at all, so the six search tests mirrored into it are where that arithmetic is now pinned for the disk-backed store.Two deliberate divergences from the Python half (Python: [BREAKING] Add file_access_read_lines and move the line-numbering contract onto AgentFileStore #7669):
Line-rule parity note: Python addresses a trailing empty line on
"a\nb\n"; .NET has two lines there, because .NET's line editor never had that phantom line. Each language stays self-consistent, which is what the grep → read → edit round trip actually depends on.Related Issue
#7571 — linked without a closing keyword on purpose: the Python half ships as #7669, and the issue should stay open until both land. Will change to
Closesin the last one.Note that #7571's body still says no store-protocol change is needed; that predates the review discussion above and is no longer accurate for either language.
Contribution Checklist
--warnaserroracross all five TFMs, Package Validation includedAgentFileStoreContractTestsbreaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.