Plain text ingestion WIP: read frames of input with trailing lines

[nblumhardt]

This PR establishes a way of reading logs in real-time from STDIN; the eventual usage will be:

./myapp | seqcli ingest --plain

The major requirement dealt with here is the need to ingest multi-line messages; we do this by:

• Establishing what the start of a message ("frame") will look like, via a TextParser<TextSpan>
• Reading from the start of a message until we see the start of the next message, or, until a timed deadline elapses

This ensures that output like:

Error: System.InvalidOperationException: failed!
at a.cs
at b.cs

can be read into a single frame, without waiting for another message to confirm that no further stack trace lines are coming.

Not the most efficient implementation, but does successfully avoid synchronization.

One snag, the Console.In.ReadLineAsync() method blocks without returning a Task, at least on .NET Core/Windows, so an additional Task.Run() was required.

[nblumhardt]

This was necessary to deal with the change from While() to WithoutAny() a few lines earlier. The former created zero-length matches, while the latter doesn't.

We can revert to the earlier approach when something like Span.MatchedBy(Character.Except('"').Many()) can be expressed in Superpower.

[nblumhardt]

This PR establishes a way of reading logs in real-time from STDIN; the eventual usage will be:

./myapp | seqcli ingest --plain

The major requirement dealt with here is the need to ingest multi-line messages; we do this by:

• Establishing what the start of a message ("frame") will look like, via a TextParser<TextSpan>
• Reading from the start of a message until we see the start of the next message, or, until a timed deadline elapses

This ensures that output like:

Error: System.InvalidOperationException: failed!
at a.cs
at b.cs

can be read into a single frame, without waiting for another message to confirm that no further stack trace lines are coming.

Not the most efficient implementation, but does successfully avoid synchronization.

One snag, the Console.In.ReadLineAsync() method blocks without returning a Task, at least on .NET Core/Windows, so an additional Task.Run() was required.

[nblumhardt]

This was necessary to deal with the change from While() to WithoutAny() a few lines earlier. The former created zero-length matches, while the latter doesn't.

We can revert to the earlier approach when something like Span.MatchedBy(Character.Except('"').Many()) can be expressed in Superpower.

[nblumhardt]

This PR establishes a way of reading logs in real-time from STDIN; the eventual usage will be:

./myapp | seqcli ingest --plain

The major requirement dealt with here is the need to ingest multi-line messages; we do this by:

• Establishing what the start of a message ("frame") will look like, via a TextParser<TextSpan>
• Reading from the start of a message until we see the start of the next message, or, until a timed deadline elapses

This ensures that output like:

Error: System.InvalidOperationException: failed!
at a.cs
at b.cs

can be read into a single frame, without waiting for another message to confirm that no further stack trace lines are coming.

Not the most efficient implementation, but does successfully avoid synchronization.

One snag, the Console.In.ReadLineAsync() method blocks without returning a Task, at least on .NET Core/Windows, so an additional Task.Run() was required.

[nblumhardt]

This was necessary to deal with the change from While() to WithoutAny() a few lines earlier. The former created zero-length matches, while the latter doesn't.

We can revert to the earlier approach when something like Span.MatchedBy(Character.Except('"').Many()) can be expressed in Superpower.

[nblumhardt]

This PR establishes a way of reading logs in real-time from STDIN; the eventual usage will be:

./myapp | seqcli ingest --plain

The major requirement dealt with here is the need to ingest multi-line messages; we do this by:

• Establishing what the start of a message ("frame") will look like, via a TextParser<TextSpan>
• Reading from the start of a message until we see the start of the next message, or, until a timed deadline elapses

This ensures that output like:

Error: System.InvalidOperationException: failed!
at a.cs
at b.cs

can be read into a single frame, without waiting for another message to confirm that no further stack trace lines are coming.

Not the most efficient implementation, but does successfully avoid synchronization.

One snag, the Console.In.ReadLineAsync() method blocks without returning a Task, at least on .NET Core/Windows, so an additional Task.Run() was required.

[nblumhardt]

This was necessary to deal with the change from While() to WithoutAny() a few lines earlier. The former created zero-length matches, while the latter doesn't.

We can revert to the earlier approach when something like Span.MatchedBy(Character.Except('"').Many()) can be expressed in Superpower.

[nblumhardt]

This PR establishes a way of reading logs in real-time from STDIN; the eventual usage will be:

./myapp | seqcli ingest --plain

The major requirement dealt with here is the need to ingest multi-line messages; we do this by:

• Establishing what the start of a message ("frame") will look like, via a TextParser<TextSpan>
• Reading from the start of a message until we see the start of the next message, or, until a timed deadline elapses

This ensures that output like:

Error: System.InvalidOperationException: failed!
at a.cs
at b.cs

can be read into a single frame, without waiting for another message to confirm that no further stack trace lines are coming.

Not the most efficient implementation, but does successfully avoid synchronization.

One snag, the Console.In.ReadLineAsync() method blocks without returning a Task, at least on .NET Core/Windows, so an additional Task.Run() was required.

[nblumhardt]

This was necessary to deal with the change from While() to WithoutAny() a few lines earlier. The former created zero-length matches, while the latter doesn't.

We can revert to the earlier approach when something like Span.MatchedBy(Character.Except('"').Many()) can be expressed in Superpower.

[nblumhardt]

This PR establishes a way of reading logs in real-time from STDIN; the eventual usage will be:

./myapp | seqcli ingest --plain

The major requirement dealt with here is the need to ingest multi-line messages; we do this by:

• Establishing what the start of a message ("frame") will look like, via a TextParser<TextSpan>
• Reading from the start of a message until we see the start of the next message, or, until a timed deadline elapses

This ensures that output like:

Error: System.InvalidOperationException: failed!
at a.cs
at b.cs

can be read into a single frame, without waiting for another message to confirm that no further stack trace lines are coming.

Not the most efficient implementation, but does successfully avoid synchronization.

One snag, the Console.In.ReadLineAsync() method blocks without returning a Task, at least on .NET Core/Windows, so an additional Task.Run() was required.

[nblumhardt]

This was necessary to deal with the change from While() to WithoutAny() a few lines earlier. The former created zero-length matches, while the latter doesn't.

We can revert to the earlier approach when something like Span.MatchedBy(Character.Except('"').Many()) can be expressed in Superpower.

[nblumhardt]

This PR establishes a way of reading logs in real-time from STDIN; the eventual usage will be:

./myapp | seqcli ingest --plain

The major requirement dealt with here is the need to ingest multi-line messages; we do this by:

• Establishing what the start of a message ("frame") will look like, via a TextParser<TextSpan>
• Reading from the start of a message until we see the start of the next message, or, until a timed deadline elapses

This ensures that output like:

Error: System.InvalidOperationException: failed!
at a.cs
at b.cs

can be read into a single frame, without waiting for another message to confirm that no further stack trace lines are coming.

Not the most efficient implementation, but does successfully avoid synchronization.

One snag, the Console.In.ReadLineAsync() method blocks without returning a Task, at least on .NET Core/Windows, so an additional Task.Run() was required.

[nblumhardt]

This was necessary to deal with the change from While() to WithoutAny() a few lines earlier. The former created zero-length matches, while the latter doesn't.

We can revert to the earlier approach when something like Span.MatchedBy(Character.Except('"').Many()) can be expressed in Superpower.

[nblumhardt]

This PR establishes a way of reading logs in real-time from STDIN; the eventual usage will be:

./myapp | seqcli ingest --plain

The major requirement dealt with here is the need to ingest multi-line messages; we do this by:

• Establishing what the start of a message ("frame") will look like, via a TextParser<TextSpan>
• Reading from the start of a message until we see the start of the next message, or, until a timed deadline elapses

This ensures that output like:

Error: System.InvalidOperationException: failed!
at a.cs
at b.cs

can be read into a single frame, without waiting for another message to confirm that no further stack trace lines are coming.

Not the most efficient implementation, but does successfully avoid synchronization.

One snag, the Console.In.ReadLineAsync() method blocks without returning a Task, at least on .NET Core/Windows, so an additional Task.Run() was required.

[nblumhardt]

This was necessary to deal with the change from While() to WithoutAny() a few lines earlier. The former created zero-length matches, while the latter doesn't.

We can revert to the earlier approach when something like Span.MatchedBy(Character.Except('"').Many()) can be expressed in Superpower.