Annotate System.Data.{Odbc,OleDb} for nullability by roji · Pull Request #39597 · dotnet/runtime

roji · 2020-07-18T12:13:16Z

As before, no behavior changes were done, only annotations (but see comments below)
Nullability warnings are disabled for OleDb, see https://github.com/roji/runtime/tree/SystemDataOleDbBuildHack for a csproj hack to make these appear
Unfortunately there don't seem to be any warnings about discrepancies between implementation (src) and interface (ref). I did the best I could to keep them in sync, but it would be better to have automatic verification.
Also marked System.Data.DataSetExtensions as annotated - it's a facade with no actual types.

ghost · 2020-07-18T12:50:14Z

Tagging subscribers to this area: @roji, @ajcvickers
Notify danmosemsft if you want to be subscribed.

roji · 2020-07-19T09:37:55Z

@stephentoub @jeffhandley the build failures seem to be because System.Data.Odbc is being built for older TFMs (netfx, netstandard2.0) where nothing is annotated, so Debug.Assert isn't recognized (when building locally, presumably for .NET Core, there are no errors/warnings). Nullability warnings/errors should probably be disabled for older TFMs here, right?

jeffhandley · 2020-07-21T01:38:04Z

Nullability warnings/errors should probably be disabled for older TFMs here, right?

I'm not sure myself. @ericstj, do you know?

jeffhandley

I'm at 37/200 files viewed. So far so good. And thanks for fixing the spelling errors in comments!

ericstj · 2020-07-21T15:33:48Z

Nullability warnings/errors should probably be disabled for older TFMs here, right?

I'm not sure myself. @ericstj, do you know

Looks to me like folks are suppressing the warnings on other frameworks:

runtime/src/libraries/Directory.Build.targets

Lines 9 to 10 in 3f7202b

 <NoWarnCondition="'$(TargetFrameworkIdentifier)' == '.NETFramework' or '$(TargetFrameworkIdentifier)' == '.NETStandard' or '$(TargetFramework)' == 'netcoreapp2.1'">$(NoWarn);nullable</NoWarn> 
 <NoWarnCondition="'$(GeneratePlatformNotSupportedAssembly)' == 'true' or '$(GeneratePlatformNotSupportedAssemblyMessage)' != ''">$(NoWarn);nullable</NoWarn> 

krwq · 2020-07-22T10:22:40Z

 for (int i = 0; i < restrictions.Length; ++i)
 {
- if (null != restrictions[i])
+ if (restrictions[i] is { } restriction)


what does is { } mean?

This is just a way to use pattern making to check if a value isn't null

!= null or is not null would be better.

!= null is the check being replaced, the problem is that since we're checking an array element, the compiler doesn't track the nullability status of that, and any additional reference is flagged again (that's why I made this change).
is not null is C# 9, right? Is it OK to use this already in runtime? If not, I can just use a local variable here instead (or replace this with is not null when that becomes available).

FYI am going to merge as-is, but we can definitely replace these with the nicer is not null once we can use C# 9 here.

There's a tiny benefit of capturing restriction, that will be realized more when using is not null.

Sure, but neither { } nor is not null are necessary to achieve that:
if(restrictions[i]isstringrestriction){ ...}

Ah, yeah that'd be better indeed.

The slight disadvantage of restrictions[i] is string restriction is that it's ambiguous with regards to whether we're checking that something isn't null, or that it's a string (as opposed to something else). FWIW Resharper/Rider warn about this.
I tried restrictions[i] is not null restriction, assuming that's how the C# 9 feature looks like, but it doesn't seem to compile. I'll just switch to restrictions[i] is string restriction for now as suggested.

If you see it as a disadvantage, then just do the super simple thing:
foreach(string?restrictioninrestrictions){if(restriction!=null){tmp[count++]=restriction;}}
or
for(inti=0;i<restrictions.Length;i++){string?restriction=restrictions[i];if(restriction!=null){tmp[count++]=restriction;}}
or just suppress the warning:
for(inti=0;i<restrictions.Length;i++){if(restrictions[i]!=null){tmp[count++]=restrictions[i]!;// TODO-NULLABLE: Indexer nullability tracked (https://github.com/dotnet/roslyn/issues/34644)}}
as we do in dozens of other places in dotnet/runtime... the more we work around it, the less likely it'll be the compiler "fixes" it.
cc: @cston
Not everything needs to use fancy syntax.

krwq · 2020-07-22T12:23:05Z

+ public override long GetBytes(int ordinal, long dataIndex, byte[]? buffer, int bufferIndex, int length) { throw null; }
 public override char GetChar(int ordinal) { throw null; }
- public override long GetChars(int ordinal, long dataIndex, char[] buffer, int bufferIndex, int length) { throw null; }
+ public override long GetChars(int ordinal, long dataIndex, char[]? buffer, int bufferIndex, int length) { throw null; }


is null buffer actually allowed or is it only for zero length?

A null buffer makes the method just return the number of characters in the column, without returning it in any way (this is even in the docs... doesn't happen that often in System.Data :)).

krwq · 2020-07-22T12:38:43Z

I'm at 65/120 and have not seen anything blocking so far - my eyes got a bit blurry with question marks flying so will likely not finish today.

The only comment is to replace all TODO with TODO-NULLABLE for easier tracking

roji · 2020-07-22T16:06:41Z

@ericstj (or anyone else) any idea about the nullability errors in the build? It's odd that they're only in that one config, and I don't get them when building locally...

stephentoub · 2020-07-22T16:47:13Z

(or anyone else) any idea about the nullability errors in the build? It's odd that they're only in that one config, and I don't get them when building locally...

The reason we suppress nullability warnings in

runtime/src/libraries/Directory.Build.targets

Lines 9 to 10 in 3f7202b

 <NoWarnCondition="'$(TargetFrameworkIdentifier)' == '.NETFramework' or '$(TargetFrameworkIdentifier)' == '.NETStandard' or '$(TargetFramework)' == 'netcoreapp2.1'">$(NoWarn);nullable</NoWarn> 
 <NoWarnCondition="'$(GeneratePlatformNotSupportedAssembly)' == 'true' or '$(GeneratePlatformNotSupportedAssemblyMessage)' != ''">$(NoWarn);nullable</NoWarn> 

for downlevel targets is because those surface areas don't have nullable annotations / attributes. That includes Debug.Assert, which means if you have code like Debug.Assert(foo != null); foo.Blah();, without the [DoesNotReturnIf(false)] attribute on the assert, the compiler doesn't know that foo.Blah() is valid and warns that foo might be null. So, the most likely cause here is this library is building against a target that is not yet included in that Directory.Build.targets exclusion, and the fix would be to extend those conditions to make sure it's included.

ericstj · 2020-07-22T16:58:35Z

Are you doing an -allconfigurations build locally? The other legs are all for our vertical configurations where we execute tests, whereas allconfigurations includes other configurations which ship in the package. From the looks of the log the framework you're failing on is 'netcoreapp2.0-FreeBSD`.

We should probably just change the condition on nowarn to suppress on '$(TargetFrameworkIdentifier)' == '.NETCoreApp' and $([MSBuild]::VersionLessThan($(TargetFrameworkVersion), '3.0') instead of only netcoreapp2.1 as it does today.

ericstj · 2020-07-22T20:04:01Z

I went ahead and pushed a commit to suppress nullable warnings on all old netcoreapp frameworks as I suggested.

roji · 2020-07-23T05:09:54Z

Thanks @ericstj!

ericstj · 2020-07-23T16:27:10Z

Thanks @ericstj!

Sure thing! @krwq @jeffhandley I think this is still pending review of annotations.

jeffhandley

I'm at 80/121 files (although I started moving bottom up here recently). Will keep going tomorrow.

jeffhandley

A bit more progress reviewing but still not done.

jeffhandley

Finished reviewed. There were several places where you might want to use TODO-NULLABLE instead of TODO (following the pattern seen in other places in this PR). Just suggestions though.

I think what @ericstj spotted in DbMetaDataFactory.cs is a lurking regression. I don't know if there's existing test coverage that should have caught it though.

Nothing to actually annotated as it's a full facade assembly, all types have been moved to System.Data

roji · 2020-07-25T22:12:37Z

@jeffhandley @krwq and the others, thanks for the thorough reviewing, I've integrated all your comments.

jeffhandley

Thank you for all of the effort you put into this!!

Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>

* Annotate System.Data.Odbc for nullability * Annotate System.Data.OleDb for nullability * Suppress nullable warnings on netcoreapp < 3.0 * Mark System.Data.DataSetExtensions as null-annotated * Fix null-check bug in System.Data.OleDb

roji requested a review from stephentoub July 18, 2020 12:13

Dotnet-GitSync-Bot added the area-Meta label Jul 18, 2020

roji commented Jul 18, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.OleDb/src/OleDbCommand.cs

roji commented Jul 18, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.Odbc/src/System/Data/Odbc/OdbcDataReader.cs Outdated

roji commented Jul 18, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.Odbc/src/System/Data/Odbc/OdbcDataReader.cs

roji commented Jul 18, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.OleDb/src/OleDbDataReader.cs

jkotas added the area-System.Data label Jul 18, 2020

jkotas removed the area-Meta label Jul 18, 2020

jeffhandley reviewed Jul 21, 2020
View reviewed changes

Comment threadsrc/libraries/Common/src/System/Data/Common/AdapterUtil.cs
Comment threadsrc/libraries/Common/src/System/Data/ProviderBase/DbConnectionInternal.cs
Comment threadsrc/libraries/System.Data.Common/src/System/Data/Common/DbDataAdapter.cs

krwq reviewed Jul 22, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.Odbc/src/System/Data/Odbc/OdbcConnectionOpen.cs Outdated

krwq reviewed Jul 22, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.Odbc/src/System/Data/Odbc/OdbcDataReader.cs Outdated

krwq reviewed Jul 22, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.Odbc/src/System/Data/Odbc/OdbcDataReader.cs Outdated

krwq reviewed Jul 22, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.OleDb/src/ColumnBinding.cs Outdated

jaredpar mentioned this pull request Jul 22, 2020
OSX machines are de-provisioned during CI / PR runs leading to failures #34472
Closed

jeffhandley reviewed Jul 24, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.OleDb/src/OleDbConnectionFactory.cs Outdated
Comment threadsrc/libraries/System.Data.OleDb/src/OleDbConnectionInternal.cs Outdated

jeffhandley requested changes Jul 24, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.OleDb/src/OleDbDataReader.cs
Comment threadsrc/libraries/System.Data.OleDb/src/OleDbDataReader.cs Outdated

rojiand others added 5 commits July 25, 2020 21:34

Mark System.Data.DataSetExtensions as null-annotated
c7c7726
Nothing to actually annotated as it's a full facade assembly, all types have been moved to System.Data

Suppress nullable warnings on netcoreapp < 3.0
41722cd

Annotate System.Data.Odbc for nullability
5081504

Annotate System.Data.OleDb for nullability
455049c

Fix null-check bug in System.Data.OleDb
b21b2a2

roji force-pushed the SystemDataNullability3 branch from ada6f2a to b21b2a2Compare July 25, 2020 19:11

jeffhandley approved these changes Jul 26, 2020
View reviewed changes

Comment threadsrc/libraries/System.Data.OleDb/src/OleDbDataReader.cs Outdated

rojiand others added 2 commits July 26, 2020 17:50

Update src/libraries/System.Data.OleDb/src/OleDbDataReader.cs
0443dfb
Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>

Move away from pattern matching syntax "is { }"
d97c819

jeffhandley approved these changes Jul 27, 2020
View reviewed changes

roji merged commit ea96ee3 into dotnet:masterJul 27, 2020

karelz added this to the 5.0.0 milestone Aug 18, 2020

roji mentioned this pull request Sep 4, 2020
Ensure <Nullable>enable<Nullable> on System.IO.Packaging and S.Resources.Extensions #41731
Merged

ghost locked as resolved and limited conversation to collaborators Dec 8, 2020

Uh oh!

Conversation

roji commented Jul 18, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ghost commented Jul 18, 2020

Uh oh!

roji commented Jul 19, 2020

Uh oh!

jeffhandley commented Jul 21, 2020

Uh oh!

jeffhandley left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ericstj commented Jul 21, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rojiJul 22, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rojiJul 22, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

krwq commented Jul 22, 2020

Uh oh!

roji commented Jul 22, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephentoub commented Jul 22, 2020

Uh oh!

ericstj commented Jul 22, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ericstj commented Jul 22, 2020

Uh oh!

roji commented Jul 23, 2020

Uh oh!

ericstj commented Jul 23, 2020

Uh oh!

jeffhandley left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jeffhandley left a comment

Choose a reason for hiding this comment

rojiJul 22, 2020•
edited
Loading

rojiJul 22, 2020•
edited
Loading

roji commented Jul 22, 2020•
edited
Loading

ericstj commented Jul 22, 2020•
edited
Loading