Skip to content

fixes for warnings related to unit tests and nan comparisons - #1657

Merged
shoyer merged 12 commits into
pydata:masterfrom
jhamman:fix/1652
Oct 29, 2017
Merged

fixes for warnings related to unit tests and nan comparisons#1657
shoyer merged 12 commits into
pydata:masterfrom
jhamman:fix/1652

Conversation

@jhamman

@jhammanjhamman commented Oct 25, 2017

Copy link
Copy Markdown
Member

This first commit just includes the fixes for the warnings issued in our unit tests. I've temporarily added -W error to the travis builds so we can see the warnings/errors for all environments. Next step is to address the warnings in the xarray code itself.

@jhamman

Copy link
Copy Markdown
MemberAuthor

Down to 96 (was 372) warnings in my py36 test environment.

Two changes that I could uses some input on:

1 - Numpy element wise comparison
defassertEqual(self, a1, a2):
>asserta1==a2or (a1!=a1anda2!=a2)
EDeprecationWarning: elementwise==comparisonfailed; thiswillraiseanerrorinthefuture.

@shoyer - you and I put together the assertEqual method a while back. Do we need to check the size of arrays before comparing the values?

2 - Numpy invalid value comparison

e.g.:

RuntimeWarning: invalidvalueencounteredingreater_equal

I'm hoping there is a succinct way of wrapping many of our nan comparing functions in numpy.errstat. Thoughts here? I've run out of time for tonight but will pick this up again tomorrow.

@jhamman
jhamman requested a review from shoyerOctober 27, 2017 19:30
@jhamman

Copy link
Copy Markdown
MemberAuthor

@shoyer - I'm wondering if we should attack #1652 in stages. The remaining warnings are going to take a bit more effort and this first block included a lot of line changes. I'm a bit concerned that if we let this sit, we will end up with a million merge conflicts.

@shoyer

Copy link
Copy Markdown
Member

I'm wondering if we should attack #1652 in stages

Yes, absolutely. Let's do it in stages.

@jhamman

Copy link
Copy Markdown
MemberAuthor

Okay, so this knocks off the lowest hanging fruit. I'll pull the pytest error out of the travis build and we'll move forward with this.

@shoyershoyer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh my... I forgot that assertRaisesRegexp was deprecated!

Comment threadxarray/core/dataarray.py Outdated
variable = (f(self.variable, other_variable)
if not reflexive
else f(other_variable, self.variable))
with np.errstate(all='ignore'):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this here -- this should be caught at the level of the Variable wrapper.

Comment threadxarray/core/dataarray.py Outdated
other_variable = getattr(other, 'variable', other)
with self.coords._merge_inplace(other_coords):
f(self.variable, other_variable)
with np.errstate(all='ignore'):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also probably unneeded

@shoyershoyer mentioned this pull request Oct 27, 2017
13 tasks
@shoyer

Copy link
Copy Markdown
Member

@jhamman sorry, looks like you'll need to merge in master, too -- I just put in some conflicting changes with my unicode to netCDF3 PR.

@jhamman

Copy link
Copy Markdown
MemberAuthor

@shoyer - updated. If all the tests pass, I'll merge tonight.

@shoyer

Copy link
Copy Markdown
Member

This looks good to me. Can you add a note to "what's new" (under bug fixes) about removing the warning when comparing with NaN?

@jhammanjhamman changed the title fixes for warnings related to unit testsfixes for warnings related to unit tests and nan comparisonsOct 28, 2017

with self.assertRaisesRegexp(TypeError, 'pip install netcdf4'):
with raises_regex(TypeError, 'pip install netcdf4'):
open_dataset(tmp_file, engine='scipy')

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we aren't closing this file properly, so I'm not too surprised that Windows is complaining. I don't know why it's only showing up now, though.

For now, just add allow_cleanup_failure=True to the create_tmp_file() call. We'll tackle this later in #1668.

Comment threadxarray/tests/test_backends.py Outdated
with open_example_dataset('example_1.nc.gz') as expected:
with open_example_dataset('example_1.nc') as actual:
self.assertDatasetIdentical(expected, actual)
if sys.version_info[:2] < (2, 7):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert to my changes here


with self.assertRaisesRegexp(TypeError, 'pip install netcdf4'):
with raises_regex(TypeError, 'pip install netcdf4'):
open_dataset(tmp_file, engine='scipy')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we aren't closing this file properly, so I'm not too surprised that Windows is complaining. I don't know why it's only showing up now, though.

For now, just add allow_cleanup_failure=True to the create_tmp_file() call. We'll tackle this later in #1668.

@jhamman

Copy link
Copy Markdown
MemberAuthor

@shoyer - I think this is all good now. We seem to have acquired another unrelated build failures though...

@shoyer
shoyer merged commit d016ea7 into pydata:masterOct 29, 2017
@shoyer

Copy link
Copy Markdown
Member

Everything passed after the pandas 0.21 fixes in #1669

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Resolve warnings issued in the xarray test suite

2 participants

@jhamman@shoyer