diff --git a/docs/src/userguide/cube_maths.rst b/docs/src/userguide/cube_maths.rst index d2d4d84b68..35e672eaf6 100644 --- a/docs/src/userguide/cube_maths.rst +++ b/docs/src/userguide/cube_maths.rst @@ -157,7 +157,7 @@ Let's extend this example slightly, by taking a slice from the middle air_temperature / (K) (longitude: 49; time: 240) Compared to our original time-series, the *air_temp_T_slice* cube has one -less dimension *and* it's shape if different. However, this doesn't prevent +less dimension *and* its shape is different. However, this doesn't prevent us from performing cube arithmetic with it, thanks to the extended cube broadcasting behaviour:: @@ -237,7 +237,7 @@ by a cube with unit ``'1'`` will preserve units, so the cube ``temperature`` will be given the same units as are in ``pot_temperature``. It should be noted that some combinations of units, particularly those involving power operations, will not result in a valid unit and will cause the calculation -to fail. For example, a cube ``a`` had units ``'m'`` then ``a ** 0.5`` +to fail. For example, if a cube ``a`` had units ``'m'`` then ``a ** 0.5`` would result in an error since the square root of a meter has no meaningful unit (if ``a`` had units ``'m2'`` then ``a ** 0.5`` would result in a cube with units ``'m'``). diff --git a/docs/src/whatsnew/3.0.1.rst b/docs/src/whatsnew/3.0.1.rst index 05bf41ce18..11ad75d95c 100644 --- a/docs/src/whatsnew/3.0.1.rst +++ b/docs/src/whatsnew/3.0.1.rst @@ -24,12 +24,11 @@ This document explains the changes made to Iris for this release factory, and the associated derived coordinate will be missing. (:pull:`3965`) -.. dropdown:: :opticon:`report` Release Highlights +.. dropdown:: :opticon:`report` v3.0.0 Release Highlights :container: + shadow :title: text-primary text-center font-weight-bold :body: bg-light :animate: fade-in - :open: The highlights for this major release of Iris include: diff --git a/docs/src/whatsnew/3.0.2.rst b/docs/src/whatsnew/3.0.2.rst new file mode 100644 index 0000000000..1d1415b1cf --- /dev/null +++ b/docs/src/whatsnew/3.0.2.rst @@ -0,0 +1,571 @@ +.. include:: ../common_links.inc + +v3.0.2 (27 May 2021) +******************** + +This document explains the changes made to Iris for this release +(:doc:`View all changes `.) + + +.. dropdown:: :opticon:`alert` v3.0.2 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + :open: + + The patches included in this release include: + + 🐛 **Bugs Fixed** + + #. `@jonseddon`_ handled a malformed ``um_stash_source`` CF variable attribute in + a netCDF file rather than raising a ``ValueError``. (:pull:`4035`) + + #. `@rcomer`_ fixed :meth:`~iris.cube.Cube.intersection` for special cases + where one cell's bounds align with the requested maximum and minimum, as + reported in :issue:`3391`. (:pull:`4059`) + + #. `@bjlittle`_ resolved a regression in arithmetic behaviour between a coordinate + and a cube which resulted in a ``NotYetImplementedError`` being raised, as reported + in :issue:`4000`. This fix supports ``+``, ``-``, ``*``, and ``/`` operations + between a coordinate and a cube, and for convenience additionally includes + :meth:`iris.cube.Cube.__neg__` support. (:pull:`4159`) + + 📚 **Documentation** + + #. `@bjlittle`_ updated the ``intersphinx_mapping`` and fixed documentation + to use ``stable`` URLs for `matplotlib`_. (:pull:`4003`) [``pre-v3.1.0``] + + 💼 **Internal** + + #. `@jamesp`_ updated a test to the latest numpy version (:pull:`3977`) [``pre-v3.1.0``] + + #. `@bjlittle`_ enabled `cirrus-ci`_ compute credits for non-draft pull-requests + from collaborators targeting the Iris ``master`` branch. (:pull:`4007`) + [``pre-v3.1.0``] + + #. `@bjlittle`_ added conditional task execution to `.cirrus.yml`_ to allow + developers to easily disable `cirrus-ci`_ tasks. See + :ref:`skipping Cirrus-CI tasks`. (:pull:`4019`) [``pre-v3.1.0``] + + #. `@pp-mo`_ adjusted the use of :func:`dask.array.from_array` in :func:`iris._lazy_data.as_lazy_data`, + to avoid the dask 'test access'. This makes loading of netcdf files with a + large number of variables significantly faster. (:pull:`4135`) + + #. `@pp-mo`_ reverted a change made previously in (:pull:`3659`) to + :meth:`iris.fileformats.pp.PPDataProxy.__getitem__`. The check for empty slicings + is no longer needed since (:pull:`4135`) was added. (:pull:`4141`) + + Note that, the above contributions labelled with ``pre-v3.1.0`` are part of the forthcoming + Iris v3.1.0 release, but require to be included in this patch release. + + +.. dropdown:: :opticon:`alert` v3.0.1 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The patches included in this release include: + + 💼 **Internal** + + #. `@bjlittle`_ gracefully promote formula terms within :mod:`~iris.aux_factory` that have ``units`` of ``unknown`` + to ``units`` of ``1`` (dimensionless), where the formula term **must** have dimensionless ``units``. Without this + graceful treatment of ``units`` the resulting :class:`~iris.cube.Cube` will **not** contain the expected auxiliary + factory, and the associated derived coordinate will be missing. (:pull:`3965`) + + +.. dropdown:: :opticon:`report` v3.0.0 Release Highlights + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The highlights for this major release of Iris include: + + * We've finally dropped support for ``Python 2``, so welcome to ``Iris 3`` + and ``Python 3``! + * We've extended our coverage of the `CF Conventions and Metadata`_ by + introducing support for `CF Ancillary Data`_ and `Quality Flags`_, + * Lazy regridding is now available for several regridding schemes, + * Managing and manipulating metadata within Iris is now easier and more + consistent thanks to the introduction of a new common metadata API, + * :ref:`Cube arithmetic ` has been significantly improved with + regards to extended broadcasting, auto-transposition and a more lenient + behaviour towards handling metadata and coordinates, + * Our :ref:`documentation ` has been refreshed, + restructured, revitalised and rehosted on `readthedocs`_, + * It's now easier than ever to :ref:`install Iris ` + as a user or a developer, and the newly revamped developers guide walks + you though how you can :ref:`get involved ` + and contribute to Iris, + * Also, this is a major release of Iris, so please be aware of the + :ref:`incompatible changes ` and + :ref:`deprecations `. + + And finally, get in touch with us on `GitHub`_ if you have any issues or + feature requests for improving Iris. Enjoy! + + +📢 Announcements +================ + +#. Congratulations to `@bouweandela`_, `@jvegasbsc`_, and `@zklaus`_ who + recently became Iris core developers. They bring a wealth of expertise to the + team, and are using Iris to underpin `ESMValTool`_ - "*A community diagnostic + and performance metrics tool for routine evaluation of Earth system models + in CMIP*". Welcome aboard! 🎉 + +#. Congratulations also goes to `@jonseddon`_ who recently became an Iris core + developer. We look forward to seeing more of your awesome contributions! 🎉 + + +✨ Features +=========== + +#. `@MoseleyS`_ greatly enhanced the :mod:`~iris.fileformats.nimrod` + module to provide richer meta-data translation when loading ``Nimrod`` data + into cubes. This covers most known operational use-cases. (:pull:`3647`) + +#. `@stephenworsley`_ improved the handling of + :class:`iris.coords.CellMeasure`\ s in the :class:`~iris.cube.Cube` + statistical operations :meth:`~iris.cube.Cube.collapsed`, + :meth:`~iris.cube.Cube.aggregated_by` and + :meth:`~iris.cube.Cube.rolling_window`. These previously removed every + :class:`~iris.coords.CellMeasure` attached to the cube. Now, a + :class:`~iris.coords.CellMeasure` will only be removed if it is associated + with an axis over which the statistic is being run. (:pull:`3549`) + +#. `@stephenworsley`_, `@pp-mo`_ and `@abooton`_ added support for + `CF Ancillary Data`_ variables. These are created as + :class:`iris.coords.AncillaryVariable`, and appear as components of cubes + much like :class:`~iris.coords.AuxCoord`\ s, with the new + :class:`~iris.cube.Cube` methods + :meth:`~iris.cube.Cube.add_ancillary_variable`, + :meth:`~iris.cube.Cube.remove_ancillary_variable`, + :meth:`~iris.cube.Cube.ancillary_variable`, + :meth:`~iris.cube.Cube.ancillary_variables` and + :meth:`~iris.cube.Cube.ancillary_variable_dims`. + They are loaded from and saved to NetCDF-CF files. Special support for + `Quality Flags`_ is also provided, to ensure they load and save with + appropriate units. (:pull:`3800`) + +#. `@bouweandela`_ implemented lazy regridding for the + :class:`~iris.analysis.Linear`, :class:`~iris.analysis.Nearest`, and + :class:`~iris.analysis.AreaWeighted` regridding schemes. (:pull:`3701`) + +#. `@bjlittle`_ added `logging`_ support within :mod:`iris.analysis.maths`, + :mod:`iris.common.metadata`, and :mod:`iris.common.resolve`. Each module + defines a :class:`logging.Logger` instance called ``logger`` with a default + ``level`` of ``INFO``. To enable ``DEBUG`` logging use + ``logger.setLevel("DEBUG")``. (:pull:`3785`) + +#. `@bjlittle`_ added the :mod:`iris.common.resolve` module, which provides + infrastructure to support the analysis, identification and combination + of metadata common between two :class:`~iris.cube.Cube` operands into a + single resultant :class:`~iris.cube.Cube` that will be auto-transposed, + and with the appropriate broadcast shape. (:pull:`3785`) + +#. `@bjlittle`_ added the :ref:`common metadata API `, which provides + a unified treatment of metadata across Iris, and allows users to easily + manage and manipulate their metadata in a consistent way. (:pull:`3785`) + +#. `@bjlittle`_ added :ref:`lenient metadata ` support, to + allow users to control **strict** or **lenient** metadata equivalence, + difference and combination. (:pull:`3785`) + +#. `@bjlittle`_ added :ref:`lenient cube maths ` support and + resolved several long standing major issues with cube arithmetic regarding + a more robust treatment of cube broadcasting, cube dimension auto-transposition, + and preservation of common metadata and coordinates during cube math operations. + Resolves :issue:`1887`, :issue:`2765`, and :issue:`3478`. (:pull:`3785`) + +#. `@pp-mo`_ and `@TomekTrzeciak`_ enhanced :meth:`~iris.cube.Cube.collapse` to allow a 1-D weights array when + collapsing over a single dimension. + Previously, the weights had to be the same shape as the whole cube, which could cost a lot of memory in some cases. + The 1-D form is supported by most weighted array statistics (such as :meth:`np.average`), so this now works + with the corresponding Iris schemes (in that case, :const:`~iris.analysis.MEAN`). (:pull:`3943`) + + +🐛 Bugs Fixed +============= + +#. `@stephenworsley`_ fixed :meth:`~iris.cube.Cube.remove_coord` to now also + remove derived coordinates by removing aux_factories. (:pull:`3641`) + +#. `@jonseddon`_ fixed ``isinstance(cube, collections.Iterable)`` to now behave + as expected if a :class:`~iris.cube.Cube` is iterated over, while also + ensuring that ``TypeError`` is still raised. (Fixed by setting the + ``__iter__()`` method in :class:`~iris.cube.Cube` to ``None``). + (:pull:`3656`) + +#. `@stephenworsley`_ enabled cube concatenation along an axis shared by cell + measures; these cell measures are now concatenated together in the resulting + cube. Such a scenario would previously cause concatenation to inappropriately + fail. (:pull:`3566`) + +#. `@stephenworsley`_ newly included :class:`~iris.coords.CellMeasure`\ s in + :class:`~iris.cube.Cube` copy operations. Previously copying a + :class:`~iris.cube.Cube` would ignore any attached + :class:`~iris.coords.CellMeasure`. (:pull:`3546`) + +#. `@bjlittle`_ set a :class:`~iris.coords.CellMeasure`'s + ``measure`` attribute to have a default value of ``area``. + Previously, the ``measure`` was provided as a keyword argument to + :class:`~iris.coords.CellMeasure` with a default value of ``None``, which + caused a ``TypeError`` when no ``measure`` was provided, since ``area`` or + ``volume`` are the only accepted values. (:pull:`3533`) + +#. `@trexfeathers`_ set **all** plot types in :mod:`iris.plot` to now use + :obj:`matplotlib.dates.date2num` to format date/time coordinates for use on a plot + axis (previously :meth:`~iris.plot.pcolor` and :meth:`~iris.plot.pcolormesh` + did not include this behaviour). (:pull:`3762`) + +#. `@trexfeathers`_ changed date/time axis labels in :mod:`iris.quickplot` to + now **always** be based on the ``epoch`` used in :obj:`matplotlib.dates.date2num` + (previously would take the unit from a time coordinate, if present, even + though the coordinate's value had been changed via ``date2num``). + (:pull:`3762`) + +#. `@pp-mo`_ newly included attributes of cell measures in NETCDF-CF + file loading; they were previously being discarded. They are now available on + the :class:`~iris.coords.CellMeasure` in the loaded :class:`~iris.cube.Cube`. + (:pull:`3800`) + +#. `@pp-mo`_ fixed the netcdf loader to now handle any grid-mapping + variables with missing ``false_easting`` and ``false_northing`` properties, + which was previously failing for some coordinate systems. See :issue:`3629`. + (:pull:`3804`) + +#. `@stephenworsley`_ changed the way tick labels are assigned from string coords. + Previously, the first tick label would occasionally be duplicated. This also + removes the use of the deprecated `matplotlib`_ ``IndexFormatter``. (:pull:`3857`) + +#. `@znicholls`_ fixed :meth:`~iris.quickplot._title` to only check + ``units.is_time_reference`` if the ``units`` symbol is not used. (:pull:`3902`) + +#. `@rcomer`_ fixed a bug whereby numpy array type attributes on a cube's + coordinates could prevent printing it. See :issue:`3921`. (:pull:`3922`) + +.. _whatsnew 3.0.2 changes: + +💣 Incompatible Changes +======================= + +#. `@pp-mo`_ rationalised :class:`~iris.cube.CubeList` extraction + methods: + + The former method ``iris.cube.CubeList.extract_strict``, and the ``strict`` + keyword of the :meth:`~iris.cube.CubeList.extract` method have been removed, + and are replaced by the new routines :meth:`~iris.cube.CubeList.extract_cube` + and :meth:`~iris.cube.CubeList.extract_cubes`. + The new routines perform the same operation, but in a style more like other + ``Iris`` functions such as :meth:`~iris.load_cube` and :meth:`~iris.load_cubes`. + Unlike ``strict`` extraction, the type of return value is now completely + consistent : :meth:`~iris.cube.CubeList.extract_cube` always returns a + :class:`~iris.cube.Cube`, and :meth:`~iris.cube.CubeList.extract_cubes` + always returns an :class:`iris.cube.CubeList` of a length equal to the + number of constraints. (:pull:`3715`) + +#. `@pp-mo`_ removed the former function + ``iris.analysis.coord_comparison``. (:pull:`3562`) + +#. `@bjlittle`_ moved the + :func:`iris.experimental.equalise_cubes.equalise_attributes` function from + the :mod:`iris.experimental` module into the :mod:`iris.util` module. Please + use the :func:`iris.util.equalise_attributes` function instead. + (:pull:`3527`) + +#. `@bjlittle`_ removed the module ``iris.experimental.concatenate``. In + ``v1.6.0`` the experimental ``concatenate`` functionality was moved to the + :meth:`iris.cube.CubeList.concatenate` method. Since then, calling the + :func:`iris.experimental.concatenate.concatenate` function raised an + exception. (:pull:`3523`) + +#. `@stephenworsley`_ changed the default units of :class:`~iris.coords.DimCoord` + and :class:`~iris.coords.AuxCoord` from `"1"` to `"unknown"`. (:pull:`3795`) + +#. `@stephenworsley`_ changed Iris objects loaded from NetCDF-CF files to have + ``units='unknown'`` where the corresponding NetCDF variable has no ``units`` + property. Previously these cases defaulted to ``units='1'``. + This affects loading of coordinates whose file variable has no "units" + attribute (not valid, under `CF units rules`_): These will now have units + of `"unknown"`, rather than `"1"`, which **may prevent the creation of + a hybrid vertical coordinate**. While these cases used to "work", this was + never really correct behaviour. (:pull:`3795`) + +#. `@SimonPeatman`_ added attribute ``var_name`` to coordinates created by the + :func:`iris.analysis.trajectory.interpolate` function. This prevents + duplicate coordinate errors in certain circumstances. (:pull:`3718`) + +#. `@bjlittle`_ aligned the :func:`iris.analysis.maths.apply_ufunc` with the + rest of the :mod:`iris.analysis.maths` API by changing its keyword argument + from ``other_cube`` to ``other``. (:pull:`3785`) + +#. `@bjlittle`_ changed the :meth:`iris.analysis.maths.IFunc.__call__` to ignore + any surplus ``other`` keyword argument for a ``data_func`` that requires + **only one** argument. This aligns the behaviour of + :meth:`iris.analysis.maths.IFunc.__call__` with + :func:`~iris.analysis.maths.apply_ufunc`. Previously a ``ValueError`` + exception was raised. (:pull:`3785`) + + +.. _whatsnew 3.0.2 deprecations: + +🔥 Deprecations +=============== + +#. `@stephenworsley`_ removed the deprecated :class:`iris.Future` flags + ``cell_date_time_objects``, ``netcdf_promote``, ``netcdf_no_unlimited`` and + ``clip_latitudes``. (:pull:`3459`) + +#. `@stephenworsley`_ changed :attr:`iris.fileformats.pp.PPField.lbproc` to be an + ``int``. The deprecated attributes ``flag1``, ``flag2`` etc. have been + removed from it. (:pull:`3461`) + +#. `@bjlittle`_ deprecated :func:`~iris.util.as_compatible_shape` in preference + for :class:`~iris.common.resolve.Resolve` e.g., ``Resolve(src, tgt)(tgt.core_data())``. + The :func:`~iris.util.as_compatible_shape` function will be removed in a future + release of Iris. (:pull:`3892`) + + +🔗 Dependencies +=============== + +#. `@stephenworsley`_, `@trexfeathers`_ and `@bjlittle`_ removed ``Python2`` + support, modernising the codebase by switching to exclusive ``Python3`` + support. (:pull:`3513`) + +#. `@bjlittle`_ improved the developer set up process. Configuring Iris and + :ref:`installing_from_source` as a developer with all the required package + dependencies is now easier with our curated conda environment YAML files. + (:pull:`3812`) + +#. `@stephenworsley`_ pinned Iris to require `Dask`_ ``>=2.0``. (:pull:`3460`) + +#. `@stephenworsley`_ and `@trexfeathers`_ pinned Iris to require + `Cartopy`_ ``>=0.18``, in order to remain compatible with the latest version + of `matplotlib`_. (:pull:`3762`) + +#. `@bjlittle`_ unpinned Iris to use the latest version of `matplotlib`_. + Supporting ``Iris`` for both ``Python2`` and ``Python3`` had resulted in + pinning our dependency on `matplotlib`_ at ``v2.x``. But this is no longer + necessary now that ``Python2`` support has been dropped. (:pull:`3468`) + +#. `@stephenworsley`_ and `@trexfeathers`_ unpinned Iris to use the latest version + of `Proj`_. (:pull:`3762`) + +#. `@stephenworsley`_ and `@trexfeathers`_ removed GDAL from the extensions + dependency group. We no longer consider it to be an extension. (:pull:`3762`) + + +.. _whatsnew 3.0.2 docs: + +📚 Documentation +================ + +#. `@tkknight`_ moved the + :ref:`sphx_glr_generated_gallery_oceanography_plot_orca_projection.py` + from the general part of the gallery to oceanography. (:pull:`3761`) + +#. `@tkknight`_ updated documentation to use a modern sphinx theme and be + served from https://scitools-iris.readthedocs.io/en/latest/. (:pull:`3752`) + +#. `@bjlittle`_ added support for the `black`_ code formatter. This is + now automatically checked on GitHub PRs, replacing the older, unittest-based + ``iris.tests.test_coding_standards.TestCodeFormat``. Black provides automatic + code format correction for most IDEs. See the new developer guide section on + :ref:`code_formatting`. (:pull:`3518`) + +#. `@tkknight`_ and `@trexfeathers`_ refreshed the :ref:`whats_new_contributions` + for the :ref:`iris_whatsnew`. This includes always creating the ``latest`` + what's new page so it appears on the latest documentation at + https://scitools-iris.readthedocs.io/en/latest/whatsnew. This resolves + :issue:`2104`, :issue:`3451`, :issue:`3818`, :issue:`3837`. Also updated the + :ref:`iris_development_releases_steps` to follow when making a release. + (:pull:`3769`, :pull:`3838`, :pull:`3843`) + +#. `@tkknight`_ enabled the PDF creation of the documentation on the + `Read the Docs`_ service. The PDF may be accessed by clicking on the version + at the bottom of the side bar, then selecting ``PDF`` from the ``Downloads`` + section. (:pull:`3765`) + +#. `@stephenworsley`_ added a warning to the + :func:`iris.analysis.cartography.project` function regarding its behaviour on + projections with non-rectangular boundaries. (:pull:`3762`) + +#. `@stephenworsley`_ added the :ref:`cube_maths_combining_units` section to the + user guide to clarify how ``Units`` are handled during cube arithmetic. + (:pull:`3803`) + +#. `@tkknight`_ overhauled the :ref:`developers_guide` including information on + getting involved in becoming a contributor and general structure of the + guide. This resolves :issue:`2170`, :issue:`2331`, :issue:`3453`, + :issue:`314`, :issue:`2902`. (:pull:`3852`) + +#. `@rcomer`_ added argument descriptions to the :class:`~iris.coords.DimCoord` + docstring. (:pull:`3681`) + +#. `@tkknight`_ added two url's to be ignored for the ``make linkcheck``. This + will ensure the Iris github project is not repeatedly hit during the + linkcheck for issues and pull requests as it can result in connection + refused and thus travis-ci_ job failures. For more information on linkcheck, + see :ref:`contributing.documentation.testing`. (:pull:`3873`) + +#. `@tkknight`_ enabled the napolean_ package that is used by sphinx_ to cater + for the existing google style docstrings and to also allow for `numpy`_ + docstrings. This resolves :issue:`3841`. (:pull:`3871`) + +#. `@tkknight`_ configured ``sphinx-build`` to promote warnings to errors when + building the documentation via ``make html``. This will minimise technical + debt accruing for the documentation. (:pull:`3877`) + +#. `@tkknight`_ updated :ref:`installing_iris` to include a reference to + Windows Subsystem for Linux. (:pull:`3885`) + +#. `@tkknight`_ updated the :ref:`iris_docs` homepage to include panels so the + links are more visible to users. This uses the sphinx-panels_ extension. + (:pull:`3884`) + +#. `@bjlittle`_ created the :ref:`Further topics ` section and + included documentation for :ref:`metadata`, :ref:`lenient metadata`, and + :ref:`lenient maths`. (:pull:`3890`) + +#. `@jonseddon`_ updated the CF version of the netCDF saver in the + :ref:`saving_iris_cubes` section and in the equivalent function docstring. + (:pull:`3925`) + +#. `@bjlittle`_ applied `Title Case Capitalization`_ to the documentation. + (:pull:`3940`) + + +💼 Internal +=========== + +#. `@pp-mo`_ and `@lbdreyer`_ removed all Iris test dependencies on `iris-grib`_ + by transferring all relevant content to the `iris-grib`_ repository. (:pull:`3662`, + :pull:`3663`, :pull:`3664`, :pull:`3665`, :pull:`3666`, :pull:`3669`, + :pull:`3670`, :pull:`3671`, :pull:`3672`, :pull:`3742`, :pull:`3746`) + +#. `@lbdreyer`_ and `@pp-mo`_ overhauled the handling of dimensional + metadata to remove duplication. (:pull:`3422`, :pull:`3551`) + +#. `@trexfeathers`_ simplified the standard license header for all files, which + removes the need to repeatedly update year numbers in the header. + (:pull:`3489`) + +#. `@stephenworsley`_ changed the numerical values in tests involving the + Robinson projection due to improvements made in + `Proj`_. (:pull:`3762`) (see also `Proj#1292`_ and `Proj#2151`_) + +#. `@stephenworsley`_ changed tests to account for more detailed descriptions of + projections in `GDAL`_. (:pull:`3762`) (see also `GDAL#1185`_) + +#. `@stephenworsley`_ changed tests to account for `GDAL`_ now saving fill values + for data without masked points. (:pull:`3762`) + +#. `@trexfeathers`_ changed every graphics test that includes `Cartopy's coastlines`_ + to account for new adaptive coastline scaling. (:pull:`3762`) + (see also `Cartopy#1105`_) + +#. `@trexfeathers`_ changed graphics tests to account for some new default + grid-line spacing in `Cartopy`_. (:pull:`3762`) (see also `Cartopy#1117`_) + +#. `@trexfeathers`_ added additional acceptable graphics test targets to account + for very minor changes in `matplotlib`_ version ``3.3`` (colormaps, fonts and + axes borders). (:pull:`3762`) + +#. `@rcomer`_ corrected the `matplotlib`_ backend in Iris tests to ignore + :obj:`matplotlib.rcdefaults`, instead the tests will **always** use ``agg``. + (:pull:`3846`) + +#. `@bjlittle`_ migrated the `black`_ support from ``19.10b0`` to ``20.8b1``. + (:pull:`3866`) + +#. `@lbdreyer`_ updated the CF standard name table to the latest version: `v75`_. + (:pull:`3867`) + +#. `@bjlittle`_ added :pep:`517` and :pep:`518` support for building and + installing Iris, in particular to handle the `PyKE`_ package dependency. + (:pull:`3812`) + +#. `@bjlittle`_ added metadata support for comparing :attr:`~iris.cube.Cube.attributes` + dictionaries that contain `numpy`_ arrays using `xxHash`_, an extremely fast + non-cryptographic hash algorithm, running at RAM speed limits. + +#. `@bjlittle`_ added the ``iris.tests.assertDictEqual`` method to override + :meth:`unittest.TestCase.assertDictEqual` in order to cope with testing + metadata :attr:`~iris.cube.Cube.attributes` dictionary comparison where + the value of a key may be a `numpy`_ array. (:pull:`3785`) + +#. `@bjlittle`_ added the :func:`~iris.config.get_logger` function for creating + a generic :class:`logging.Logger` with a :class:`logging.StreamHandler` and + custom :class:`logging.Formatter`. (:pull:`3785`) + +#. `@owena11`_ identified and optimised a bottleneck in ``FieldsFile`` header + loading due to the use of :func:`numpy.fromfile`. (:pull:`3791`) + +#. `@znicholls`_ added a test for plotting with the label being taken from the unit's symbol, + see :meth:`~iris.tests.test_quickplot.TestLabels.test_pcolormesh_str_symbol` (:pull:`3902`). + +#. `@znicholls`_ made :func:`~iris.tests.idiff.step_over_diffs` robust to hyphens (``-``) in + the input path (i.e. the ``result_dir`` argument) (:pull:`3902`). + +#. `@bjlittle`_ migrated the CIaaS from `travis-ci`_ to `cirrus-ci`_, and removed `stickler-ci`_ + support. (:pull:`3928`) + +#. `@bjlittle`_ introduced `nox`_ as a common and easy entry-point for test automation. + It can be used both from `cirrus-ci`_ in the cloud, and locally by the developer to + run the Iris tests, the doc-tests, the gallery doc-tests, and lint Iris + with `flake8`_ and `black`_. (:pull:`3928`) + +.. _Read the Docs: https://scitools-iris.readthedocs.io/en/latest/ +.. _CF units rules: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#units +.. _CF Ancillary Data: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#ancillary-data +.. _Quality Flags: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#flags +.. _iris-grib: https://github.com/SciTools/iris-grib +.. _Cartopy: https://github.com/SciTools/cartopy +.. _Cartopy's coastlines: https://scitools.org.uk/cartopy/docs/latest/matplotlib/geoaxes.html?highlight=coastlines#cartopy.mpl.geoaxes.GeoAxes.coastlines +.. _Cartopy#1105: https://github.com/SciTools/cartopy/pull/1105 +.. _Cartopy#1117: https://github.com/SciTools/cartopy/pull/1117 +.. _Dask: https://github.com/dask/dask +.. _Proj: https://github.com/OSGeo/PROJ +.. _black: https://black.readthedocs.io/en/stable/ +.. _Proj#1292: https://github.com/OSGeo/PROJ/pull/1292 +.. _Proj#2151: https://github.com/OSGeo/PROJ/pull/2151 +.. _GDAL: https://github.com/OSGeo/gdal +.. _GDAL#1185: https://github.com/OSGeo/gdal/pull/1185 +.. _@MoseleyS: https://github.com/MoseleyS +.. _@stephenworsley: https://github.com/stephenworsley +.. _@pp-mo: https://github.com/pp-mo +.. _@abooton: https://github.com/abooton +.. _@bouweandela: https://github.com/bouweandela +.. _@bjlittle: https://github.com/bjlittle +.. _@trexfeathers: https://github.com/trexfeathers +.. _@jonseddon: https://github.com/jonseddon +.. _@tkknight: https://github.com/tkknight +.. _@lbdreyer: https://github.com/lbdreyer +.. _@SimonPeatman: https://github.com/SimonPeatman +.. _@TomekTrzeciak: https://github.com/TomekTrzeciak +.. _@rcomer: https://github.com/rcomer +.. _@jvegasbsc: https://github.com/jvegasbsc +.. _@zklaus: https://github.com/zklaus +.. _@znicholls: https://github.com/znicholls +.. _ESMValTool: https://github.com/ESMValGroup/ESMValTool +.. _v75: https://cfconventions.org/Data/cf-standard-names/75/build/cf-standard-name-table.html +.. _sphinx-panels: https://sphinx-panels.readthedocs.io/en/latest/ +.. _logging: https://docs.python.org/3/library/logging.html +.. _numpy: https://github.com/numpy/numpy +.. _xxHash: https://github.com/Cyan4973/xxHash +.. _PyKE: https://pypi.org/project/scitools-pyke/ +.. _@owena11: https://github.com/owena11 +.. _GitHub: https://github.com/SciTools/iris/issues/new/choose +.. _readthedocs: https://readthedocs.org/ +.. _CF Conventions and Metadata: https://cfconventions.org/ +.. _flake8: https://flake8.pycqa.org/en/stable/ +.. _nox: https://nox.thea.codes/en/stable/ +.. _Title Case Capitalization: https://apastyle.apa.org/style-grammar-guidelines/capitalization/title-case +.. _travis-ci: https://travis-ci.org/github/SciTools/iris +.. _stickler-ci: https://stickler-ci.com/ +.. _@jamesp: https://github.com/jamesp diff --git a/docs/src/whatsnew/3.0.3.rst b/docs/src/whatsnew/3.0.3.rst new file mode 100644 index 0000000000..412ebff6b6 --- /dev/null +++ b/docs/src/whatsnew/3.0.3.rst @@ -0,0 +1,586 @@ +.. include:: ../common_links.inc + +v3.0.3 (07 July 2021) +********************* + +This document explains the changes made to Iris for this release +(:doc:`View all changes `.) + + +.. dropdown:: :opticon:`alert` v3.0.3 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + :open: + + The patches included in this release include: + + 🐛 **Bugs Fixed** + + #. `@lbdreyer`_ modified :meth:`~iris.cube.Cube.intersection` to use a tolerant + equality check, when looking for cells that straddle the wrapping point. + (:pull:`4220`) + + +.. dropdown:: :opticon:`alert` v3.0.2 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The patches included in this release include: + + 🐛 **Bugs Fixed** + + #. `@jonseddon`_ handled a malformed ``um_stash_source`` CF variable attribute in + a netCDF file rather than raising a ``ValueError``. (:pull:`4035`) + + #. `@rcomer`_ fixed :meth:`~iris.cube.Cube.intersection` for special cases + where one cell's bounds align with the requested maximum and minimum, as + reported in :issue:`3391`. (:pull:`4059`) + + #. `@bjlittle`_ resolved a regression in arithmetic behaviour between a coordinate + and a cube which resulted in a ``NotYetImplementedError`` being raised, as reported + in :issue:`4000`. This fix supports ``+``, ``-``, ``*``, and ``/`` operations + between a coordinate and a cube, and for convenience additionally includes + :meth:`iris.cube.Cube.__neg__` support. (:pull:`4159`) + + 📚 **Documentation** + + #. `@bjlittle`_ updated the ``intersphinx_mapping`` and fixed documentation + to use ``stable`` URLs for `matplotlib`_. (:pull:`4003`) [``pre-v3.1.0``] + + 💼 **Internal** + + #. `@jamesp`_ updated a test to the latest numpy version (:pull:`3977`) [``pre-v3.1.0``] + + #. `@bjlittle`_ enabled `cirrus-ci`_ compute credits for non-draft pull-requests + from collaborators targeting the Iris ``master`` branch. (:pull:`4007`) + [``pre-v3.1.0``] + + #. `@bjlittle`_ added conditional task execution to `.cirrus.yml`_ to allow + developers to easily disable `cirrus-ci`_ tasks. See + :ref:`skipping Cirrus-CI tasks`. (:pull:`4019`) [``pre-v3.1.0``] + + #. `@pp-mo`_ adjusted the use of :func:`dask.array.from_array` in :func:`iris._lazy_data.as_lazy_data`, + to avoid the dask 'test access'. This makes loading of netcdf files with a + large number of variables significantly faster. (:pull:`4135`) + + #. `@pp-mo`_ reverted a change made previously in (:pull:`3659`) to + :meth:`iris.fileformats.pp.PPDataProxy.__getitem__`. The check for empty slicings + is no longer needed since (:pull:`4135`) was added. (:pull:`4141`) + + Note that, the above contributions labelled with ``pre-v3.1.0`` are part of the forthcoming + Iris v3.1.0 release, but require to be included in this patch release. + + +.. dropdown:: :opticon:`alert` v3.0.1 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The patches included in this release include: + + 💼 **Internal** + + #. `@bjlittle`_ gracefully promote formula terms within :mod:`~iris.aux_factory` that have ``units`` of ``unknown`` + to ``units`` of ``1`` (dimensionless), where the formula term **must** have dimensionless ``units``. Without this + graceful treatment of ``units`` the resulting :class:`~iris.cube.Cube` will **not** contain the expected auxiliary + factory, and the associated derived coordinate will be missing. (:pull:`3965`) + + +.. dropdown:: :opticon:`report` v3.0.0 Release Highlights + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The highlights for this major release of Iris include: + + * We've finally dropped support for ``Python 2``, so welcome to ``Iris 3`` + and ``Python 3``! + * We've extended our coverage of the `CF Conventions and Metadata`_ by + introducing support for `CF Ancillary Data`_ and `Quality Flags`_, + * Lazy regridding is now available for several regridding schemes, + * Managing and manipulating metadata within Iris is now easier and more + consistent thanks to the introduction of a new common metadata API, + * :ref:`Cube arithmetic ` has been significantly improved with + regards to extended broadcasting, auto-transposition and a more lenient + behaviour towards handling metadata and coordinates, + * Our :ref:`documentation ` has been refreshed, + restructured, revitalised and rehosted on `readthedocs`_, + * It's now easier than ever to :ref:`install Iris ` + as a user or a developer, and the newly revamped developers guide walks + you though how you can :ref:`get involved ` + and contribute to Iris, + * Also, this is a major release of Iris, so please be aware of the + :ref:`incompatible changes ` and + :ref:`deprecations `. + + And finally, get in touch with us on `GitHub`_ if you have any issues or + feature requests for improving Iris. Enjoy! + + +📢 Announcements +================ + +#. Congratulations to `@bouweandela`_, `@jvegasbsc`_, and `@zklaus`_ who + recently became Iris core developers. They bring a wealth of expertise to the + team, and are using Iris to underpin `ESMValTool`_ - "*A community diagnostic + and performance metrics tool for routine evaluation of Earth system models + in CMIP*". Welcome aboard! 🎉 + +#. Congratulations also goes to `@jonseddon`_ who recently became an Iris core + developer. We look forward to seeing more of your awesome contributions! 🎉 + + +✨ Features +=========== + +#. `@MoseleyS`_ greatly enhanced the :mod:`~iris.fileformats.nimrod` + module to provide richer meta-data translation when loading ``Nimrod`` data + into cubes. This covers most known operational use-cases. (:pull:`3647`) + +#. `@stephenworsley`_ improved the handling of + :class:`iris.coords.CellMeasure`\ s in the :class:`~iris.cube.Cube` + statistical operations :meth:`~iris.cube.Cube.collapsed`, + :meth:`~iris.cube.Cube.aggregated_by` and + :meth:`~iris.cube.Cube.rolling_window`. These previously removed every + :class:`~iris.coords.CellMeasure` attached to the cube. Now, a + :class:`~iris.coords.CellMeasure` will only be removed if it is associated + with an axis over which the statistic is being run. (:pull:`3549`) + +#. `@stephenworsley`_, `@pp-mo`_ and `@abooton`_ added support for + `CF Ancillary Data`_ variables. These are created as + :class:`iris.coords.AncillaryVariable`, and appear as components of cubes + much like :class:`~iris.coords.AuxCoord`\ s, with the new + :class:`~iris.cube.Cube` methods + :meth:`~iris.cube.Cube.add_ancillary_variable`, + :meth:`~iris.cube.Cube.remove_ancillary_variable`, + :meth:`~iris.cube.Cube.ancillary_variable`, + :meth:`~iris.cube.Cube.ancillary_variables` and + :meth:`~iris.cube.Cube.ancillary_variable_dims`. + They are loaded from and saved to NetCDF-CF files. Special support for + `Quality Flags`_ is also provided, to ensure they load and save with + appropriate units. (:pull:`3800`) + +#. `@bouweandela`_ implemented lazy regridding for the + :class:`~iris.analysis.Linear`, :class:`~iris.analysis.Nearest`, and + :class:`~iris.analysis.AreaWeighted` regridding schemes. (:pull:`3701`) + +#. `@bjlittle`_ added `logging`_ support within :mod:`iris.analysis.maths`, + :mod:`iris.common.metadata`, and :mod:`iris.common.resolve`. Each module + defines a :class:`logging.Logger` instance called ``logger`` with a default + ``level`` of ``INFO``. To enable ``DEBUG`` logging use + ``logger.setLevel("DEBUG")``. (:pull:`3785`) + +#. `@bjlittle`_ added the :mod:`iris.common.resolve` module, which provides + infrastructure to support the analysis, identification and combination + of metadata common between two :class:`~iris.cube.Cube` operands into a + single resultant :class:`~iris.cube.Cube` that will be auto-transposed, + and with the appropriate broadcast shape. (:pull:`3785`) + +#. `@bjlittle`_ added the :ref:`common metadata API `, which provides + a unified treatment of metadata across Iris, and allows users to easily + manage and manipulate their metadata in a consistent way. (:pull:`3785`) + +#. `@bjlittle`_ added :ref:`lenient metadata ` support, to + allow users to control **strict** or **lenient** metadata equivalence, + difference and combination. (:pull:`3785`) + +#. `@bjlittle`_ added :ref:`lenient cube maths ` support and + resolved several long standing major issues with cube arithmetic regarding + a more robust treatment of cube broadcasting, cube dimension auto-transposition, + and preservation of common metadata and coordinates during cube math operations. + Resolves :issue:`1887`, :issue:`2765`, and :issue:`3478`. (:pull:`3785`) + +#. `@pp-mo`_ and `@TomekTrzeciak`_ enhanced :meth:`~iris.cube.Cube.collapse` to allow a 1-D weights array when + collapsing over a single dimension. + Previously, the weights had to be the same shape as the whole cube, which could cost a lot of memory in some cases. + The 1-D form is supported by most weighted array statistics (such as :meth:`np.average`), so this now works + with the corresponding Iris schemes (in that case, :const:`~iris.analysis.MEAN`). (:pull:`3943`) + + +🐛 Bugs Fixed +============= + +#. `@stephenworsley`_ fixed :meth:`~iris.cube.Cube.remove_coord` to now also + remove derived coordinates by removing aux_factories. (:pull:`3641`) + +#. `@jonseddon`_ fixed ``isinstance(cube, collections.Iterable)`` to now behave + as expected if a :class:`~iris.cube.Cube` is iterated over, while also + ensuring that ``TypeError`` is still raised. (Fixed by setting the + ``__iter__()`` method in :class:`~iris.cube.Cube` to ``None``). + (:pull:`3656`) + +#. `@stephenworsley`_ enabled cube concatenation along an axis shared by cell + measures; these cell measures are now concatenated together in the resulting + cube. Such a scenario would previously cause concatenation to inappropriately + fail. (:pull:`3566`) + +#. `@stephenworsley`_ newly included :class:`~iris.coords.CellMeasure`\ s in + :class:`~iris.cube.Cube` copy operations. Previously copying a + :class:`~iris.cube.Cube` would ignore any attached + :class:`~iris.coords.CellMeasure`. (:pull:`3546`) + +#. `@bjlittle`_ set a :class:`~iris.coords.CellMeasure`'s + ``measure`` attribute to have a default value of ``area``. + Previously, the ``measure`` was provided as a keyword argument to + :class:`~iris.coords.CellMeasure` with a default value of ``None``, which + caused a ``TypeError`` when no ``measure`` was provided, since ``area`` or + ``volume`` are the only accepted values. (:pull:`3533`) + +#. `@trexfeathers`_ set **all** plot types in :mod:`iris.plot` to now use + :obj:`matplotlib.dates.date2num` to format date/time coordinates for use on a plot + axis (previously :meth:`~iris.plot.pcolor` and :meth:`~iris.plot.pcolormesh` + did not include this behaviour). (:pull:`3762`) + +#. `@trexfeathers`_ changed date/time axis labels in :mod:`iris.quickplot` to + now **always** be based on the ``epoch`` used in :obj:`matplotlib.dates.date2num` + (previously would take the unit from a time coordinate, if present, even + though the coordinate's value had been changed via ``date2num``). + (:pull:`3762`) + +#. `@pp-mo`_ newly included attributes of cell measures in NETCDF-CF + file loading; they were previously being discarded. They are now available on + the :class:`~iris.coords.CellMeasure` in the loaded :class:`~iris.cube.Cube`. + (:pull:`3800`) + +#. `@pp-mo`_ fixed the netcdf loader to now handle any grid-mapping + variables with missing ``false_easting`` and ``false_northing`` properties, + which was previously failing for some coordinate systems. See :issue:`3629`. + (:pull:`3804`) + +#. `@stephenworsley`_ changed the way tick labels are assigned from string coords. + Previously, the first tick label would occasionally be duplicated. This also + removes the use of the deprecated `matplotlib`_ ``IndexFormatter``. (:pull:`3857`) + +#. `@znicholls`_ fixed :meth:`~iris.quickplot._title` to only check + ``units.is_time_reference`` if the ``units`` symbol is not used. (:pull:`3902`) + +#. `@rcomer`_ fixed a bug whereby numpy array type attributes on a cube's + coordinates could prevent printing it. See :issue:`3921`. (:pull:`3922`) + +.. _whatsnew 3.0.3 changes: + +💣 Incompatible Changes +======================= + +#. `@pp-mo`_ rationalised :class:`~iris.cube.CubeList` extraction + methods: + + The former method ``iris.cube.CubeList.extract_strict``, and the ``strict`` + keyword of the :meth:`~iris.cube.CubeList.extract` method have been removed, + and are replaced by the new routines :meth:`~iris.cube.CubeList.extract_cube` + and :meth:`~iris.cube.CubeList.extract_cubes`. + The new routines perform the same operation, but in a style more like other + ``Iris`` functions such as :meth:`~iris.load_cube` and :meth:`~iris.load_cubes`. + Unlike ``strict`` extraction, the type of return value is now completely + consistent : :meth:`~iris.cube.CubeList.extract_cube` always returns a + :class:`~iris.cube.Cube`, and :meth:`~iris.cube.CubeList.extract_cubes` + always returns an :class:`iris.cube.CubeList` of a length equal to the + number of constraints. (:pull:`3715`) + +#. `@pp-mo`_ removed the former function + ``iris.analysis.coord_comparison``. (:pull:`3562`) + +#. `@bjlittle`_ moved the + :func:`iris.experimental.equalise_cubes.equalise_attributes` function from + the :mod:`iris.experimental` module into the :mod:`iris.util` module. Please + use the :func:`iris.util.equalise_attributes` function instead. + (:pull:`3527`) + +#. `@bjlittle`_ removed the module ``iris.experimental.concatenate``. In + ``v1.6.0`` the experimental ``concatenate`` functionality was moved to the + :meth:`iris.cube.CubeList.concatenate` method. Since then, calling the + :func:`iris.experimental.concatenate.concatenate` function raised an + exception. (:pull:`3523`) + +#. `@stephenworsley`_ changed the default units of :class:`~iris.coords.DimCoord` + and :class:`~iris.coords.AuxCoord` from `"1"` to `"unknown"`. (:pull:`3795`) + +#. `@stephenworsley`_ changed Iris objects loaded from NetCDF-CF files to have + ``units='unknown'`` where the corresponding NetCDF variable has no ``units`` + property. Previously these cases defaulted to ``units='1'``. + This affects loading of coordinates whose file variable has no "units" + attribute (not valid, under `CF units rules`_): These will now have units + of `"unknown"`, rather than `"1"`, which **may prevent the creation of + a hybrid vertical coordinate**. While these cases used to "work", this was + never really correct behaviour. (:pull:`3795`) + +#. `@SimonPeatman`_ added attribute ``var_name`` to coordinates created by the + :func:`iris.analysis.trajectory.interpolate` function. This prevents + duplicate coordinate errors in certain circumstances. (:pull:`3718`) + +#. `@bjlittle`_ aligned the :func:`iris.analysis.maths.apply_ufunc` with the + rest of the :mod:`iris.analysis.maths` API by changing its keyword argument + from ``other_cube`` to ``other``. (:pull:`3785`) + +#. `@bjlittle`_ changed the :meth:`iris.analysis.maths.IFunc.__call__` to ignore + any surplus ``other`` keyword argument for a ``data_func`` that requires + **only one** argument. This aligns the behaviour of + :meth:`iris.analysis.maths.IFunc.__call__` with + :func:`~iris.analysis.maths.apply_ufunc`. Previously a ``ValueError`` + exception was raised. (:pull:`3785`) + + +.. _whatsnew 3.0.3 deprecations: + +🔥 Deprecations +=============== + +#. `@stephenworsley`_ removed the deprecated :class:`iris.Future` flags + ``cell_date_time_objects``, ``netcdf_promote``, ``netcdf_no_unlimited`` and + ``clip_latitudes``. (:pull:`3459`) + +#. `@stephenworsley`_ changed :attr:`iris.fileformats.pp.PPField.lbproc` to be an + ``int``. The deprecated attributes ``flag1``, ``flag2`` etc. have been + removed from it. (:pull:`3461`) + +#. `@bjlittle`_ deprecated :func:`~iris.util.as_compatible_shape` in preference + for :class:`~iris.common.resolve.Resolve` e.g., ``Resolve(src, tgt)(tgt.core_data())``. + The :func:`~iris.util.as_compatible_shape` function will be removed in a future + release of Iris. (:pull:`3892`) + + +🔗 Dependencies +=============== + +#. `@stephenworsley`_, `@trexfeathers`_ and `@bjlittle`_ removed ``Python2`` + support, modernising the codebase by switching to exclusive ``Python3`` + support. (:pull:`3513`) + +#. `@bjlittle`_ improved the developer set up process. Configuring Iris and + :ref:`installing_from_source` as a developer with all the required package + dependencies is now easier with our curated conda environment YAML files. + (:pull:`3812`) + +#. `@stephenworsley`_ pinned Iris to require `Dask`_ ``>=2.0``. (:pull:`3460`) + +#. `@stephenworsley`_ and `@trexfeathers`_ pinned Iris to require + `Cartopy`_ ``>=0.18``, in order to remain compatible with the latest version + of `matplotlib`_. (:pull:`3762`) + +#. `@bjlittle`_ unpinned Iris to use the latest version of `matplotlib`_. + Supporting ``Iris`` for both ``Python2`` and ``Python3`` had resulted in + pinning our dependency on `matplotlib`_ at ``v2.x``. But this is no longer + necessary now that ``Python2`` support has been dropped. (:pull:`3468`) + +#. `@stephenworsley`_ and `@trexfeathers`_ unpinned Iris to use the latest version + of `Proj`_. (:pull:`3762`) + +#. `@stephenworsley`_ and `@trexfeathers`_ removed GDAL from the extensions + dependency group. We no longer consider it to be an extension. (:pull:`3762`) + + +.. _whatsnew 3.0.3 docs: + +📚 Documentation +================ + +#. `@tkknight`_ moved the + :ref:`sphx_glr_generated_gallery_oceanography_plot_orca_projection.py` + from the general part of the gallery to oceanography. (:pull:`3761`) + +#. `@tkknight`_ updated documentation to use a modern sphinx theme and be + served from https://scitools-iris.readthedocs.io/en/latest/. (:pull:`3752`) + +#. `@bjlittle`_ added support for the `black`_ code formatter. This is + now automatically checked on GitHub PRs, replacing the older, unittest-based + ``iris.tests.test_coding_standards.TestCodeFormat``. Black provides automatic + code format correction for most IDEs. See the new developer guide section on + :ref:`code_formatting`. (:pull:`3518`) + +#. `@tkknight`_ and `@trexfeathers`_ refreshed the :ref:`whats_new_contributions` + for the :ref:`iris_whatsnew`. This includes always creating the ``latest`` + what's new page so it appears on the latest documentation at + https://scitools-iris.readthedocs.io/en/latest/whatsnew. This resolves + :issue:`2104`, :issue:`3451`, :issue:`3818`, :issue:`3837`. Also updated the + :ref:`iris_development_releases_steps` to follow when making a release. + (:pull:`3769`, :pull:`3838`, :pull:`3843`) + +#. `@tkknight`_ enabled the PDF creation of the documentation on the + `Read the Docs`_ service. The PDF may be accessed by clicking on the version + at the bottom of the side bar, then selecting ``PDF`` from the ``Downloads`` + section. (:pull:`3765`) + +#. `@stephenworsley`_ added a warning to the + :func:`iris.analysis.cartography.project` function regarding its behaviour on + projections with non-rectangular boundaries. (:pull:`3762`) + +#. `@stephenworsley`_ added the :ref:`cube_maths_combining_units` section to the + user guide to clarify how ``Units`` are handled during cube arithmetic. + (:pull:`3803`) + +#. `@tkknight`_ overhauled the :ref:`developers_guide` including information on + getting involved in becoming a contributor and general structure of the + guide. This resolves :issue:`2170`, :issue:`2331`, :issue:`3453`, + :issue:`314`, :issue:`2902`. (:pull:`3852`) + +#. `@rcomer`_ added argument descriptions to the :class:`~iris.coords.DimCoord` + docstring. (:pull:`3681`) + +#. `@tkknight`_ added two url's to be ignored for the ``make linkcheck``. This + will ensure the Iris github project is not repeatedly hit during the + linkcheck for issues and pull requests as it can result in connection + refused and thus travis-ci_ job failures. For more information on linkcheck, + see :ref:`contributing.documentation.testing`. (:pull:`3873`) + +#. `@tkknight`_ enabled the napolean_ package that is used by sphinx_ to cater + for the existing google style docstrings and to also allow for `numpy`_ + docstrings. This resolves :issue:`3841`. (:pull:`3871`) + +#. `@tkknight`_ configured ``sphinx-build`` to promote warnings to errors when + building the documentation via ``make html``. This will minimise technical + debt accruing for the documentation. (:pull:`3877`) + +#. `@tkknight`_ updated :ref:`installing_iris` to include a reference to + Windows Subsystem for Linux. (:pull:`3885`) + +#. `@tkknight`_ updated the :ref:`iris_docs` homepage to include panels so the + links are more visible to users. This uses the sphinx-panels_ extension. + (:pull:`3884`) + +#. `@bjlittle`_ created the :ref:`Further topics ` section and + included documentation for :ref:`metadata`, :ref:`lenient metadata`, and + :ref:`lenient maths`. (:pull:`3890`) + +#. `@jonseddon`_ updated the CF version of the netCDF saver in the + :ref:`saving_iris_cubes` section and in the equivalent function docstring. + (:pull:`3925`) + +#. `@bjlittle`_ applied `Title Case Capitalization`_ to the documentation. + (:pull:`3940`) + + +💼 Internal +=========== + +#. `@pp-mo`_ and `@lbdreyer`_ removed all Iris test dependencies on `iris-grib`_ + by transferring all relevant content to the `iris-grib`_ repository. (:pull:`3662`, + :pull:`3663`, :pull:`3664`, :pull:`3665`, :pull:`3666`, :pull:`3669`, + :pull:`3670`, :pull:`3671`, :pull:`3672`, :pull:`3742`, :pull:`3746`) + +#. `@lbdreyer`_ and `@pp-mo`_ overhauled the handling of dimensional + metadata to remove duplication. (:pull:`3422`, :pull:`3551`) + +#. `@trexfeathers`_ simplified the standard license header for all files, which + removes the need to repeatedly update year numbers in the header. + (:pull:`3489`) + +#. `@stephenworsley`_ changed the numerical values in tests involving the + Robinson projection due to improvements made in + `Proj`_. (:pull:`3762`) (see also `Proj#1292`_ and `Proj#2151`_) + +#. `@stephenworsley`_ changed tests to account for more detailed descriptions of + projections in `GDAL`_. (:pull:`3762`) (see also `GDAL#1185`_) + +#. `@stephenworsley`_ changed tests to account for `GDAL`_ now saving fill values + for data without masked points. (:pull:`3762`) + +#. `@trexfeathers`_ changed every graphics test that includes `Cartopy's coastlines`_ + to account for new adaptive coastline scaling. (:pull:`3762`) + (see also `Cartopy#1105`_) + +#. `@trexfeathers`_ changed graphics tests to account for some new default + grid-line spacing in `Cartopy`_. (:pull:`3762`) (see also `Cartopy#1117`_) + +#. `@trexfeathers`_ added additional acceptable graphics test targets to account + for very minor changes in `matplotlib`_ version ``3.3`` (colormaps, fonts and + axes borders). (:pull:`3762`) + +#. `@rcomer`_ corrected the `matplotlib`_ backend in Iris tests to ignore + :obj:`matplotlib.rcdefaults`, instead the tests will **always** use ``agg``. + (:pull:`3846`) + +#. `@bjlittle`_ migrated the `black`_ support from ``19.10b0`` to ``20.8b1``. + (:pull:`3866`) + +#. `@lbdreyer`_ updated the CF standard name table to the latest version: `v75`_. + (:pull:`3867`) + +#. `@bjlittle`_ added :pep:`517` and :pep:`518` support for building and + installing Iris, in particular to handle the `PyKE`_ package dependency. + (:pull:`3812`) + +#. `@bjlittle`_ added metadata support for comparing :attr:`~iris.cube.Cube.attributes` + dictionaries that contain `numpy`_ arrays using `xxHash`_, an extremely fast + non-cryptographic hash algorithm, running at RAM speed limits. + +#. `@bjlittle`_ added the ``iris.tests.assertDictEqual`` method to override + :meth:`unittest.TestCase.assertDictEqual` in order to cope with testing + metadata :attr:`~iris.cube.Cube.attributes` dictionary comparison where + the value of a key may be a `numpy`_ array. (:pull:`3785`) + +#. `@bjlittle`_ added the :func:`~iris.config.get_logger` function for creating + a generic :class:`logging.Logger` with a :class:`logging.StreamHandler` and + custom :class:`logging.Formatter`. (:pull:`3785`) + +#. `@owena11`_ identified and optimised a bottleneck in ``FieldsFile`` header + loading due to the use of :func:`numpy.fromfile`. (:pull:`3791`) + +#. `@znicholls`_ added a test for plotting with the label being taken from the unit's symbol, + see :meth:`~iris.tests.test_quickplot.TestLabels.test_pcolormesh_str_symbol` (:pull:`3902`). + +#. `@znicholls`_ made :func:`~iris.tests.idiff.step_over_diffs` robust to hyphens (``-``) in + the input path (i.e. the ``result_dir`` argument) (:pull:`3902`). + +#. `@bjlittle`_ migrated the CIaaS from `travis-ci`_ to `cirrus-ci`_, and removed `stickler-ci`_ + support. (:pull:`3928`) + +#. `@bjlittle`_ introduced `nox`_ as a common and easy entry-point for test automation. + It can be used both from `cirrus-ci`_ in the cloud, and locally by the developer to + run the Iris tests, the doc-tests, the gallery doc-tests, and lint Iris + with `flake8`_ and `black`_. (:pull:`3928`) + +.. _Read the Docs: https://scitools-iris.readthedocs.io/en/latest/ +.. _CF units rules: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#units +.. _CF Ancillary Data: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#ancillary-data +.. _Quality Flags: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#flags +.. _iris-grib: https://github.com/SciTools/iris-grib +.. _Cartopy: https://github.com/SciTools/cartopy +.. _Cartopy's coastlines: https://scitools.org.uk/cartopy/docs/latest/matplotlib/geoaxes.html?highlight=coastlines#cartopy.mpl.geoaxes.GeoAxes.coastlines +.. _Cartopy#1105: https://github.com/SciTools/cartopy/pull/1105 +.. _Cartopy#1117: https://github.com/SciTools/cartopy/pull/1117 +.. _Dask: https://github.com/dask/dask +.. _Proj: https://github.com/OSGeo/PROJ +.. _black: https://black.readthedocs.io/en/stable/ +.. _Proj#1292: https://github.com/OSGeo/PROJ/pull/1292 +.. _Proj#2151: https://github.com/OSGeo/PROJ/pull/2151 +.. _GDAL: https://github.com/OSGeo/gdal +.. _GDAL#1185: https://github.com/OSGeo/gdal/pull/1185 +.. _@MoseleyS: https://github.com/MoseleyS +.. _@stephenworsley: https://github.com/stephenworsley +.. _@pp-mo: https://github.com/pp-mo +.. _@abooton: https://github.com/abooton +.. _@bouweandela: https://github.com/bouweandela +.. _@bjlittle: https://github.com/bjlittle +.. _@trexfeathers: https://github.com/trexfeathers +.. _@jonseddon: https://github.com/jonseddon +.. _@tkknight: https://github.com/tkknight +.. _@lbdreyer: https://github.com/lbdreyer +.. _@SimonPeatman: https://github.com/SimonPeatman +.. _@TomekTrzeciak: https://github.com/TomekTrzeciak +.. _@rcomer: https://github.com/rcomer +.. _@jvegasbsc: https://github.com/jvegasbsc +.. _@zklaus: https://github.com/zklaus +.. _@znicholls: https://github.com/znicholls +.. _ESMValTool: https://github.com/ESMValGroup/ESMValTool +.. _v75: https://cfconventions.org/Data/cf-standard-names/75/build/cf-standard-name-table.html +.. _sphinx-panels: https://sphinx-panels.readthedocs.io/en/latest/ +.. _logging: https://docs.python.org/3/library/logging.html +.. _numpy: https://github.com/numpy/numpy +.. _xxHash: https://github.com/Cyan4973/xxHash +.. _PyKE: https://pypi.org/project/scitools-pyke/ +.. _@owena11: https://github.com/owena11 +.. _GitHub: https://github.com/SciTools/iris/issues/new/choose +.. _readthedocs: https://readthedocs.org/ +.. _CF Conventions and Metadata: https://cfconventions.org/ +.. _flake8: https://flake8.pycqa.org/en/stable/ +.. _nox: https://nox.thea.codes/en/stable/ +.. _Title Case Capitalization: https://apastyle.apa.org/style-grammar-guidelines/capitalization/title-case +.. _travis-ci: https://travis-ci.org/github/SciTools/iris +.. _stickler-ci: https://stickler-ci.com/ +.. _@jamesp: https://github.com/jamesp diff --git a/docs/src/whatsnew/3.0.4.rst b/docs/src/whatsnew/3.0.4.rst new file mode 100644 index 0000000000..de0cec8315 --- /dev/null +++ b/docs/src/whatsnew/3.0.4.rst @@ -0,0 +1,620 @@ +.. include:: ../common_links.inc + +v3.0.4 (22 July 2021) +********************* + +This document explains the changes made to Iris for this release +(:doc:`View all changes `.) + + +.. dropdown:: :opticon:`alert` v3.0.4 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + :open: + + The patches included in this release include: + + 🐛 **Bugs Fixed** + + #. `@pp-mo`_ fixed 2 bugs in cube printout: + Firstly, ancillary-variables or cell-measures with long names can now widen the cube "dimensions map" to fit, + whereas previously printing these cases caused an Exception. + Secondly, cube units are now always printed, whereas previously they were missed out any time that the + "dimensions map" was widened to accomodate long coordinate names. + (:pull:`4233`)(:pull:`4238`) + + 💼 **Internal** + + #. `@bjlittle`_ Unpinned the `cftime`_ package dependency within Iris in order + to allow use of the latest versions of `cftime`_, `cf-units`_ and `nc-time-axis`_. + (:pull:`4222`) + + #. `@rcomer`_ modified test modules so they run consistently under ``pytest`` and + ``nose``, and also fixed some minor issues with :class:`~iris.time.PartialDateTime`. + (:pull:`4249`) + + Note that, we are forced to drop support for ``Python 3.6`` in this patch due to + the third-party package dependencies required by (:pull:`4222`). + + +.. dropdown:: :opticon:`alert` v3.0.3 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The patches included in this release include: + + 🐛 **Bugs Fixed** + + #. `@lbdreyer`_ modified :meth:`~iris.cube.Cube.intersection` to use a tolerant + equality check, when looking for cells that straddle the wrapping point. + (:pull:`4220`) + + +.. dropdown:: :opticon:`alert` v3.0.2 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The patches included in this release include: + + 🐛 **Bugs Fixed** + + #. `@jonseddon`_ handled a malformed ``um_stash_source`` CF variable attribute in + a netCDF file rather than raising a ``ValueError``. (:pull:`4035`) + + #. `@rcomer`_ fixed :meth:`~iris.cube.Cube.intersection` for special cases + where one cell's bounds align with the requested maximum and minimum, as + reported in :issue:`3391`. (:pull:`4059`) + + #. `@bjlittle`_ resolved a regression in arithmetic behaviour between a coordinate + and a cube which resulted in a ``NotYetImplementedError`` being raised, as reported + in :issue:`4000`. This fix supports ``+``, ``-``, ``*``, and ``/`` operations + between a coordinate and a cube, and for convenience additionally includes + :meth:`iris.cube.Cube.__neg__` support. (:pull:`4159`) + + 📚 **Documentation** + + #. `@bjlittle`_ updated the ``intersphinx_mapping`` and fixed documentation + to use ``stable`` URLs for `matplotlib`_. (:pull:`4003`) [``pre-v3.1.0``] + + 💼 **Internal** + + #. `@jamesp`_ updated a test to the latest numpy version (:pull:`3977`) [``pre-v3.1.0``] + + #. `@bjlittle`_ enabled `cirrus-ci`_ compute credits for non-draft pull-requests + from collaborators targeting the Iris ``master`` branch. (:pull:`4007`) + [``pre-v3.1.0``] + + #. `@bjlittle`_ added conditional task execution to `.cirrus.yml`_ to allow + developers to easily disable `cirrus-ci`_ tasks. See + :ref:`skipping Cirrus-CI tasks`. (:pull:`4019`) [``pre-v3.1.0``] + + #. `@pp-mo`_ adjusted the use of :func:`dask.array.from_array` in :func:`iris._lazy_data.as_lazy_data`, + to avoid the dask 'test access'. This makes loading of netcdf files with a + large number of variables significantly faster. (:pull:`4135`) + + #. `@pp-mo`_ reverted a change made previously in (:pull:`3659`) to + :meth:`iris.fileformats.pp.PPDataProxy.__getitem__`. The check for empty slicings + is no longer needed since (:pull:`4135`) was added. (:pull:`4141`) + + Note that, the above contributions labelled with ``pre-v3.1.0`` are part of the forthcoming + Iris v3.1.0 release, but require to be included in this patch release. + + +.. dropdown:: :opticon:`alert` v3.0.1 Patches + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The patches included in this release include: + + 💼 **Internal** + + #. `@bjlittle`_ gracefully promote formula terms within :mod:`~iris.aux_factory` that have ``units`` of ``unknown`` + to ``units`` of ``1`` (dimensionless), where the formula term **must** have dimensionless ``units``. Without this + graceful treatment of ``units`` the resulting :class:`~iris.cube.Cube` will **not** contain the expected auxiliary + factory, and the associated derived coordinate will be missing. (:pull:`3965`) + + +.. dropdown:: :opticon:`report` v3.0.0 Release Highlights + :container: + shadow + :title: text-primary text-center font-weight-bold + :body: bg-light + :animate: fade-in + + The highlights for this major release of Iris include: + + * We've finally dropped support for ``Python 2``, so welcome to ``Iris 3`` + and ``Python 3``! + * We've extended our coverage of the `CF Conventions and Metadata`_ by + introducing support for `CF Ancillary Data`_ and `Quality Flags`_, + * Lazy regridding is now available for several regridding schemes, + * Managing and manipulating metadata within Iris is now easier and more + consistent thanks to the introduction of a new common metadata API, + * :ref:`Cube arithmetic ` has been significantly improved with + regards to extended broadcasting, auto-transposition and a more lenient + behaviour towards handling metadata and coordinates, + * Our :ref:`documentation ` has been refreshed, + restructured, revitalised and rehosted on `readthedocs`_, + * It's now easier than ever to :ref:`install Iris ` + as a user or a developer, and the newly revamped developers guide walks + you though how you can :ref:`get involved ` + and contribute to Iris, + * Also, this is a major release of Iris, so please be aware of the + :ref:`incompatible changes ` and + :ref:`deprecations `. + + And finally, get in touch with us on `GitHub`_ if you have any issues or + feature requests for improving Iris. Enjoy! + + +📢 Announcements +================ + +#. Congratulations to `@bouweandela`_, `@jvegasbsc`_, and `@zklaus`_ who + recently became Iris core developers. They bring a wealth of expertise to the + team, and are using Iris to underpin `ESMValTool`_ - "*A community diagnostic + and performance metrics tool for routine evaluation of Earth system models + in CMIP*". Welcome aboard! 🎉 + +#. Congratulations also goes to `@jonseddon`_ who recently became an Iris core + developer. We look forward to seeing more of your awesome contributions! 🎉 + + +✨ Features +=========== + +#. `@MoseleyS`_ greatly enhanced the :mod:`~iris.fileformats.nimrod` + module to provide richer meta-data translation when loading ``Nimrod`` data + into cubes. This covers most known operational use-cases. (:pull:`3647`) + +#. `@stephenworsley`_ improved the handling of + :class:`iris.coords.CellMeasure`\ s in the :class:`~iris.cube.Cube` + statistical operations :meth:`~iris.cube.Cube.collapsed`, + :meth:`~iris.cube.Cube.aggregated_by` and + :meth:`~iris.cube.Cube.rolling_window`. These previously removed every + :class:`~iris.coords.CellMeasure` attached to the cube. Now, a + :class:`~iris.coords.CellMeasure` will only be removed if it is associated + with an axis over which the statistic is being run. (:pull:`3549`) + +#. `@stephenworsley`_, `@pp-mo`_ and `@abooton`_ added support for + `CF Ancillary Data`_ variables. These are created as + :class:`iris.coords.AncillaryVariable`, and appear as components of cubes + much like :class:`~iris.coords.AuxCoord`\ s, with the new + :class:`~iris.cube.Cube` methods + :meth:`~iris.cube.Cube.add_ancillary_variable`, + :meth:`~iris.cube.Cube.remove_ancillary_variable`, + :meth:`~iris.cube.Cube.ancillary_variable`, + :meth:`~iris.cube.Cube.ancillary_variables` and + :meth:`~iris.cube.Cube.ancillary_variable_dims`. + They are loaded from and saved to NetCDF-CF files. Special support for + `Quality Flags`_ is also provided, to ensure they load and save with + appropriate units. (:pull:`3800`) + +#. `@bouweandela`_ implemented lazy regridding for the + :class:`~iris.analysis.Linear`, :class:`~iris.analysis.Nearest`, and + :class:`~iris.analysis.AreaWeighted` regridding schemes. (:pull:`3701`) + +#. `@bjlittle`_ added `logging`_ support within :mod:`iris.analysis.maths`, + :mod:`iris.common.metadata`, and :mod:`iris.common.resolve`. Each module + defines a :class:`logging.Logger` instance called ``logger`` with a default + ``level`` of ``INFO``. To enable ``DEBUG`` logging use + ``logger.setLevel("DEBUG")``. (:pull:`3785`) + +#. `@bjlittle`_ added the :mod:`iris.common.resolve` module, which provides + infrastructure to support the analysis, identification and combination + of metadata common between two :class:`~iris.cube.Cube` operands into a + single resultant :class:`~iris.cube.Cube` that will be auto-transposed, + and with the appropriate broadcast shape. (:pull:`3785`) + +#. `@bjlittle`_ added the :ref:`common metadata API `, which provides + a unified treatment of metadata across Iris, and allows users to easily + manage and manipulate their metadata in a consistent way. (:pull:`3785`) + +#. `@bjlittle`_ added :ref:`lenient metadata ` support, to + allow users to control **strict** or **lenient** metadata equivalence, + difference and combination. (:pull:`3785`) + +#. `@bjlittle`_ added :ref:`lenient cube maths ` support and + resolved several long standing major issues with cube arithmetic regarding + a more robust treatment of cube broadcasting, cube dimension auto-transposition, + and preservation of common metadata and coordinates during cube math operations. + Resolves :issue:`1887`, :issue:`2765`, and :issue:`3478`. (:pull:`3785`) + +#. `@pp-mo`_ and `@TomekTrzeciak`_ enhanced :meth:`~iris.cube.Cube.collapse` to allow a 1-D weights array when + collapsing over a single dimension. + Previously, the weights had to be the same shape as the whole cube, which could cost a lot of memory in some cases. + The 1-D form is supported by most weighted array statistics (such as :meth:`np.average`), so this now works + with the corresponding Iris schemes (in that case, :const:`~iris.analysis.MEAN`). (:pull:`3943`) + + +🐛 Bugs Fixed +============= + +#. `@stephenworsley`_ fixed :meth:`~iris.cube.Cube.remove_coord` to now also + remove derived coordinates by removing aux_factories. (:pull:`3641`) + +#. `@jonseddon`_ fixed ``isinstance(cube, collections.Iterable)`` to now behave + as expected if a :class:`~iris.cube.Cube` is iterated over, while also + ensuring that ``TypeError`` is still raised. (Fixed by setting the + ``__iter__()`` method in :class:`~iris.cube.Cube` to ``None``). + (:pull:`3656`) + +#. `@stephenworsley`_ enabled cube concatenation along an axis shared by cell + measures; these cell measures are now concatenated together in the resulting + cube. Such a scenario would previously cause concatenation to inappropriately + fail. (:pull:`3566`) + +#. `@stephenworsley`_ newly included :class:`~iris.coords.CellMeasure`\ s in + :class:`~iris.cube.Cube` copy operations. Previously copying a + :class:`~iris.cube.Cube` would ignore any attached + :class:`~iris.coords.CellMeasure`. (:pull:`3546`) + +#. `@bjlittle`_ set a :class:`~iris.coords.CellMeasure`'s + ``measure`` attribute to have a default value of ``area``. + Previously, the ``measure`` was provided as a keyword argument to + :class:`~iris.coords.CellMeasure` with a default value of ``None``, which + caused a ``TypeError`` when no ``measure`` was provided, since ``area`` or + ``volume`` are the only accepted values. (:pull:`3533`) + +#. `@trexfeathers`_ set **all** plot types in :mod:`iris.plot` to now use + :obj:`matplotlib.dates.date2num` to format date/time coordinates for use on a plot + axis (previously :meth:`~iris.plot.pcolor` and :meth:`~iris.plot.pcolormesh` + did not include this behaviour). (:pull:`3762`) + +#. `@trexfeathers`_ changed date/time axis labels in :mod:`iris.quickplot` to + now **always** be based on the ``epoch`` used in :obj:`matplotlib.dates.date2num` + (previously would take the unit from a time coordinate, if present, even + though the coordinate's value had been changed via ``date2num``). + (:pull:`3762`) + +#. `@pp-mo`_ newly included attributes of cell measures in NETCDF-CF + file loading; they were previously being discarded. They are now available on + the :class:`~iris.coords.CellMeasure` in the loaded :class:`~iris.cube.Cube`. + (:pull:`3800`) + +#. `@pp-mo`_ fixed the netcdf loader to now handle any grid-mapping + variables with missing ``false_easting`` and ``false_northing`` properties, + which was previously failing for some coordinate systems. See :issue:`3629`. + (:pull:`3804`) + +#. `@stephenworsley`_ changed the way tick labels are assigned from string coords. + Previously, the first tick label would occasionally be duplicated. This also + removes the use of the deprecated `matplotlib`_ ``IndexFormatter``. (:pull:`3857`) + +#. `@znicholls`_ fixed :meth:`~iris.quickplot._title` to only check + ``units.is_time_reference`` if the ``units`` symbol is not used. (:pull:`3902`) + +#. `@rcomer`_ fixed a bug whereby numpy array type attributes on a cube's + coordinates could prevent printing it. See :issue:`3921`. (:pull:`3922`) + +.. _whatsnew 3.0.4 changes: + +💣 Incompatible Changes +======================= + +#. `@pp-mo`_ rationalised :class:`~iris.cube.CubeList` extraction + methods: + + The former method ``iris.cube.CubeList.extract_strict``, and the ``strict`` + keyword of the :meth:`~iris.cube.CubeList.extract` method have been removed, + and are replaced by the new routines :meth:`~iris.cube.CubeList.extract_cube` + and :meth:`~iris.cube.CubeList.extract_cubes`. + The new routines perform the same operation, but in a style more like other + ``Iris`` functions such as :meth:`~iris.load_cube` and :meth:`~iris.load_cubes`. + Unlike ``strict`` extraction, the type of return value is now completely + consistent : :meth:`~iris.cube.CubeList.extract_cube` always returns a + :class:`~iris.cube.Cube`, and :meth:`~iris.cube.CubeList.extract_cubes` + always returns an :class:`iris.cube.CubeList` of a length equal to the + number of constraints. (:pull:`3715`) + +#. `@pp-mo`_ removed the former function + ``iris.analysis.coord_comparison``. (:pull:`3562`) + +#. `@bjlittle`_ moved the + :func:`iris.experimental.equalise_cubes.equalise_attributes` function from + the :mod:`iris.experimental` module into the :mod:`iris.util` module. Please + use the :func:`iris.util.equalise_attributes` function instead. + (:pull:`3527`) + +#. `@bjlittle`_ removed the module ``iris.experimental.concatenate``. In + ``v1.6.0`` the experimental ``concatenate`` functionality was moved to the + :meth:`iris.cube.CubeList.concatenate` method. Since then, calling the + :func:`iris.experimental.concatenate.concatenate` function raised an + exception. (:pull:`3523`) + +#. `@stephenworsley`_ changed the default units of :class:`~iris.coords.DimCoord` + and :class:`~iris.coords.AuxCoord` from `"1"` to `"unknown"`. (:pull:`3795`) + +#. `@stephenworsley`_ changed Iris objects loaded from NetCDF-CF files to have + ``units='unknown'`` where the corresponding NetCDF variable has no ``units`` + property. Previously these cases defaulted to ``units='1'``. + This affects loading of coordinates whose file variable has no "units" + attribute (not valid, under `CF units rules`_): These will now have units + of `"unknown"`, rather than `"1"`, which **may prevent the creation of + a hybrid vertical coordinate**. While these cases used to "work", this was + never really correct behaviour. (:pull:`3795`) + +#. `@SimonPeatman`_ added attribute ``var_name`` to coordinates created by the + :func:`iris.analysis.trajectory.interpolate` function. This prevents + duplicate coordinate errors in certain circumstances. (:pull:`3718`) + +#. `@bjlittle`_ aligned the :func:`iris.analysis.maths.apply_ufunc` with the + rest of the :mod:`iris.analysis.maths` API by changing its keyword argument + from ``other_cube`` to ``other``. (:pull:`3785`) + +#. `@bjlittle`_ changed the :meth:`iris.analysis.maths.IFunc.__call__` to ignore + any surplus ``other`` keyword argument for a ``data_func`` that requires + **only one** argument. This aligns the behaviour of + :meth:`iris.analysis.maths.IFunc.__call__` with + :func:`~iris.analysis.maths.apply_ufunc`. Previously a ``ValueError`` + exception was raised. (:pull:`3785`) + + +.. _whatsnew 3.0.4 deprecations: + +🔥 Deprecations +=============== + +#. `@stephenworsley`_ removed the deprecated :class:`iris.Future` flags + ``cell_date_time_objects``, ``netcdf_promote``, ``netcdf_no_unlimited`` and + ``clip_latitudes``. (:pull:`3459`) + +#. `@stephenworsley`_ changed :attr:`iris.fileformats.pp.PPField.lbproc` to be an + ``int``. The deprecated attributes ``flag1``, ``flag2`` etc. have been + removed from it. (:pull:`3461`) + +#. `@bjlittle`_ deprecated :func:`~iris.util.as_compatible_shape` in preference + for :class:`~iris.common.resolve.Resolve` e.g., ``Resolve(src, tgt)(tgt.core_data())``. + The :func:`~iris.util.as_compatible_shape` function will be removed in a future + release of Iris. (:pull:`3892`) + + +🔗 Dependencies +=============== + +#. `@stephenworsley`_, `@trexfeathers`_ and `@bjlittle`_ removed ``Python2`` + support, modernising the codebase by switching to exclusive ``Python3`` + support. (:pull:`3513`) + +#. `@bjlittle`_ improved the developer set up process. Configuring Iris and + :ref:`installing_from_source` as a developer with all the required package + dependencies is now easier with our curated conda environment YAML files. + (:pull:`3812`) + +#. `@stephenworsley`_ pinned Iris to require `Dask`_ ``>=2.0``. (:pull:`3460`) + +#. `@stephenworsley`_ and `@trexfeathers`_ pinned Iris to require + `Cartopy`_ ``>=0.18``, in order to remain compatible with the latest version + of `matplotlib`_. (:pull:`3762`) + +#. `@bjlittle`_ unpinned Iris to use the latest version of `matplotlib`_. + Supporting ``Iris`` for both ``Python2`` and ``Python3`` had resulted in + pinning our dependency on `matplotlib`_ at ``v2.x``. But this is no longer + necessary now that ``Python2`` support has been dropped. (:pull:`3468`) + +#. `@stephenworsley`_ and `@trexfeathers`_ unpinned Iris to use the latest version + of `Proj`_. (:pull:`3762`) + +#. `@stephenworsley`_ and `@trexfeathers`_ removed GDAL from the extensions + dependency group. We no longer consider it to be an extension. (:pull:`3762`) + + +.. _whatsnew 3.0.4 docs: + +📚 Documentation +================ + +#. `@tkknight`_ moved the + :ref:`sphx_glr_generated_gallery_oceanography_plot_orca_projection.py` + from the general part of the gallery to oceanography. (:pull:`3761`) + +#. `@tkknight`_ updated documentation to use a modern sphinx theme and be + served from https://scitools-iris.readthedocs.io/en/latest/. (:pull:`3752`) + +#. `@bjlittle`_ added support for the `black`_ code formatter. This is + now automatically checked on GitHub PRs, replacing the older, unittest-based + ``iris.tests.test_coding_standards.TestCodeFormat``. Black provides automatic + code format correction for most IDEs. See the new developer guide section on + :ref:`code_formatting`. (:pull:`3518`) + +#. `@tkknight`_ and `@trexfeathers`_ refreshed the :ref:`whats_new_contributions` + for the :ref:`iris_whatsnew`. This includes always creating the ``latest`` + what's new page so it appears on the latest documentation at + https://scitools-iris.readthedocs.io/en/latest/whatsnew. This resolves + :issue:`2104`, :issue:`3451`, :issue:`3818`, :issue:`3837`. Also updated the + :ref:`iris_development_releases_steps` to follow when making a release. + (:pull:`3769`, :pull:`3838`, :pull:`3843`) + +#. `@tkknight`_ enabled the PDF creation of the documentation on the + `Read the Docs`_ service. The PDF may be accessed by clicking on the version + at the bottom of the side bar, then selecting ``PDF`` from the ``Downloads`` + section. (:pull:`3765`) + +#. `@stephenworsley`_ added a warning to the + :func:`iris.analysis.cartography.project` function regarding its behaviour on + projections with non-rectangular boundaries. (:pull:`3762`) + +#. `@stephenworsley`_ added the :ref:`cube_maths_combining_units` section to the + user guide to clarify how ``Units`` are handled during cube arithmetic. + (:pull:`3803`) + +#. `@tkknight`_ overhauled the :ref:`developers_guide` including information on + getting involved in becoming a contributor and general structure of the + guide. This resolves :issue:`2170`, :issue:`2331`, :issue:`3453`, + :issue:`314`, :issue:`2902`. (:pull:`3852`) + +#. `@rcomer`_ added argument descriptions to the :class:`~iris.coords.DimCoord` + docstring. (:pull:`3681`) + +#. `@tkknight`_ added two url's to be ignored for the ``make linkcheck``. This + will ensure the Iris github project is not repeatedly hit during the + linkcheck for issues and pull requests as it can result in connection + refused and thus travis-ci_ job failures. For more information on linkcheck, + see :ref:`contributing.documentation.testing`. (:pull:`3873`) + +#. `@tkknight`_ enabled the napolean_ package that is used by sphinx_ to cater + for the existing google style docstrings and to also allow for `numpy`_ + docstrings. This resolves :issue:`3841`. (:pull:`3871`) + +#. `@tkknight`_ configured ``sphinx-build`` to promote warnings to errors when + building the documentation via ``make html``. This will minimise technical + debt accruing for the documentation. (:pull:`3877`) + +#. `@tkknight`_ updated :ref:`installing_iris` to include a reference to + Windows Subsystem for Linux. (:pull:`3885`) + +#. `@tkknight`_ updated the :ref:`iris_docs` homepage to include panels so the + links are more visible to users. This uses the sphinx-panels_ extension. + (:pull:`3884`) + +#. `@bjlittle`_ created the :ref:`Further topics ` section and + included documentation for :ref:`metadata`, :ref:`lenient metadata`, and + :ref:`lenient maths`. (:pull:`3890`) + +#. `@jonseddon`_ updated the CF version of the netCDF saver in the + :ref:`saving_iris_cubes` section and in the equivalent function docstring. + (:pull:`3925`) + +#. `@bjlittle`_ applied `Title Case Capitalization`_ to the documentation. + (:pull:`3940`) + + +💼 Internal +=========== + +#. `@pp-mo`_ and `@lbdreyer`_ removed all Iris test dependencies on `iris-grib`_ + by transferring all relevant content to the `iris-grib`_ repository. (:pull:`3662`, + :pull:`3663`, :pull:`3664`, :pull:`3665`, :pull:`3666`, :pull:`3669`, + :pull:`3670`, :pull:`3671`, :pull:`3672`, :pull:`3742`, :pull:`3746`) + +#. `@lbdreyer`_ and `@pp-mo`_ overhauled the handling of dimensional + metadata to remove duplication. (:pull:`3422`, :pull:`3551`) + +#. `@trexfeathers`_ simplified the standard license header for all files, which + removes the need to repeatedly update year numbers in the header. + (:pull:`3489`) + +#. `@stephenworsley`_ changed the numerical values in tests involving the + Robinson projection due to improvements made in + `Proj`_. (:pull:`3762`) (see also `Proj#1292`_ and `Proj#2151`_) + +#. `@stephenworsley`_ changed tests to account for more detailed descriptions of + projections in `GDAL`_. (:pull:`3762`) (see also `GDAL#1185`_) + +#. `@stephenworsley`_ changed tests to account for `GDAL`_ now saving fill values + for data without masked points. (:pull:`3762`) + +#. `@trexfeathers`_ changed every graphics test that includes `Cartopy's coastlines`_ + to account for new adaptive coastline scaling. (:pull:`3762`) + (see also `Cartopy#1105`_) + +#. `@trexfeathers`_ changed graphics tests to account for some new default + grid-line spacing in `Cartopy`_. (:pull:`3762`) (see also `Cartopy#1117`_) + +#. `@trexfeathers`_ added additional acceptable graphics test targets to account + for very minor changes in `matplotlib`_ version ``3.3`` (colormaps, fonts and + axes borders). (:pull:`3762`) + +#. `@rcomer`_ corrected the `matplotlib`_ backend in Iris tests to ignore + :obj:`matplotlib.rcdefaults`, instead the tests will **always** use ``agg``. + (:pull:`3846`) + +#. `@bjlittle`_ migrated the `black`_ support from ``19.10b0`` to ``20.8b1``. + (:pull:`3866`) + +#. `@lbdreyer`_ updated the CF standard name table to the latest version: `v75`_. + (:pull:`3867`) + +#. `@bjlittle`_ added :pep:`517` and :pep:`518` support for building and + installing Iris, in particular to handle the `PyKE`_ package dependency. + (:pull:`3812`) + +#. `@bjlittle`_ added metadata support for comparing :attr:`~iris.cube.Cube.attributes` + dictionaries that contain `numpy`_ arrays using `xxHash`_, an extremely fast + non-cryptographic hash algorithm, running at RAM speed limits. + +#. `@bjlittle`_ added the ``iris.tests.assertDictEqual`` method to override + :meth:`unittest.TestCase.assertDictEqual` in order to cope with testing + metadata :attr:`~iris.cube.Cube.attributes` dictionary comparison where + the value of a key may be a `numpy`_ array. (:pull:`3785`) + +#. `@bjlittle`_ added the :func:`~iris.config.get_logger` function for creating + a generic :class:`logging.Logger` with a :class:`logging.StreamHandler` and + custom :class:`logging.Formatter`. (:pull:`3785`) + +#. `@owena11`_ identified and optimised a bottleneck in ``FieldsFile`` header + loading due to the use of :func:`numpy.fromfile`. (:pull:`3791`) + +#. `@znicholls`_ added a test for plotting with the label being taken from the unit's symbol, + see :meth:`~iris.tests.test_quickplot.TestLabels.test_pcolormesh_str_symbol` (:pull:`3902`). + +#. `@znicholls`_ made :func:`~iris.tests.idiff.step_over_diffs` robust to hyphens (``-``) in + the input path (i.e. the ``result_dir`` argument) (:pull:`3902`). + +#. `@bjlittle`_ migrated the CIaaS from `travis-ci`_ to `cirrus-ci`_, and removed `stickler-ci`_ + support. (:pull:`3928`) + +#. `@bjlittle`_ introduced `nox`_ as a common and easy entry-point for test automation. + It can be used both from `cirrus-ci`_ in the cloud, and locally by the developer to + run the Iris tests, the doc-tests, the gallery doc-tests, and lint Iris + with `flake8`_ and `black`_. (:pull:`3928`) + +.. _Read the Docs: https://scitools-iris.readthedocs.io/en/latest/ +.. _CF units rules: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#units +.. _CF Ancillary Data: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#ancillary-data +.. _Quality Flags: https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#flags +.. _iris-grib: https://github.com/SciTools/iris-grib +.. _Cartopy: https://github.com/SciTools/cartopy +.. _Cartopy's coastlines: https://scitools.org.uk/cartopy/docs/latest/matplotlib/geoaxes.html?highlight=coastlines#cartopy.mpl.geoaxes.GeoAxes.coastlines +.. _Cartopy#1105: https://github.com/SciTools/cartopy/pull/1105 +.. _Cartopy#1117: https://github.com/SciTools/cartopy/pull/1117 +.. _Dask: https://github.com/dask/dask +.. _Proj: https://github.com/OSGeo/PROJ +.. _black: https://black.readthedocs.io/en/stable/ +.. _Proj#1292: https://github.com/OSGeo/PROJ/pull/1292 +.. _Proj#2151: https://github.com/OSGeo/PROJ/pull/2151 +.. _GDAL: https://github.com/OSGeo/gdal +.. _GDAL#1185: https://github.com/OSGeo/gdal/pull/1185 +.. _@MoseleyS: https://github.com/MoseleyS +.. _@stephenworsley: https://github.com/stephenworsley +.. _@pp-mo: https://github.com/pp-mo +.. _@abooton: https://github.com/abooton +.. _@bouweandela: https://github.com/bouweandela +.. _@bjlittle: https://github.com/bjlittle +.. _@trexfeathers: https://github.com/trexfeathers +.. _@jonseddon: https://github.com/jonseddon +.. _@tkknight: https://github.com/tkknight +.. _@lbdreyer: https://github.com/lbdreyer +.. _@SimonPeatman: https://github.com/SimonPeatman +.. _@TomekTrzeciak: https://github.com/TomekTrzeciak +.. _@rcomer: https://github.com/rcomer +.. _@jvegasbsc: https://github.com/jvegasbsc +.. _@zklaus: https://github.com/zklaus +.. _@znicholls: https://github.com/znicholls +.. _ESMValTool: https://github.com/ESMValGroup/ESMValTool +.. _v75: https://cfconventions.org/Data/cf-standard-names/75/build/cf-standard-name-table.html +.. _sphinx-panels: https://sphinx-panels.readthedocs.io/en/latest/ +.. _logging: https://docs.python.org/3/library/logging.html +.. _numpy: https://github.com/numpy/numpy +.. _xxHash: https://github.com/Cyan4973/xxHash +.. _PyKE: https://pypi.org/project/scitools-pyke/ +.. _@owena11: https://github.com/owena11 +.. _GitHub: https://github.com/SciTools/iris/issues/new/choose +.. _readthedocs: https://readthedocs.org/ +.. _CF Conventions and Metadata: https://cfconventions.org/ +.. _flake8: https://flake8.pycqa.org/en/stable/ +.. _nox: https://nox.thea.codes/en/stable/ +.. _Title Case Capitalization: https://apastyle.apa.org/style-grammar-guidelines/capitalization/title-case +.. _travis-ci: https://travis-ci.org/github/SciTools/iris +.. _stickler-ci: https://stickler-ci.com/ +.. _@jamesp: https://github.com/jamesp +.. _cftime: https://github.com/Unidata/cftime +.. _cf-units: https://github.com/SciTools/cf-units +.. _nc-time-axis: https://github.com/SciTools/nc-time-axis diff --git a/docs/src/whatsnew/index.rst b/docs/src/whatsnew/index.rst index 257674718a..cadb7c224a 100644 --- a/docs/src/whatsnew/index.rst +++ b/docs/src/whatsnew/index.rst @@ -11,6 +11,9 @@ Iris versions. :maxdepth: 1 latest.rst + 3.0.4.rst + 3.0.3.rst + 3.0.2.rst 3.0.1.rst 3.0.rst 2.4.rst diff --git a/lib/iris/_lazy_data.py b/lib/iris/_lazy_data.py index e197afe549..038f9d9337 100644 --- a/lib/iris/_lazy_data.py +++ b/lib/iris/_lazy_data.py @@ -191,7 +191,9 @@ def as_lazy_data(data, chunks=None, asarray=False): if isinstance(data, ma.core.MaskedConstant): data = ma.masked_array(data.data, mask=data.mask) if not is_lazy_data(data): - data = da.from_array(data, chunks=chunks, asarray=asarray) + data = da.from_array( + data, chunks=chunks, asarray=asarray, meta=np.ndarray + ) return data diff --git a/lib/iris/coords.py b/lib/iris/coords.py index 473062840f..2247469f56 100644 --- a/lib/iris/coords.py +++ b/lib/iris/coords.py @@ -403,28 +403,15 @@ def __binary_operator__(self, other, mode_constant): # Note: this method includes bounds handling code, but it only runs # within Coord type instances, as only these allow bounds to be set. - if isinstance(other, _DimensionalMetadata) or not isinstance( - other, (int, float, np.number) - ): - - def typename(obj): - if isinstance(obj, Coord): - result = "Coord" - else: - # We don't really expect this, but do something anyway. - result = self.__class__.__name__ - return result - - emsg = "{selftype} {operator} {othertype}".format( - selftype=typename(self), - operator=self._MODE_SYMBOL[mode_constant], - othertype=typename(other), + if isinstance(other, _DimensionalMetadata): + emsg = ( + f"{self.__class__.__name__} " + f"{self._MODE_SYMBOL[mode_constant]} " + f"{other.__class__.__name__}" ) raise iris.exceptions.NotYetImplementedError(emsg) - else: - # 'Other' is an array type : adjust points, and bounds if any. - result = NotImplemented + if isinstance(other, (int, float, np.number)): def op(values): if mode_constant == self._MODE_ADD: @@ -441,8 +428,14 @@ def op(values): new_values = op(self._values_dm.core_data()) result = self.copy(new_values) + if self.has_bounds(): result.bounds = op(self._bounds_dm.core_data()) + else: + # must return NotImplemented to ensure invocation of any + # associated reflected operator on the "other" operand + # see https://docs.python.org/3/reference/datamodel.html#emulating-numeric-types + result = NotImplemented return result @@ -461,8 +454,7 @@ def __div__(self, other): def __truediv__(self, other): return self.__binary_operator__(other, self._MODE_DIV) - def __radd__(self, other): - return self + other + __radd__ = __add__ def __rsub__(self, other): return (-self) + other @@ -473,8 +465,7 @@ def __rdiv__(self, other): def __rtruediv__(self, other): return self.__binary_operator__(other, self._MODE_RDIV) - def __rmul__(self, other): - return self * other + __rmul__ = __mul__ def __neg__(self): values = -self._core_values() diff --git a/lib/iris/cube.py b/lib/iris/cube.py index dc6f481d3c..7d4f1dc33d 100644 --- a/lib/iris/cube.py +++ b/lib/iris/cube.py @@ -3193,24 +3193,26 @@ def _intersect_modulus( # and call the new bounds = the new points + the difference. pre_wrap_delta = np.diff(coord.bounds[inside_indices]) post_wrap_delta = np.diff(bounds[inside_indices]) - close_enough = np.allclose(pre_wrap_delta, post_wrap_delta) - if not close_enough: - split_cell_indices, _ = np.where( - pre_wrap_delta != post_wrap_delta - ) - - # Recalculate the extended minimum. + split_cell_indices, _ = np.where( + ~np.isclose(pre_wrap_delta, post_wrap_delta) + ) + if split_cell_indices.size: indices = inside_indices[split_cell_indices] cells = bounds[indices] - cells_delta = np.diff(coord.bounds[indices]) - - # Watch out for ascending/descending bounds - if cells_delta[0, 0] > 0: - cells[:, 0] = cells[:, 1] - cells_delta[:, 0] - minimum = np.min(cells[:, 0]) - else: - cells[:, 1] = cells[:, 0] + cells_delta[:, 0] - minimum = np.min(cells[:, 1]) + if maximum % modulus not in cells: + # Recalculate the extended minimum only if the output bounds + # do not span the requested (minimum, maximum) range. If + # they do span that range, this adjustment would give unexpected + # results (see #3391). + cells_delta = np.diff(coord.bounds[indices]) + + # Watch out for ascending/descending bounds. + if cells_delta[0, 0] > 0: + cells[:, 0] = cells[:, 1] - cells_delta[:, 0] + minimum = np.min(cells[:, 0]) + else: + cells[:, 1] = cells[:, 0] + cells_delta[:, 0] + minimum = np.min(cells[:, 1]) points = wrap_lons(coord.points, minimum, modulus) @@ -3764,37 +3766,49 @@ def __ne__(self, other): def __hash__(self): return hash(id(self)) - def __add__(self, other): - return iris.analysis.maths.add(self, other) + __add__ = iris.analysis.maths.add def __iadd__(self, other): return iris.analysis.maths.add(self, other, in_place=True) __radd__ = __add__ - def __sub__(self, other): - return iris.analysis.maths.subtract(self, other) + __sub__ = iris.analysis.maths.subtract def __isub__(self, other): return iris.analysis.maths.subtract(self, other, in_place=True) + def __rsub__(self, other): + return (-self) + other + __mul__ = iris.analysis.maths.multiply - __rmul__ = iris.analysis.maths.multiply def __imul__(self, other): return iris.analysis.maths.multiply(self, other, in_place=True) + __rmul__ = __mul__ + __div__ = iris.analysis.maths.divide def __idiv__(self, other): return iris.analysis.maths.divide(self, other, in_place=True) - __truediv__ = iris.analysis.maths.divide + def __rdiv__(self, other): + data = 1 / self.core_data() + reciprocal = self.copy(data=data) + return iris.analysis.maths.multiply(reciprocal, other) - def __itruediv__(self, other): - return iris.analysis.maths.divide(self, other, in_place=True) + __truediv__ = __div__ + + __itruediv__ = __idiv__ + + __rtruediv__ = __rdiv__ __pow__ = iris.analysis.maths.exponentiate + + def __neg__(self): + return self.copy(data=-self.core_data()) + # END OPERATOR OVERLOADS def collapsed(self, coords, aggregator, **kwargs): diff --git a/lib/iris/fileformats/_nc_load_rules/actions.py b/lib/iris/fileformats/_nc_load_rules/actions.py index 5813c5bca7..e3b63d41d2 100644 --- a/lib/iris/fileformats/_nc_load_rules/actions.py +++ b/lib/iris/fileformats/_nc_load_rules/actions.py @@ -43,11 +43,15 @@ from functools import wraps import warnings +from iris.config import get_logger import iris.fileformats.cf import iris.fileformats.pp as pp from . import helpers as hh +# Configure the logger. +logger = get_logger(__name__, fmt="[%(funcName)s]") + def _default_rulenamesfunc(func_name): # A simple default function to deduce the rules-name from an action-name. @@ -385,13 +389,23 @@ def action_ukmo_stash(engine): attr_name = "ukmo__um_stash_source" attr_value = getattr(var, attr_name, None) if attr_value is None: - attr_altname = "um_stash_source" # legacy form - attr_value = getattr(var, attr_altname, None) + attr_name = "um_stash_source" # legacy form + attr_value = getattr(var, attr_name, None) if attr_value is None: rule_name += "(NOT-TRIGGERED)" else: # No helper routine : just do it - engine.cube.attributes["STASH"] = pp.STASH.from_msi(attr_value) + try: + stash_code = pp.STASH.from_msi(attr_value) + except (TypeError, ValueError): + engine.cube.attributes[attr_name] = attr_value + msg = ( + "Unable to set attribute STASH as not a valid MSI " + f'string "mXXsXXiXXX", got "{attr_value}"' + ) + logger.debug(msg) + else: + engine.cube.attributes["STASH"] = stash_code return rule_name diff --git a/lib/iris/fileformats/name_loaders.py b/lib/iris/fileformats/name_loaders.py index d0571090ce..c3db0e98de 100644 --- a/lib/iris/fileformats/name_loaders.py +++ b/lib/iris/fileformats/name_loaders.py @@ -478,7 +478,7 @@ def _generate_cubes( coord_units = _parse_units("FL") if coord.name == "time": coord_units = time_unit - pts = time_unit.date2num(coord.values) + pts = np.float_(time_unit.date2num(coord.values)) if coord.dimension is not None: if coord.name == "longitude": @@ -505,7 +505,7 @@ def _generate_cubes( ): dt = coord.values - field_headings["Av or Int period"] bnds = time_unit.date2num(np.vstack((dt, coord.values)).T) - icoord.bounds = bnds + icoord.bounds = np.float_(bnds) else: icoord.guess_bounds() cube.add_dim_coord(icoord, coord.dimension) @@ -522,7 +522,7 @@ def _generate_cubes( ): dt = coord.values - field_headings["Av or Int period"] bnds = time_unit.date2num(np.vstack((dt, coord.values)).T) - icoord.bounds = bnds[i, :] + icoord.bounds = np.float_(bnds[i, :]) cube.add_aux_coord(icoord) # Headings/column headings which are encoded elsewhere. @@ -1250,7 +1250,7 @@ def load_NAMEIII_trajectory(filename): long_name = units = None if isinstance(values[0], datetime.datetime): - values = time_unit.date2num(values) + values = np.float_(time_unit.date2num(values)) units = time_unit if name == "Time": name = "time" diff --git a/lib/iris/fileformats/netcdf.py b/lib/iris/fileformats/netcdf.py index 14dbab8054..158cb6723d 100644 --- a/lib/iris/fileformats/netcdf.py +++ b/lib/iris/fileformats/netcdf.py @@ -1033,7 +1033,7 @@ def write( dtype(i.e. 'i2', 'short', 'u4') or a dict of packing parameters as described below. This provides support for netCDF data packing as described in - http://www.unidata.ucar.edu/software/netcdf/documentation/NUG/best_practices.html#bp_Packed-Data-Values + https://www.unidata.ucar.edu/software/netcdf/documentation/NUG/best_practices.html#bp_Packed-Data-Values If this argument is a type (or type string), appropriate values of scale_factor and add_offset will be automatically calculated based on `cube.data` and possible masking. For more control, pass a dict @@ -2579,7 +2579,7 @@ def save( (i.e. 'i2', 'short', 'u4') or a dict of packing parameters as described below or an iterable of such types, strings, or dicts. This provides support for netCDF data packing as described in - http://www.unidata.ucar.edu/software/netcdf/documentation/NUG/best_practices.html#bp_Packed-Data-Values + https://www.unidata.ucar.edu/software/netcdf/documentation/NUG/best_practices.html#bp_Packed-Data-Values If this argument is a type (or type string), appropriate values of scale_factor and add_offset will be automatically calculated based on `cube.data` and possible masking. For more control, pass a dict with diff --git a/lib/iris/fileformats/pp.py b/lib/iris/fileformats/pp.py index 0a02dbb911..142d03c187 100644 --- a/lib/iris/fileformats/pp.py +++ b/lib/iris/fileformats/pp.py @@ -37,7 +37,6 @@ import iris.fileformats.pp_load_rules from iris.fileformats.pp_save_rules import verify import iris.fileformats.rules -from iris.util import _array_slice_ifempty try: import mo_pack @@ -593,23 +592,18 @@ def ndim(self): return len(self.shape) def __getitem__(self, keys): - # Check for 'empty' slicings, in which case don't fetch the data. - # Because, since Dask v2, 'dask.array.from_array' performs an empty - # slicing and we must not fetch the data at that time. - result = _array_slice_ifempty(keys, self.shape, self.dtype) - if result is None: - with open(self.path, "rb") as pp_file: - pp_file.seek(self.offset, os.SEEK_SET) - data_bytes = pp_file.read(self.data_len) - data = _data_bytes_to_shaped_array( - data_bytes, - self.lbpack, - self.boundary_packing, - self.shape, - self.src_dtype, - self.mdi, - ) - result = data.__getitem__(keys) + with open(self.path, "rb") as pp_file: + pp_file.seek(self.offset, os.SEEK_SET) + data_bytes = pp_file.read(self.data_len) + data = _data_bytes_to_shaped_array( + data_bytes, + self.lbpack, + self.boundary_packing, + self.shape, + self.src_dtype, + self.mdi, + ) + result = data.__getitem__(keys) return np.asanyarray(result, dtype=self.dtype) @@ -1481,8 +1475,18 @@ def t1(self): """ if not hasattr(self, "_t1"): + has_year_zero = self.lbyr == 0 + calendar = ( + None if self.lbmon == 0 or self.lbdat == 0 else self.calendar + ) self._t1 = cftime.datetime( - self.lbyr, self.lbmon, self.lbdat, self.lbhr, self.lbmin + self.lbyr, + self.lbmon, + self.lbdat, + self.lbhr, + self.lbmin, + calendar=calendar, + has_year_zero=has_year_zero, ) return self._t1 @@ -1505,8 +1509,18 @@ def t2(self): """ if not hasattr(self, "_t2"): + has_year_zero = self.lbyrd == 0 + calendar = ( + None if self.lbmond == 0 or self.lbdatd == 0 else self.calendar + ) self._t2 = cftime.datetime( - self.lbyrd, self.lbmond, self.lbdatd, self.lbhrd, self.lbmind + self.lbyrd, + self.lbmond, + self.lbdatd, + self.lbhrd, + self.lbmind, + calendar=calendar, + has_year_zero=has_year_zero, ) return self._t2 @@ -1542,6 +1556,10 @@ def t1(self): """ if not hasattr(self, "_t1"): + has_year_zero = self.lbyr == 0 + calendar = ( + None if self.lbmon == 0 or self.lbdat == 0 else self.calendar + ) self._t1 = cftime.datetime( self.lbyr, self.lbmon, @@ -1549,6 +1567,8 @@ def t1(self): self.lbhr, self.lbmin, self.lbsec, + calendar=calendar, + has_year_zero=has_year_zero, ) return self._t1 @@ -1571,6 +1591,10 @@ def t2(self): """ if not hasattr(self, "_t2"): + has_year_zero = self.lbyrd == 0 + calendar = ( + None if self.lbmond == 0 or self.lbdatd == 0 else self.calendar + ) self._t2 = cftime.datetime( self.lbyrd, self.lbmond, @@ -1578,6 +1602,8 @@ def t2(self): self.lbhrd, self.lbmind, self.lbsecd, + calendar=calendar, + has_year_zero=has_year_zero, ) return self._t2 diff --git a/lib/iris/fileformats/pp_save_rules.py b/lib/iris/fileformats/pp_save_rules.py index 22acebca46..76ac6226c8 100644 --- a/lib/iris/fileformats/pp_save_rules.py +++ b/lib/iris/fileformats/pp_save_rules.py @@ -123,7 +123,7 @@ def _general_time_rules(cube, pp): pp.lbtim.ia = 0 pp.lbtim.ib = 0 pp.t1 = time_coord.units.num2date(time_coord.points[0]) - pp.t2 = cftime.datetime(0, 0, 0) + pp.t2 = cftime.datetime(0, 0, 0, calendar=None, has_year_zero=True) # Forecast. if ( diff --git a/lib/iris/plot.py b/lib/iris/plot.py index a49957accf..0086a05ead 100644 --- a/lib/iris/plot.py +++ b/lib/iris/plot.py @@ -604,7 +604,8 @@ def _fixup_dates(coord, values): r = [ nc_time_axis.CalendarDateTime( - cftime.datetime(*date), coord.units.calendar + cftime.datetime(*date, calendar=coord.units.calendar), + coord.units.calendar, ) for date in dates ] diff --git a/lib/iris/tests/integration/plot/test_netcdftime.py b/lib/iris/tests/integration/plot/test_netcdftime.py index 02dda76fd5..340f37dda7 100644 --- a/lib/iris/tests/integration/plot/test_netcdftime.py +++ b/lib/iris/tests/integration/plot/test_netcdftime.py @@ -44,6 +44,7 @@ def test_360_day_calendar(self): atime.hour, atime.minute, atime.second, + calendar=calendar, ) for atime in times ] diff --git a/lib/iris/tests/system_test.py b/lib/iris/tests/system_test.py index 29589321f9..36573362dd 100644 --- a/lib/iris/tests/system_test.py +++ b/lib/iris/tests/system_test.py @@ -22,7 +22,7 @@ class SystemInitialTest(tests.IrisTest): - def system_test_supported_filetypes(self): + def test_supported_filetypes(self): nx, ny = 60, 60 data = np.arange(nx * ny, dtype=">f4").reshape(nx, ny) @@ -74,7 +74,7 @@ def horiz_cs(): new_cube, ("system", "supported_filetype_%s.cml" % filetype) ) - def system_test_imports_general(self): + def test_imports_general(self): if tests.MPL_AVAILABLE: import matplotlib # noqa import netCDF4 # noqa diff --git a/lib/iris/tests/test_basic_maths.py b/lib/iris/tests/test_basic_maths.py index 4f2b5e102a..e753adbae8 100644 --- a/lib/iris/tests/test_basic_maths.py +++ b/lib/iris/tests/test_basic_maths.py @@ -16,8 +16,9 @@ import iris import iris.analysis.maths -import iris.coords -import iris.exceptions +from iris.coords import AuxCoord, DimCoord +from iris.cube import Cube +from iris.exceptions import NotYetImplementedError import iris.tests.stock @@ -112,12 +113,12 @@ def test_minus_coord(self): xdim = a.ndim - 1 ydim = a.ndim - 2 - c_x = iris.coords.DimCoord( + c_x = DimCoord( points=np.arange(a.shape[xdim]), long_name="x_coord", units=self.cube.units, ) - c_y = iris.coords.AuxCoord( + c_y = AuxCoord( points=np.arange(a.shape[ydim]), long_name="y_coord", units=self.cube.units, @@ -150,12 +151,12 @@ def test_addition_coord(self): xdim = a.ndim - 1 ydim = a.ndim - 2 - c_x = iris.coords.DimCoord( + c_x = DimCoord( points=np.arange(a.shape[xdim]), long_name="x_coord", units=self.cube.units, ) - c_y = iris.coords.AuxCoord( + c_y = AuxCoord( points=np.arange(a.shape[ydim]), long_name="y_coord", units=self.cube.units, @@ -195,12 +196,12 @@ def test_addition_fail(self): xdim = a.ndim - 1 ydim = a.ndim - 2 - c_axis_length_fail = iris.coords.DimCoord( + c_axis_length_fail = DimCoord( points=np.arange(a.shape[ydim]), long_name="x_coord", units=self.cube.units, ) - c_unit_fail = iris.coords.AuxCoord( + c_unit_fail = AuxCoord( points=np.arange(a.shape[xdim]), long_name="x_coord", units="volts" ) @@ -208,7 +209,7 @@ def test_addition_fail(self): ValueError, iris.analysis.maths.add, a, c_axis_length_fail ) self.assertRaises( - iris.exceptions.NotYetImplementedError, + NotYetImplementedError, iris.analysis.maths.add, a, c_unit_fail, @@ -464,7 +465,7 @@ def test_divide_by_coordinate_dim2(self): def test_divide_by_singular_coordinate(self): a = self.cube - coord = iris.coords.DimCoord(points=2, long_name="foo", units="1") + coord = DimCoord(points=2, long_name="foo", units="1") c = iris.analysis.maths.divide(a, coord) self.assertCML(c, ("analysis", "division_by_singular_coord.cml")) @@ -474,7 +475,7 @@ def test_divide_by_singular_coordinate(self): def test_divide_by_different_len_coord(self): a = self.cube - coord = iris.coords.DimCoord( + coord = DimCoord( points=np.arange(10) * 2 + 5, standard_name="longitude", units="degrees", @@ -686,12 +687,12 @@ def setUp(self): self.data_1u = np.array([[9, 9, 9], [8, 8, 8]], dtype=np.uint64) self.data_2u = np.array([[3, 3, 3], [2, 2, 2]], dtype=np.uint64) - self.cube_1f = iris.cube.Cube(self.data_1f) - self.cube_2f = iris.cube.Cube(self.data_2f) - self.cube_1i = iris.cube.Cube(self.data_1i) - self.cube_2i = iris.cube.Cube(self.data_2i) - self.cube_1u = iris.cube.Cube(self.data_1u) - self.cube_2u = iris.cube.Cube(self.data_2u) + self.cube_1f = Cube(self.data_1f) + self.cube_2f = Cube(self.data_2f) + self.cube_1i = Cube(self.data_1i) + self.cube_2i = Cube(self.data_2i) + self.cube_1u = Cube(self.data_1u) + self.cube_2u = Cube(self.data_2u) self.ops = (operator.add, operator.sub, operator.mul, operator.truediv) self.iops = ( @@ -701,6 +702,22 @@ def setUp(self): operator.itruediv, ) + def common_neg(self, cube, data): + result1 = -cube + result2 = -data + self.assertIsInstance(result1, Cube) + self.assertIsNot(result1, cube) + self.assertArrayAlmostEqual(result1.data, result2) + + def test_neg_f(self): + self.common_neg(self.cube_1f, self.data_1f) + + def test_neg_i(self): + self.common_neg(self.cube_1i, self.data_1i) + + def test_neg_u(self): + self.common_neg(self.cube_1u, self.data_1u) + def test_operator(self): for test_op in self.ops: result1 = test_op(self.cube_1f, self.cube_2f) @@ -849,7 +866,7 @@ def setUp(self): mask=[[0, 1, 0], [0, 0, 1]], dtype=np.float64, ) - self.cube = iris.cube.Cube(self.data) + self.cube = Cube(self.data) def test_incompatible_dimensions(self): data3 = ma.MaskedArray( @@ -863,9 +880,61 @@ def test_increase_cube_dimensionality(self): with self.assertRaises(ValueError): # This would increase the dimensionality of the cube # due to auto-broadcasting. - cube_x = iris.cube.Cube(ma.MaskedArray([[9]], mask=[[0]])) + cube_x = Cube(ma.MaskedArray([[9]], mask=[[0]])) cube_x + ma.MaskedArray([[3, 3, 3, 3]], mask=[[0, 1, 0, 1]]) +class TestCoordMathOperations(tests.IrisTest): + def setUp(self): + self.value_cube = 100 + self.value = 10 + self.cube = Cube([self.value_cube]) + self.dim = DimCoord([self.value]) + self.aux = AuxCoord([self.value]) + self.i = int(self.value) + self.f = float(self.value) + self.np_i = np.int32(self.value) + self.np_f = np.float32(self.value) + self.numbers = (self.i, self.f, self.np_i, self.np_f) + self.ops = (operator.add, operator.sub, operator.mul, operator.truediv) + self.symbols = ("+", "-", "*", "/") + + def test_coord_op_coord__fail(self): + for op, symbol in zip(self.ops, self.symbols): + emsg = f"AuxCoord \{symbol} DimCoord" # noqa: W605 + with self.assertRaisesRegex(NotYetImplementedError, emsg): + _ = op(self.aux, self.dim) + + def test_coord_op_number(self): + for op in self.ops: + for number in self.numbers: + actual = op(self.aux, number) + expected = op(self.value, number) + self.assertIsInstance(actual, AuxCoord) + self.assertEqual(expected, actual.points) + + def test_number_op_coord(self): + for op in self.ops: + for number in self.numbers: + actual = op(number, self.dim) + expected = op(number, self.value) + self.assertIsInstance(actual, DimCoord) + self.assertEqual(expected, actual.points) + + def test_coord_op_cube(self): + for op in self.ops: + actual = op(self.aux, self.cube) + expected = op(self.value, self.value_cube) + self.assertIsInstance(actual, Cube) + self.assertEqual(expected, actual.data) + + def test_cube_op_coord(self): + for op in self.ops: + actual = op(self.cube, self.dim) + expected = op(self.value_cube, self.value) + self.assertIsInstance(actual, Cube) + self.assertEqual(actual.data, expected) + + if __name__ == "__main__": tests.main() diff --git a/lib/iris/tests/test_merge.py b/lib/iris/tests/test_merge.py index 269fda24ad..c209d68da0 100644 --- a/lib/iris/tests/test_merge.py +++ b/lib/iris/tests/test_merge.py @@ -26,7 +26,7 @@ import iris.tests.stock -class TestMixin: +class MergeMixin: """ Mix-in class for attributes & utilities common to these test cases. @@ -56,7 +56,7 @@ def test_duplication(self): @tests.skip_data -class TestSingleCube(tests.IrisTest, TestMixin): +class TestSingleCube(tests.IrisTest, MergeMixin): def setUp(self): self._data_path = tests.get_data_path(("PP", "globClim1", "theta.pp")) self._num_cubes = 1 @@ -64,7 +64,7 @@ def setUp(self): @tests.skip_data -class TestMultiCube(tests.IrisTest, TestMixin): +class TestMultiCube(tests.IrisTest, MergeMixin): def setUp(self): self._data_path = tests.get_data_path( ("PP", "globClim1", "dec_subset.pp") diff --git a/lib/iris/tests/test_netcdf.py b/lib/iris/tests/test_netcdf.py index 36e06202d1..61dfc73580 100644 --- a/lib/iris/tests/test_netcdf.py +++ b/lib/iris/tests/test_netcdf.py @@ -499,6 +499,146 @@ def test_default_units(self): cubes[0].cell_measure("areas").units, as_unit("unknown") ) + def test_um_stash_source(self): + """Test that um_stash_source is converted into a STASH code""" + # Note: using a CDL string as a test data reference, rather than a binary file. + ref_cdl = """ + netcdf cm_attr { + dimensions: + axv = 3 ; + ayv = 2 ; + variables: + int64 qqv(ayv, axv) ; + qqv:long_name = "qq" ; + qqv:ancillary_variables = "my_av" ; + qqv:cell_measures = "area: my_areas" ; + qqv:um_stash_source = "m01s02i003" ; + int64 ayv(ayv) ; + ayv:long_name = "y" ; + int64 axv(axv) ; + axv:units = "1" ; + axv:long_name = "x" ; + double my_av(axv) ; + my_av:long_name = "refs" ; + double my_areas(ayv, axv) ; + my_areas:long_name = "areas" ; + data: + axv = 11, 12, 13; + ayv = 21, 22; + my_areas = 110., 120., 130., 221., 231., 241.; + } + """ + self.tmpdir = tempfile.mkdtemp() + cdl_path = os.path.join(self.tmpdir, "tst.cdl") + nc_path = os.path.join(self.tmpdir, "tst.nc") + # Write CDL string into a temporary CDL file. + with open(cdl_path, "w") as f_out: + f_out.write(ref_cdl) + # Use ncgen to convert this into an actual (temporary) netCDF file. + command = "ncgen -o {} {}".format(nc_path, cdl_path) + check_call(command, shell=True) + # Load with iris.fileformats.netcdf.load_cubes, and check expected content. + cubes = list(nc_load_cubes(nc_path)) + self.assertEqual(len(cubes), 1) + self.assertEqual( + cubes[0].attributes["STASH"], iris.fileformats.pp.STASH(1, 2, 3) + ) + + def test_ukmo__um_stash_source_priority(self): + """ + Test that ukmo__um_stash_source is converted into a STASH code with a + higher priority than um_stash_source. + """ + # Note: using a CDL string as a test data reference, rather than a binary file. + ref_cdl = """ + netcdf cm_attr { + dimensions: + axv = 3 ; + ayv = 2 ; + variables: + int64 qqv(ayv, axv) ; + qqv:long_name = "qq" ; + qqv:ancillary_variables = "my_av" ; + qqv:cell_measures = "area: my_areas" ; + qqv:um_stash_source = "m01s02i003" ; + qqv:ukmo__um_stash_source = "m09s08i007" ; + int64 ayv(ayv) ; + ayv:long_name = "y" ; + int64 axv(axv) ; + axv:units = "1" ; + axv:long_name = "x" ; + double my_av(axv) ; + my_av:long_name = "refs" ; + double my_areas(ayv, axv) ; + my_areas:long_name = "areas" ; + data: + axv = 11, 12, 13; + ayv = 21, 22; + my_areas = 110., 120., 130., 221., 231., 241.; + } + """ + self.tmpdir = tempfile.mkdtemp() + cdl_path = os.path.join(self.tmpdir, "tst.cdl") + nc_path = os.path.join(self.tmpdir, "tst.nc") + # Write CDL string into a temporary CDL file. + with open(cdl_path, "w") as f_out: + f_out.write(ref_cdl) + # Use ncgen to convert this into an actual (temporary) netCDF file. + command = "ncgen -o {} {}".format(nc_path, cdl_path) + check_call(command, shell=True) + # Load with iris.fileformats.netcdf.load_cubes, and check expected content. + cubes = list(nc_load_cubes(nc_path)) + self.assertEqual(len(cubes), 1) + self.assertEqual( + cubes[0].attributes["STASH"], iris.fileformats.pp.STASH(9, 8, 7) + ) + + def test_bad_um_stash_source(self): + """Test that um_stash_source not in strict MSI form is kept""" + # Note: using a CDL string as a test data reference, rather than a binary file. + ref_cdl = """ + netcdf cm_attr { + dimensions: + axv = 3 ; + ayv = 2 ; + variables: + int64 qqv(ayv, axv) ; + qqv:long_name = "qq" ; + qqv:ancillary_variables = "my_av" ; + qqv:cell_measures = "area: my_areas" ; + qqv:um_stash_source = "10*m01s02i003" ; + int64 ayv(ayv) ; + ayv:long_name = "y" ; + int64 axv(axv) ; + axv:units = "1" ; + axv:long_name = "x" ; + double my_av(axv) ; + my_av:long_name = "refs" ; + double my_areas(ayv, axv) ; + my_areas:long_name = "areas" ; + data: + axv = 11, 12, 13; + ayv = 21, 22; + my_areas = 110., 120., 130., 221., 231., 241.; + } + """ + self.tmpdir = tempfile.mkdtemp() + cdl_path = os.path.join(self.tmpdir, "tst.cdl") + nc_path = os.path.join(self.tmpdir, "tst.nc") + # Write CDL string into a temporary CDL file. + with open(cdl_path, "w") as f_out: + f_out.write(ref_cdl) + # Use ncgen to convert this into an actual (temporary) netCDF file. + command = "ncgen -o {} {}".format(nc_path, cdl_path) + check_call(command, shell=True) + # Load with iris.fileformats.netcdf.load_cubes, and check expected content. + cubes = list(nc_load_cubes(nc_path)) + self.assertEqual(len(cubes), 1) + self.assertFalse(hasattr(cubes[0].attributes, "STASH")) + self.assertEqual( + cubes[0].attributes["um_stash_source"], "10*m01s02i003" + ) + def test_units(self): # Test exercising graceful cube and coordinate units loading. cube0, cube1 = sorted( diff --git a/lib/iris/tests/test_pp_module.py b/lib/iris/tests/test_pp_module.py index 6c916e166a..7eebfb07b3 100644 --- a/lib/iris/tests/test_pp_module.py +++ b/lib/iris/tests/test_pp_module.py @@ -154,9 +154,11 @@ def test_lbtim_access(self): self.assertEqual(self.r[0].lbtim.ic, 2) def test_t1_t2_access(self): + field = self.r[0] + calander = "360_day" self.assertEqual( - self.r[0].t1.timetuple(), - cftime.datetime(1994, 12, 1, 0, 0).timetuple(), + field.t1.timetuple(), + cftime.datetime(1994, 12, 1, 0, 0, calendar=calander).timetuple(), ) def test_save_single(self): diff --git a/lib/iris/tests/unit/analysis/scipy_interpolate/__init__.py b/lib/iris/tests/unit/analysis/scipy_interpolate/__init__.py new file mode 100644 index 0000000000..67218194c2 --- /dev/null +++ b/lib/iris/tests/unit/analysis/scipy_interpolate/__init__.py @@ -0,0 +1,6 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the LGPL license. +# See COPYING and COPYING.LESSER in the root of the repository for full +# licensing details. +"""Unit tests for the :mod:`iris.analysis.scipy_interpolate` module.""" diff --git a/lib/iris/tests/unit/cube/test_Cube.py b/lib/iris/tests/unit/cube/test_Cube.py index 4f78d697b6..e6e973c58a 100644 --- a/lib/iris/tests/unit/cube/test_Cube.py +++ b/lib/iris/tests/unit/cube/test_Cube.py @@ -1781,6 +1781,14 @@ def test_aligned_exclusive(self): self.assertEqual(result.data[0, 0, 0], 171) self.assertEqual(result.data[0, 0, -1], 189) + def test_aligned_bounds_at_modulus(self): + cube = create_cube(-179.5, 180.5, bounds=True) + result = cube.intersection(longitude=(0, 360)) + self.assertArrayEqual(result.coord("longitude").bounds[0], [0, 1]) + self.assertArrayEqual(result.coord("longitude").bounds[-1], [359, 360]) + self.assertEqual(result.data[0, 0, 0], 180) + self.assertEqual(result.data[0, 0, -1], 179) + def test_negative_misaligned_points_inside(self): cube = create_cube(0, 360, bounds=True) result = cube.intersection(longitude=(-10.25, 10.25)) @@ -1856,8 +1864,35 @@ def test_numerical_tolerance(self): # modulus wrapping cube = create_cube(28.5, 68.5, bounds=True) result = cube.intersection(longitude=(27.74, 68.61)) - self.assertAlmostEqual(result.coord("longitude").points[0], 28.5) - self.assertAlmostEqual(result.coord("longitude").points[-1], 67.5) + result_lons = result.coord("longitude") + self.assertAlmostEqual(result_lons.points[0], 28.5) + self.assertAlmostEqual(result_lons.points[-1], 67.5) + dtype = result_lons.dtype + np.testing.assert_array_almost_equal( + result_lons.bounds[0], np.array([28.0, 29.0], dtype=dtype) + ) + np.testing.assert_array_almost_equal( + result_lons.bounds[-1], np.array([67.0, 68.0], dtype=dtype) + ) + + def test_numerical_tolerance_wrapped(self): + # test the tolerance on the coordinate value causes modulus wrapping + # where appropriate + cube = create_cube(0.5, 3600.5, bounds=True) + lons = cube.coord("longitude") + lons.points = lons.points / 10 + lons.bounds = lons.bounds / 10 + result = cube.intersection(longitude=(-60, 60)) + result_lons = result.coord("longitude") + self.assertAlmostEqual(result_lons.points[0], -60.05) + self.assertAlmostEqual(result_lons.points[-1], 60.05) + dtype = result_lons.dtype + np.testing.assert_array_almost_equal( + result_lons.bounds[0], np.array([-60.1, -60.0], dtype=dtype) + ) + np.testing.assert_array_almost_equal( + result_lons.bounds[-1], np.array([60.0, 60.1], dtype=dtype) + ) def unrolled_cube(): diff --git a/lib/iris/tests/unit/fileformats/nc_load_rules/actions/test__miscellaneous.py b/lib/iris/tests/unit/fileformats/nc_load_rules/actions/test__miscellaneous.py index 4ed90fd79a..a8e44747dd 100644 --- a/lib/iris/tests/unit/fileformats/nc_load_rules/actions/test__miscellaneous.py +++ b/lib/iris/tests/unit/fileformats/nc_load_rules/actions/test__miscellaneous.py @@ -83,14 +83,16 @@ def test_stash_altname(self): self.check_result(cube, stashcode=self.stashcode) def test_stash_empty(self): - msg = "Expected STASH code MSI string" - with self.assertRaisesRegex(ValueError, msg): - self.run_testcase(ukmo__um_stash_source="") + value = "" + cube = self.run_testcase(ukmo__um_stash_source=value) + self.assertNotIn("STASH", cube.attributes) + self.assertEqual(cube.attributes["ukmo__um_stash_source"], value) def test_stash_invalid(self): - msg = "Expected STASH code MSI string" - with self.assertRaisesRegex(ValueError, msg): - self.run_testcase(ukmo__um_stash_source="XXX") + value = "XXX" + cube = self.run_testcase(ukmo__um_stash_source="XXX") + self.assertNotIn("STASH", cube.attributes) + self.assertEqual(cube.attributes["ukmo__um_stash_source"], value) def test_processflags_single(self): cube = self.run_testcase(ukmo__process_flags="this") diff --git a/lib/iris/tests/unit/fileformats/pp/test_PPDataProxy.py b/lib/iris/tests/unit/fileformats/pp/test_PPDataProxy.py index b33779b2cf..d70e573296 100644 --- a/lib/iris/tests/unit/fileformats/pp/test_PPDataProxy.py +++ b/lib/iris/tests/unit/fileformats/pp/test_PPDataProxy.py @@ -11,8 +11,6 @@ from unittest import mock -import numpy as np - from iris.fileformats.pp import PPDataProxy, SplittableInt @@ -35,128 +33,5 @@ def test_lbpack_raw(self): self.assertEqual(proxy.lbpack.n4, lbpack // 1000 % 10) -class SliceTranslator: - """ - Class to translate an array-indexing expression into a tuple of keys. - - An instance just returns the argument of its __getitem__ call. - - """ - - def __getitem__(self, keys): - return keys - - -# A multidimensional-indexable object that returns its index keys, so we can -# use multidimensional-indexing notation to specify a slicing expression. -Slices = SliceTranslator() - - -class Test__getitem__slicing(tests.IrisTest): - def _check_slicing( - self, test_shape, indices, result_shape, data_was_fetched=True - ): - # Check behaviour of the getitem call with specific slicings. - # Especially: check cases where a fetch does *not* read from the file. - # This is necessary because, since Dask 2.0, the "from_array" function - # takes a zero-length slice of its array argument, to capture array - # metadata, and in those cases we want to avoid file access. - test_dtype = np.dtype(np.float32) - proxy = PPDataProxy( - shape=test_shape, - src_dtype=test_dtype, - path=None, - offset=None, - data_len=None, - lbpack=0, # Note: a 'real' value is needed. - boundary_packing=None, - mdi=None, - ) - - # Mock out the file-open call, to see if the file would be read. - builtin_open_func_name = "builtins.open" - mock_fileopen = self.patch(builtin_open_func_name) - - # Also mock out the 'databytes_to_shaped_array' call, to fake minimal - # operation in the cases where file-open *does* get called. - fake_data = np.zeros(test_shape, dtype=test_dtype) - self.patch( - "iris.fileformats.pp._data_bytes_to_shaped_array", - mock.MagicMock(return_value=fake_data), - ) - - # Test the requested indexing operation. - result = proxy.__getitem__(indices) - - # Check the behaviour and results were as expected. - self.assertEqual(mock_fileopen.called, data_was_fetched) - self.assertIsInstance(result, np.ndarray) - self.assertEqual(result.dtype, test_dtype) - self.assertEqual(result.shape, result_shape) - - def test_slicing_1d_normal(self): - # A 'normal' 1d testcase with no empty slices. - self._check_slicing( - test_shape=(3,), - indices=Slices[1:10], - result_shape=(2,), - data_was_fetched=True, - ) - - def test_slicing_1d_empty(self): - # A 1d testcase with an empty slicing. - self._check_slicing( - test_shape=(3,), - indices=Slices[0:0], - result_shape=(0,), - data_was_fetched=False, - ) - - def test_slicing_2d_normal(self): - # A 2d testcase with no empty slices. - self._check_slicing( - test_shape=(3, 4), - indices=Slices[2, :3], - result_shape=(3,), - data_was_fetched=True, - ) - - def test_slicing_2d_allempty(self): - # A 2d testcase with all empty slices. - self._check_slicing( - test_shape=(3, 4), - indices=Slices[0:0, 0:0], - result_shape=(0, 0), - data_was_fetched=False, - ) - - def test_slicing_2d_empty_dim0(self): - # A 2d testcase with an empty slice. - self._check_slicing( - test_shape=(3, 4), - indices=Slices[0:0], - result_shape=(0, 4), - data_was_fetched=False, - ) - - def test_slicing_2d_empty_dim1(self): - # A 2d testcase with an empty slice, and an integer index. - self._check_slicing( - test_shape=(3, 4), - indices=Slices[1, 0:0], - result_shape=(0,), - data_was_fetched=False, - ) - - def test_slicing_complex(self): - # Multiple dimensions with multiple empty slices. - self._check_slicing( - test_shape=(3, 4, 2, 5, 6, 3, 7), - indices=Slices[1:3, 2, 0:0, :, 1:1, :100], - result_shape=(2, 0, 5, 0, 3, 7), - data_was_fetched=False, - ) - - if __name__ == "__main__": tests.main() diff --git a/lib/iris/tests/unit/fileformats/pp/test_save.py b/lib/iris/tests/unit/fileformats/pp/test_save.py index 1f3d982bb7..45012dc8bd 100644 --- a/lib/iris/tests/unit/fileformats/pp/test_save.py +++ b/lib/iris/tests/unit/fileformats/pp/test_save.py @@ -240,7 +240,7 @@ def test_t2_time_mean(self): def test_t2_no_time_mean(self): cube = _get_single_time_cube(set_time_mean=False) - expected = cftime.datetime(0, 0, 0) + expected = cftime.datetime(0, 0, 0, calendar=None, has_year_zero=True) with mock.patch( "iris.fileformats.pp.PPField3", autospec=True diff --git a/lib/iris/tests/unit/fileformats/pp_load_rules/test__convert_time_coords.py b/lib/iris/tests/unit/fileformats/pp_load_rules/test__convert_time_coords.py index 3db6c167e8..d975884cb0 100644 --- a/lib/iris/tests/unit/fileformats/pp_load_rules/test__convert_time_coords.py +++ b/lib/iris/tests/unit/fileformats/pp_load_rules/test__convert_time_coords.py @@ -15,7 +15,7 @@ import unittest -from cf_units import CALENDAR_GREGORIAN, Unit +from cf_units import CALENDAR_360_DAY, CALENDAR_GREGORIAN, Unit from cftime import datetime as nc_datetime import numpy as np @@ -48,7 +48,9 @@ class TestLBTIMx0x_SingleTimepoint(TestField): def _check_timepoint(self, lbcode, expect_match=True): lbtim = _lbtim(ib=0, ic=1) t1 = nc_datetime(1970, 1, 1, hour=6, minute=0, second=0) - t2 = nc_datetime(0, 0, 0) # not used in result + t2 = nc_datetime( + 0, 0, 0, calendar=None, has_year_zero=True + ) # not used in result lbft = None # unused coords_and_dims = _convert_time_coords( lbcode=lbcode, @@ -297,8 +299,8 @@ def test_time_cross_section(self): class TestLBTIMx2x_ZeroYear(TestField): def test_(self): lbtim = _lbtim(ib=2, ic=1) - t1 = nc_datetime(0, 1, 1) - t2 = nc_datetime(0, 1, 31, 23, 59, 00) + t1 = nc_datetime(0, 1, 1, has_year_zero=True) + t2 = nc_datetime(0, 1, 31, 23, 59, 00, has_year_zero=True) lbft = 0 lbcode = _lbcode(1) coords_and_dims = _convert_time_coords( @@ -315,8 +317,8 @@ def test_(self): class TestLBTIMxxx_Unhandled(TestField): def test_unrecognised(self): lbtim = _lbtim(ib=4, ic=1) - t1 = nc_datetime(0, 0, 0) - t2 = nc_datetime(0, 0, 0) + t1 = nc_datetime(0, 0, 0, calendar=None, has_year_zero=True) + t2 = nc_datetime(0, 0, 0, calendar=None, has_year_zero=True) lbft = None lbcode = _lbcode(0) coords_and_dims = _convert_time_coords( @@ -334,8 +336,13 @@ class TestLBCODE3xx(TestField): def test(self): lbcode = _lbcode(value=31323) lbtim = _lbtim(ib=2, ic=2) - t1 = nc_datetime(1970, 1, 3, hour=0, minute=0, second=0) - t2 = nc_datetime(1970, 1, 4, hour=0, minute=0, second=0) + calendar = CALENDAR_360_DAY + t1 = nc_datetime( + 1970, 1, 3, hour=0, minute=0, second=0, calendar=calendar + ) + t2 = nc_datetime( + 1970, 1, 4, hour=0, minute=0, second=0, calendar=calendar + ) lbft = 24 * 4 coords_and_dims = _convert_time_coords( lbcode=lbcode, diff --git a/lib/iris/tests/unit/fileformats/pp_load_rules/test__epoch_date_hours.py b/lib/iris/tests/unit/fileformats/pp_load_rules/test__epoch_date_hours.py index 8f4a13c95a..2877d6ea89 100644 --- a/lib/iris/tests/unit/fileformats/pp_load_rules/test__epoch_date_hours.py +++ b/lib/iris/tests/unit/fileformats/pp_load_rules/test__epoch_date_hours.py @@ -30,27 +30,28 @@ class TestEpochHours__gregorian(tests.IrisTest): def setUp(self): - self.hrs_unit = Unit( - "hours since epoch", calendar=cf_units.CALENDAR_GREGORIAN - ) + self.calendar = cf_units.CALENDAR_GREGORIAN + self.hrs_unit = Unit("hours since epoch", calendar=self.calendar) def test_1970_1_1(self): - test_date = nc_datetime(1970, 1, 1) + test_date = nc_datetime(1970, 1, 1, calendar=self.calendar) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, 0.0) def test_ymd_1_1_1(self): - test_date = nc_datetime(1, 1, 1) + test_date = nc_datetime(1, 1, 1, calendar=self.calendar) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17259936.0) def test_year_0(self): - test_date = nc_datetime(0, 1, 1) + test_date = nc_datetime( + 0, 1, 1, calendar=self.calendar, has_year_zero=True + ) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17268720.0) def test_ymd_0_0_0(self): - test_date = nc_datetime(0, 0, 0) + test_date = nc_datetime(0, 0, 0, calendar=None, has_year_zero=True) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17269488.0) @@ -60,7 +61,15 @@ def test_ymd_0_preserves_timeofday(self): hrs + 1.0 / 60 * mins + 1.0 / 3600 * secs + (1.0e-6) / 3600 * usecs ) test_date = nc_datetime( - 0, 0, 0, hour=hrs, minute=mins, second=secs, microsecond=usecs + 0, + 0, + 0, + hour=hrs, + minute=mins, + second=secs, + microsecond=usecs, + calendar=None, + has_year_zero=True, ) result = epoch_hours_call(self.hrs_unit, test_date) # NOTE: the calculation is only accurate to approx +/- 0.5 seconds @@ -73,66 +82,69 @@ def test_ymd_0_preserves_timeofday(self): class TestEpochHours__360day(tests.IrisTest): def setUp(self): - self.hrs_unit = Unit( - "hours since epoch", calendar=cf_units.CALENDAR_360_DAY - ) + self.calendar = cf_units.CALENDAR_360_DAY + self.hrs_unit = Unit("hours since epoch", calendar=self.calendar) def test_1970_1_1(self): - test_date = nc_datetime(1970, 1, 1) + test_date = nc_datetime(1970, 1, 1, calendar=self.calendar) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, 0.0) def test_ymd_1_1_1(self): - test_date = nc_datetime(1, 1, 1) + test_date = nc_datetime(1, 1, 1, calendar=self.calendar) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17012160.0) def test_year_0(self): - test_date = nc_datetime(0, 1, 1) + test_date = nc_datetime( + 0, 1, 1, calendar=self.calendar, has_year_zero=True + ) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17020800.0) def test_ymd_0_0_0(self): - test_date = nc_datetime(0, 0, 0) + test_date = nc_datetime(0, 0, 0, calendar=None, has_year_zero=True) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17021544.0) class TestEpochHours__365day(tests.IrisTest): def setUp(self): - self.hrs_unit = Unit( - "hours since epoch", calendar=cf_units.CALENDAR_365_DAY - ) + self.calendar = cf_units.CALENDAR_365_DAY + self.hrs_unit = Unit("hours since epoch", calendar=self.calendar) def test_1970_1_1(self): - test_date = nc_datetime(1970, 1, 1) + test_date = nc_datetime(1970, 1, 1, calendar=self.calendar) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, 0.0) def test_ymd_1_1_1(self): - test_date = nc_datetime(1, 1, 1) + test_date = nc_datetime(1, 1, 1, calendar=self.calendar) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17248440.0) def test_year_0(self): - test_date = nc_datetime(0, 1, 1) + test_date = nc_datetime( + 0, 1, 1, calendar=self.calendar, has_year_zero=True + ) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17257200.0) def test_ymd_0_0_0(self): - test_date = nc_datetime(0, 0, 0) + test_date = nc_datetime(0, 0, 0, calendar=None, has_year_zero=True) result = epoch_hours_call(self.hrs_unit, test_date) self.assertEqual(result, -17257968.0) class TestEpochHours__invalid_calendar(tests.IrisTest): def test_bad_calendar(self): + self.calendar = cf_units.CALENDAR_ALL_LEAP # Setup a unit with an unrecognised calendar - hrs_unit = Unit( - "hours since epoch", calendar=cf_units.CALENDAR_ALL_LEAP - ) + hrs_unit = Unit("hours since epoch", calendar=self.calendar) # Test against a date with year=0, which requires calendar correction. - test_date = nc_datetime(0, 1, 1) + test_date = nc_datetime( + 0, 1, 1, calendar=self.calendar, has_year_zero=True + ) # Check that this causes an error. with self.assertRaisesRegex(ValueError, "unrecognised calendar"): epoch_hours_call(hrs_unit, test_date) diff --git a/lib/iris/tests/unit/lazy_data/test_co_realise_cubes.py b/lib/iris/tests/unit/lazy_data/test_co_realise_cubes.py index f746f4847d..0c10d69c16 100644 --- a/lib/iris/tests/unit/lazy_data/test_co_realise_cubes.py +++ b/lib/iris/tests/unit/lazy_data/test_co_realise_cubes.py @@ -69,12 +69,12 @@ def test_combined_access(self): cube_e = Cube(derived_e) co_realise_cubes(cube_a, cube_b, cube_c, cube_d, cube_e) # Though used more than once, the source data should only get fetched - # twice by dask. Once when dask performs an initial data access with - # no data payload to ascertain the metadata associated with the - # dask.array (this access is specific to dask 2+, see - # dask.array.utils.meta_from_array), and again when the whole data is - # accessed. - self.assertEqual(wrapped_array.access_count, 2) + # once by dask, when the whole data is accessed. + # This also ensures that dask does *not* perform an initial data + # access with no data payload to ascertain the metadata associated with + # the dask.array (this access is specific to dask 2+, + # see dask.array.utils.meta_from_array). + self.assertEqual(wrapped_array.access_count, 1) if __name__ == "__main__": diff --git a/lib/iris/tests/unit/plot/test__fixup_dates.py b/lib/iris/tests/unit/plot/test__fixup_dates.py index a932c321a7..157780dcae 100644 --- a/lib/iris/tests/unit/plot/test__fixup_dates.py +++ b/lib/iris/tests/unit/plot/test__fixup_dates.py @@ -43,13 +43,14 @@ def test_gregorian_calendar_sub_second(self): @tests.skip_nc_time_axis def test_360_day_calendar(self): - unit = Unit("days since 2000-02-25 00:00:00", calendar="360_day") + calendar = "360_day" + unit = Unit("days since 2000-02-25 00:00:00", calendar=calendar) coord = AuxCoord([3, 4, 5], "time", units=unit) result = _fixup_dates(coord, coord.points) expected_datetimes = [ - cftime.datetime(2000, 2, 28), - cftime.datetime(2000, 2, 29), - cftime.datetime(2000, 2, 30), + cftime.datetime(2000, 2, 28, calendar=calendar), + cftime.datetime(2000, 2, 29, calendar=calendar), + cftime.datetime(2000, 2, 30, calendar=calendar), ] self.assertArrayEqual( [cdt.datetime for cdt in result], expected_datetimes @@ -57,13 +58,14 @@ def test_360_day_calendar(self): @tests.skip_nc_time_axis def test_365_day_calendar(self): - unit = Unit("minutes since 2000-02-25 00:00:00", calendar="365_day") + calendar = "365_day" + unit = Unit("minutes since 2000-02-25 00:00:00", calendar=calendar) coord = AuxCoord([30, 60, 150], "time", units=unit) result = _fixup_dates(coord, coord.points) expected_datetimes = [ - cftime.datetime(2000, 2, 25, 0, 30), - cftime.datetime(2000, 2, 25, 1, 0), - cftime.datetime(2000, 2, 25, 2, 30), + cftime.datetime(2000, 2, 25, 0, 30, calendar=calendar), + cftime.datetime(2000, 2, 25, 1, 0, calendar=calendar), + cftime.datetime(2000, 2, 25, 2, 30, calendar=calendar), ] self.assertArrayEqual( [cdt.datetime for cdt in result], expected_datetimes @@ -71,10 +73,11 @@ def test_365_day_calendar(self): @tests.skip_nc_time_axis def test_360_day_calendar_attribute(self): - unit = Unit("days since 2000-02-01 00:00:00", calendar="360_day") + calendar = "360_day" + unit = Unit("days since 2000-02-01 00:00:00", calendar=calendar) coord = AuxCoord([0, 3, 6], "time", units=unit) result = _fixup_dates(coord, coord.points) - self.assertEqual(result[0].calendar, "360_day") + self.assertEqual(result[0].calendar, calendar) if __name__ == "__main__": diff --git a/lib/iris/tests/unit/time/__init__.py b/lib/iris/tests/unit/time/__init__.py new file mode 100644 index 0000000000..3483b92e62 --- /dev/null +++ b/lib/iris/tests/unit/time/__init__.py @@ -0,0 +1,6 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the LGPL license. +# See COPYING and COPYING.LESSER in the root of the repository for full +# licensing details. +"""Unit tests for the :mod:`iris.time` module.""" diff --git a/lib/iris/tests/unit/time/test_PartialDateTime.py b/lib/iris/tests/unit/time/test_PartialDateTime.py index 43c1984f46..cfffafea2c 100644 --- a/lib/iris/tests/unit/time/test_PartialDateTime.py +++ b/lib/iris/tests/unit/time/test_PartialDateTime.py @@ -224,12 +224,12 @@ def setUp(self): self.expected_value = EQ_EXPECTATIONS def test_cftime_equal(self): - pdt = PartialDateTime(month=3, microsecond=2) + pdt = PartialDateTime(month=3, second=2) other = cftime.datetime(year=2013, month=3, day=20, second=2) self.assertTrue(pdt == other) def test_cftime_not_equal(self): - pdt = PartialDateTime(month=3, microsecond=2) + pdt = PartialDateTime(month=3, second=2) other = cftime.datetime(year=2013, month=4, day=20, second=2) self.assertFalse(pdt == other) diff --git a/lib/iris/time.py b/lib/iris/time.py index cd2b780421..51aac3d46d 100644 --- a/lib/iris/time.py +++ b/lib/iris/time.py @@ -155,8 +155,17 @@ def __eq__(self, other): other_attr = other.microsecond if attr is not None and attr != other_attr: result = False + except AttributeError: - result = NotImplemented + result = other.__eq__(self) + if result is NotImplemented: + # Equality is undefined between these objects. We don't + # want Python to fall back to the default `object` + # behaviour (which compares using object IDs), so we raise + # an exception here instead. + fmt = "unable to compare PartialDateTime with {}" + raise TypeError(fmt.format(type(other))) + return result def __ne__(self, other): @@ -164,14 +173,3 @@ def __ne__(self, other): if result is not NotImplemented: result = not result return result - - def __cmp__(self, other): - # Since we've defined all the rich comparison operators (via - # functools.total_ordering), we can only reach this point if - # neither this class nor the other class had a rich comparison - # that could handle the type combination. - # We don't want Python to fall back to the default `object` - # behaviour (which compares using object IDs), so we raise an - # exception here instead. - fmt = "unable to compare PartialDateTime with {}" - raise TypeError(fmt.format(type(other))) diff --git a/lib/iris/util.py b/lib/iris/util.py index 37e53fd3e0..9cadf85f0f 100644 --- a/lib/iris/util.py +++ b/lib/iris/util.py @@ -961,67 +961,6 @@ def __lt__(self, other): return NotImplemented -def _array_slice_ifempty(keys, shape, dtype): - """ - Detect cases where an array slice will contain no data, as it contains a - zero-length dimension, and produce an equivalent result for those cases. - - The function indicates 'empty' slicing cases, by returning an array equal - to the slice result in those cases. - - Args: - - * keys (indexing key, or tuple of keys): - The argument from an array __getitem__ call. - Only tuples of integers and slices are supported, in particular no - newaxis, ellipsis or array keys. - These are the types of array access usage we expect from Dask. - * shape (tuple of int): - The shape of the array being indexed. - * dtype (numpy.dtype): - The dtype of the array being indexed. - - Returns: - result (np.ndarray or None): - If 'keys' contains a slice(0, 0), this is an ndarray of the correct - resulting shape and provided dtype. - Otherwise it is None. - - .. note:: - - This is used to prevent DataProxy arraylike objects from fetching their - file data when wrapped as Dask arrays. - This is because, for Dask >= 2.0, the "dask.array.from_array" call - performs a fetch like [0:0, 0:0, ...], to 'snapshot' array metadata. - This function enables us to avoid triggering a file data fetch in those - cases : This is consistent because the result will not contain any - actual data content. - - """ - # Convert a single key into a 1-tuple, so we always have a tuple of keys. - if isinstance(keys, tuple): - keys_tuple = keys - else: - keys_tuple = (keys,) - - if any(key == slice(0, 0) for key in keys_tuple): - # An 'empty' slice is present : Return a 'fake' array instead. - target_shape = list(shape) - for i_dim, key in enumerate(keys_tuple): - if key == slice(0, 0): - # Reduce dims with empty slicing to length 0. - target_shape[i_dim] = 0 - # Create a prototype result : no memory usage, as some dims are 0. - result = np.zeros(target_shape, dtype=dtype) - # Index with original keys to produce the desired result shape. - # Note : also ok in 0-length dims, as the slice is always '0:0'. - result = result[keys] - else: - result = None - - return result - - def create_temp_filename(suffix=""): """Return a temporary file name. diff --git a/requirements/ci/nox.lock/py37-linux-64.lock b/requirements/ci/nox.lock/py37-linux-64.lock index 718af91f28..a1370c8395 100644 --- a/requirements/ci/nox.lock/py37-linux-64.lock +++ b/requirements/ci/nox.lock/py37-linux-64.lock @@ -1,6 +1,6 @@ # Generated by conda-lock. # platform: linux-64 -# env_hash: 6c711a9771dd36da4e85fae63fd584e6a1bba39d304d13e747db86a4f861804e +# env_hash: 6dc0000cc1dab34473c629d4c739a5828bcc51d769c258d205d0bbe3817e25ae @EXPLICIT https://conda.anaconda.org/conda-forge/linux-64/_libgcc_mutex-0.1-conda_forge.tar.bz2#d7c89558ba9fa0495403155b64376d81 https://conda.anaconda.org/conda-forge/linux-64/ca-certificates-2021.5.30-ha878542_0.tar.bz2#6a777890e94194dc94a29a76d2a7e721 @@ -100,7 +100,7 @@ https://conda.anaconda.org/conda-forge/linux-64/lcms2-2.12-hddcbb42_0.tar.bz2#79 https://conda.anaconda.org/conda-forge/linux-64/libcurl-7.78.0-h2574ce0_0.tar.bz2#9c06cc5692dcd0b91699413fcc18405b https://conda.anaconda.org/conda-forge/linux-64/libpq-13.3-hd57d9b9_0.tar.bz2#66ef2cacc483205b7d303f7b02601c3b https://conda.anaconda.org/conda-forge/linux-64/libwebp-1.2.0-h3452ae3_0.tar.bz2#8f4e19a8988c38feec7db41bcd0bf0d0 -https://conda.anaconda.org/conda-forge/linux-64/nss-3.67-hb5efdd6_0.tar.bz2#3f2a4bc7d5fded1327ff1b8c61faae53 +https://conda.anaconda.org/conda-forge/linux-64/nss-3.69-hb5efdd6_0.tar.bz2#da3ad4247c36e2248cfc2aadb1e423b0 https://conda.anaconda.org/conda-forge/linux-64/python-3.7.10-hffdb5ce_100_cpython.tar.bz2#7425fffa658971915f595e9110163c3c https://conda.anaconda.org/conda-forge/linux-64/xorg-libxext-1.3.4-h7f98852_1.tar.bz2#536cc5db4d0a3ba0630541aec064b5e4 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxrender-0.9.10-h7f98852_1003.tar.bz2#f59c1242cc1dd93e72c2ee2b360979eb @@ -123,10 +123,13 @@ https://conda.anaconda.org/conda-forge/noarch/imagesize-1.2.0-py_0.tar.bz2#5879b https://conda.anaconda.org/conda-forge/noarch/iris-sample-data-2.3.0-pyh9f0ad1d_0.tar.bz2#e4a33192da1a6dc4967ba18c6c765945 https://conda.anaconda.org/conda-forge/linux-64/libgd-2.3.2-h78a0170_0.tar.bz2#ac0c23e6f3bbb61569781f00b5666f97 https://conda.anaconda.org/conda-forge/noarch/locket-0.2.0-py_2.tar.bz2#709e8671651c7ec3d1ad07800339ff1d +https://conda.anaconda.org/conda-forge/noarch/mccabe-0.6.1-py_1.tar.bz2#a326cb400c1ccd91789f3e7d02124d61 https://conda.anaconda.org/conda-forge/noarch/nose-1.3.7-py_1006.tar.bz2#382019d5f8e9362ef6f60a8d4e7bce8f https://conda.anaconda.org/conda-forge/noarch/olefile-0.46-pyh9f0ad1d_1.tar.bz2#0b2e68acc8c78c8cc392b90983481f58 https://conda.anaconda.org/conda-forge/linux-64/proj-7.2.0-h277dcde_2.tar.bz2#db654ee11298d3463bad67445707654c +https://conda.anaconda.org/conda-forge/noarch/pycodestyle-2.7.0-pyhd8ed1ab_0.tar.bz2#0234673eb2ecfbdf4e54574ab4d95f81 https://conda.anaconda.org/conda-forge/noarch/pycparser-2.20-pyh9f0ad1d_2.tar.bz2#aa798d50ffd182a0f6f31478c7f434f6 +https://conda.anaconda.org/conda-forge/noarch/pyflakes-2.3.1-pyhd8ed1ab_0.tar.bz2#01e9ada82bd261ee2b6366aa832018cc https://conda.anaconda.org/conda-forge/noarch/pyparsing-2.4.7-pyh9f0ad1d_0.tar.bz2#626c4f20d5bf06dcec9cf2eaa31725c7 https://conda.anaconda.org/conda-forge/noarch/pyshp-2.1.3-pyh44b312d_0.tar.bz2#2d1867b980785eb44b8122184d8b42a6 https://conda.anaconda.org/conda-forge/linux-64/python_abi-3.7-2_cp37m.tar.bz2#afff88bf9a7048da740c70aeb8cdbb82 @@ -178,7 +181,7 @@ https://conda.anaconda.org/conda-forge/linux-64/pyyaml-5.4.1-py37h5e8e339_0.tar. https://conda.anaconda.org/conda-forge/linux-64/tornado-6.1-py37h5e8e339_1.tar.bz2#92449128c4639feae48d731ef2186099 https://conda.anaconda.org/conda-forge/noarch/zict-2.0.0-py_0.tar.bz2#4750152be22f24d695b3004c5e1712d3 https://conda.anaconda.org/conda-forge/linux-64/brotlipy-0.7.0-py37h5e8e339_1001.tar.bz2#871eed4ba322e7b3f200956a096b34e7 -https://conda.anaconda.org/conda-forge/linux-64/cftime-1.2.1-py37h161383b_1.tar.bz2#314ca8b00ed742f8c46a6cc68d84d90f +https://conda.anaconda.org/conda-forge/linux-64/cftime-1.5.0-py37h6f94858_0.tar.bz2#e252867ed3798d946b16576a575ccaa6 https://conda.anaconda.org/conda-forge/linux-64/click-8.0.1-py37h89c1867_0.tar.bz2#bb1ad97b5d8626f662b753f620c3c913 https://conda.anaconda.org/conda-forge/linux-64/cryptography-3.4.7-py37h5d9358c_0.tar.bz2#d811fb6a96ae0cf8c0a17457a8e67ff4 https://conda.anaconda.org/conda-forge/noarch/dask-core-2021.7.2-pyhd8ed1ab_0.tar.bz2#249982c77565b1040d8cc27dadf9dd89 @@ -202,6 +205,7 @@ https://conda.anaconda.org/conda-forge/linux-64/bokeh-2.1.1-py37hc8dfbb8_0.tar.b https://conda.anaconda.org/conda-forge/linux-64/cf-units-3.0.1.post0-py37hb1e94ed_0.tar.bz2#3e2728b1107710a5897324afd82bc78a https://conda.anaconda.org/conda-forge/linux-64/distributed-2021.7.2-py37h89c1867_0.tar.bz2#ecf2bb78c07203e6c23bd216ff5aae1e https://conda.anaconda.org/conda-forge/linux-64/esmf-8.1.1-mpi_mpich_h3dcaa78_100.tar.bz2#5b4bab1017226f2c03ba0fe02b783316 +https://conda.anaconda.org/conda-forge/noarch/flake8-3.9.2-pyhd8ed1ab_0.tar.bz2#37d685abea0a25c921431edda02ad143 https://conda.anaconda.org/conda-forge/linux-64/gtk2-2.24.33-h539f30e_1.tar.bz2#606777b4da3664d5c9415f5f165349fd https://conda.anaconda.org/conda-forge/noarch/identify-2.2.13-pyhd8ed1ab_0.tar.bz2#1889223441d67f7c30327c97a78b980f https://conda.anaconda.org/conda-forge/noarch/imagehash-4.2.1-pyhd8ed1ab_0.tar.bz2#01cc8698b6e1a124dc4f585516c27643 @@ -217,8 +221,8 @@ https://conda.anaconda.org/conda-forge/linux-64/cartopy-0.19.0.post1-py37h0c48da https://conda.anaconda.org/conda-forge/noarch/dask-2021.7.2-pyhd8ed1ab_0.tar.bz2#998ae0ba5626d98bb972359f776bc394 https://conda.anaconda.org/conda-forge/linux-64/esmpy-8.1.1-mpi_mpich_py37hf719a8e_100.tar.bz2#d608536dd44b60da923950c60619583d https://conda.anaconda.org/conda-forge/linux-64/graphviz-2.48.0-h85b4f2f_0.tar.bz2#286099ec11970542c294f8be9c43871e -https://conda.anaconda.org/conda-forge/noarch/nc-time-axis-1.2.0-pyhd8ed1ab_2.tar.bz2#ca7d092db9ebbd9a0102710e8d4b4a28 -https://conda.anaconda.org/conda-forge/linux-64/pre-commit-2.13.0-py37h89c1867_0.tar.bz2#a2ddf76626c4e7481f106fa08d5d77c5 +https://conda.anaconda.org/conda-forge/noarch/nc-time-axis-1.3.1-pyhd8ed1ab_2.tar.bz2#d37916d7bf9519a3e3abb902a4c06b69 +https://conda.anaconda.org/conda-forge/linux-64/pre-commit-2.14.0-py37h89c1867_0.tar.bz2#126bf6b478227945bf57824a8f15b4c5 https://conda.anaconda.org/conda-forge/linux-64/pyqtchart-5.12-py37he336c9b_7.tar.bz2#2b1959f3a87b5ad66690340ef921323c https://conda.anaconda.org/conda-forge/linux-64/pyqtwebengine-5.12.1-py37he336c9b_7.tar.bz2#15f5cbcafb4889bb41da2a0a0e338f2a https://conda.anaconda.org/conda-forge/noarch/pyugrid-0.3.1-py_2.tar.bz2#7d7361886fbcf2be663fd185bf6d244d diff --git a/requirements/ci/nox.lock/py38-linux-64.lock b/requirements/ci/nox.lock/py38-linux-64.lock index 84a1b19e5f..6918bac897 100644 --- a/requirements/ci/nox.lock/py38-linux-64.lock +++ b/requirements/ci/nox.lock/py38-linux-64.lock @@ -1,6 +1,6 @@ # Generated by conda-lock. # platform: linux-64 -# env_hash: fbb4556337a9f497fb9d021a1e2d2e8eae9f52dfc1d7859ee0f63d66e1842b0c +# env_hash: b18cda575ddf26d4adbc2e8fa98625982e3a50c991b0082207b9a5bd493af9b1 @EXPLICIT https://conda.anaconda.org/conda-forge/linux-64/_libgcc_mutex-0.1-conda_forge.tar.bz2#d7c89558ba9fa0495403155b64376d81 https://conda.anaconda.org/conda-forge/linux-64/ca-certificates-2021.5.30-ha878542_0.tar.bz2#6a777890e94194dc94a29a76d2a7e721 @@ -96,11 +96,10 @@ https://conda.anaconda.org/conda-forge/linux-64/gdk-pixbuf-2.42.6-h04a7f16_0.tar https://conda.anaconda.org/conda-forge/linux-64/glib-tools-2.68.3-h9c3ff4c_0.tar.bz2#2e9275303dd09a2e245faf31770a1416 https://conda.anaconda.org/conda-forge/linux-64/gstreamer-1.18.4-h76c114f_2.tar.bz2#5db765d4974fa89f64c1544eb2a552cb https://conda.anaconda.org/conda-forge/linux-64/gts-0.7.6-h64030ff_2.tar.bz2#112eb9b5b93f0c02e59aea4fd1967363 -https://conda.anaconda.org/conda-forge/linux-64/lcms2-2.12-hddcbb42_0.tar.bz2#797117394a4aa588de6d741b06fad80f https://conda.anaconda.org/conda-forge/linux-64/libcurl-7.78.0-h2574ce0_0.tar.bz2#9c06cc5692dcd0b91699413fcc18405b https://conda.anaconda.org/conda-forge/linux-64/libpq-13.3-hd57d9b9_0.tar.bz2#66ef2cacc483205b7d303f7b02601c3b https://conda.anaconda.org/conda-forge/linux-64/libwebp-1.2.0-h3452ae3_0.tar.bz2#8f4e19a8988c38feec7db41bcd0bf0d0 -https://conda.anaconda.org/conda-forge/linux-64/nss-3.67-hb5efdd6_0.tar.bz2#3f2a4bc7d5fded1327ff1b8c61faae53 +https://conda.anaconda.org/conda-forge/linux-64/nss-3.69-hb5efdd6_0.tar.bz2#da3ad4247c36e2248cfc2aadb1e423b0 https://conda.anaconda.org/conda-forge/linux-64/python-3.8.10-h49503c6_1_cpython.tar.bz2#69f7d6ef1f00c3a109b1b06279e6d6a9 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxext-1.3.4-h7f98852_1.tar.bz2#536cc5db4d0a3ba0630541aec064b5e4 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxrender-0.9.10-h7f98852_1003.tar.bz2#f59c1242cc1dd93e72c2ee2b360979eb @@ -123,10 +122,13 @@ https://conda.anaconda.org/conda-forge/noarch/imagesize-1.2.0-py_0.tar.bz2#5879b https://conda.anaconda.org/conda-forge/noarch/iris-sample-data-2.3.0-pyh9f0ad1d_0.tar.bz2#e4a33192da1a6dc4967ba18c6c765945 https://conda.anaconda.org/conda-forge/linux-64/libgd-2.3.2-h78a0170_0.tar.bz2#ac0c23e6f3bbb61569781f00b5666f97 https://conda.anaconda.org/conda-forge/noarch/locket-0.2.0-py_2.tar.bz2#709e8671651c7ec3d1ad07800339ff1d +https://conda.anaconda.org/conda-forge/noarch/mccabe-0.6.1-py_1.tar.bz2#a326cb400c1ccd91789f3e7d02124d61 https://conda.anaconda.org/conda-forge/noarch/nose-1.3.7-py_1006.tar.bz2#382019d5f8e9362ef6f60a8d4e7bce8f https://conda.anaconda.org/conda-forge/noarch/olefile-0.46-pyh9f0ad1d_1.tar.bz2#0b2e68acc8c78c8cc392b90983481f58 https://conda.anaconda.org/conda-forge/linux-64/proj-7.2.0-h277dcde_2.tar.bz2#db654ee11298d3463bad67445707654c +https://conda.anaconda.org/conda-forge/noarch/pycodestyle-2.7.0-pyhd8ed1ab_0.tar.bz2#0234673eb2ecfbdf4e54574ab4d95f81 https://conda.anaconda.org/conda-forge/noarch/pycparser-2.20-pyh9f0ad1d_2.tar.bz2#aa798d50ffd182a0f6f31478c7f434f6 +https://conda.anaconda.org/conda-forge/noarch/pyflakes-2.3.1-pyhd8ed1ab_0.tar.bz2#01e9ada82bd261ee2b6366aa832018cc https://conda.anaconda.org/conda-forge/noarch/pyparsing-2.4.7-pyh9f0ad1d_0.tar.bz2#626c4f20d5bf06dcec9cf2eaa31725c7 https://conda.anaconda.org/conda-forge/noarch/pyshp-2.1.3-pyh44b312d_0.tar.bz2#2d1867b980785eb44b8122184d8b42a6 https://conda.anaconda.org/conda-forge/linux-64/python_abi-3.8-2_cp38.tar.bz2#bfbb29d517281e78ac53e48d21e6e860 @@ -143,8 +145,8 @@ https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-serializinghtml-1.1. https://conda.anaconda.org/conda-forge/noarch/tblib-1.7.0-pyhd8ed1ab_0.tar.bz2#3d4afc31302aa7be471feb6be048ed76 https://conda.anaconda.org/conda-forge/noarch/toml-0.10.2-pyhd8ed1ab_0.tar.bz2#f832c45a477c78bebd107098db465095 https://conda.anaconda.org/conda-forge/noarch/toolz-0.11.1-py_0.tar.bz2#d1e66b58cb00b3817ad9f05eec098c00 -https://conda.anaconda.org/conda-forge/noarch/typing_extensions-3.10.0.0-pyha770c72_0.tar.bz2#67c0cba6533b641f28946d7c16f361c8 https://conda.anaconda.org/conda-forge/noarch/wheel-0.36.2-pyhd3deb0d_0.tar.bz2#768bfbe026426d0e76b377997d1f2b98 +https://conda.anaconda.org/conda-forge/noarch/zipp-3.5.0-pyhd8ed1ab_0.tar.bz2#f9dd05a5ed6b81c91f097e3739107a74 https://conda.anaconda.org/conda-forge/linux-64/antlr-python-runtime-4.7.2-py38h578d9bd_1002.tar.bz2#2b2207e2c8a05fc0bc5b62fc32c355e6 https://conda.anaconda.org/conda-forge/noarch/babel-2.9.1-pyh44b312d_0.tar.bz2#74136ed39bfea0832d338df1e58d013e https://conda.anaconda.org/conda-forge/linux-64/certifi-2021.5.30-py38h578d9bd_0.tar.bz2#a2e14464711f8e76010cd7e0c49bc4ae @@ -157,6 +159,7 @@ https://conda.anaconda.org/conda-forge/linux-64/cytoolz-0.11.0-py38h497a2fe_3.ta https://conda.anaconda.org/conda-forge/linux-64/dbus-1.13.6-h48d8840_2.tar.bz2#eba672c69baf366fdedd1c6f702dbb81 https://conda.anaconda.org/conda-forge/linux-64/docutils-0.16-py38h578d9bd_3.tar.bz2#a7866449fb9e5e4008a02df276549d34 https://conda.anaconda.org/conda-forge/linux-64/harfbuzz-2.8.2-h83ec7ef_0.tar.bz2#5dbbfef72138c2849d7486f5b17db010 +https://conda.anaconda.org/conda-forge/linux-64/importlib-metadata-4.6.3-py38h578d9bd_0.tar.bz2#6b4ce5de2f9df86ca03b6f17a68533b7 https://conda.anaconda.org/conda-forge/linux-64/kiwisolver-1.3.1-py38h1fd1430_1.tar.bz2#01488c80daae318ed5c17e7bb12af64e https://conda.anaconda.org/conda-forge/linux-64/libnetcdf-4.8.0-mpi_mpich_hf07302c_2.tar.bz2#d76a3f327eb8e26b5ce6b042ac1abeb3 https://conda.anaconda.org/conda-forge/linux-64/markupsafe-2.0.1-py38h497a2fe_0.tar.bz2#d075babffd68330d81b0488a45435698 @@ -165,7 +168,7 @@ https://conda.anaconda.org/conda-forge/linux-64/msgpack-python-1.0.2-py38h1fd143 https://conda.anaconda.org/conda-forge/linux-64/numpy-1.21.1-py38h9894fe3_0.tar.bz2#84ab79d83d91541e159a8e89113567f7 https://conda.anaconda.org/conda-forge/noarch/packaging-21.0-pyhd8ed1ab_0.tar.bz2#45cfb8e482b5cce8f07c87e0e19a592c https://conda.anaconda.org/conda-forge/noarch/partd-1.2.0-pyhd8ed1ab_0.tar.bz2#0c32f563d7f22e3a34c95cad8cc95651 -https://conda.anaconda.org/conda-forge/linux-64/pillow-6.2.2-py38h9776b28_0.tar.bz2#bd527d652ba06fb2aae61640bcf7c435 +https://conda.anaconda.org/conda-forge/linux-64/pillow-6.2.1-py38hd70f55b_1.tar.bz2#80d719bee2b77a106b199150c0829107 https://conda.anaconda.org/conda-forge/noarch/pockets-0.9.1-py_0.tar.bz2#1b52f0c42e8077e5a33e00fe72269364 https://conda.anaconda.org/conda-forge/linux-64/psutil-5.8.0-py38h497a2fe_1.tar.bz2#3c465545aa3cec37f8f1341546677956 https://conda.anaconda.org/conda-forge/linux-64/pyqt5-sip-4.19.18-py38h709712a_7.tar.bz2#e012838bbbe92f6a458c2584634830f1 @@ -177,7 +180,7 @@ https://conda.anaconda.org/conda-forge/linux-64/tornado-6.1-py38h497a2fe_1.tar.b https://conda.anaconda.org/conda-forge/linux-64/virtualenv-20.4.7-py38h578d9bd_0.tar.bz2#24b5e0295c43de15a51afb00f93a41de https://conda.anaconda.org/conda-forge/noarch/zict-2.0.0-py_0.tar.bz2#4750152be22f24d695b3004c5e1712d3 https://conda.anaconda.org/conda-forge/linux-64/brotlipy-0.7.0-py38h497a2fe_1001.tar.bz2#56753dd777a6517b34966ddcb39af734 -https://conda.anaconda.org/conda-forge/linux-64/cftime-1.2.1-py38hab2c0dc_1.tar.bz2#777186ded2d850f3eab4ce7131c6c17c +https://conda.anaconda.org/conda-forge/linux-64/cftime-1.5.0-py38hb5d20a5_0.tar.bz2#ef7d29510b551767f7584df1ce150b53 https://conda.anaconda.org/conda-forge/linux-64/cryptography-3.4.7-py38ha5dfef3_0.tar.bz2#a8b014aba670157256dabdc885f71af4 https://conda.anaconda.org/conda-forge/noarch/dask-core-2021.7.2-pyhd8ed1ab_0.tar.bz2#249982c77565b1040d8cc27dadf9dd89 https://conda.anaconda.org/conda-forge/linux-64/editdistance-s-1.0.0-py38h1fd1430_1.tar.bz2#03bbd69539712a691b0a43bd4a49976e @@ -198,6 +201,7 @@ https://conda.anaconda.org/conda-forge/linux-64/bokeh-2.1.1-py38h32f6830_0.tar.b https://conda.anaconda.org/conda-forge/linux-64/cf-units-3.0.1.post0-py38h6c62de6_0.tar.bz2#450b1b71703889cbda24c0181a4ce89a https://conda.anaconda.org/conda-forge/linux-64/distributed-2021.7.2-py38h578d9bd_0.tar.bz2#ae1d71fc7637e2210f38941e881cfdb6 https://conda.anaconda.org/conda-forge/linux-64/esmf-8.1.1-mpi_mpich_h3dcaa78_100.tar.bz2#5b4bab1017226f2c03ba0fe02b783316 +https://conda.anaconda.org/conda-forge/noarch/flake8-3.9.2-pyhd8ed1ab_0.tar.bz2#37d685abea0a25c921431edda02ad143 https://conda.anaconda.org/conda-forge/linux-64/gtk2-2.24.33-h539f30e_1.tar.bz2#606777b4da3664d5c9415f5f165349fd https://conda.anaconda.org/conda-forge/noarch/identify-2.2.13-pyhd8ed1ab_0.tar.bz2#1889223441d67f7c30327c97a78b980f https://conda.anaconda.org/conda-forge/noarch/imagehash-4.2.1-pyhd8ed1ab_0.tar.bz2#01cc8698b6e1a124dc4f585516c27643 @@ -213,8 +217,8 @@ https://conda.anaconda.org/conda-forge/linux-64/cartopy-0.19.0.post1-py38hc9c980 https://conda.anaconda.org/conda-forge/noarch/dask-2021.7.2-pyhd8ed1ab_0.tar.bz2#998ae0ba5626d98bb972359f776bc394 https://conda.anaconda.org/conda-forge/linux-64/esmpy-8.1.1-mpi_mpich_py38h7f78e9f_100.tar.bz2#ce0ac0d6f5e6c5e7e0c613b08b3a0960 https://conda.anaconda.org/conda-forge/linux-64/graphviz-2.48.0-h85b4f2f_0.tar.bz2#286099ec11970542c294f8be9c43871e -https://conda.anaconda.org/conda-forge/noarch/nc-time-axis-1.2.0-pyhd8ed1ab_2.tar.bz2#ca7d092db9ebbd9a0102710e8d4b4a28 -https://conda.anaconda.org/conda-forge/linux-64/pre-commit-2.13.0-py38h578d9bd_0.tar.bz2#1013dff06f574377c64f11efa7e2c016 +https://conda.anaconda.org/conda-forge/noarch/nc-time-axis-1.3.1-pyhd8ed1ab_2.tar.bz2#d37916d7bf9519a3e3abb902a4c06b69 +https://conda.anaconda.org/conda-forge/linux-64/pre-commit-2.14.0-py38h578d9bd_0.tar.bz2#cd9ef44c5b77792b3709b4fea2757c18 https://conda.anaconda.org/conda-forge/linux-64/pyqtchart-5.12-py38h7400c14_7.tar.bz2#3003444b4f41742a33b7afdeb3260cbc https://conda.anaconda.org/conda-forge/linux-64/pyqtwebengine-5.12.1-py38h7400c14_7.tar.bz2#1c17944e118b314ff4d0bfc05f03a5e1 https://conda.anaconda.org/conda-forge/noarch/pyugrid-0.3.1-py_2.tar.bz2#7d7361886fbcf2be663fd185bf6d244d diff --git a/requirements/ci/py37.yml b/requirements/ci/py37.yml index fac21560a4..cd6e303bce 100644 --- a/requirements/ci/py37.yml +++ b/requirements/ci/py37.yml @@ -4,24 +4,24 @@ channels: - conda-forge dependencies: - - python=3.7 + - python =3.7 # Setup dependencies. - - setuptools>=40.8.0 + - setuptools >=40.8.0 # Core dependencies. - - cartopy>=0.18 - - cf-units>=2 - - cftime<1.3.0 - - dask>=2 + - cartopy >=0.18 + - cf-units + - cftime >=1.5 + - dask >=2 - matplotlib - netcdf4 - - numpy>=1.14 + - numpy >=1.14 - python-xxhash - scipy # Optional dependencies. - - esmpy>=7.0 + - esmpy >=7.0 - graphviz - iris-sample-data - mo_pack @@ -34,9 +34,10 @@ dependencies: # Test dependencies. - asv - filelock - - imagehash>=4.0 + - flake8 + - imagehash >=4.0 - nose - - pillow<7 + - pillow <7 - pre-commit - requests diff --git a/requirements/ci/py38.yml b/requirements/ci/py38.yml index 4be43fdba6..86555ef76c 100644 --- a/requirements/ci/py38.yml +++ b/requirements/ci/py38.yml @@ -4,24 +4,24 @@ channels: - conda-forge dependencies: - - python=3.8 + - python =3.8 # Setup dependencies. - - setuptools>=40.8.0 + - setuptools >=40.8.0 # Core dependencies. - - cartopy>=0.18 - - cf-units>=2 - - cftime<1.3.0 - - dask>=2 + - cartopy >=0.18 + - cf-units + - cftime >=1.5 + - dask >=2 - matplotlib - netcdf4 - - numpy>=1.14 + - numpy >=1.14 - python-xxhash - scipy # Optional dependencies. - - esmpy>=7.0 + - esmpy >=7.0 - graphviz - iris-sample-data - mo_pack @@ -34,9 +34,10 @@ dependencies: # Test dependencies. - asv - filelock - - imagehash>=4.0 + - flake8 + - imagehash >=4.0 - nose - - pillow<7 + - pillow <7 - pre-commit - requests diff --git a/setup.cfg b/setup.cfg index 5cb918502c..b885872765 100644 --- a/setup.cfg +++ b/setup.cfg @@ -48,8 +48,8 @@ version = attr: iris.__version__ include_package_data = True install_requires = cartopy>=0.18 - cf-units>=2 - cftime<1.3.0 + cf-units + cftime>=1.5.0 dask[array]>=2 matplotlib netcdf4