Improve performance for backend datetime handling

[Illviljan]

Was hunting some low-hanging performance fruits when reading in files.

• Use Variable(..., fastpath=True) for cases when a Variable has been unpacked and modified slightly.
• Don't check if variable is variable in decode_cf_variable
• Don't import DataArray until necessary in as_compatible_data.
• Add typing to touched files to make sure only Variables are used.

• Closes #xxxx
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst
• New functions/methods are listed in api.rst

[Illviljan]

This previously would've crashed when trying to use attrs["coordinates"].split() on a non-string value.
Might be a functionality change? Should this raise an error instead if it's not a string?

[dcherian]

Yes it would be good to raise a nice error. "coordinates" is expected to be a string with variable names separated by spaces: http://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#attribute-appendix

[headtr1ck]

+100 for all the typing :)

Do we need some benchmark for this?

[Illviljan]

Benchmark are improved if I understand the logs correctly. Unfortunately not significant enough to make ASV report it though. The ratio has to be >1.5 and the improvements on .time_open_dataset are around 1.3-1.4.

# PR:
[ 50.85%] ··· dataset_io.IOReadCustomEngine.time_open_dataset ok
[ 50.85%] ··· ======== =========
chunks -------- ---------
None 130±1ms {} 689±6ms ======== =========
[ 54.69%] ··· dataset_io.IOReadSingleFile.time_read_dataset ok
[ 54.69%] ··· ========= ============= =============
-- chunks --------- ---------------------------
engine None {} ========= ============= =============
scipy 5.48±0.04ms 6.91±0.01ms netcdf4 2.93±0.04ms 4.32±0.02ms ========= ============= =============
# Baseline:
[ 75.85%] ··· dataset_io.IOReadCustomEngine.time_open_dataset ok
[ 75.85%] ··· ======== ===========
chunks -------- -----------
None 177±0.5ms {} 737±3ms ======== ===========
[ 79.69%] ··· dataset_io.IOReadSingleFile.time_read_dataset ok
[ 79.69%] ··· ========= ============ ============
-- chunks --------- -------------------------
engine None {} ========= ============ ============
scipy 4.47±0.6ms 5.74±0.7ms netcdf4 4.39±0.7ms 5.82±0.6ms ========= ============ ============

# PR:
[ 50.85%] ··· dataset_io.IOReadCustomEngine.time_open_dataset ok
[ 50.85%] ··· ======== ==========
chunks -------- ----------
None 149±4ms {} 797±20ms ======== ==========
[ 54.69%] ··· dataset_io.IOReadSingleFile.time_read_dataset ok
[ 54.69%] ··· ========= ============ =============
-- chunks --------- --------------------------
engine None {} ========= ============ =============
scipy 6.57±0.2ms 7.77±0.01ms netcdf4 3.71±0.1ms 6.17±0.5ms ========= ============ =============
# Baseline:
[ 75.85%] ··· dataset_io.IOReadCustomEngine.time_open_dataset ok
[ 75.85%] ··· ======== ==========
chunks -------- ----------
None 204±2ms {} 857±20ms ======== ==========
[ 79.69%] ··· dataset_io.IOReadSingleFile.time_read_dataset ok
[ 79.69%] ··· ========= ============ ============
-- chunks --------- -------------------------
engine None {} ========= ============ ============
scipy 5.53±1ms 7.12±0.8ms netcdf4 4.96±0.6ms 6.74±0.8ms ========= ============ ============

# PR:
[ 50.85%] ··· dataset_io.IOReadCustomEngine.time_open_dataset ok
[ 50.85%] ··· ======== ============
chunks -------- ------------
None 204±8ms {} 1.20±0.04s ======== ============
[ 54.69%] ··· dataset_io.IOReadSingleFile.time_read_dataset ok
[ 54.69%] ··· ========= ============ ============
-- chunks --------- -------------------------
engine None {} ========= ============ ============
netcdf4 6.86±0.7ms 9.81±0.6ms scipy 6.74±1ms 9.10±1ms ========= ============ ============
# Baseline:
[ 75.85%] ··· dataset_io.IOReadCustomEngine.time_open_dataset ok
[ 75.85%] ··· ======== ============
chunks -------- ------------
None 282±5ms {} 1.20±0.04s ======== ============
[ 79.69%] ··· dataset_io.IOReadSingleFile.time_read_dataset ok
[ 79.69%] ··· ========= ============ ============
-- chunks --------- -------------------------
engine None {} ========= ============ ============
netcdf4 6.91±1ms 9.77±0.7ms scipy 6.91±0.7ms 9.11±1ms ========= ============ ============

[dcherian]

Looks like an improvement on my machine

 before after ratio
[4f3128bb] [898b8728]
<main> <mypy_conventions>
- 244±7ms 184±3ms 0.76 dataset_io.IOReadCustomEngine.time_open_dataset(None)

IOReadSingleFile hasn't changed significantly

[dcherian]

Thanks @Illviljan

[headtr1ck]

Great, thanks!