Redesign UnitSystem to support non-SI systems and configurable default units

[lipchev]

(WIP) for #651

• 1. Redesign/Remove BaseUnits type and remove related from JSON and UnitSystem (breaking change)

• 2. UnitSystem: Add mapping of default unit, such as QuantityType.Length => LengthUnit.Centimeter

• 3. UnitSystem: Add mapping of default abbreviation per unit enum value, such as LengthUnit.Feet => "ft" (optional- also a breaking change if UnitAbbreviationCache is to become part of the UnitSystem)

• 4. Length.ctor(double, UnitSystem) constructs with unit system's default unit for Length

• 5. double val = myLength.As(UnitSystem) converts to unit system's default unit for Length

• 6. UnitSystem should remain immutable, so to change its configuration at runtime you create a new instance either manually or by cloning an existing system with extension methods for adding/modifying mappings like in use case 5. Then you inject this instance into your view or whatever.

• 7. Add UnitSystem.SI and UnitSystem.EnglishEngineering with mappings

• 8. Add tests for the the new public methods in UnitSystem (got one in LenghtTests- but none that test for the "exceptional" cases)

[codecov-io]

Codecov Report

Merging #709 (e45efe8) into master (0f39bec) will increase coverage by 84%.
The diff coverage is 83%.

❗ Current head e45efe8 differs from pull request most recent head dfe0bd2. Consider uploading reports for the commit dfe0bd2 to get more accurate results

@@ Coverage Diff @@## master #709 +/- ##
=========================================
+ Coverage 0 84% +84% 
=========================================
Files 0 308 +308 Lines 0 43886 +43886 =========================================
+ Hits 0 36938 +36938 - Misses 0 6948 +6948 

Impacted Files	Coverage Δ
...tsNet/GeneratedCode/Quantities/AmplitudeRatio.g.cs	79% <80%> (ø)
UnitsNet/GeneratedCode/Quantities/Angle.g.cs	83% <80%> (ø)
...tsNet/GeneratedCode/Quantities/ApparentEnergy.g.cs	79% <80%> (ø)
...itsNet/GeneratedCode/Quantities/ApparentPower.g.cs	79% <80%> (ø)
UnitsNet/GeneratedCode/Quantities/AreaDensity.g.cs	76% <80%> (ø)
.../GeneratedCode/Quantities/AreaMomentOfInertia.g.cs	80% <80%> (ø)
UnitsNet/GeneratedCode/Quantities/BitRate.g.cs	86% <80%> (ø)
...dCode/Quantities/BrakeSpecificFuelConsumption.g.cs	79% <80%> (ø)
UnitsNet/GeneratedCode/Quantities/Capacitance.g.cs	81% <80%> (ø)
...Code/Quantities/CoefficientOfThermalExpansion.g.cs	79% <80%> (ø)
... and 354 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[lipchev]

Initially I went with a dictionary (as proposed in #651) but being stunned by the lack of improvement in performance compared with the test in #708 I dropped it in favor of an array (of QuantityInfo), indexed by the (int)QuantityType - 1, which gave me some improvement- but not nearly as much as I expected initially..

[lipchev]

Method	Mean	Error	StdDev	Median	Gen 0	Gen 1	Gen 2	Allocated
Constructor	10.261 ns	0.2568 ns	0.3683 ns	10.102 ns	-	-	-	-
Constructor_SI	341.489 ns	20.1771 ns	57.5664 ns	300.000 ns	-	-	-	-
FromMethod	23.169 ns	0.1087 ns	0.0907 ns	23.156 ns	-	-	-	-
ToProperty	7.105 ns	0.0894 ns	0.0836 ns	7.111 ns	-	-	-	-
As	7.171 ns	0.1547 ns	0.1447 ns	7.095 ns	-	-	-	-
As_SI	336.364 ns	19.1489 ns	56.1604 ns	300.000 ns	-	-	-	-
ToUnit	16.206 ns	0.1705 ns	0.1424 ns	16.205 ns	-	-	-	-
ToUnit_SI	456.667 ns	23.4713 ns	65.4286 ns	400.000 ns	-	-	-	-
ToStringTest	2,022.562 ns	14.2156 ns	13.2973 ns	2,020.125 ns	0.1945	-	-	832 B
Parse	51,561.651 ns	990.9471 ns	1,179.6520 ns	51,123.584 ns	10.1929	-	-	43000 B
TryParseValid	49,676.343 ns	808.6956 ns	756.4544 ns	49,425.769 ns	10.1929	-	-	42976 B
TryParseInvalid	52,532.164 ns	612.5532 ns	543.0125 ns	52,483.185 ns	10.1318	-	-	42576 B
QuantityFrom	68.093 ns	1.4228 ns	1.5814 ns	67.222 ns	0.0132	-	-	56 B
IQuantity_As	15.401 ns	0.2076 ns	0.1942 ns	15.320 ns	0.0057	-	-	24 B
IQuantity_As_SI	380.723 ns	14.8686 ns	39.6873 ns	400.000 ns	-	-	-	-
IQuantity_ToUnit	23.538 ns	0.2679 ns	0.2506 ns	23.453 ns	0.0133	-	-	56 B
IQuantity_ToStringTest	2,037.505 ns	39.4539 ns	61.4250 ns	2,013.519 ns	0.1945	-	-	832 B

Note that the performance of my machine is slightly lower today (I think it's either the room temperature or some process eating up my resources).

[lipchev]

Still some things I'm not sure about:

• what would be the expected equality comparison for two UnitSystems (currently depends on the BaseType)
• previous tests state that for example "AmplitudeRatio is unitless. Can't have any matches" (when constructing AmplituedRatio using SI)- yet it seems there are 4 "dimensionless units" to chose from when selecting a default unit
• I struggled to validate within WithDefaultUnit(QuantityType, UnitInfo) that UnitInfo corresponds to the provided QuantityType (any ideas? lookup the list of Info's again? change the signiture?)
• I struggled to create the generic overload corresponding to WithDefaultUnit(QuantityType, UnitInfo):
  WithDefaultUnit<TUnit>(UnitInfo<TUnit>) // how do I get the QuantityType value from this

[angularsen]

We can't merge this into master branch until all breaking changes are ready, so I just created release/v5 branch. Please point this PR to that branch instead, or create a new PR if it can't be changed.

Other breaking changes: #666

[angularsen]

which gave me some improvement- but not nearly as much as I expected initially..

I think this basically comes down to the fact that we have ~100 quantities today, and enumerating 100 items with a simple condition check is really really fast to begin with. At any rate, a lookup of sorts seems reasonable anyway.

[angularsen]

Still some things I'm not sure about:

• what would be the expected equality comparison for two UnitSystems (currently depends on the BaseType)

I propose using the default implementation that class brings, which checks if they are the exact same object reference. We could compare all the mappings and any values, but I really question the value of this.

• previous tests state that for example "AmplitudeRatio is unitless. Can't have any matches" (when constructing AmplituedRatio using SI)- yet it seems there are 4 "dimensionless units" to chose from when selecting a default unit

I believe you are referencing this test:
https://github.com/angularsen/UnitsNet/blob/master/UnitsNet.Tests/CustomCode/LengthTests.cs#L189-L194

First, I don't understand why we are testing AmplitudeRatio in LengthTests.
Second, it's likely a limitation of the old unit system design where it's not possible to uniquely map from SI base units to dimensionless units. I don't see why we shouldn't preserve this behavior in the new design, so this test can be removed I think.

• I struggled to validate within WithDefaultUnit(QuantityType, UnitInfo) that UnitInfo corresponds to the provided QuantityType (any ideas? lookup the list of Info's again? change the signiture?)

I'll have to look closer on this one.

• I struggled to create the generic overload corresponding to WithDefaultUnit(QuantityType, UnitInfo):
  WithDefaultUnit<TUnit>(UnitInfo<TUnit>) // how do I get the QuantityType value from this

I'll have to look closer on this one.

[angularsen]

I know this is work in progress, but here are some early comments after reading through the first time.

[angularsen]

1. Preserve returning null if no mapping is defined.
2. AmplitudeRatio should have a mapping defined in SI, I think.

[lipchev]

Right- so, in theory, the only way to get a null here would be if one creates a custom unit system, explicitly setting the association to null (as presumable the original unit system would have all of its associations defined).

[angularsen]

Sounds about right. We should also create some unit tests that breaks as soon as a new quantity is added and if one forgets to add a mapping of default unit for included unit systems such as SI and EnglishEngineering. I don't think we need to add a mapping for default abbreviation, since I think it's natural to default to the first abbreviation defined in JSON as we already do.

[angularsen]

Same here, aren't we removing BaseUnits?

[angularsen]

BaseUnits is going away here too, right?

[lipchev]

Regarding all comments about BaseUnits going away- you are right- they should be going away in v5.

I just thought I might make this a two step PR (as to not make it too difficult to review)- where the first part would be the 'non-breaking' change (that allows for issue like the one described in #700 to be patched in the client code).

However, now that I think of it- I'm not sure how good a fix is to pollute one's code with something like SI_Ex (having correctly mapped all necessary defaults by hand) just to have it fixed in the next major release (and going about changing back to SI once more)...

[angularsen]

Regarding all comments about BaseUnits going away- you are right- they should be going away in v5.

I just thought I might make this a two step PR (as to not make it too difficult to review)- where the first part would be the 'non-breaking' change (that allows for issue like the one described in #700 to be patched in the client code).

However, now that I think of it- I'm not sure how good a fix is to pollute one's code with something like SI_Ex (having correctly mapped all necessary defaults by hand) just to have it fixed in the next major release (and going about changing back to SI once more)...

Got it, good thinking, but I suspect the PR might be easier to review if we do a clean break to begin with instead of having both old and new designs in our head at the same time. It's up to you, just do whatever you feel makes the most sense.

[lipchev]

Here is an example of the modified unit definition schema (haven't removed the BaseUnits from the CubicHectometer & ImperialPint as to not trigger the code-regeneration)- tell me what you think.

The two associations would be used to construct the corresponding list of UnitInfo's and default UnitInfo for a given quantity and unit system.

Modifying the whole list of UnitDefinitions in such a way would be quite the task- but as long as there is some review process- I think we can manage.

As we do that- what I think would be really cool is to associate units & quantities with their actual ontological references: like for instance our MassFraction is defined in UO as MassPercentage and again as MassFraction in OM.

We could thus (maybe in the future) run some ontological alignment validations on our unit definitions, as described in this awesome paper: Comparison and Evaluation of Ontologies for Units of Measurement.

Not to mention the nice extension methods that one could think of..

[angularsen]

I like it. Explicit and much simpler to reason about than before. I originally thought about defining unit system as its own file, but keeping it per quantity is actually a much better idea since it's much more visible and just a single JSON file to edit when adding new units.

It's still something new that contributors will have to think about, but with some updated steps in the wiki I think we should manage.

Also, unit systems are really opt-in the way I see it. Meaning, if we decide to not map all units they should still continue to work for everything else but methods that take a unit system as argument.

[lipchev]

Ok, so I refactored UnitSystem- I couldn't bear to remove the BaseUnits so I moved it the BaseUnitSystem extension of the UnitSystem (along with the IEquatable interface). It is thus still present for UnitSystem.SI (which is a BaseUnitSystem)- I don't yet know how we could is use- but I don't think it hurts to keep it for now. I've also left out the original BaseUnits constructor (and the corresponding mapping behavior)- marked as obsolete. Thus it seems we are mostly backward-compatible. (still I presume we will move this to v5?)

I've replaced the previous a UnitInfo[] mapping (in the UnitSystem) with a UnitSystemInfo- that includes the default unit (optional) and list of "common" (also "derived" or "named") units for a quantity in a unit system. This class itself is currently not exposed (maybe make internal)- only the UnitInfos are exposed via the GetDefaultUnitInfo(QuantityType quantityType) & GetCommonUnitsInfo(QuantityType quantityType) methods in the UnitSystem.

Next I updated the JSON types & added UnitSystem generator- that completes the partial part of the concrete UnitSystem implementations (SI, GGS, EE etc..) and finally added several more UnitSystem mappings to the UnitDefinitions of Acceleration, AmountOfSubstance, Area, Energy & Length (pretty much the same thing as with Volume- note however that CGS often maps default unit to prefixed-quantities).

Still some tests need to be added (note for instance how it's not really possible to argument check the Lazy in the constructor). Also I've commented(temporary) parts of the Ctor_WithValueAndSIUnitSystem_ReturnsQuantityWithSIUnitOrThrowsArgum, and two of my own tests for the WithDefaultUnit registration- that used to test for the BaseUnit- haven't created an overload for it in BaseUnitSystem yet)

[lipchev]

Also- the benchmark showed some unexpected results- I didn't expect to see any change over the previous tests (we are still doing the same array lookup) - yet the results turned up even faster than the original As/ToUnit methods- what, am I not doing enough argument checking somewhere? Or maybe it is something worse :)

[lipchev]

Method	Mean	Error	StdDev	Median	Gen 0	Gen 1	Gen 2	Allocated
Constructor	9.703 ns	0.1571 ns	0.1469 ns	9.747 ns	-	-	-	-
Constructor_SI	14.399 ns	0.2823 ns	0.3138 ns	14.327 ns	-	-	-	-
FromMethod	22.979 ns	0.3424 ns	0.3036 ns	22.930 ns	-	-	-	-
ToProperty	7.132 ns	0.1730 ns	0.2188 ns	7.109 ns	-	-	-	-
As	7.117 ns	0.1795 ns	0.2631 ns	7.158 ns	-	-	-	-
As_SI	8.965 ns	0.1758 ns	0.1727 ns	8.984 ns	-	-	-	-
ToUnit	16.113 ns	0.3489 ns	0.5634 ns	16.020 ns	-	-	-	-
ToUnit_SI	19.220 ns	0.5484 ns	1.5909 ns	18.631 ns	-	-	-	-
ToStringTest	2,143.655 ns	42.0048 ns	72.4562 ns	2,124.925 ns	0.1945	-	-	832 B
Parse	50,824.466 ns	1,167.4845 ns	1,674.3713 ns	50,253.683 ns	10.1929	-	-	43000 B
TryParseValid	50,085.794 ns	1,109.7561 ns	1,519.0467 ns	49,460.004 ns	10.1929	-	-	42976 B
TryParseInvalid	51,682.055 ns	1,388.3917 ns	1,946.3312 ns	51,121.497 ns	10.1318	-	-	42576 B
QuantityFrom	64.901 ns	1.1769 ns	1.0433 ns	64.964 ns	0.0132	-	-	56 B
IQuantity_As	14.626 ns	0.2431 ns	0.2274 ns	14.591 ns	0.0057	-	-	24 B
IQuantity_As_SI	10.176 ns	0.1887 ns	0.1576 ns	10.154 ns	-	-	-	-
IQuantity_ToUnit	23.791 ns	0.4649 ns	0.4348 ns	23.867 ns	0.0133	-	-	56 B
IQuantity_ToStringTest	1,985.983 ns	34.6024 ns	27.0153 ns	1,994.777 ns	0.1945	-	-	832 B

[angularsen]

I've kind of forgotten to follow up on this. Will try to get to it sometime soon.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[lipchev]

Sorry for the long delay - I'm just so late on my main project that I've had to imposed myself a restriction on working on other solutions for the time being.. :)
Regardless - I've had a little time to consider a couple of things with respect to this feature, the BaseUnits and the exact unit representations (#714 and #666 ). Here's what I think:

1. Given that every SI derived quantity has a default unit (in SI), it seems to me like it should be possible to define/generate the Multiplication/Division operators between any 3 quantities (TResult, TLeft, TRight) where:

• {TResult, TLeft, TRight} are all part of SI (lets assume this for- more on this later)
• TResult.BaseDimension == TLeft.BaseDimensions * TRight.BaseDimensions (for multiplication)
• TResult.BaseDimension == TLeft.BaseDimensions / TRight.BaseDimensions (for division)

2. Such operators would naturally result in TResult having the DefaultUnit as per the UnitSystem (SI in the case under consideration). I guess this is fine for the most part (like as a default/fail-back behavior)- but it's not ideal for the following two use cases:

• 1 mm * 1 mm -> (0.001 m * 0.001 m) - > 1E-6 m^2 (SI) // ideally you would get 1 mm^2)
• 1 ft * 1 ft -> (??? m * ??? m) -> ??? m^2 (SI) // ideally you would get 1 ft^2
• most integer multiplications (as per Creating generic version of structs (WIP) #714) would fail // ideally the result is Area< int >

3. As it currently stands- users adding/updating units write operators by hand- putting whatever unit the like in the result- thus imposing a convention for writing operators that are compliant with use cases stated above- seems like a tall order. Let alone having to do so for more than one value type (int,double,decimal).

...let me grab another coffee and will try to outline the proposal..

[lipchev]

Lets assume that the operation TResult = TLeft * TRight makes sense (more on that later): it should be possible to obtain the triplet <TLeftUnit, TRightUnit, TResultUnit> such that we can:
return new TResult(Multiply(TLeft.As(TLeftUnit), TRight.As(TRightUnit), TResultUnit)) // as per #714
You probably have guessed by now that I'm looking at the BaseUnits (not the Dimensions) for the rescue. My suggestion is that we construct those correctly for each quantity/unit/prefix. Lets start unrolling this slowly:

1. Let's consider a basic SI unit with no prefixes and no specific information with regards to the BasicUnits: say Density (ignoring the prefixes for now)- we can safely construct the BaseUnits for it using the DefaultUnit for SI (which is required) giving us something like {Mass = Killogram, Length=Meter}
2. Prefixed units (SI only) with no BaseUnits information - here we have to consider the problem outlined a few times already (see Extending the functionality of BaseUnits & UnitSystem #651 point 3.) about selecting appropriate BaseUnits for something like Deciliter : 1 dl = 1 ? (0.1 dm3 | 100 cm3). In this case, I don't see any problem in simply selecting the bigger one- e.g. 1 dl = 100 cm3. The correction factor (100 instead of the implicit value of 1) should either be included in the BaseUnits definition (or somehow accounting for it by the means of the Quantity.Dimensions)
3. Another option would be to restructure the BaseUnits- such that it can represent a map of {Quantity, Unit} with the entries representing any of the Quantity/Units in the system (not only the ones in SI). Given that the list contains the prefixed units as well- it shouldn't be a problem to create a non-SI base unit for Density - e.g. {Mass = Gram, Volume= Liter}
4. As it stands (in the current version of this feature)- we have the list of associated UnitInfos for each quantity inside each UnitSystem - it shouldn't be a problem to allow for different UnitInfo instances per different UnitSystem (containing the customized BaseUnits like for the Density given before).
5. However, in order to benefit from (4) we have to preserve the UnitSystem that was used for the construction of a given quantity...

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[lipchev]

Stale - but not abandoned..

[angularsen]

I added the pinned label to prevent bot from stale-closing it.

[lipchev]

This is almost done- still need to complete mapping in the JSON files, but as it stands I can only see one client call that is breaking : new UnitSystem(baseUnits).BaseUnits - the constructor works as before(marked as obsolete)- it is only the BaseUnits property that was moved to BaseUnitSystem- so UnitSystem.SI.BaseUnits is still valid.

I've kept the equality contract for BaseUnitSystem as before- only comparing the default SI base units, where as for all other unit systems (not derived from BaseUnitSystem) we compare all unit associations.

I've added tests akin to the ones you've suggested in #844.
There are a few more tests that can be added to UnitSystemTests (I don't think that those that I've marked as Obsolete contribute to the coverage- while the constructor they relied on is indeed obsolete- the tested behavior is still valid).

[angularsen]

Awesome, I'll try to take a closer look this weekend.

[angularsen]

@lipchev I have totally neglected this one, but stumbled over it today. Would you like me to review this still?

[angularsen]

Also, related #864.

Here we deprecate QuantityType in favor of strings, which is more extensible. It seems like something this PR should take into account as well.

[inflectionpoint]

I would suggest the naming the alternative Unit System of "EnglishEngineering" units be defined as "USCustomary" as I believe that is the proper name. I believe the "English" or "Imperial" is historical naming and not used anymore.

[angularsen]

Thank you @inflectionpoint , I didn't know that!

[lipchev]

@angularsen I've merged everything from upstream, removing the SupportsSIUnitSystem tests in favor of the auto-generated equivalents.
There is one small thing that's been bugging me for a while- that is the Undefined unit. As far as I can tell this is only used for the SI BaseUnit definitions (where I think it could easily be replaced it with a nullable property). Do you know of any other reason for having it? Wouldn't it be better if we made BaseUnit the zero-index (default) unit for all quantities? At the very least- this would allow us to remove the nullable _unit field (well maybe not remove it, but at least make it non-nullable :)).

[angularsen]

@lipchev I think the reasoning for Undefined was to catch bugs where someone didn't specify a value, such as when deserializing JSON.
I'm fine with making it nullable instead since we are already making breaking changes, and I don't expect anyone to use Undefined for anything meaningful that can't be replaced with null.

[lipchev]

I don't know where to put this (maybe a discussion?) but here's a great explanation of the linear algebra behind the BaseUnits of different UnitSystems.