Skip to content

Core (LV::AudioConvert): Rewrite audio sample conversion - #381

Draft
kaixiong wants to merge 1 commit into
masterfrom
rewrite-audio-convert
Draft

Core (LV::AudioConvert): Rewrite audio sample conversion#381
kaixiong wants to merge 1 commit into
masterfrom
rewrite-audio-convert

Conversation

@kaixiong

@kaixiongkaixiong commented Jan 18, 2025

Copy link
Copy Markdown
Member

Did extensive testing of the audio sample conversion and found a tonne of egregious numerical errors.

This PR fixes all of the known bugs. Where the conversion accuracy is sacrificed for performance, I have quantified the errors (with proofs!).

I don't know if they were introduced when I ported the original C macros to C++ generics, or during its refinements over time with modern C++-isms. If the bugs are present in the C original, I can backport the fixes over to 0.4.x.

More (gory) details to follow.

@kaixiong

Copy link
Copy Markdown
MemberAuthor

Here is the test comparing all three implementations. The 0.5.x (master) and 0.4.x were adapted slightly to work with the test runner.

General observations

  • PR works in all cases
  • 0.5.x works incorrectly when converting between signed and unsigned integer samples involved narrowing/widening, and overflows when mapping 1.0 float values to integers.
  • 0.4.x works incorrectly in all cases except same-type conversions, largely down to careless mistakes: mixing up directionality converting to/from floats, wrong shifts (wrong units i.e. bit vs byte). It also has underflow/overflow issues with 32-bit integers.

All 49 possible conversions were tested. For each test, 8 input values spread across the input range are used. The output values are presented in the following order: PR, 0.5.x, 0.4.x.

int8_t -> uint8_t
[-128, 127] -> [0, 255]
-128 -> 0 vs 0 vs 0
-91 -> 37 vs 37 vs 37
-55 -> 73 vs 73 vs 73
-18 -> 110 vs 110 vs 110
17 -> 145 vs 145 vs 145
54 -> 182 vs 182 vs 182
90 -> 218 vs 218 vs 218
127 -> 255 vs 255 vs 255
int8_t -> int8_t
[-128, 127] -> [-128, 127]
-128 -> -128 vs -128 vs -128
-91 -> -91 vs -91 vs -91
-55 -> -55 vs -55 vs -55
-18 -> -18 vs -18 vs -18
17 -> 17 vs 17 vs 17
54 -> 54 vs 54 vs 54
90 -> 90 vs 90 vs 90
127 -> 127 vs 127 vs 127
int8_t -> uint16_t
[-128, 127] -> [0, 65535]
-128 -> 0 vs 0 vs 33024
-91 -> 9472 vs 9472 vs 33098
-55 -> 18688 vs 18688 vs 33170
-18 -> 28160 vs 28160 vs 33244
17 -> 37120 vs 37120 vs 32802
54 -> 46592 vs 46592 vs 32876
90 -> 55808 vs 55808 vs 32948
127 -> 65280 vs 65280 vs 33022
int8_t -> int16_t
[-128, 127] -> [-32768, 32767]
-128 -> -32768 vs -32768 vs 256
-91 -> -23296 vs -23296 vs 330
-55 -> -14080 vs -14080 vs 402
-18 -> -4608 vs -4608 vs 476
17 -> 4352 vs 4352 vs 34
54 -> 13824 vs 13824 vs 108
90 -> 23040 vs 23040 vs 180
127 -> 32512 vs 32512 vs 254
int8_t -> uint32_t
[-128, 127] -> [0, 4294967295]
-128 -> 0 vs 0 vs 1024
-91 -> 620756992 vs 620756992 vs 1320
-55 -> 1224736768 vs 1224736768 vs 1608
-18 -> 1845493760 vs 1845493760 vs 1904
17 -> 2432696320 vs 2432696320 vs 136
54 -> 3053453312 vs 3053453312 vs 432
90 -> 3657433088 vs 3657433088 vs 720
127 -> 4278190080 vs 4278190080 vs 1016
int8_t -> int32_t
[-128, 127] -> [-2147483648, 2147483647]
-128 -> -2147483648 vs -2147483648 vs 1024
-91 -> -1526726656 vs -1526726656 vs 1320
-55 -> -922746880 vs -922746880 vs 1608
-18 -> -301989888 vs -301989888 vs 1904
17 -> 285212672 vs 285212672 vs 136
54 -> 905969664 vs 905969664 vs 432
90 -> 1509949440 vs 1509949440 vs 720
127 -> 2130706432 vs 2130706432 vs 1016
int8_t -> float
[-128, 127] -> [-1, 1]
-128 -> -1 vs -1.007874 vs 0
-91 -> -0.70980394 vs -0.71653545 vs 0
-55 -> -0.42745095 vs -0.43307087 vs 0
-18 -> -0.13725483 vs -0.14173228 vs 0
17 -> 0.13725495 vs 0.13385826 vs 0
54 -> 0.427451 vs 0.42519686 vs 0
90 -> 0.70980406 vs 0.70866144 vs 5.877472e-39
127 -> 1 vs 1 vs 0
uint8_t -> uint8_t
[0, 255] -> [0, 255]
0 -> 0 vs 0 vs 0
36 -> 36 vs 36 vs 36
72 -> 72 vs 72 vs 72
109 -> 109 vs 109 vs 109
145 -> 145 vs 145 vs 145
182 -> 182 vs 182 vs 182
218 -> 218 vs 218 vs 218
255 -> 255 vs 255 vs 255
uint8_t -> int8_t
[0, 255] -> [-128, 127]
0 -> -128 vs -128 vs -128
36 -> -92 vs -92 vs -92
72 -> -56 vs -56 vs -56
109 -> -19 vs -19 vs -19
145 -> 17 vs 17 vs 17
182 -> 54 vs 54 vs 54
218 -> 90 vs 90 vs 90
255 -> 127 vs 127 vs 127
uint8_t -> uint16_t
[0, 255] -> [0, 65535]
0 -> 0 vs 0 vs 0
36 -> 9216 vs 9216 vs 72
72 -> 18432 vs 18432 vs 144
109 -> 27904 vs 27904 vs 218
145 -> 37120 vs 37120 vs 290
182 -> 46592 vs 46592 vs 364
218 -> 55808 vs 55808 vs 436
255 -> 65280 vs 65280 vs 510
uint8_t -> int16_t
[0, 255] -> [-32768, 32767]
0 -> -32768 vs 0 vs -128
36 -> -23552 vs 9216 vs -56
72 -> -14336 vs 18432 vs 16
109 -> -4864 vs 27904 vs 90
145 -> 4352 vs -28416 vs -350
182 -> 13824 vs -18944 vs -276
218 -> 23040 vs -9728 vs -204
255 -> 32512 vs -256 vs -130
uint8_t -> uint32_t
[0, 255] -> [0, 4294967295]
0 -> 0 vs 0 vs 0
36 -> 603979776 vs 603979776 vs 288
72 -> 1207959552 vs 1207959552 vs 576
109 -> 1828716544 vs 1828716544 vs 872
145 -> 2432696320 vs 2432696320 vs 1160
182 -> 3053453312 vs 3053453312 vs 1456
218 -> 3657433088 vs 3657433088 vs 1744
255 -> 4278190080 vs 4278190080 vs 2040
uint8_t -> int32_t
[0, 255] -> [-2147483648, 2147483647]
0 -> -2147483648 vs 0 vs -128
36 -> -1543503872 vs 603979776 vs 160
72 -> -939524096 vs 1207959552 vs 448
109 -> -318767104 vs 1828716544 vs 744
145 -> 285212672 vs -1862270976 vs -1016
182 -> 905969664 vs -1241513984 vs -720
218 -> 1509949440 vs -637534208 vs -432
255 -> 2130706432 vs -16777216 vs -136
uint8_t -> float
[0, 255] -> [-1, 1]
0 -> -1 vs -1 vs -1.1754944e-38
36 -> -0.7176471 vs -0.71875 vs -1.180104e-38
72 -> -0.4352941 vs -0.4375 vs -1.1754944e-38
109 -> -0.14509797 vs -0.1484375 vs -1.180104e-38
145 -> 0.13725495 vs 0.1328125 vs 1.1663287e-38
182 -> 0.427451 vs 0.421875 vs -1.180104e-38
218 -> 0.70980406 vs 0.703125 vs 1.1755123e-38
255 -> 1 vs 0.9921875 vs -1.180104e-38
int16_t -> int8_t
[-32768, 32767] -> [-128, 127]
-32768 -> -128 vs -128 vs 0
-23405 -> -92 vs -92 vs 73
-14043 -> -55 vs -55 vs -110
-4681 -> -19 vs -19 vs -37
4680 -> 18 vs 18 vs 36
14042 -> 54 vs 54 vs 109
23404 -> 91 vs 91 vs -74
32767 -> 127 vs 127 vs -1
int16_t -> uint8_t
[-32768, 32767] -> [0, 255]
-32768 -> 0 vs 0 vs 128
-23405 -> 36 vs 36 vs 201
-14043 -> 73 vs 73 vs 18
-4681 -> 109 vs 109 vs 91
4680 -> 146 vs 146 vs 164
14042 -> 182 vs 182 vs 237
23404 -> 219 vs 219 vs 54
32767 -> 255 vs 255 vs 127
int16_t -> uint16_t
[-32768, 32767] -> [0, 65535]
-32768 -> 0 vs 0 vs 0
-23405 -> 9363 vs 9363 vs 9363
-14043 -> 18725 vs 18725 vs 18725
-4681 -> 28087 vs 28087 vs 28087
4680 -> 37448 vs 37448 vs 37448
14042 -> 46810 vs 46810 vs 46810
23404 -> 56172 vs 56172 vs 56172
32767 -> 65535 vs 65535 vs 65535
int16_t -> int16_t
[-32768, 32767] -> [-32768, 32767]
-32768 -> -32768 vs -32768 vs -32768
-23405 -> -23405 vs -23405 vs -23405
-14043 -> -14043 vs -14043 vs -14043
-4681 -> -4681 vs -4681 vs -4681
4680 -> 4680 vs 4680 vs 4680
14042 -> 14042 vs 14042 vs 14042
23404 -> 23404 vs 23404 vs 23404
32767 -> 32767 vs 32767 vs 32767
int16_t -> uint32_t
[-32768, 32767] -> [0, 4294967295]
-32768 -> 0 vs 0 vs 131072
-23405 -> 613613568 vs 613613568 vs 168524
-14043 -> 1227161600 vs 1227161600 vs 205972
-4681 -> 1840709632 vs 1840709632 vs 243420
4680 -> 2454192128 vs 2454192128 vs 18720
14042 -> 3067740160 vs 3067740160 vs 56168
23404 -> 3681288192 vs 3681288192 vs 93616
32767 -> 4294901760 vs 4294901760 vs 131068
int16_t -> int32_t
[-32768, 32767] -> [-2147483648, 2147483647]
-32768 -> -2147483648 vs -2147483648 vs 131072
-23405 -> -1533870080 vs -1533870080 vs 168524
-14043 -> -920322048 vs -920322048 vs 205972
-4681 -> -306774016 vs -306774016 vs 243420
4680 -> 306708480 vs 306708480 vs 18720
14042 -> 920256512 vs 920256512 vs 56168
23404 -> 1533804544 vs 1533804544 vs 93616
32767 -> 2147418112 vs 2147418112 vs 131068
int16_t -> float
[-32768, 32767] -> [-1, 1]
-32768 -> -1 vs -1.0000305 vs 0
-23405 -> -0.71425956 vs -0.71428573 vs 0
-14043 -> -0.42854965 vs -0.42857143 vs 0
-4681 -> -0.14283973 vs -0.14285715 vs 0
4680 -> 0.14283967 vs 0.14282662 vs 0
14042 -> 0.42854965 vs 0.42854092 vs 0
23404 -> 0.7142595 vs 0.7142552 vs 0
32767 -> 1 vs 1 vs 0
uint16_t -> uint8_t
[0, 65535] -> [0, 255]
0 -> 0 vs 0 vs 0
9362 -> 36 vs 36 vs 73
18724 -> 73 vs 73 vs 146
28086 -> 109 vs 109 vs 219
37448 -> 146 vs 146 vs 36
46810 -> 182 vs 182 vs 109
56172 -> 219 vs 219 vs 182
65535 -> 255 vs 255 vs 255
uint16_t -> int8_t
[0, 65535] -> [-128, 127]
0 -> -128 vs 0 vs 0
9362 -> -92 vs 36 vs 73
18724 -> -55 vs 73 vs -110
28086 -> -19 vs 109 vs -37
37448 -> 18 vs -110 vs 36
46810 -> 54 vs -74 vs 109
56172 -> 91 vs -37 vs -74
65535 -> 127 vs -1 vs -1
uint16_t -> uint16_t
[0, 65535] -> [0, 65535]
0 -> 0 vs 0 vs 0
9362 -> 9362 vs 9362 vs 9362
18724 -> 18724 vs 18724 vs 18724
28086 -> 28086 vs 28086 vs 28086
37448 -> 37448 vs 37448 vs 37448
46810 -> 46810 vs 46810 vs 46810
56172 -> 56172 vs 56172 vs 56172
65535 -> 65535 vs 65535 vs 65535
uint16_t -> int16_t
[0, 65535] -> [-32768, 32767]
0 -> -32768 vs -32768 vs -32768
9362 -> -23406 vs -23406 vs -23406
18724 -> -14044 vs -14044 vs -14044
28086 -> -4682 vs -4682 vs -4682
37448 -> 4680 vs 4680 vs 4680
46810 -> 14042 vs 14042 vs 14042
56172 -> 23404 vs 23404 vs 23404
65535 -> 32767 vs 32767 vs 32767
uint16_t -> uint32_t
[0, 65535] -> [0, 4294967295]
0 -> 0 vs 0 vs 0
9362 -> 613548032 vs 613548032 vs 37448
18724 -> 1227096064 vs 1227096064 vs 74896
28086 -> 1840644096 vs 1840644096 vs 112344
37448 -> 2454192128 vs 2454192128 vs 149792
46810 -> 3067740160 vs 3067740160 vs 187240
56172 -> 3681288192 vs 3681288192 vs 224688
65535 -> 4294901760 vs 4294901760 vs 262140
uint16_t -> int32_t
[0, 65535] -> [-2147483648, 2147483647]
0 -> -2147483648 vs 0 vs -32768
9362 -> -1533935616 vs 613548032 vs 4680
18724 -> -920387584 vs 1227096064 vs 42128
28086 -> -306839552 vs 1840644096 vs 79576
37448 -> 306708480 vs -1840775168 vs -145120
46810 -> 920256512 vs -1227227136 vs -107672
56172 -> 1533804544 vs -613679104 vs -70224
65535 -> 2147418112 vs -65536 vs -32772
uint16_t -> float
[0, 65535] -> [-1, 1]
0 -> -1 vs -1 vs 4.5918e-41
9362 -> -0.7142901 vs -0.71429443 vs 4.5916e-41
18724 -> -0.42858016 vs -0.42858887 vs -4.5918e-41
28086 -> -0.14287025 vs -0.1428833 vs -4.5918e-41
37448 -> 0.14283967 vs 0.14282227 vs 4.5918e-41
46810 -> 0.42854965 vs 0.42852783 vs 4.5918e-41
56172 -> 0.7142595 vs 0.7142334 vs -4.5918e-41
65535 -> 1 vs 0.9999695 vs -4.5918e-41
int32_t -> uint8_t
[-2147483648, 2147483647] -> [0, 255]
-2147483648 -> 0 vs 0 vs 128
-1533916891 -> 36 vs 36 vs 164
-920350135 -> 73 vs 73 vs 201
-306783378 -> 109 vs 109 vs 237
306783377 -> 146 vs 146 vs 18
920350134 -> 182 vs 182 vs 54
1533916890 -> 219 vs 219 vs 91
2147483647 -> 255 vs 255 vs 127
int32_t -> int8_t
[-2147483648, 2147483647] -> [-128, 127]
-2147483648 -> -128 vs -128 vs 0
-1533916891 -> -92 vs -92 vs 36
-920350135 -> -55 vs -55 vs 73
-306783378 -> -19 vs -19 vs 109
306783377 -> 18 vs 18 vs -110
920350134 -> 54 vs 54 vs -74
1533916890 -> 91 vs 91 vs -37
2147483647 -> 127 vs 127 vs -1
int32_t -> uint16_t
[-2147483648, 2147483647] -> [0, 65535]
-2147483648 -> 0 vs 0 vs 32768
-1533916891 -> 9362 vs 9362 vs 4681
-920350135 -> 18724 vs 18724 vs 42130
-306783378 -> 28086 vs 28086 vs 14043
306783377 -> 37449 vs 37449 vs 51492
920350134 -> 46811 vs 46811 vs 23405
1533916890 -> 56173 vs 56173 vs 60854
2147483647 -> 65535 vs 65535 vs 32767
int32_t -> int16_t
[-2147483648, 2147483647] -> [-32768, 32767]
-2147483648 -> -32768 vs -32768 vs 0
-1533916891 -> -23406 vs -23406 vs -28087
-920350135 -> -14044 vs -14044 vs 9362
-306783378 -> -4682 vs -4682 vs -18725
306783377 -> 4681 vs 4681 vs 18724
920350134 -> 14043 vs 14043 vs -9363
1533916890 -> 23405 vs 23405 vs 28086
2147483647 -> 32767 vs 32767 vs -1
int32_t -> uint32_t
[-2147483648, 2147483647] -> [0, 4294967295]
-2147483648 -> 0 vs 0 vs 2147483648
-1533916891 -> 613566757 vs 613566757 vs 2761050405
-920350135 -> 1227133513 vs 1227133513 vs 3374617161
-306783378 -> 1840700270 vs 1840700270 vs 3988183918
306783377 -> 2454267025 vs 2454267025 vs 306783377
920350134 -> 3067833782 vs 3067833782 vs 920350134
1533916890 -> 3681400538 vs 3681400538 vs 1533916890
2147483647 -> 4294967295 vs 4294967295 vs 2147483647
int32_t -> int32_t
[-2147483648, 2147483647] -> [-2147483648, 2147483647]
-2147483648 -> -2147483648 vs -2147483648 vs -2147483648
-1533916891 -> -1533916891 vs -1533916891 vs -1533916891
-920350135 -> -920350135 vs -920350135 vs -920350135
-306783378 -> -306783378 vs -306783378 vs -306783378
306783377 -> 306783377 vs 306783377 vs 306783377
920350134 -> 920350134 vs 920350134 vs 920350134
1533916890 -> 1533916890 vs 1533916890 vs 1533916890
2147483647 -> 2147483647 vs 2147483647 vs 2147483647
int32_t -> float
[-2147483648, 2147483647] -> [-1, 1]
-2147483648 -> -1 vs -1 vs 0
-1533916891 -> -0.71428573 vs -0.71428573 vs 0
-920350135 -> -0.42857143 vs -0.42857143 vs 0
-306783378 -> -0.14285715 vs -0.14285715 vs 0
306783377 -> 0.14285715 vs 0.14285715 vs 0
920350134 -> 0.42857143 vs 0.42857143 vs 0
1533916890 -> 0.71428573 vs 0.71428573 vs 0
2147483647 -> 1 vs 1 vs -0
uint32_t -> uint8_t
[0, 4294967295] -> [0, 255]
0 -> 0 vs 0 vs 0
613566756 -> 36 vs 36 vs 36
1227133512 -> 73 vs 73 vs 73
1840700269 -> 109 vs 109 vs 109
2454267025 -> 146 vs 146 vs 146
3067833782 -> 182 vs 182 vs 182
3681400538 -> 219 vs 219 vs 219
4294967295 -> 255 vs 255 vs 255
uint32_t -> int8_t
[0, 4294967295] -> [-128, 127]
0 -> -128 vs 0 vs 0
613566756 -> -92 vs 36 vs 36
1227133512 -> -55 vs 73 vs 73
1840700269 -> -19 vs 109 vs 109
2454267025 -> 18 vs -110 vs -110
3067833782 -> 54 vs -74 vs -74
3681400538 -> 91 vs -37 vs -37
4294967295 -> 127 vs -1 vs -1
uint32_t -> uint16_t
[0, 4294967295] -> [0, 65535]
0 -> 0 vs 0 vs 0
613566756 -> 9362 vs 9362 vs 37449
1227133512 -> 18724 vs 18724 vs 9362
1840700269 -> 28086 vs 28086 vs 46811
2454267025 -> 37449 vs 37449 vs 18724
3067833782 -> 46811 vs 46811 vs 56173
3681400538 -> 56173 vs 56173 vs 28086
4294967295 -> 65535 vs 65535 vs 65535
uint32_t -> int16_t
[0, 4294967295] -> [-32768, 32767]
0 -> -32768 vs 0 vs 0
613566756 -> -23406 vs 9362 vs -28087
1227133512 -> -14044 vs 18724 vs 9362
1840700269 -> -4682 vs 28086 vs -18725
2454267025 -> 4681 vs -28087 vs 18724
3067833782 -> 14043 vs -18725 vs -9363
3681400538 -> 23405 vs -9363 vs 28086
4294967295 -> 32767 vs -1 vs -1
uint32_t -> uint32_t
[0, 4294967295] -> [0, 4294967295]
0 -> 0 vs 0 vs 0
613566756 -> 613566756 vs 613566756 vs 613566756
1227133512 -> 1227133512 vs 1227133512 vs 1227133512
1840700269 -> 1840700269 vs 1840700269 vs 1840700269
2454267025 -> 2454267025 vs 2454267025 vs 2454267025
3067833782 -> 3067833782 vs 3067833782 vs 3067833782
3681400538 -> 3681400538 vs 3681400538 vs 3681400538
4294967295 -> 4294967295 vs 4294967295 vs 4294967295
uint32_t -> int32_t
[0, 4294967295] -> [-2147483648, 2147483647]
0 -> -2147483648 vs -2147483648 vs 0
613566756 -> -1533916892 vs -1533916892 vs 613566756
1227133512 -> -920350136 vs -920350136 vs 1227133512
1840700269 -> -306783379 vs -306783379 vs 1840700269
2454267025 -> 306783377 vs 306783377 vs -1840700271
3067833782 -> 920350134 vs 920350134 vs -1227133514
3681400538 -> 1533916890 vs 1533916890 vs -613566758
4294967295 -> 2147483647 vs 2147483647 vs -1
uint32_t -> float
[0, 4294967295] -> [-1, 1]
0 -> -1 vs -1 vs 0
613566756 -> -0.71428573 vs -0.71428573 vs 0
1227133512 -> -0.42857143 vs -0.4285714 vs 0
1840700269 -> -0.14285715 vs -0.14285713 vs 0
2454267025 -> 0.14285715 vs 0.1428572 vs 0
3067833782 -> 0.42857143 vs 0.42857146 vs 0
3681400538 -> 0.71428573 vs 0.71428573 vs 0
4294967295 -> 1 vs 1 vs 0
float -> uint8_t
[-1, 1] -> [0, 255]
-1 -> 0 vs 0 vs 0
-0.71428573 -> 36 vs 36 vs 0
-0.42857143 -> 72 vs 73 vs 128
-0.14285715 -> 109 vs 109 vs 191
0.14285715 -> 145 vs 146 vs 0
0.42857143 -> 182 vs 182 vs 0
0.71428573 -> 218 vs 219 vs 128
1 -> 255 vs 0 vs 191
float -> int8_t
[-1, 1] -> [-128, 127]
-1 -> -128 vs -127 vs 0
-0.71428573 -> -92 vs -90 vs 0
-0.42857143 -> -56 vs -54 vs 0
-0.14285715 -> -19 vs -18 vs 0
0.14285715 -> 17 vs 18 vs 0
0.42857143 -> 54 vs 54 vs 0
0.71428573 -> 90 vs 90 vs 0
1 -> 127 vs 127 vs 0
float -> uint16_t
[-1, 1] -> [0, 65535]
-1 -> 0 vs 0 vs 0
-0.71428573 -> 9362 vs 9362 vs 49024
-0.42857143 -> 18724 vs 18724 vs 0
-0.14285715 -> 28086 vs 28086 vs 16126
0.14285715 -> 37448 vs 37449 vs 56320
0.42857143 -> 46810 vs 46811 vs 16182
0.71428573 -> 56172 vs 56173 vs 55296
1 -> 65535 vs 0 vs 16124
float -> int16_t
[-1, 1] -> [-32768, 32767]
-1 -> -32768 vs -32767 vs 0
-0.71428573 -> -23406 vs -23405 vs 0
-0.42857143 -> -14044 vs -14043 vs 0
-0.14285715 -> -4682 vs -4681 vs -16639
0.14285715 -> 4680 vs 4681 vs 18432
0.42857143 -> 14042 vs 14043 vs -16750
0.71428573 -> 23404 vs 23405 vs -27648
1 -> 32767 vs 32767 vs -16639
float -> int32_t
[-1, 1] -> [-2147483648, 2147483647]
-1 -> -2147483648 vs -2147483648 vs -8388608
-0.71428573 -> -1533916929 vs -1533916928 vs -8388608
-0.42857143 -> -920350145 vs -920350144 vs -8388608
-0.14285715 -> -306783393 vs -306783392 vs -8388608
0.14285715 -> 306783391 vs 306783392 vs 2139095040
0.42857143 -> 920350143 vs 920350144 vs 2139095040
0.71428573 -> 1533916927 vs 1533916928 vs 2139095040
1 -> 2147483647 vs -2147483648 vs 2139095040
float -> uint32_t
[-1, 1] -> [0, 4294967295]
-1 -> 0 vs 0 vs 2139095040
-0.71428573 -> 613566719 vs 613566720 vs 2139095040
-0.42857143 -> 1227133503 vs 1227133440 vs 2139095040
-0.14285715 -> 1840700255 vs 1840700288 vs 2139095040
0.14285715 -> 2454267039 vs 2454267136 vs 2139095040
0.42857143 -> 3067833791 vs 3067833856 vs 2139095040
0.71428573 -> 3681400575 vs 3681400576 vs 2139095040
1 -> 4294967295 vs 0 vs 2139095040
float -> float
[-1, 1] -> [-1, 1]
-1 -> -1 vs -1 vs 0
-0.71428573 -> -0.71428573 vs -0.71428573 vs 0
-0.42857143 -> -0.42857143 vs -0.42857143 vs 0
-0.14285715 -> -0.14285715 vs -0.14285715 vs 0
0.14285715 -> 0.14285715 vs 0.14285715 vs 0
0.42857143 -> 0.42857143 vs 0.42857143 vs 0
0.71428573 -> 0.71428573 vs 0.71428573 vs 0
1 -> 1 vs 1 vs 0

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@kaixiong@hartwork