Error in calculating power function

[shaoziyang]

Describe the bug
I calculate a expression in makecode, then send the value to serial port. In simulator it is ok, but in real serial port, it will get wrong number.

let P = 100240
basic.forever(function () {
    serial.writeValue("T", (P / 101325) ** (1 / 5.255))
    basic.pause(1000)
})

In simulator, it will show:

T:0.997953407897062

But in real serial port it will show:

T: 1

[shaoziyang]

If Multiply by 100

let P = 100240
basic.forever(function () {
    serial.writeValue("T", (P / 101325) ** (1 / 5.255) * 100)
    basic.pause(1000)
})

Simulator will show:

T:99.7953407897062

And real serial port will show:

T: 100

[shaoziyang]

I have make a simple test, get same result:

https://makecode.microbit.org/_ixr8jjaYTeM6

[whaleygeek]

Implementation of floats differ slightly between the Typescript and the micro:bit runtime library, are these just 'small rounding differences'?

Might be related to: #1781

[whaleygeek]

Hmmm, Abhijith Chatra (@abchatra) looks suspiciously more like a type conversion error, given that the number on the real micro:bit comes out without any decimal places?

[mmoskal]

The power function on microbit is rounding the exponent to an integer. Regular pow() function brings in 4k of code, for which I think we don't have space in flash. This is controlled by PXT_POWI configuration macro.

We should definitely align sim with hw. Maybe we should just error on the ** operator when PXT_POWI is defined.

Note that this is the case only on micro:bit (and in light mode on maker, which currently only applies to bluepill). Other targets, including Arcade and CPX, use real pow function.

[mmoskal]

Also, there should be no differences in the actual values of numbers between simulator and hardware; hardware may round them differently but only when displaying, the intermediate results should be exactly the same.

BTW, it seems the fix for displaying numbers never made it into microbit release branch.

[whaleygeek]

FYI, the fix for displaying correctly rounded numbers is quite important, I have had lots of people moan at me about how the micro:bit got harder to use when floating point was added, and the root cause seems to be display rounding.

…

On Thu, 25 Jul 2019, 17:51 Michał Moskal, ***@***.***> wrote: Also, there should be no differences in the actual values of numbers between simulator and hardware; hardware may round them differently but only when displaying, the intermediate results should be exactly the same. BTW, it seems the fix for displaying numbers <microsoft/pxt-common-packages#862> never made it into microbit release branch. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2192?email_source=notifications&email_token=AA7FYANQZXKXZBAGXROGNTDQBHKZTA5CNFSM4HX4AMP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD22CBLQ#issuecomment-515121326>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA7FYAOYY47PLPJNUDZCGDDQBHKZTANCNFSM4HX4AMPQ> .