Decimal points and rounding input/output

Post Reply
Erg2
Posts: 18
Joined: Sat Dec 10, 2016 8:58 pm
My devices: ipad2
Flag: Australia

Decimal points and rounding input/output

Post by Erg2 »

Hi
Can anyone help please.

When I run below, it will round to decimal place as per input, however after the the 9th and onwards rounding the output does not follow the same logic for rounding and random numbers start to appear when compared to the original input. What is causing this?

ps. I've added the last print statement so it's easier to compare on the output screen the behaviour of result versus actual

'Decimal points and rounding input/output
y = 123456789.1234567890123456
input "Decimal Points" : dp
a$ = "#.####################"
b$ = left$ (a$,dp + 2)
print "result is. "; b$:y
print "Actuaul input 123456789.1234567890123456"

Or is there a global setting in Smart Basic that controls what I'm trying to achieve
Thank U

User avatar
Mr. Kibernetik
Site Admin
Posts: 4786
Joined: Mon Nov 19, 2012 10:16 pm
My devices: iPhone, iPad, MacBook
Location: Russia
Flag: Russia

Re: Decimal points and rounding input/output

Post by Mr. Kibernetik »

Computers have limit of numbers precision, and this is a good example of this limit.

User avatar
sarossell
Posts: 195
Joined: Sat Nov 05, 2016 6:31 pm
My devices: iPad Mini 2, iPhone 5, MacBook Air, MacBook Pro
Flag: United States of America
Contact:

Re: Decimal points and rounding input/output

Post by sarossell »

As I understand it (and please correct me if I'm wrong, Mr. K), smart BASIC uses 53 bits for numeric operations which allows for sufficiently large numbers and decimal places but if the number of total digits is over 16 or so, it will not have enough binary digits to represent them accurately and will have to round. It looks like you found the event horizon.

This is called "fixed point precision mathematics". Some programming languages, created specifically for mathematical accuracy and scientific applications, are written with "arbitrary precision mathematics" which allow for infinitely long numbers, limited only by the amount of memory in the computer, but they are much more difficult to design and rarely necessary outside of engineering or science disciplines.
smart BASIC Rocks!

- Scott : San Diego, California

Post Reply