Hi
Can anyone help please.
When I run below, it will round to decimal place as per input, however after the the 9th and onwards rounding the output does not follow the same logic for rounding and random numbers start to appear when compared to the original input. What is causing this?
ps. I've added the last print statement so it's easier to compare on the output screen the behaviour of result versus actual
'Decimal points and rounding input/output
y = 123456789.1234567890123456
input "Decimal Points" : dp
a$ = "#.####################"
b$ = left$ (a$,dp + 2)
print "result is. "; b$:y
print "Actuaul input 123456789.1234567890123456"
Or is there a global setting in Smart Basic that controls what I'm trying to achieve
Thank U
Decimal points and rounding input/output
- Mr. Kibernetik
- Site Admin
- Posts: 4786
- Joined: Mon Nov 19, 2012 10:16 pm
- My devices: iPhone, iPad, MacBook
- Location: Russia
- Flag:
Re: Decimal points and rounding input/output
Computers have limit of numbers precision, and this is a good example of this limit.
- sarossell
- Posts: 195
- Joined: Sat Nov 05, 2016 6:31 pm
- My devices: iPad Mini 2, iPhone 5, MacBook Air, MacBook Pro
- Flag:
- Contact:
Re: Decimal points and rounding input/output
As I understand it (and please correct me if I'm wrong, Mr. K), smart BASIC uses 53 bits for numeric operations which allows for sufficiently large numbers and decimal places but if the number of total digits is over 16 or so, it will not have enough binary digits to represent them accurately and will have to round. It looks like you found the event horizon.
This is called "fixed point precision mathematics". Some programming languages, created specifically for mathematical accuracy and scientific applications, are written with "arbitrary precision mathematics" which allow for infinitely long numbers, limited only by the amount of memory in the computer, but they are much more difficult to design and rarely necessary outside of engineering or science disciplines.
This is called "fixed point precision mathematics". Some programming languages, created specifically for mathematical accuracy and scientific applications, are written with "arbitrary precision mathematics" which allow for infinitely long numbers, limited only by the amount of memory in the computer, but they are much more difficult to design and rarely necessary outside of engineering or science disciplines.
smart BASIC Rocks!
- Scott : San Diego, California
- Scott : San Diego, California