|
|||
|
Hello To All, I am using ADS version 10 to sum up some values in SQL
the values are 1636.36, 22.61 the total I am getting is 1658.98 when it should be 1658.97 I am using the sum function in an sql statement with VO2.8. All other grouping are correct except this group. Can someone give me some help please!! One Million Thanks!! John Gosub |
|
|
||||
|
||||
|
|
|
|||
|
John,
This kind of thing has nothing to do with it being ADS, DBF or SQL, it is purely your misunderstanding of what you are looking at and the precision in question. What is visually presented to you will be the result of formatting for display as opposed to the underlying data content, which you need to know. I suspect the data types you are dealing with store more than two decimal places but your visual formatting rounds to 2. For example, let us say the actual data is this: 1636.364 22.614 When you review both items to 2 decimals you see 1636.36 and 22.61 but when you add these two numbers and then review the result, you MUST get the answer 1658.98 - that is correct. So this is really Mathematics 101. You need to decide whether you are going to round and store or keep precision and round the result. You need to pick a strategy and keep it consistent throughout the application but these contentions will always exist. Another huge error SQL writers often commit is to compare a float value to zero: SOMETOT <> 0. The trouble here is that if SOMETOT actually held 0.0000000000000001, visually it looks like zero but in actuality, it isn't. This issue has been around since 1993 when VO first came out. Geoff "John Gosub" <servia@tstt.net.tt> wrote in message news:a931e52b-5cea-4839-89d4-c3857d986899@m1g2000yqo.googlegroups.com: > Hello To All, I am using ADS version 10 to sum up some values in SQL > the values are 1636.36, 22.61 the total I am getting is 1658.98 when > it should be 1658.97 I am using the sum function in an sql statement > with VO2.8. > All other grouping are correct except this group. Can someone give me > some help please!! > > One Million Thanks!! > John Gosub |
|
|
![]() |
| Thread Tools | |
| Display Modes | |
|
|