Advice: A numerical problem under Vb.net

kaoni1024

I have a program segment: For loop
dim y1BeginData as single = -4.0f
dim y1KdValue as singel = 0.8f
...
for ykdIdx as integer = 0 to 10
  dim ykdV as single = y1BeginData + ykdIdx * y1KdValue
  ...
next

When running in this block, when ykdIdx = 3, ykdV = -1.5999999
When ykdIdx = 4, ykdV = -0.799999952; when ykdIdx = 5, ykdV = 0.0000000596046448; when ykdIdx = 6, ykdV = 0.8000001; when ykdIdx = 7, ykdV = 1.60000014; other values ​​are normal, but in fact these are The values ​​should be: -1.6, -0.8, 0, 0.8, 1.6; ask, what is going on in this situation and how to solve it? Thank you

kaoni1024

After waiting for a long time, I finally solved it by myself. I changed the single to double to solve it. It is estimated that it was a problem in numerical conversion.

kaoni1024

However, I haven't figured out why. I hope you can help and figure out why this is the case.

wangfan45

This is a common problem with computers: to know that computers use binary numbers to represent any value, naturally, the style is studied: not all decimal numbers can be accurately represented by binary floating-point numbers, but expressed as the closest to it Close to the decimal fraction!

nijiaoshaa

Floating-point numbers have a precision problem. When you don't specify precision (number of decimal places), the result may be inaccurate. If you want to leave one digit after the decimal point, you can do this: multiply your number by 10, round it, and finally Then divide by 10, and the result is correct.

CInt (yournumber * 10) / 10

Check out, you will solve the problem if you follow my instructions!

kaoni1024

Thank younijiaoshaafor the friend above, but the problem is still not solved. Single, Double, and Double are calculated correctly, but this problem occurs in Single calculation.

xiang7997

wangfan45positive solution

clcvictor

When using floating-point numbers (Single data type (Visual Basic) and Double data type (Visual Basic)), remember that they are stored as binary fractions. This means that for any number of non-binary fractions (of the form k / (2 ^ n, where k and n are integers), floating-point numbers cannot store their exact representation. For example, 0.5 (= 1/ 2) and 0.3125 (= 5/16) can be saved as accurate values, while 0.2 (= 1/5) and 0.3 (= 3/10) can only be approximate values.

Because of this imprecision, you cannot expect accurate results when dealing with floating-point values. Specifically, two values ​​that are theoretically equal may have slightly different representations.

Compare floating point numbers
Use the Abs method of the Math class in the System namespace to calculate the absolute value of the difference between the quantities.

Determine the maximum acceptable difference so that if the difference between the two quantities does not exceed this value, you can consider that the two quantities are actually equal.

Compare the absolute value of the difference with the acceptable difference.

The following example demonstrates a true and false comparison of two Double values.

Visual Basic Copy Code
Dim oneThird As Double = 1.0 / 3.0
Dim pointThrees As Double = 0.333333333333333

'The following comparison does not indicate equality.
Dim exactlyEqual As Boolean = (oneThird = pointThrees)

'The following comparison indicates equality.
Dim closeEnough As Double = 0.000000000000001
Dim absoluteDifference As Double = Math.Abs(oneThird-pointThrees)
Dim practicallyEqual As Boolean = (absoluteDifference <closeEnough)

MsgBox("1.0 / 3.0 is represented as "&oneThird.ToString("G17") _
&vbCrLf&"0.333333333333333 is represented as "_
&pointThrees.ToString("G17") _
&vbCrLf&"Exact comparison generates "&CStr(exactlyEqual) _
&vbCrLf&"Acceptable difference comparison generates "_
&CStr(practicallyEqual))



The above example uses the ToString method of the Double structure, so it can specify a higher precision than when using the CStr keyword. The default precision is 15 digits, but the "G17" format expands it to 17 digits.

The Mod operator does not return accurate results
Due to the inaccuracy of floating-point storage, if at least one operand is a floating-point number, the Mod operator (Visual Basic) may return unexpected results.

The Decimal data type (Visual Basic) does not use floating-point representation. Many numbers that are imprecise in Single and Double are exact numbers in Decimal (for example, 0.2 and 0.3). Although the floating-point representation is faster in arithmetic operations than the Decimal representation, it may be worth the performance degradation in exchange for the precision improvement.

Find the integer remainder of the floating point number
Declare the variable as Decimal.

Use the text type character D to coerce the text to Decimal in case the value of the text is too large relative to the Long data type.

The following example demonstrates the potential inaccuracy of floating-point operands.

Visual Basic Copy Code
Dim two As Double = 2.0
Dim zeroPointTwo As Double = 0.2
Dim quotient As Double = two / zeroPointTwo
Dim doubleRemainder As Double = two Mod zeroPointTwo

MsgBox("2.0 is represented as "&two.ToString("G17") _
&vbCrLf&"0.2 is represented as "&zeroPointTwo.ToString("G17") _
&vbCrLf&"2.0 / 0.2 generates "&quotient.ToString("G17") _
&vbCrLf&"2.0 Mod 0.2 generates "_
&doubleRemainder.ToString("G17"))

Dim decimalRemainder As Decimal = 2D Mod 0.2D
MsgBox("2.0D Mod 0.2D generates "&CStr(decimalRemainder))



The above example uses the ToString method of the Double structure, so it can specify a higher precision than when using the CStr keyword. The default precision is 15 digits, but the "G17" format expands it to 17 digits.

Since zeroPointTwo is Double, the value 0.2 it represents is an infinitely repeated binary fraction (the stored value is 0.20000000000000001). Dividing 2.0 by this amount will give 9.9999999999999995 with a remainder of 0.19999999999999991.

In the expression of decimalRemainder, the text type character D coerces both operands to Decimal, so that 0.2 will have an accurate representation. Therefore, the Mod operator will get the expected remainder 0.0.

Note that declaring decimalRemainder as Decimal is not enough. You must also cast the text to Decimal, otherwise they will default to Double, and decimalRemainder will receive inaccurate values ​​just like doubleRemainder.


MSDN 2005
ms-help://MS.MSDNQTR.v80.chs/MS.MSDN.v80/MS.VisualStudio.v80.chs/dv_vbalr/html/90040d67-b630-4125-a6ae-37195b079042.htm

彼岸花

The answer upstairs is in place, it should be a question of binary fraction storage
learned
hoho collection