Data Type Conversions

[Heaven]

Im using visual studio.net pro, and this is occuring when Im using VB

Hope some one has the answer to this one.
I just want to know the Why this happens.

Heres my code:

Dim l as Long
Dim S as Single

l = Long.MaxValue set the long variable to its maximum value
Debug.WriteLine(l) display the results to output (which are 922337203684775807)

s= Convert.ToString(l) convert the long to a string
s -=1000000000000 subtract 1 trillion form the value
l= Convert.toInt64(s) convert the answer back to a long

Debug.WriteLine(l) display the results to ourput (which are 9223370937343148032)

end of code

now as you can see you dont get what you would expect. a single should have just rounded off at 7 places and returned the answer of 9223371 + a lot of trailing zeros.

Ive looked at the .net library and the only thing I can find is that the results would be "inaccurate" duh.... I am trying to find out why . any ideas??? im sure it has something to do with overflow and possibly PositiveInfinity.

 

[Robby]

if you set Option Strict On you cannot do this...

s -=1000000000000 subtract 1 trillion form the value

 

[Derek Stone]

You take a 64-bit integer value, convert it to a string, set a 32-bit equal to it (the string value) and then proceed to perform subtraction on a string value...

You might, just might, want to rethink that very poorly written code.

 

[Heaven]

re; data conversion

I Know that it is poorly written code. as a matter of a fact.. its not a good to convert from a long back to a single. but.... thats not what Im trying to find out. Im trying to find out what is happening to the result. why is it coming back as inaccurately as it is. I think (and I may be wrong) that it might have something to do with when the non integer number is converted to binary from base 10 and then back to decimal for display. im wondering if maybe the fact that some base 10 non integer numbers, when converted to binary act more like an irrational number and go to positive infinity (like as an example pi. or 0.2 ) if anyone knows what is happening with the numbers, Id sure like to know (its turned into a vendeta for me.)

 

[Robby]

One thing I dont understand is why you declared S as Single, then you Convert it to String.
Did you mean to declare S as String ?

Is so, then this will work...

Dim l As Long
Dim S As String

l = Long.MaxValue
S = l.ToString
S = (Convert.ToInt64(S) - 1000000000000).ToString
l = Convert.ToInt64(S)

MessageBox.Show(Long.MaxValue & ControlChars.NewLine & l.ToString)