Clever Geek Handbook

LINQ to SQL Math.Round problem

I am having a problem with a query I wrote where, for some reason, the variable I am using to store the decimal value returns 6 values ​​after the decimal point (they are mostly 0).

I've tried the following (and different combinations using Math.Round) with no luck.

Sales =
          (from invhed in INVHEAD 
           ... // Joins here
           orderby cust.State ascending
           select new Sale
           {
                InvoiceLine = inv.InvoiceLine,
                InvoiceNum = inv.InvoiceNum,
                ...
                NetPrice = Math.Round((inv.ExtPrice - inv.Discount) * (Decimal) (qsales.RepSplit / 100.0), 2, MidpointRounding.ToEven),
           }).ToList<Sale>();

NetPrice member has values ​​like 300.000000, 5000.500000, 3245.250000, etc.

Any hints? I can't seem to find anything on the internet for this issue.

EDIT:

Decimal.Round did the trick (I forgot to mention that the NetPrice item was of decimal type). See my answer below.

+2


source to share


3 answers


I got it to work using Decimal.Round () with the same arguments as before. :)

It looks like the problem is somewhat related to what Paul was saying, where decimal types behave differently and it would seem that Math.Round doesn't work with them as one might expect ...



Thanks for all the input.

0


source


Trailing zeros can be displayed in .ToString output for decimal type. You need to specify the number of digits after the decimal point that you want to display using the correct format string. For example: -

 var s = dec.ToString("#.00");

display 2 decimal places.

Internally, the decimal type uses an integer and decimal scale factor. Its a scaling factor that ends up being 0. If you start with a decimal type with a scaling factor of 2, you get 2 digits after the decimal point, even if they are 0.

Adding and subtracting decimal places will result in a decimal value having a scaling factor that is the maximum number of decimal places. Therefore, subtracting one decimal number with a scaling factor of 2 from another with the same final decimal number also has a factor of 2.



Multiplying and dividing decimal places will result in a decimal value having a scaling factor, which is the sum of the scaling factors of the two decimal operands. Multiplying the decimal places by a scaling factor of 2 results in a new decimal point with a scaling factor of 4.

Try this: -

var x = new decimal(1.01) - (decimal)0.01;
var y = new decimal(2.02) - (decimal)0.02;
Console.WriteLine(x * y * x * x);

You get 2.00000000.

+2


source


System.Decimal

preserves trailing zeros by design. In other words, 1m

and 1.00m

are two different decimal

(although they will be compared as equal) and can be interpreted as rounded to a different number of decimal places - for example, Math.Round(1.001m)

give 1m

and Math.Round(1.001m, 2)

give 1.00m

. Arithmetic operators treat them differently - +

and -

will lead to a result that has the same number of places as the operand, which most of them have (s 1.50m + 0.5m == 1.10m

), *

and /

will have the sum of the number of places for their operands (s 1.00m * 1.000m == 1.00000m

).

+2


source






More articles:


All Articles