6

.Net Gotcha: Decimals track their number of decimal places

 3 years ago
source link: https://stevedrivendevelopment.com/2015/07/10/net-gotcha-decimals-track-their-number-of-decimal-places/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

.Net Gotcha: Decimals track their number of decimal places

tl:dr; System.Decimal values in .Net track their decimal places, so that 12.0 is not precisely the same as 12.00000. This will cause you trouble if you expect the string representation to be the same.

The other day I was calculating a key signature for some records. A key signature is a way to combine a compound primary key of any type and create a representative string value;

// the record has the primary key { 42, 99 }
signature == "42|99"

The idea being that you can compare records using simple string comparison. Simplifying comparison means that operations like sorting, duplicate detection, or JOIN-like operations can now be implemented by working on sequences of strings.

The difficulty is this — how do you reduce arbitrary value types to strings? A straightforward approach is just to call Convert.ToString on the value, and that was my first approach;

string CalculateSignature1(object[] keyValues) {
    var sb = new StringBuilder();
    foreach(var value in keyValues)
    {
        if (value == null) {
            sb.Append("<null>");
        } else {
            sb.Append(Convert.ToString(value));
        }
        sb.Append("|");
    }
    if (keyValues.Length > 0) { sb.Length -= 1; }
    return sb.ToString();
}

Which works like so;

CalculateSignature1(new object[] { 1, "hello", null, 99 }) == "1|hello|<null>|99";

This is reasonable and works fairly well until you come to Decimals.

Decimals cause you problems because they track their decimal place. By that I mean that these two objects are different

var sb = new StringBuilder();
Decimal a = 12.0d; // Convert.ToString returns "12.0"
Decimal b = 12.000d; // Convert.ToString returns "12.000"

So you can see that Convert.ToString is returning different representations.

This was a beast to track down, because the expression a==b returns true. So tests, debuggers, etc seem to show them as identical values. The values, types, etc all tell you it’s the same value.

What you need to do is use String.Format or StringBuilder.AppendFormat to reduce to a common format — here’s the updated version;

string CalculateSignature2(object[] keyValues) {
    var sb = new StringBuilder();
    foreach(var value in keyValues)
    {
        if (value == null) {
            sb.Append("<null>");
        } else if (value is Decimal) {
            // decimals store their accuracy, so that 2.0M and 2.000M are not the same number. Here we
            // toString in such a way that numbers like 2.0M and 2.000M give the same representation in the
            // signature. To do otherwise is to calculate different signatures from different values, and
            // therefore 'miss' an appropriate join between source and target records. 
            var d = (decimal)value;
            sb.AppendFormat("{0:0000000000000000.0000000000000000}", d);
        } else {
            sb.Append(Convert.ToString(value));
        }
        sb.Append("|");
    }
    if (keyValues.Length > 0) { sb.Length -= 1; }
    return sb.ToString();
}

So watch out for that — it can sneak up on you!

decimal-gotchas.csx — scriptcs code;

void AssertEqual(string a, string b) {
	Console.Write(a == b ? "PASS: " : "FAIL: "); 
	Console.Write(a);
	Console.Write(" == ");
	Console.WriteLine(b);
}

void AssertNotEqual(string a, string b) {
	Console.Write(a != b ? "PASS: " : "FAIL: "); 
	Console.Write(a);
	Console.Write(" != ");
	Console.WriteLine(b);
}


string CalculateSignature1(object[] keyValues) {
	var sb = new StringBuilder();
	foreach(var value in keyValues)
	{
		if (value == null) {
			sb.Append("<null>");
		} else {
			sb.Append(Convert.ToString(value));
		}
	    sb.Append("|");
	}
	if (keyValues.Length > 0) { sb.Length -= 1; }
	return sb.ToString();
}

string CalculateSignature2(object[] keyValues) {
	var sb = new StringBuilder();
	foreach(var value in keyValues)
	{
		if (value == null) {
			sb.Append("<null>");
		} else if (value is Decimal) {
			// decimals store their accuracy, so that 2.0M and 2.000M are not the same number. Here we
            // toString in such a way that numbers like 2.0M and 2.000M give the same representation in the
            // signature. To do otherwise is to calculate different signatures from different values, and
            // therefore 'miss' an appropriate join between source and target records. 
            var d = (decimal)value;
            sb.AppendFormat("{0:0000000000000000.0000000000000000}", d);
	    } else {
			sb.Append(Convert.ToString(value));
		}
	    sb.Append("|");
	}
	if (keyValues.Length > 0) { sb.Length -= 1; }
	return sb.ToString();
}

AssertEqual(CalculateSignature1(new object[] { 1, "hello", null, 99 }), "1|hello|<null>|99");

Decimal a = 12.0m;
Decimal b = 12.000m;

Console.WriteLine(a == b);

AssertNotEqual(CalculateSignature1(new object[] { a }), CalculateSignature1(new object[] { b }));
AssertEqual(CalculateSignature2(new object[] { a }), CalculateSignature2(new object[] { b }));


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK