Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Float64 conversion #174

Open
dcullender-cb opened this issue Nov 9, 2021 · 1 comment
Open

Float64 conversion #174

dcullender-cb opened this issue Nov 9, 2021 · 1 comment

Comments

@dcullender-cb
Copy link

dcullender-cb commented Nov 9, 2021

When converting to float64 using Float64(), the value can differ from the input string while shopspring/decimal seems to produce the correct result. for example

image

Appreciate that the value cannot fit into a float64 as per ericOk but curious why the shopspring value matches?

@ericlagergren
Copy link
Owner

This is because shopspring converts their decimal to a big.Rat, then converts the Rat to a float64. My library converts as-is using float64.

The upside to shopspring’s method is that it won’t round worse than the decimal’s current precision during the conversion. The downside is that converting to a rational number is expensive: you end up with huge numerators and denominators. Plus, you have to rescale the decimal to create the rational number.

The upside to my method is it’s fast and allocation-free (for small floats, that is). The downside is you occasionally have rounding issues.

There might be a better way of performing the conversion (more bit twiddling instead of float division/multiplication), but I haven’t looked too closely at it. I’m not entirely sure why people are using the Float64 method—it seems to defeat the purpose of using a decimal library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants