Real is a pure 64-bit floating point value (Java double), so it does not have a precision. That is what Decimal is for. What about its string representation doesn’t work?
It may look like you get back 3.23, but since this is a binary floating point representation, that may not be what you really have. For example:
(0.1 + 0.2 == 0.3)
Actually evaluates to false, not true.
There is nothing wrong with the string representation inside the platform. But I need the resulting JSON object that I am going to plug the number into to not have quotes around it.
So if I did something like this:
var number = toDecimal((randomFloat() * (max – min) + min ), 2)
return {val:number}
I might get back something like
{
“val”: “24.41”
}
If I send it to another service or if I called it via an API externally, I get back a string.`
- You must be logged in to reply to this topic.
Is there a way to set the precision of a Real without having to convert it to string and then back.
For example
var myNumber = 3.23323424234
var result = toReal(format(“{0,number,######.##}”, myNumber))
will give me back 3.23. It seems odd I would have to do it this way. While toDecimal has precision, this won’t work for me because of its String representation and what I’m trying to do.
Am I stuck with this conversion?