[metapost] Long numerical tokens with new number systems
luigi.scarso at gmail.com
Tue Oct 6 12:43:01 CEST 2015
On Sat, Aug 15, 2015 at 9:52 PM, Toby Thurston <thurston at eml.cc> wrote:
> The syntax for <numeric token> that MP inherits from Metafont allows for
> "one or more" decimal digits in the fractional part of a number. So it's
> legal to write:
> numeric pi;
> pi = 3.141592653589793238462643383279502884197169399375105820974944592;
> show pi;
> This works ok with all the number systems except "decimal" which complains
> about excessive precision.
> Here's what I get with each number system
> >> "scaled"
> >> 3.14159 )
> >> "double"
> >> 3.1415926535897931 )
> >> "binary"
> >> 3.1415926535897932384626433832795028 )
> >> "decimal"
> ! Number is too precise (numberprecision = 34).
> l.80 ...643383279502884197169399375105820974944592
> >> 3.141592653589793238462643383279503 )
> Doesn't this behaviour seem to be a bit inconsistent?
> Clearly, with decimal it would be possible to avoid the error message by
> writing this sort of thing:
> if numbersystem="decimal": numberprecision := 64; fi
> before defining the constant, but it would be nice to have a consistent
> syntax across all number systems.
> One other thing that I notice is that the final digit rounding looks wrong
> for double and binary.
> Any comments?
> Thanks, Toby
Rev. 2070 has now a patch based on
"27 bits are not enough for 8-digit accuracy" by I. Bennett Goldberg
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the metapost