Timothy wrote:
In the first case it is Garmin that is arguably “wrong” (I am not sure I agree) and SD right.
SD is always wrong, and Garmin is merely inaccurate (but nothing that warrants concern IMO). Case closed (as they say 
) On the other hand EasyVFR got these things correct.
Markuus wrote:
Skydemon decided to stay a non-professional assistance tool for hobby pilots. We should treat it as this and not interpret anything serious into it.
Agree 100% with that.
LeSving wrote:
which gives an accuracy of a few NMs, pure and simple, basic math when doing GC calculations.
The thing is that people don’t use ‘basic math’: in this kind of calculations.
How can a lack of floating point precision account for the short legs in the original example?
We are not talking about a 5000nm leg and a few nm at the end of it.
It’s explained in that wiki article. You have to take the cosine of a number that is greater than 0.99999999 but less than 1.00000000. The distance between those numbers are a few kilometers, hence the accuracy is a few kilometers (don’t know the exact numbers).
OK, yes:
On computer systems with low floating-point precision, the spherical law of cosines formula can have large rounding errors if the distance is small (if the two points are a kilometer apart on the surface of the Earth, the cosine of the central angle is near 0.99999999). For modern 64-bit floating-point numbers, the spherical law of cosines formula, given above, does not have serious rounding errors for distances larger than a few meters on the surface of the Earth.3 The haversine formula is numerically better-conditioned for small distances
Amazing… this is the first case I have found in over 40 years of embedded systems design where double floats are needed 
This is simply because very few things in the physical world are known to such a precision.
I am also surprised that a different formula has not been developed which gets around this issue. After all, this is nothing new, and the double float execution time is of the order of hundreds of ms on the earliest microprocessors, which is inconvenient to say the least.
LeSving wrote:
It’s explained in that wiki article. You have to take the cosine of a number that is greater than 0.99999999 but less than 1.00000000. The distance between those numbers are a few kilometers, hence the accuracy is a few kilometers (don’t know the exact numbers).
Yes. This is a very well known issue in numerical analysis. A nonzero result changes very slowly with input meaning that there will be a huge loss of significant figures. Another example of the same phenomenon is taking the difference of two almost equal numbers. Such formulae are said to be ill conditioned.
The trick in such a situation is to reformulate the computation so that this doesn’t happen. Then you can get good accuracy without adding more bits to the computation. In the GC calculation this means that you should reformulate to not use arccos.
Please read a bit further in the Wikipedia article. It goes on to show two other formulae that don’t use arccos. One uses arcsin, the other arctan. For either of these functions the problem simply does not exist. Since both sin and tan (and thus arcsin and arctan) are very close to their argument for small angles, the loss of significant figures simply doesn’t happen. The Wikipedia actually spells out that the formula using arcsin is “better-conditioned for small distances” while it says that the formula using arctan is “accurate for all distances”.
Peter wrote:
Amazing… this is the first case I have found in over 40 years of embedded systems design where double floats are needed
It does not. See my post above.
Indeed; my immediate Q was: how did they do it in the days of Apollo? They had damn clever people who were well able to come up with different formulae, suitable for fixed point integer calculations. They didn’t have floats until much later into the programme, and they were really slow, implemented with an interpreted language.
Apps like SD, probably written in C++, have easy access to double floats anyway. Unless you are doing some dodgy casting, you just put “double” in front of everything
As the GTN FPL distances appear to depend on groundspeed, and are correct when stationary, I think you are barking up the wrong tree.
Well, in your first post, was that during flight? They are wrong, both GTN and SD.
