Windy.com has really changed the landscape in this area of flying.
Firstly it has brought the ECMWF and later the ICON weather models, which are clearly more accurate than the GFS which has served GA for many years because the European data was inaccessible due to commercialisation.
It presents the weather with nice graphics.
The detailed graphics can give the impression of great accuracy, but this is an illusion as much as any other wx forecast has always been… it is all probabilistic in the end. A lot of people look at the Rain/Thunder layer and cancel a trip, rather than doing what they used to do and wait until the morning of the flight and check the tafs and metars or, for the better informed, the radar and IR images.
Im not an IFR type, so I tend to make all my trips with inbuilt flexibility and opt-outs anyway and am not scared to “see how far I can get” on occasion, but I actually think that the new weather forecasting accuracy combined with easily readable graphics make planning trips much easier so you can be more confident than ever before. Additionally the internet and the proliferation of webcams means that you can often see the conditions at destination and enroute before departure and unless flying at FL-nosebleed often get inflight realtime updates over mobile devices.
Regards, SD..
I would agree with the title of the thread but hasten to add that this is not only an aviation problem.
The various apps which partly give forecasts up to 2 weeks in advance with a implied accuracy which is totally ridikulous for anyone who knows meteorology have lead to a total change in public perception about weather forecasting. As a result, many weather service beancounters are salivating over the high cost forecasters and figure they can be retired soon, so that models would automatize just about everything about weather forecasting.
For me, that is a very wrong direction in which meteorology is moving, but it corresponds with the Zeitgeist trying to more and more eliminate the human element in favour of automatisation, even if the end product is not necessarily better but sellable and flashy.
Yes, it is a problem that these sites are available and are used by people who have little up to no experience with weather models. Consequently the accuracy of those forecasts is heavily disputed often enough, which is sometimes a result of simply way too great forecast periods or lack of experience in interpreting models. The sites do suggest that the model output is “reality” while in fact it is a more or less educated machine generated guess.
Of course models have been much improved over the years. If I remember the forecasts we had in 2001 when I started with my current job, the flood of information we have now is mindblowing. Some of the expert models have become very good indeed, as long as the weather situation corresponds to what they “know”. In some situations they however turn out complete hysteria or ignore stuff which should induce total panic but doesn’t as the models don’t know that particular setting.
The most important thing when dealing with those models is to learn when to trust them and when not. Which model will yield best results for which situation. When do you have to revert to the big picture as opposed to pin point forecasts, e.g. in convective situations where all you will know for certain is that there’s gonna be a lot of bowling in St.Pete’s alley but where the individual cells will hit is anyone’s guess until afterwards. Or the famed cut-off situations where my old teacher told me to report sick when one of those started to develop as the behaviour of a cut off over Europe has been described in a rather politically incorrect way as the one of “a mother in law in menopause having a menstruational crisis”. Yea I know but then I did not make that one up (actually rumor has it that it was a woman who did and a whopping good scientist she was in her days) and having lived through a few cut-off situations I’d say the comparison is crude but not totally inaccurate.
Now try to make a model predict that behaviour…. and then tell a guy who bought 2000 roast sausages based on a model forecast which looked like the open air event he was hired to cater for would happen only to deliver them to the met office in utter frustration on the Monday after. Took a few weeks until they were eaten too.
Personally, I’d not take anything more than 2-3 days ahead as gospel and as we know even gospels are inaccurate at times. The rest is a faint hint at things to come. I more than once had to find out myself that being tempted to trust long range forecasts can bite worse than a pitbull on LSD.
Windy is just a “visualisation tool” (e.g. not a numerical solver that runs partial differtial equations models), so it shows complex raw outputs for forcast/actual weather in nice formats, actually I find it the best for water sports and flying
Actually one can check metar/taf there where you have history, trend and geographic points but I find the visualisation useful to get a consistent full picture while planning (accurate or not who cares? you will not land/depart if weather deviate too much on the wrong side from your planning), I have no clue how people could fly based on TAF/METARs while a simple live webcam picture tells you everything these days,
The best weather tool is the one that show uncertainty bounds and worst case way out rather than getting it 100% right or wrong, also weather is complex so having all data even rubish ones helps to understand but weather decisions should be really simple and I agree too much info could misleading here…
Ibra wrote:
The best weather tool is the one that show uncertainty bounds and worst case way out rather than getting it 100% right or wrong,
If you show worst case, you can basically forget flying in many places in Europe.
What I’d see as a very interesting development is if Ensemble forecasts are used to give a confidence index to the forecasts. I’ve seen this in a few applications but in none of the ones shown to the public, as it would be admitting they don’t know either :) but ensembles are really great to see what the spread is. The bigger the spread of “opinions” in an ensemble, the less it’s worth looking at the end products. With a confidence index of 10% or so, why bother. And looking at the CI vs time, it goes from 80% on day one to about 10% in day 7. So 14 day forecasts….. build your own opinion about those. Unless you live in a place where the weather is always the same and they last changed the rain gauge in 1984 or so,…..
Maybe some PhD student could do a big analysis comparing the different models’ forecasts, at different timescales, against METARs. We’d soon have a very good idea of the level of trust that can be placed in each model at what timescale. Actually, it’s probably been done.
In his really excellent seminar (highly recommended), Alan South says that forecasts achieve 97% accuracy at three days, and doesn’t recommend even glancing at the forecast any earlier than that.
Mooney_Driver wrote:
If you show worst case, you can basically forget flying in many places in Europe
Yes, if you live on the cold lee side of the valley with warm sea on the other side you are better off flying locally for 15min max 
Or just put more fuel and fly somewhere else where the forecast works ok 
Timothy wrote:
Maybe some PhD student could do a big analysis comparing the different models’ forecasts, at different timescales, against METARs. We’d soon have a very good idea of the level of trust that can be placed in each model at what timescale. Actually, it’s probably been done.
There are various weather model validation reports but they very boring thing to read and most of time their % accuracy have been aggregated in space/time/season to be useful for GA mission it will be good to be able to tweak the score for a given airport/time/season and skew accuracy for bad weather vs pilot currency, AFAIK weather in Southend is always better than Stansted/Luton while all sits in the same “weather cell” and have the same % accuracy score
On OP, a “cautious pilot” (inexperienced or very experienced) will loose confidence/currency over time unless he takes few risks on actual weather, that has nothing to do with weather forecast % accuracy, say there are only few good CAVOK days to fly in even if you have the dates at the start of the year and there are 10 of them, you will have to fly in BKN one day to fly 11 days a year…
Timothy wrote:
Maybe some PhD student could do a big analysis comparing the different models’ forecasts, at different timescales, against METARs. We’d soon have a very good idea of the level of trust that can be placed in each model at what timescale. Actually, it’s probably been done.
You are darn right it is being done, continuously. The guys working on those models and the verifications thereof are probably the brightest people in any of these services. We occasionally get briefs by them about what they are up to. IMHO it’s not Rocket Science, it is way beyond that. The top model guys measure up to the ladies and gentlemen working in places like Cern or PSI, only that I got it from one of the nuclear scientists of the latter that he thinks that meteorological modelling is something nuclear scientists are looking up to… about the only thing.
So PhD students indeed sometimes do verification work but most of the folks working models are beyond PhD level. They got to be, otherwise they’d run out of headache pills before breakfast. And verification is a quite large part of that. It is based on verification results that changes to the models will be made to try to address those factors which are known to be deficient without breaking something which works. The “fun” bit about those models is that changing one minor factor can have fundamental consequences. Like one guy put it: The beat of the wing of a butterfly in Maryland can have an influence on a thunderstorm in Scotland. I suppose some of those guys might find bomb defusing an entertaining past time.
Ibra wrote:
Yes, if you live on the cold lee side of the valley with warm sea on the other side you are better off flying locally for 15min max
Or just put more fuel and fly somewhere else where the forecast works ok
The factors are treefold: Weather, time and availability.
I have done a GAFOR analysis once for some Swiss core routes, East-West and North-South. In short, East – West is VFR planable an average of about 70% over the year, North-South about 30% due to the Alps. So broken down over a month, you should expect to be able to fly north of the Alps at about 21 days during a calendar month while the North-South routes are available about 9 days a month. If you calculate that an average employee has 8 days off per month, then you can wagger that his chances of flying from Zurich to Geneva on one of those days off are quite good, while it is pretty well possible that North-South won’t happen at all. And GAFORs are man made forecasts based on huge experience.
Looking at model forecasts when trying to determine whether it’s gonna be a beach day at Albenga or a burger in La Chaux de Fonds as the darn Alps are shut again, the waggers may even be worse. If I look at automatical METAR generation, they are generally erring on the “safe” side, that means to the bad. If we were trying to do GAFOR forecasts automated (and they are thinking about them) I would expect the ratio of OD vs MX to go even further downwards. The same effect is there if you look at the raw model stuff, visualized or not, which is not verified on generation like a normal GAFOR is, where model output is crosschecked with real world data.
The next head scratcher is that the model runs base on analysis of current conditions which are up to now presented by meteorological observers mixed with automated data. Taking into account what I said earlier about Autometars which, while capable of discerning VMC/IMC and some of those parameters well enough fail brutally in cloud base and visibility just to mention two factors which come out fishier than 10 day old sushi, the principle of garbage in-garbage out will be a huge problem for model calcs in the future and partially already is in some of those places where automatic observations are the only thing available.
Timothy wrote:
Alan South says that forecasts achieve 97% accuracy at three days, and doesn’t recommend even glancing at the forecast any earlier than that.
That is what my experience tells me as well for the average day and general conditions. It is a sight worse for e.g. convective activities onto a specific location, fog forecasting and some other such niceties. One of my “favorite” things at work are those guys who call from their smartphones while watching Windy or Weather Pro and ask me to give them the probability of rain in (insert a postcode, yes, as a meteorologist some people expect you to know all post codes by heart as well) in 3 weeks time at a specific time of the day. Particularly in Summer, they can ask the same question on the morning of the day and if there is convectivity you still can’t tell them with the degree of accuracy they demand whether their concrete will dry for sure before the next rain or become a soggy mess, so in order to save your bacon and their money you’ll overstate the actual probability with the risk that that particular real estate won’t get a single drop, while neigboring villages get free swimming pools in their cellars and cars take on the form of Junkers airfoils after hail is done with them.
So it is always interesting to know what these 97% mean. General meteorological situation, I believe that yes. Particular forecast on spot on time, maybe in the UK that works but it certainly doesn’t in alpine territory.
Timothy wrote:
Maybe some PhD student could do a big analysis comparing the different models’ forecasts, at different timescales, against METARs
There are very few weather models. Typically each country/region has one single model (number cruncher), and what these weather apps do is simple to display the results (at best). METARs on the other hand, is actual weather + forecasted temporaries, and may also be very “wrong” at one particular point in time, but more accurate when averaging them up.
There is weather and there is weather. Sunshine is obviously nicer than rain (most of the time), but say very little about weather conditions, especially local weather conditions wrt go/no go. Webcams are better tools than weather forecasts IMO. In Norway we now got this professional system that ambulance helicopters use (HemsWX). Actual weather (visibility) in real time.
