Sitting here again tonight analysing the models (as I have done since 2002) and a question has popped into my head...is model reliability improving ?

T+144 was generally considered as FI in 2002, T+144 is still generally considered as FI in 2013.

In 11 years technology has come on leaps on bounds at a frightening rate but have the models ?

Despite all of the global super computers and a vast network of reliable starting data being fed into them are things better?

It is generally considered that a 2 or 3 day forecast in the 70's had about the same reliability as a 5 day forecast these days. Progress has been made but have we reached a limit ?

Despite the progress it would still seem that the progging of energy and direction of the jet is extremely difficult in certain stages, even at just 72 hours.

LRF's are educated guesswork at best and seasonal forecasts are still very much in their infancy. It will be many many years before these have any credibility. In the meantime can short range forecast be improved further ?

Thoughts welcome.

 

 

---

I think with the weather being a natural phenomenon it will always be difficult to accurately forecast it 100%. I think that's what makes it so interesting in that it can make fools of us all. That's not to say we cannot improve but there may be a limit on just how accurate. It's a bit like Space and Astronomy I mean we been playing around with it seriously for 50+ years now but never sent humans further than the moon, we've only gone so far.

---

Hi Steve,
A few years ago I felt anything coming in from the west was pretty well nailed on in terms of the forecast. Models picked up LP’s that weren’t even formed off the coast of the states and usually more or less got the timing right when they hit our shores.

Being the prevailing wind, there must be stacks of data on this to enable such good forecasting. Could be that these more accurate forecasts were made later on in the year after any hurricane activity had complicated matters, which may happen at this time of year so makes forecasting less predictable.

Always however I have felt any forecast from the north, south and east are less reliable, given there is less data on weather from these directions.

There are so many variables to forecasting and a number we may not even understand or have discovered yet. In a way, the lack of overall knowledge and the unpredictability at times can provide excitement on rare occasions for those interested in weather.

What makes me laugh are LRF's of two weeks and beyond. Not worth the time.

Well this week the models have certainly been appalling.

I've been looking at the output since 2004 and I have to say I have seen no improvement, we seem to have hit a brick wall really but we have seen a decline of westerly winds as well.

Long range forecasts are generally a waste of time unfortunately as well, to the credit of CFS though it can pick up some signals at long range (CFS last March with the northern blocking for example) but with a climate as varied as that of the UK even the smallest changes to large scale circulations (eg NAO) have big implications.

In terms of much longer range forecasting, We need to look into past climates more (i.e 1700's to early 1800's) and see if we can find out more about the AMO and other potential cycles. 

We still have a long way to go but thats what makes weather and climate so interesting and such an important area to research.

---

I think the main problem lies with posters following each and every run, that and the belief that models control the weather and climate.

It's the Jet stream. It's fickle, like second guessing government policy, you don't know which way it will head next...

---

Some really good replies here.  

I agree with Phil, in the fact that with westerlies model reliability has improved drastically. 

The reduction in westerlies as mentioned by yorkshireladin recent years is apparent and this is why maybe we are seeing so much indecisiveness when weather patterns are more meridonal.

 

---

No it haven't improvied but got worse like now they correctly showing very warm BH weekend then suddenly change to wash out cool unsettled one without a REASON stated, then now they slowly restoring back to near orginal warm BH.  It shouldn't have drop that warm settled BH in the first place.

I think the data speak for themselves...

http://www.emc.ncep.noaa.gov/gmb/STATS/html/new_aczhist.html 

Yes, they change, as the data input based on obsevrations etc. change, and this will always be the case with NWP output (surely)? And no the models do not always get it spot on, especially a few days out but there is little doubt they are much better than they were ten plus years ago. (Ten years of improvements are not negated by a few different output options for one weekend 😉 )

---

I suspect one of the problems we have now is access to too much data (4 model runs a day to go over...) This, coupled with a poor understanding by some of the use of ensembles, and what an ensemble run tells us, and sadly a degree of hope forecasting (i.e. the tendency to think/assume that the particular chart/run showing heavy snow/35 deg heat/-20 temps etc is the RIGHT one, even at 7 days away.

There are always going to be easier periods to forecast and harder ones. The long settled spell a few weeks ago was easy. When the models start to struggle is usually associated with pattern changes, and we see this a lot in winter.

On a related point I do worry a bit about the BBC forecasts, and why the recent rain forecasts have been wrong - are they deliberately overestimating how wet conditions might be so that when it isn't we feel better?

Fundamentally it's down to data volume. There is no debate about processing and algorithm improvements - that has continued to enhance the reliability and peformance - but you only get out what you put in. That is why forecasts in a westerly regime tend to be more reliable as the data stream is biased towards the Atlantic with n amount of input from planes, ships, buoys, etc. The data grid is more sparse in other directions, such as the far North Atlantic and Arctic so you would expect the models to be less reliable with developments in these areas.
There is also an argument that the algorithms have some learning capabilities using best fit scenarios from previous experience so westerly weather, usually more prevalent for the UK, is easier to model.

---

A good topic and I broadly agree that the NWP index of all the worlds top models have improved over the last 10 years - in some cases significantly so. The problem remains though that an improvement in NWP index scores does not necessarily translate into an improved weather forecast in our town, village or back yard.

Model resolution continues to vary in time and space. To model the atmosphere across the globe at the resolution required to provide a significant improvement on what we have at the moment would cost a huge, mind boggling sum of money. The question is who will pay for this ?  Do we really need to model the upper air in the western Atlantic at the same resolution we run the limited area or high resolution models which generate our weather forecasts in the UK? It may well be that those who use the upper air data thousands of miles away in the western Atlantic - most notably Airlines are quite happy with the performance of the models in forecasting Jetstream position and strength +/- 10% ...IE its good enough for their planning purposes. Of couse this is not necessarily the case for people like ourselves who need each and every little kink and jetsteak modelled at a much higher resolution in order to predict whether an Atlantic storm will develop or not across the eastern Altantic 48 to 72 hours later. So Im afraid its simply not yet cost effective to model the upper air across the globe at the same resolution 3 to 5 days out as it is to model it 6 to 12 or 12 to 24 hours out. No doubt as computing power continues to improve - we'll see further improvements in modelling at longer range that will hopefully ultimately improve that all important weather forecast in our back garden.

---

 

---