TV Weathercaster Evaluation: An Atmospheric Sciences 101 Class Project

For nine weeks this autumn, over 100 students in the UW Atmospheric Sciences 101 class taught by Professor Clifford Mass watched TV weather segments on late-evening local news broadcasts in order to evaluate the relative skill of TV weathercasters and the National Weather Service for maxiimum temperature forecasts at Seattle-Tacoma Airport.

Major questions included:

How rapidly does forecast skill deteriorate as a function of forecast projection? That is, how much worse are 4-5 day forecasts compared to 1-2 day forecasts?

Does skill vary among local TV stations?

Are some TV stations consistently more skillful in forecasting maximum temperature?

Do any significantly improve upon the NWS forecasts?

Precipitation forecasts were not verified since TV stations in the Seattle market do not provide any quantifiable prediction s of precipitation (using graphics to show collections of drops, etc when rain is forecast).

Method:

Two to three students were assigned to each broadcast to reduce the possibility of transcribing bad data. The late night (10 PM or 11 PM) broadcasts were used. For the NWS forecasts for the first two days into the forecast period we used the official forecast for Seattle that is released at approximately 9:30 PM. Since there is no official NWS forecast for days 3-5 we used the forecasts provided by the National Weather Service model output statistics package for the Medium Range Forecast model. All of the above data was collected daily and the collection was entered into computer files by Scott Judson, an undergraduate. The verification site for the maximum temperature forecast was Seattle Tacoma Airport--the site generally used for verification purposes in the central sound. Mr. Judson and Ernie Recker, a department staff member, wrote a simple computer program to analyze the information--which is presented below. Below are the raw results; conclusion are presented at the end. It is important to note that small differences in forecast skill (less than approximately .5F) are probably not statistically significant.


Day 1: Mean Maximum Temperature Errors

National Weather Service Mean Error: 2.5F

    Error (F)

    Stations

    2.3 to 2.7

    NWS, KOMO,

    KING, KIRO

    2.8 to 3.2

    KSTW


Day 2: Mean Maximum Temperature Errors

National Weather Service Mean Error: 2.8F

    Error (F)

    Stations

    2.8 to 3.2

    NWS, KOMO,

    KING, KIRO,

    KSTW

    3.3 and higher

    none


Day 3: Mean Maximum Temperature Errors

National Weather Service Mean Error: 3.3F

    Error (F)

    Stations

    3.3 to 3.7

    NWS, KOMO,

    KING, KIRO,

    KSTW

    3.8 and higher

    none


Day 4: Mean Maximum Temperature Errors

National Weather Service Mean Error: 3.2F

    Error (F)

    Stations

    3.2

    NWS, KOMO

    3.3 to 3.7

    KIRO

    3.8 to 4.2

    KING, KSTW


Day 5: Mean Maximum Temperature Errors

National Weather Service Mean Error: 4.9F

    Error (F)

    Stations

    3.8 to 4.2

    KIRO

    4.3 to 4.7

    KING

    4.8 to 5.2

    NWS, KSTW


Conclusions: