Alternative Verification using MAE as a metric, Forecast Hour 72


Explanation of notation used in tables below
Map of stations held out for bias correction (50%), verification of bias correction during optimization (25%), and final verification (25%)

Verification of all settings below versus the same held-out subset (25%) of obs

25-day, simple bias correction Mass-Wedam-Steed settings Optimized settings
(with 50% obs for bc, 25% obs for bc verification,
and 25% obs for final verification)
Annual (Jul-2004 - Jun-2005)
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =      57500
mae_nobc    =   2.565378
mae_bc      =   2.548738
me_nobc     =  -0.728467
me_bc       =   0.008352
hpf         =   0.330087
htf         =   0.324922
mae_improve =   0.648637

Metric, MAE =   2.548738
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =      57500
mae_nobc    =   2.565378
mae_bc      =   2.432314
me_nobc     =  -0.728467
me_bc       =  -0.200363
hpf         =   0.281513
htf         =   0.212870
mae_improve =   5.186916

Metric, MAE =   2.432314
      
ndts          = 54
nalg          = 3
nobs          = 8
tolin         = 3.5
limin         = 5
max_dist      = 744
max_elev_diff = 290

ngood       =      57500
mae_nobc    =   2.565378
mae_bc      =   2.381337
me_nobc     =  -0.728467
me_bc       =  -0.229720
hpf         =   0.336278
htf         =   0.229130
mae_improve =   7.174030

Metric, MAE =   2.381337
      
Optimization plot
July 2004
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       4671
mae_nobc    =   3.215045
mae_bc      =   2.576944
me_nobc     =  -2.322670
me_bc       =  -0.635043
hpf         =   0.507814
htf         =   0.242560
mae_improve =  19.847343

Metric, MAE =   2.576944
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       4671
mae_nobc    =   3.215045
mae_bc      =   2.555845
me_nobc     =  -2.322670
me_bc       =  -1.024344
hpf         =   0.462428
htf         =   0.183258
mae_improve =  20.503601

Metric, MAE =   2.555845
      
ndts          = 24
nalg          = 8
nobs          = 7
tolin         = 5.5
limin         = 10
max_dist      = 840
max_elev_diff = 430

ngood       =       4671
mae_nobc    =   3.215045
mae_bc      =   2.429131
me_nobc     =  -2.322670
me_bc       =  -0.723909
hpf         =   0.534789
htf         =   0.213445
mae_improve =  24.444883

Metric, MAE =   2.429131
      
Optimization plot
August 2004
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       4771
mae_nobc    =   2.662369
mae_bc      =   2.583830
me_nobc     =  -1.216867
me_bc       =   0.751921
hpf         =   0.404737
htf         =   0.389017
mae_improve =   2.949967

Metric, MAE =   2.583830
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       4771
mae_nobc    =   2.662369
mae_bc      =   2.373605
me_nobc     =  -1.216867
me_bc       =   0.041897
hpf         =   0.356529
htf         =   0.243764
mae_improve =  10.846130

Metric, MAE =   2.373605
      
ndts          = 74
nalg          = 8
nobs          = 3
tolin         = 4
limin         = 4.5
max_dist      = 828
max_elev_diff = 170

ngood       =       4771
mae_nobc    =   2.662369
mae_bc      =   2.351556
me_nobc     =  -1.216867
me_bc       =   0.054884
hpf         =   0.460700
htf         =   0.291972
mae_improve =  11.674302

Metric, MAE =   2.351556
      
Optimization plot
September 2004
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       4542
mae_nobc    =   2.131480
mae_bc      =   2.311510
me_nobc     =   0.127190
me_bc       =   0.723465
hpf         =   0.241083
htf         =   0.343021
mae_improve =  -8.446244

Metric, MAE =   2.311510
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       4542
mae_nobc    =   2.131480
mae_bc      =   2.133212
me_nobc     =   0.127190
me_bc       =   0.209580
hpf         =   0.199031
htf         =   0.193967
mae_improve =  -0.081258

Metric, MAE =   2.133212
      
ndts          = 16
nalg          = 2
nobs          = 14
tolin         = 7.5
limin         = 2.5
max_dist      = 816
max_elev_diff = 410

ngood       =       4542
mae_nobc    =   2.131480
mae_bc      =   2.075821
me_nobc     =   0.127190
me_bc       =   0.181032
hpf         =   0.147732
htf         =   0.095553
mae_improve =   2.611284

Metric, MAE =   2.075821
      
Optimization plot
October 2004
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       4670
mae_nobc    =   1.923209
mae_bc      =   2.054647
me_nobc     =  -0.153231
me_bc       =   0.256788
hpf         =   0.294433
htf         =   0.363383
mae_improve =  -6.834307

Metric, MAE =   2.054647
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       4670
mae_nobc    =   1.923209
mae_bc      =   1.909752
me_nobc     =  -0.153231
me_bc       =  -0.150710
hpf         =   0.175375
htf         =   0.159315
mae_improve =   0.699716

Metric, MAE =   1.909752
      
ndts          = 37
nalg          = 7
nobs          = 11
tolin         = 4.5
limin         = 3
max_dist      = 960
max_elev_diff = 290

ngood       =       4670
mae_nobc    =   1.923209
mae_bc      =   1.889378
me_nobc     =  -0.153231
me_bc       =  -0.167353
hpf         =   0.088223
htf         =   0.060814
mae_improve =   1.759091

Metric, MAE =   1.889378
      
Optimization plot
November 2004
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       5029
mae_nobc    =   2.049784
mae_bc      =   2.097715
me_nobc     =  -0.114942
me_bc       =  -0.053605
hpf         =   0.262478
htf         =   0.285544
mae_improve =  -2.338344

Metric, MAE =   2.097715
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       5029
mae_nobc    =   2.049784
mae_bc      =   1.995638
me_nobc     =  -0.114942
me_bc       =  -0.162214
hpf         =   0.234440
htf         =   0.192484
mae_improve =   2.641547

Metric, MAE =   1.995638
      
ndts          = 65
nalg          = 13
nobs          = 6
tolin         = 4
limin         = 20
max_dist      = 804
max_elev_diff = 320

ngood       =       5029
mae_nobc    =   2.049784
mae_bc      =   1.977945
me_nobc     =  -0.114942
me_bc       =  -0.143123
hpf         =   0.238815
htf         =   0.181945
mae_improve =   3.504711

Metric, MAE =   1.977945
      
Optimization plot
December 2004
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       4740
mae_nobc    =   2.307859
mae_bc      =   2.342579
me_nobc     =  -0.144414
me_bc       =  -0.188938
hpf         =   0.227848
htf         =   0.265823
mae_improve =  -1.504425

Metric, MAE =   2.342579
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       4740
mae_nobc    =   2.307859
mae_bc      =   2.202448
me_nobc     =  -0.144414
me_bc       =  -0.085676
hpf         =   0.309916
htf         =   0.219620
mae_improve =   4.567480

Metric, MAE =   2.202448
      
ndts          = 60
nalg          = 5
nobs          = 7
tolin         = 1.5
limin         = 5.5
max_dist      = 684
max_elev_diff = 210

ngood       =       4740
mae_nobc    =   2.307859
mae_bc      =   2.215717
me_nobc     =  -0.144414
me_bc       =  -0.096525
hpf         =   0.255274
htf         =   0.170042
mae_improve =   3.992532

Metric, MAE =   2.215717
      
Optimization plot
January 2005
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       4786
mae_nobc    =   3.042965
mae_bc      =   3.051730
me_nobc     =  -0.917424
me_bc       =  -1.191191
hpf         =   0.305056
htf         =   0.315921
mae_improve =  -0.288041

Metric, MAE =   3.051730
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       4786
mae_nobc    =   3.042965
mae_bc      =   2.858850
me_nobc     =  -0.917424
me_bc       =  -0.511823
hpf         =   0.314041
htf         =   0.223569
mae_improve =   6.050513

Metric, MAE =   2.858850
      
ndts          = 60
nalg          = 1
nobs          = 23
tolin         = 2.5
limin         = 7.5
max_dist      = 888
max_elev_diff = 310

ngood       =       4786
mae_nobc    =   3.042965
mae_bc      =   2.947085
me_nobc     =  -0.917424
me_bc       =  -0.591871
hpf         =   0.228165
htf         =   0.189511
mae_improve =   3.150874

Metric, MAE =   2.947085
      
Optimization plot
February 2005
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       4299
mae_nobc    =   2.589072
mae_bc      =   2.352171
me_nobc     =  -1.619056
me_bc       =  -0.639398
hpf         =   0.411491
htf         =   0.267039
mae_improve =   9.150035

Metric, MAE =   2.352171
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       4299
mae_nobc    =   2.589072
mae_bc      =   2.231934
me_nobc     =  -1.619056
me_bc       =  -0.557113
hpf         =   0.414515
htf         =   0.212142
mae_improve =  13.794054

Metric, MAE =   2.231934
      
ndts          = 45
nalg          = 6
nobs          = 10
tolin         = 5.5
limin         = 6
max_dist      = 924
max_elev_diff = 270

ngood       =       4299
mae_nobc    =   2.589072
mae_bc      =   2.207853
me_nobc     =  -1.619056
me_bc       =  -0.640572
hpf         =   0.466155
htf         =   0.205862
mae_improve =  14.724156

Metric, MAE =   2.207853
      
Optimization plot
March 2005
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       5012
mae_nobc    =   2.839813
mae_bc      =   2.716727
me_nobc     =  -1.768621
me_bc       =   0.083608
hpf         =   0.421588
htf         =   0.348563
mae_improve =   4.334299

Metric, MAE =   2.716727
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       5012
mae_nobc    =   2.839813
mae_bc      =   2.500062
me_nobc     =  -1.768621
me_bc       =  -0.308841
hpf         =   0.389266
htf         =   0.240423
mae_improve =  11.963851

Metric, MAE =   2.500062
      
ndts          = 66
nalg          = 3
nobs          = 9
tolin         = 4.5
limin         = 6
max_dist      = 948
max_elev_diff = 450

ngood       =       5012
mae_nobc    =   2.839813
mae_bc      =   2.337386
me_nobc     =  -1.768621
me_bc       =  -0.262852
hpf         =   0.473863
htf         =   0.259777
mae_improve =  17.692256

Metric, MAE =   2.337386
      
Optimization plot
April 2005
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       4751
mae_nobc    =   2.893767
mae_bc      =   2.748900
me_nobc     =  -1.510104
me_bc       =  -0.094506
hpf         =   0.392970
htf         =   0.302463
mae_improve =   5.006174

Metric, MAE =   2.748900
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       4751
mae_nobc    =   2.893767
mae_bc      =   2.688623
me_nobc     =  -1.510104
me_bc       =  -0.598547
hpf         =   0.290886
htf         =   0.204799
mae_improve =   7.089168

Metric, MAE =   2.688623
      
ndts          = 50
nalg          = 7
nobs          = 7
tolin         = 6
limin         = 7
max_dist      = 1008
max_elev_diff = 470

ngood       =       4751
mae_nobc    =   2.893767
mae_bc      =   2.623071
me_nobc     =  -1.510104
me_bc       =  -0.531120
hpf         =   0.396759
htf         =   0.239949
mae_improve =   9.354450

Metric, MAE =   2.623071
      
Optimization plot
May 2005
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       5158
mae_nobc    =   2.711837
mae_bc      =   2.872591
me_nobc     =   0.027881
me_bc       =   0.824916
hpf         =   0.263086
htf         =   0.354207
mae_improve =  -5.927864

Metric, MAE =   2.872591
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       5158
mae_nobc    =   2.711837
mae_bc      =   2.617685
me_nobc     =   0.027881
me_bc       =   0.523857
hpf         =   0.241954
htf         =   0.194261
mae_improve =   3.471890

Metric, MAE =   2.617685
      
ndts          = 72
nalg          = 1
nobs          = 9
tolin         = 4
limin         = 4
max_dist      = 648
max_elev_diff = 270

ngood       =       5158
mae_nobc    =   2.711837
mae_bc      =   2.590210
me_nobc     =   0.027881
me_bc       =   0.180645
hpf         =   0.284413
htf         =   0.199302
mae_improve =   4.485041

Metric, MAE =   2.590210
      
Optimization plot
June 2005
ndts          = 25
nalg          = 25
nobs          = 1
tolin         = 99
limin         = 99
max_dist      = 1000
max_elev_diff = 200

ngood       =       5071
mae_nobc    =   2.408993
mae_bc      =   2.481338
me_nobc     =   0.673356
me_bc       =   0.096316
hpf         =   0.313153
htf         =   0.337014
mae_improve =  -3.003122

Metric, MAE =   2.481338
      
ndts          = 35
nalg          = 5
nobs          = 5
tolin         = 2.5
limin         = 10
max_dist      = 480
max_elev_diff = 200

ngood       =       5071
mae_nobc    =   2.408993
mae_bc      =   2.422861
me_nobc     =   0.673356
me_bc       =   0.128480
hpf         =   0.271544
htf         =   0.269769
mae_improve =  -0.575676

Metric, MAE =   2.422861
      
ndts          = 75
nalg          = 12
nobs          = 10
tolin         = 5.5
limin         = 5.5
max_dist      = 972
max_elev_diff = 460

ngood       =       5071
mae_nobc    =   2.408993
mae_bc      =   2.343623
me_nobc     =   0.673356
me_bc       =   0.380438
hpf         =   0.246303
htf         =   0.178071
mae_improve =   2.713582

Metric, MAE =   2.343623
      
Optimization plot