Monday, January 5, 2015

The Prediction Tracker

This is just a quick note to say that this season I've been submitting predictions to The Prediction Tracker.  This site tracks the performance of a number of rating / prediction sites.  Current standings as of today look like this:

                                                                  Mean        Straight    Against
                           Straight  Against    Mean     Mean     Square        Up        Spread
                              Up     Spread     Error    Bias     Error       W     L     W     L

  Opening Line               0.77321  0.52652  8.82583 -0.12012  150.521    1016  298    268   241
  Dokter Entropy             0.76173  0.52541  8.79621 -0.15383  126.594    1055  330    703   635
  Line                       0.76170   .       8.63633 -0.07785  121.524    1042  326      .     .
  Ashby AccuRatings          0.74982  0.50901  8.86946  0.51996  127.529    1034  345    650   627
  StatFox                    0.74982  0.50000  9.15083  0.18088  135.416    1034  345    629   629
  System Average             0.74946  0.49440  8.92108  0.07232  128.329    1038  347    662   677
  Sonny Moore                0.74764  0.51311  9.30695  1.43590  139.909    1031  348    685   650
  NetProphet                 0.74661  0.53379  8.92028 -0.17446  126.397     551  187    387   338
  Sagarin Rating             0.74619  0.49625  9.16057 -0.02549  134.504    1029  350    661   671
  DRatings.com               0.74566  0.49234  9.43660 -1.43057  147.705     988  337    611   630
  Sagarin Predictor          0.74547  0.49737  9.11497  0.02793  133.947    1028  351    663   670
  ComPughter Ratings         0.74032  0.51709  9.53151  0.20241  148.497     841  295    575   537
  Sagarin Golden Mean        0.72806  0.49512  9.81426  0.07640  154.552    1004  375    659   672
  Sagarin Elo Score          0.71501  0.50113  10.2929  0.17347  169.295     986  393    668   665

I optimize my predictor on RMSE, so I'm pleased to see that I have the best performance in that metric of the tracked predictors.  I'm also doing the best of the predictors against the spread, although that performance is a little higher than I'd expect from my own testing so I won't be surprised if that trends down.  It's interesting to note that my predictor is about middle-of-the-pack for predicting the winner straight up and also not very good on Mean Error.

I only submit picks once a week on Monday, so that hurts my performance a little bit.  The predictions for the Saturday games are five or six days stale, which probably costs me 0.1 or so in RMSE.  (But for all I know the other predictors have the same problem.)

No comments:

Post a Comment