Simulating the 2014 So Cal Governors Prelim Rounds

Some analysis of the SoCal Govenor Cup G01 Preliminary round predictions.  Scroll way down to see the simulations or click this --> Link to page where you can see simulations with link to team pages This simulation was run with a model based on data before the Governors Cup started.  Here are the actual results from the tournament.

So how well did the model do comparing the predictions and actual results?

• Predicting the #1 finisher. What fraction of the time brackets do you expect that the team with the highest probability would win?  To do comparison, I sum up the highest probabilities in the 1st column and divide by 100 and compare to # of times the #1 finisher was the one with the highest probability.
• If my model were perfect and there were no uncertainty in the estimates, then I expect to get 11.29 brackets right (right=I correctly predicted the #1 in bracket).
• If I chose randomly (meaning all probabilities in 1st col = 25), I expect to get .26x17=4.25 brackets right.   
• I actually got 7 right.  Better than average but definitely lower than 11 by a lot.
• So, the estimated strengths are uncertain (certainly true) and/or past performance is not a perfect predictor future performance (also true).
• Overall, my impression of my success at predicting the #1 finisher in a bracket is...'Meh'. I suspect that that is just hard to predict (which is part of what make tournaments fun---the favorites do not always finish #1)
• How well does the model predict what team gets through their bracket.  Here things look a lot better (to me).  This plot shows the model probability of getting through (probability finishing #1 + probability of finishing #2) against whether or not the team did get through (0 or 1).  You can see there is a strong correlation.  If there were none, the red line, which is the logistic regression fit, would be flat.
• If the model probability is less than 30, chances are grim BUT some do still make it and it could be your team!
• If the probability is greater than 70, chances are good but don't get complacent.  Even teams almost certain of making it through (according to my model) have been stopped in the prelims.
• Between 30 and 70, chances are about 50/50.
• Next step, compare to the National Cup logistic regression after that cup is done.  My intuition is that a higher level cup (National Cup) will be more predictable---though this is pretty spot on.

SIMULATION RESULTS

SoCal Govenors Bracket A
                                    1st 2nd 3rd 4th
Conejo Valley United White G01        0   2  10  88
Temecula United G01                  32  33  31   4  #2
West Coast SA East Valley Lions G01  37  33  27   4  #3
Westside Breakers Blue G01           31  32  32   4  #1

SoCal Govenors Bracket B
                              1st 2nd 3rd 4th
Carlsbad Lightning White G01    0   3  28  69
Murrieta Surf Blue G01          8  75  14   2  #1
Santa Monica United G01        91   9   0   0  #2
Simi Valley Eclipse White G01   0  13  58  29  #3


SoCal Govenors Bracket C
                           1st 2nd 3rd 4th
Fram Soccer Club White G01  44  38  14   4 #1
Santa Anita SC Bronze G01   47  36  13   4 #2
Baldwin Park United G01      4  11  33  52 #3
Santa Barbara SC Red G01     5  15  40  40


SoCal Govenors Bracket D
                            1st 2nd 3rd 4th
Desert Elite FC Natives G01   1  24  31  44
Legends FC White G01         95   4   0   0 #1
Vista Storm G01               2  32  35  31 #2
Wolfpack G01                  2  39  33  25 #3


SoCal Govenors Bracket E
                                              1st 2nd 3rd 4th
Downtown SC G01                                 1   3  12  84
San Luis Obispo Storm G01                      23  31  38   8 #1
Elite Soccer League - Saddleback Strikers G01  44  32  21   3 #3
Sherman Oaks Extreme G01                       33  34  28   5 #2


SoCal Govenors Bracket F
                             1st 2nd 3rd 4th
Arsenal FC SD Avengers         0   1  15  84 #3
Central Coast Condors G01      2  16  68  14
CVSC Rebels G01               18  65  16   1 #2
So Cal Athletic Stanwood G01  80  18   1   0 #1


SoCal Govenors Bracket G [I'm not sure I used the right Wolves team]
                                 1st 2nd 3rd 4th
FC Deportivo Hacienda Wolves G01  43  28  19  10
LA Galaxy South Bay HB Black G01   7  15  27  51 #2
Legacy FC G01                     26  29  27  18 #3
San Diego SC Black G01            25  28  27  20 #1


SoCal Govenors Bracket H
                            1st 2nd 3rd 4th
AYSO Matrix Bonita Blue G01   2  59  28  11 #2
Westfield SC Blue G01         0  27  43  30 #3
CVSC Rampage G01             98   2   0   0 #1
Empire SC G01                 0  12  30  58

Bracket I missing because no data on AYSO team

SoCal Govenors Bracket J
                               1st 2nd 3rd 4th
California FC Pumas G01          2   6  18  75 #3
San Diego Fusion G01            22  30  36  12
Santa Clarita Valley Green G01  37  32  25   7 #1
Simi Valley Eclipse Blue G01    40  32  22   6 #2


SoCal Govenors Bracket K
                                     1st 2nd 3rd 4th
RSF Attack Green G01                   0   3  41  56
Scripps Ranch SC Red G01               0   4  52  43 #3
Coachella Valley Soccer Strikers G01  57  40   3   0 #1
Slammers FC White G02                 42  53   5   1 #2


SoCal Govenors Bracket L
                             1st 2nd 3rd 4th
Real So Cal Black G01          8  51  31  10 #1
City SC Westchester G01        5  31  47  17
Elite Soccer League Surf G01  87  11   2   0 #2
FC Sol Black G01               0   7  20  73 #3 

Bracket N missing since no data on the AYSO team

SoCal Govenors Bracket M
                                 1st 2nd 3rd 4th
Foothill Storm G01                24  25  26  25 #2
LA Galaxy South Bay HB Green G01  28  26  24  22
San Marcos YS Revolution G01      20  22  25  33 #3
Xplosion SC Whittier G01          27  27  25  21 #1


SoCal Govenors Bracket O
                              1st 2nd 3rd 4th
Celtic Hibs G01                 0   5  19  76
Crescenta Valley SC White G01  93   6   1   0 #1
LA Premier FC White G01         4  51  35   9 #3
Poway Vaqueros White G01        2  37  45  15 #2


SoCal Govenors Bracket P
                              1st 2nd 3rd 4th
Cypress FC Premier White G01   52  37  11   0 #1
La Jolla Impact Blue G01       42  43  14   1 #2
Ventura County Fusion YSA G01   0   0   8  92
Exiles SC Red G01               6  19  67   7 #3


SoCal Govenors Bracket Q
                            1st 2nd 3rd 4th
Liverpool Premier Trask G01   4  15  56  25 #2
Pateadores HB Blue G01       35  50  14   2 #1
Madrid North G01             61  32   6   1 #3
Valley United Sedaghat G01    0   3  24  72


SoCal Govenors Bracket R
                                          1st 2nd 3rd 4th
AYSO Matrix San Elijo G01                   0  61  27  12 #2
BYSC Corona United G01                    100   0   0   0 #1
Irvine Slammers White - FC Blades Red G01   0  16  32  52
Wolfpack White G01                          0  23  41  36 #3


SoCal Govenors Bracket S
                                      1st 2nd 3rd 4th
El Segundo Gunners G01                 20  55  18   6 #2
Encinitas Express Soccer Blue G01       1  11  38  50 #1
Greater Long Beach SC Tidal Waves G01  76  20   3   1 #3
LA Galaxy South Bay White G01           2  13  41  44

Code that made the plot
glm.out = glm(P ~ made, family=binomial(logit), data=a)
plot(a$made,a$P, type="p",ylab="Probability of Making it Through",xlab="Model Probability of Making it Through")
lines(a$made[order(a$made)], glm.out$fitted[order(a$made)], col="red",lwd=4)