Wednesday, November 10, 2010

Support for Wiggins in the late ballots keeps growing

In light of the 4,477 margin of victory that Wiggins now maintains, it might appear -- at first blush -- that the late increase in support for Wiggins is softening.  One might even conclude that Sanders is mounting a late comeback.  But a quick look at today's numbers tells a different story:

First off, don't be too fooled by the the fact that Wiggins is leading by about 2,000 more ballots than predicted (and yes, I did update the prediction from earlier after King County tallied so few ballots today).  The big reason why is that Spokane also tallied very few new ballots today (3,530 instead of the projected 9,017).  That prevented Wiggins from losing 1,322 votes from his margin.

Another reason - and definitely one of the stories of today - is the evaporation of the slim margin of support that Sanders was still receiving in Snohomish county.  Sanders only picked up a net +2 votes from Snohomish (or about 50.02% of the support).  To be sure, if Sanders was going to mount a come-from-behind win, he needed Snohomish to start providing him with a bigger margin of victory.

I added several new calculations to the results tonight, and I think it's actually very informative.  In the second-to-last column, you can see what the "Progressive SMOV%" is for each county.  If you're not familiar with my previous posts on the subject, this is essentialy my own rolling prediction for Sanders' margin of victory in each particular county.  For many of the smaller counties, this number is a holdover from the first few days of results:  in the last few days, those counties simply haven't tallied enough votes for me to feel comfortable relying on newer margins of victory.  For some of the larger counties (like King), the number does change daily because enough votes are being counted -- although in light of King's reduced tallies today, that will probably stop.

I've applied a "Progressive SMOV%" prediction to each county, and that forms the basis for the "final results" prediction that you see at the bottom of the chart.  Assuming that every county has exactly the number of EBOH that it's claiming, and assuming that the final tally of the ballots are distributed exactly according to the progressive SMOV% for each county, you would see Wiggins win by some-12,700 votes.

The last column is where things get interesting.  It shows how today's net new ballots in each county comported with the expected SMOV.  (Negative) numbers in blue reflect that the ballots came in with a reduced SMOV - that the ballots were trending more towards Wiggins than we should reasonably expect.  Conversely, positive numbers in red reflect an increase in support for Sanders not predicted by the model.

Here's a real-world example.  If County X tallied 100 ballots today (and assuming a VCR of 100%), and I projected a progressive SMOV% of 52%, we would expect to see 52 votes cast for Sanders and 48 counted for Wiggins.  But if in fact Sanders only received 51 votes, and Wiggins received 49, that would be a -(1.00%) change in SMOV% in favor of Wiggins.

Turning to the actual results, several things catch my eye:

1.  Support is still continuing to trend for Wiggins by larger than expected margins.  Obviously, this isn't true in every county; however, Pierce and Walla Walla together tallied less than 10% of the total vote today.  Together, they also only project only 750 EBOH total.  So any real change in the trend is coming too late to make a difference.

2.  The Kitsap trend is a little more of a significant departure towards Sanders.  Overall, it raised the SMOV% margin from a projected 44.62% to an actual 46.99%, which still reflects a pretty strong favoring of Wiggins.  Obviously, with 6500 EBOH, Kitsap still has some potential to tilt the results slightly -- at least, as much as any county outside King can still tilt the results.

I dug into the Kitsap SMOV% a little more though to see if I could nail it down.  Here's the data and SMOV% trends from Kitsap:

Thu 11/04:     SMOV 47.14%   75,961 ballots  (total to date)
Fri 11/05:       SMOV 44.62%   12,159 ballots
Sat 11/06:      SMOV 47.22%   3,036 ballots
Mon 11/08:   SMOV 41.42%   2,720 ballots
Tue  11/09:    SMOV 47.82%   2,493 ballots
Wed 11/10:   SMOV 46.99%   2,842 ballots

Other than the dip on Monday -- which seems to be one of those statistical anomalies that Just Happens from time to time, the county has been pretty consistently in the 46-47% neighborhood - at least for 3 out of the last 4 days and in the overall total from Thursday.  The overall SMOV% for Kitsap so far, on 98,941 ballots, is 46.70%.   So what I think this means is that Kitsap really should stay at a projected 46-47% SMOV.  The model is keying in on one of the two unusual dips that does not seem to be warranted.  I'll give some thought tonight to how to update the model to automatically reflect more stable trends rather than large, more recent anomalies.

3.  Surprising extra padding from Whatcom County tonight as well.  Through last Thursday, the SMOV slightly favored Sanders at 51.54%.  However, on subsequent days, the county posted results of (47.00%, 51.50%, 47.81%, and 42.22%).  At 43.25% today, Whatcom is shifting support to Wiggins at a fairly significant margin.

Speaking of the late-shifting support for Wiggins, one final note before I sign off for the night.  The Seattle Times endorsement switch and its coverage of Sanders' race-related comments may have gained the most attention as the "October surprise".   But I would be -- and I have been -- remiss in not acknowledging other possible sources.  Rather than tell you about it, I'll let Eli do it.  It's absolutely worth the read.

King County tabulating slower than expected

King results just posted, and apparently, someone in the Elections Director's office is determined to keep me in business for the foreseeable future (thanks, but I do have an appellate brief due on Monday).

King reported counting 18,236 ballots today.  To summarize, we've gone from 71,915 counted Monday, to 49,017 counted yesterday, to 18,236 counted today.  Even being generous and assuming a 50% rate of decay each day, and assuming no new ballots are added to the EBOH total, we'll be down to counting 1 ballot per day by November 30th, one full week after the deadline to certify this race.  And there will still be over 46,000 ballots left to count.  Depressing.

But getting back on track, it looks like the SMOV% in King dropped back down to 38.31% (it was 38.32% on Monday but jumped to 38.51% yesterday).  So it's unlikely we'll see much of a change in the remainder of the King ballots.

Updating the predictive model that I posted just a minute ago, the reduced ballot count from King only provides Wiggins with a 2,902 vote pickup out of that county.  That leaves Wiggins only picking up +305 net new votes today.  Add that to the totals from earlier, and the model projects that Wiggins will end the day with a +2,492 vote lead.  Really brings King's effect on this race into perspective.

[UPDATE (4:57pm):  I assumed, without checking, that King County's VCR% rate would stay the same, as it has, around 68-69%.  Today, it jumped to 74.19%, which is about on order with King's original averages last Thursday.  This actually produced a net vote gain of 3,163 for Wiggins out of King today.  Figure him around +2,753 overall at the end of the day.]

[UPDATE #2 (5:45pm):  It turns out there *is* a reason why King County's counting has slowed down - voter stupidity mistakes.  Check out the great explanation from Goldy at HA.]

Wednesday: Results and Predictions

Given the late hour, I'm rushing the projection for today's votes (and hopefully this post lands before those results start rolling in).

I expect Sanders to continue to perform well in Spokane county - although I honestly think that my SMOV% may favor Sanders a little too heavily.  The Monday SMOV% was 53.80, so a 2% drop, assuming the same number of ballots counted, would mean a pickup of only 1,226 votes (instead of 1,897).   Assuming King again only counts about 40,000 votes, the net gain of 7,800 votes for Wiggins there should leave him up about 7,383 votes for the day.

Tuesday's Results

Sorry for the late reaction to last night's numbers.  School is a killer right now.

Here's a quick break down on the results from yesterday.  We had a total of 12 counties report results:  Yakima didn't report (although they posted new numbers this morning), and Cowlitz, Grays Harbor, and Klickitat updated their numbers unexpectedly:

Wiggins picked up 2,867 less votes than expected.  The quick answer is that Yakima didn't report, and King County reported far fewer ballots yesterday that we were expecting (49,017 instead of 71,915).  Of course, this is readily explainable if the ballots now being worked through are the more problematic ones (incorrectly filled out, write-in votes, voter affidavits, etc). 

But it's got me thinking that there's definitely some useful data in here about the speed at which different counties are working their way though the EBOH pile.  Some have been very quick, some have started fast and slowed down, and others appear to be pretty uniform.  Once the speed of the daily returns slows down, I'll probably take some time to look at that data and see if there are some discernable trends that can be modeled for the 2012 election to predict return data.

In the last column, I compared the SMOV% from yesterday to the "progressive" SMOV% model that I've been using so far to see how the results different from expected.  The 12% number in Klickitat might raise some eyebrows... but we are only talking about 74 votes, so I think we can write that off to statistical insignificance.

The Pierce number is a little misleading.  The progressive model that I've been using still has Pierce's SMOV% at 54.73%, which is based on their total from last Thursday.  There haven't been enough ballots counted on each of the successive days (10% of total) to shift that number more towards Wiggins.   But over the four following days of returns (Friday, Saturday, Monday, and Tuesday), the SMOV% has stayed pretty consistent, between 52.27% and 53.06%.  With 53.03% yesterday, Pierce continues to show the same ~2% shift toward Wiggins that nearly every other county experienced, but that trend has stabilized.  Makes me think that the progressive SMOV% model really needs to give more credit to several days of stability, as opposed to an early lump-sum number.

The shift in King County's SMOV% is interesting, but relatively insignificant.  King was at 38.32% on Monday.  Yesterday, at 38.51% (+0.19), King has reversed what was otherwise several straight days of declinining numbers.  In practical terms, this 0.19% increase moved about 64 votes from Wiggins over to Sanders.  It'll be interesting to see where the SMOV% moves in today's results -- which should be posted shortly.

Bad math in Grays Harbor

I'll post shortly on the results from yesterday, how they lined up with expectations, and what we can expect today in terms of the vote margin.  But as I was looking through the numbers, something caught my eye.  There's some unexplained discrepancies in yesterday's vote totals from Grays Harbor.

First off - a disclaimer.  I've double and triple-checked these numbers, but it's always possible that I've made a mistake.  I don't think I have, but feel free to let me know if you see it differently.

Looking back to Monday, Grays Harbor reported the following totals:
Total Ballots Counted:    25,642
Votes Cast in Race (VCR):   19,702 (76.83%)
Votes Cast for Sanders (SMOV%):  9,922 (50.44%)
Votes Cast for Wiggins: 9,780

Yesterday, Grays Harbor reported new totals:
Total Ballots Counted:    25,884
Votes Cast in Race (VCR):   20,054 (77.48%)
Votes Cast for Sanders (SMOV%):  10,105 (50.44%)
Votes Cast for Wiggins: 9,949

Some quick math reveals that Grays Harbor (which is not located next to Chicago, incidentally) counted 242 ballots yesterday, and reported 352 new votes in the Sanders-Wiggins race. 

So where did those 110 extra ballots come from?

The 352 new votes yesterday reflected an SMOV of 51.99%  This is slightly above the overall 50.36% average that Grays Harbor had been reporting until yesterday, and reflects an definite upward tick:  Friday's SMOV% was 49.02% and Monday's was 50.44%.

Evil, election-throwing scheme?  :-)  Minor tabulation error from Monday or last week that someone finally noticed and corrected?  What do you think?

Tuesday, November 9, 2010

The late trend towards Wiggins

We've still got a few hours before the results roll in this afternoon, so I wanted to take a minute to try and figure out just how much the late swing towards Wiggins impacted the end result of this election.  With apologies to Yakima and the 30 other counties, I'll mostly focus on the Big Eight given their large impact on statewide vote totals.

To start with, here's a quick look at the SMOV % trend between last Thursday and Monday:

All eight counties showed a drop in SMOV between the total results on Thursday and each of the successive days' results, through Monday.

I've already done some casual speculating on why the later ballots may have shifted towards Wiggins, and if you're reading this, I'm sure you might make the same speculations.  Incidentally, this post may be the closest we'll ever come, statistically, to measuring the late impact the Seattle Times had on this election.


So let's re-allocate all ballots counted on Friday, Saturday, and Monday, using Thursday's SMOV%.  To control for error, I'll use the actual VCR% from each day.  To complete the totals, I'll project out the latest EBOH numbers that we have, using Thursday's SMOV% and the VCR% from my standard "progressive" model.  We'll compare the "What If" totals to the actual totals, and see just how much this late swing helped Wiggins:

That's a net swing of 21,282 votes from Wiggins to Sanders.  No model (that I'm aware of) is currently predicting a win by Wiggins that will nearly equate to that.

As the results come in, the media and various political figures may laud or criticize King County as the source of Wiggins's victory.  But the true story here is the SMOV % shift.   As I noted in a previous post, Wiggins saw a 2-4% increase in his vote margins nearly across the board between Thursday and Friday, a trend that continued throughout the weekend and in to Monday.  That margin is what will eventually propel Wiggins to (projected) victory.

A quick note about recounts

Several folks have asked about the possibility of a recount in this race.  I'm sure 2004 is still in the back of everyone's minds, so let's do a quick review of Washington recount law.

An automatic machine recount will only be triggered if the difference between Sanders and Wiggins is less than one-half of one percent (0.5%) of the total votes cast in the race AND if the total margin of victory is less than 2,000 votes.  The former seems all but certain to happen; as for the latter, my projections currently put this race well outside the recount margin.   But for those keeping score, in a statewide election like this one, a difference of less than 1,000 votes and one-quarter of one percent (0.25%) automatically requires a manual recount.

Either candidate could ask for  a recount, but the request must be made within three business days after the county canvassing board or secretary of state has certified the election (which is expected to happen on Nov. 23rd).  The requesting candidate must pay for the privilege, and the cost isn't cheap.  To conduct a machine recount, the requesting candidate must deposit 15 cents per ballot to be recounted.  With about 2.6 million ballots expected to be cast in this election, that's about $390,000.   A manual recount is even more expensive:  25 cents per ballot, or about $650,000.  However, if the outcome is reversed, the cost of the recount is reimbursed back to the requesting candidate. 

For the legally inclined, full details can be found in Washington's recount statute, RCW 29A.64

The Secretary of State's website also has a great FAQ on this topic:

I'm calling the race for...

... Justice-elect Charlie Wiggins.

I've run three different statistical models to project the final outcome in the race, and the outcome is unmistakable.  In each of the three models, Wiggins wins by well outside the recount margin.

Here's the breakdown:

Model 1 uses the "average" VCR/SMOV% totals.  These averages are calculated independently for each county based on all data returned so far.  Because the vote totals from election day are treated equally with the ballots being counted today, this model excludes most of the effects of the late shift in vote totals towards Wiggins.  This is the most conservative (not politically -- statistically) of the three models.  Although there are certainly theoretical models that could show a narrower gap, I probably wouldn't find them credible.

Model 2 swings towards the other end of the spectrum of acceptable modeling techniques.  This model uses the "latest" VCR/SMOV% totals.  These numbers come from the most recent day in which the county reported new votes, and are based only on that day's votes.  Accordingly, it picks up all of the late trends towards Wiggins, but in many counties, those numbers are based on a very small, unreliable sample size.  But having said that, this race has continued to trend further towards Wiggins each day that results have been counted.  It would not be unrealistic to see that trend continue; in that case, even this model would undercalculate the potential Wiggins gain.

Model 3 uses a"progressive" methodology (again, that's statistically, not politically) for the VCR/SMOV% totals.  This is the same model I've been applying to the vote projections I posted on Monday and Tuesday.  In general, it uses the "average" number from each county, but it will use the specific averages from Monday, Saturday, or Friday (in order of preference) if the net new ballot data supplied that day is statistically significant.  For our purposes, statistically significant means that the county reported counting at least 500 new ballots, and further, it requires that the number of ballots counted on that day amount to at least 10% of the overall ballots counted from that county so far.

All things being equal, I would normally expect this model to be a fairly accurate predictor of the final results.  However, all things are not equal.  In light of the shifting Wiggins trends across the board, I would probably lean more towards somewhere in between Model 2 and 3.

So there it is.  Even the most conservative model has Wiggins winning by over 5,000 votes.  I'm always hesitant to call a race with so many ballots outstanding and such a small margin of victory -- particularly when I'm calling it for the candidate not currently leading -- but looking at this data, I'm convinced that Wiggins has the upper hand and will coast to victory.

Expect the vote count to switch around 4:30 today when King County posts updated results.  Wiggins should take the lead then, and I don't see a model where he relinquishes it.

Tuesday Projection

First off, an apology.  My excel sheet locked up and refused to save last night, and I was forced to recreate it from Friday's saved data.  Thankfully, I still had raw numbers for the weekend and for Monday, so no loss in data.  But putting everything back together has been quite time-consuming.
Moving on to today's projection.

Only 10 counties will be reporting data today.  Of the Big Eight [see Terminology in the side bar], only 5 will report numbers today: Thurston, Snohomish, and Spokane won't report again until tomorrow.    We'll be using the same basic assumptions as yesterday:

1.  Counties will count the same number of ballots today as they did yesterday UNLESS that count would put them near or over their total EBOH.  In that case, I am assuming they will count about 80% of EBOH.  Although there was some variation yesterday, this turned out to be a fairly solid guess across the board.

2.  The overall SMOV and VCR for each county will be used UNLESS the county reported data on Friday, Saturday, or Monday AND the county reported counting at least 500 new ballots AND at least 10% of their total ballots counted.  For transparency, I have included a column in the chart below "()" that indicates the source of the VCR and SMOV numbers.  (Ttl) indicates that the county total to date is being used; (F) indicates Friday, (S) indicates Saturday, and (M) indicates Monday.

Here's the projection:

I'm on the verge of calling this race for Wiggins, but with the spreadsheet problems, I haven't had time to fully project out all EBOH remaining in these 10 counties as well as the other 29.   But it's getting very difficult to see a path to victory for the Sanders team.  I'll post shortly with a projected final result.

Monday, November 8, 2010

Whatcom (& further updates) delayed until 7:30

Whatcom won't be posting results until 7:30pm, so I'll wait until later tonight to post updated projections and thoughts on the day.

As things currently stand, Sanders' margin has narrowed to 4,064.  I anticipate that Whatcom will probably add +373 net votes to Sanders Wiggins column (assuming 8,580 of the remaining 12,000 ballots are counted today).  [EDIT: Is it just me, or are red and blue starting to look like the same color?]
I've picked up on an interesting trend that I think may deserve a follow up post.  Obviously, EBOH numbers are estimates (hence the "E").  And I would expect the daily updates to include revisions of those numbers beyond just the ballots counted (ballots tossed for invalidity, ballots verified with affidavit and added back in, late military ballots, etc.)  At this point in the process, I would expect to see very few ballots showing up in the EBOH projections.

That's why I was surprised to see that SIX counties reported a double-digit percentage increase in EBOH, notwithstanding ballots counted today.  But let me back up a second and explain the math...

Let's use a fictional county ("X"), that yesterday reported 10,000 EBOH.  Now if they count 6,000 ballots today, all things being equal, we would expect to see the EBOH drop to 4,000.  If ballots are invalidated, that number might drop.  And if more ballots arrive (delayed but still postmarked correctly), that number would go up.   Let's assume this scenaro: County X reports 6,000 ballots counted today, but updates their EBOH to 4,500 (thus reflecting 500 "newly-discovered" ballots to be counted).   So if we take that 500 increase and divide it by the original 10,000 EBOH, we get an EBOH increase of 5% (based on the starting EBOH tally at the beginning of the day).  Let's call that "Increased EBOH %".  All well and good.

The reason why we should look at Increased EBOH % (as opposed to a raw number) is because some counties are vastly larger than others.  To give you an example, King County saw a net increase of 1,915 EBOH today.  But given that King had 195,000 EBOH to start the day, that's a statistically-insignificant increase of just 0.98%.

Of the 18 counties (really 17 - Cowlitz is done) that have reported so far, the Increased EBOH % ranges from -2.4% (Snohomish) to +4.7% (Pierce) for all but six of the counties:

Chelan:  +15.6%
Benton:  +20.5%
Grant: +26.3%
Franklin: +38.9%
Yakima: +52.4%
Whitman: +122.4% (???)

Now, granted, some of those counties started with relatively small EBOH.  Franklin started with 800 (counting 961 and reporting 150 more on hand).  Grant started with 1,500 (counting 494 and reporting 1,400 more on hand).  And Chelan started with 2,000 (counting 2,262 and reporting 50 more on hand).

Whitman is a little surprising though.  They started with an EBOH of 1,000, counted 1,724, and reported 500 more on hand.  I don't mean to criticize the estimation powers of the county elections office in Whitman, but a stack of 1000 ballots and a stack of 2,224 ballots look pretty different.

All six counties have been favoring Sanders in the returns (although the net new ballots today from Whitman leaned toward Sanders).  Factoring in these new ballots, and allocating based on this morning's estimated VCR and SMOV percentages gives the following result:

Benton: 1,438 net new votes, VCR 78.0%, SMOV 57.7% = +172 SAND
Chelan: 312 net new votes, VCR 73.2%, SMOV 53.5% = +16 SAND
Franklin: 311 net new votes, VCR 81.0%, SMOV 63.2% = +66 SAND
Grant: 394 net new votes, VCR 76.6%, SMOV 65.3% = +92 SAND
Whitman: 1,224 net new votes, VCR 73.7%, SMOV 51.6% = +29 SAND
Yakima: 5,236 net new votes, VCR 81.4%, SMOV 60.1% = +862 SAND

For those keeping score, that's an extra 1,238 votes for Sanders that no one saw coming.  Now granted, King's 1,915 new EBOH hands 284 votes back to Wiggins.  But this is certainly an object lesson to keep an eye on EBOH numbers, especially when they start wiggling (no pun intended).

And if anyone has an idea what happened in Whitman or Yakima county today with several thousand new ballots showing, please share.