Monday, March 17, 2008

Obama versus Clinton: Determining Popularity

After winning 11 contests in a row, the Obama team worked very hard at convincing us that their candidate was winning the popular vote. It would seem that the argument has weakened a bit after Clinton’s wins in the Texas, Ohio and Rhode Island primaries. But the argument about popularity was actually flawed from the outset. Voting and extracting meaningful information from the process is, or should be, a science, not the political mosh pit into which the primaries have evolved over the last decades. And science is predicated on the soundness of the process used to determine the ever elusive so called truth. In fact, science itself is simply a methodology, a set of intuitive rules about good procedure.

For several years, I’ve been working on improving voting systems at a fundamental level. My passion for how to interpret and deploy voting began as a software engineer. I’ve devoted the last 10 years to developing more intelligent systems. And one of the ways to make a software system smarter is to harness the power of collective decision making, which is also more commonly known as voting. When developing software that uses voting, it’s important to implement procedures that allow you to interpret the results in a meaningful way. No knowledge can be derived from votes without a clear concept of who participated and what their motives were. Everything from Google’s search engine to NASDAQ in today’s society uses votes to make important decisions. Google counts the number of links a web site gets from other sites. NASDAQ measures the votes of traders looking for a profit. Both are confronted with how to interpret their results, which becomes impossible without good universal rules. If Google’s search engine was unaware of and did not counter-calibrate mob web sites that are only there to increase the rank of some potentially unimportant web page, our daily searches would be inundated with junk. Within weeks Google would lose us as customers. NASDAQ and the S.E.C., on their hand, have to deal with problems of insider trading to preserve our ability to make intelligent market-based decisions. What about the two major U.S. parties that have been using collective decision making since their very inception?

Sadly enough, the national Democratic and Republican parties use procedures for selecting their presidential candidate that resist any meaningful interpretation. We are left only with the guesswork of talking heads who analyze exit polls predicated on people’s participation in these very flawed procedures. The situation is especially dire within the Democratic party. After Super Tuesday , I became interested in determining who was truly more popular: Clinton or Obama. What popularity weight would each pledged delegate take to the national convention? Would a pledged New York delegate represent more approval than a pledged Alaskan delegate? It seemed simple enough. All I had to do was divide the size of a state’s constituency by the number of pledged delegates from that state. This is where I confronted my major hurdle. What counted as the whole constituency? Pledged delegates are assigned to states in part based on total vote tallies for the Democratic candidate in the previous three presidential elections. But looking all the way back to 1996 did not seem a fair way of measuring the current size of a constituency, especially given the fluidity of independent voters.

So what would be a proper definition of these constituencies? All currently registered Democrats within a state? All eligible voters? Perhaps, simply, all actual voters. Or, maybe, the entire population of the state. The answer depended on whether I should consider the process an internal party affair or everyone’s concern. When I looked at the actual process to find my answer, I was blown away by how completely inconsistent the process is across our nation. Not only do some states use caucuses rather than primaries, but some are open (all registered voters can participate) and some are closed (only registered Democrats can take part). And some, like Texas, to my bafflement, use both primaries and caucuses! In the case of Washington State, only the caucus really matters, and yet a symbolic primary is held anyway. Just to confuse us even more. In some states voters do not even register party affiliation.

Clearly, not even the Democratic Party itself has made up its mind on whether or not the process for selecting their candidate is an internal matter. This decision has been left largely to each state. It’s bad enough that we mix caucuses, which are well-suited for party activists, and primaries, which are well-suited for the general public. When you add the fact that a state can choose to have an open primary or caucus, you are left with nothing but a big mess. And this schism percolates all the way to the national convention. Superdelegates are thought of as the Establishment vote, whereas pledged delegates are thought of as an expression of the popular will (which, given the aforementioned circumstances, is quite a stretch). Now a vicious argument is being waged inside the Democratic Party about whether the superdelegates should, as originally specified, be free agents. If they are, in fact, morally bound by what is incorrectly deemed to be the “popular will”, one has to ask why they were introduced at all. If they were just created to overwhelmingly confirm the slightest majority of pledged delegates, then the superdelegates are nothing but a deceitful psychological sleight of hand. I think the simple truth is that superdelegates exist because the Democratic Party could not reconcile its impulse to make the presidential primaries a public service with the Party’s need to retain control of its own identity. Adding superdelegates to the mix may have been a fruitless attempt to close the Pandora’s Box opened by the McGovern-Fraser Commission’s 1969 effort to open the nomination process to a wider group of interests.

Changes since the McGovern-Fraser Commission have pushed the process of selecting candidates for the presidency out of the hands of the parties and into the public arena. But not entirely. The result is an inconsistent and flawed system which cannot decide whether it should abide by federalist or centralist principles, and whether it’s a private or public enterprise. Anyone who claims they can extract meaning from this system has not considered its fractured rules. Everyone, whether Republican, Democrat or independent should take heed. Given that we live in a de facto two-party system, these are the rules that determine our limited choices for president. The point of having a democratic, small “r” republic is so We the People can make a meaningful and intelligent decision on who should be our head of state and commander-in-chief. This does not necessarily mean that the straight-forward national vote tally, the simple majority so to say, should decide. We are, after all, a federation in which votes of constituencies from small states are boosted and the powers of large states are properly tempered. But being a republican democracy means that our system for choosing the President of our union has to be consistent, universal and easy to interpret. Unfortunately, how we currently choose the next President is more governed by the arbitrariness of political information cascades than it is by the power of making meaningful decisions through voting.