Editor's note: Today's will be part 1 of a special two-day Building a Voter File extravaganza! Quite frankly, this topic is just too awesome to be tackled in one entry, so check back tomorrow for the thrilling conclusion.

Nate Silver had an interesting conjecture recently, in a post about how exactly Norm Coleman got his list of "improperly rejected ballots" in the never-ending Minnesota Senate race:

What I suspect Coleman did to come up with his list of 650 is something like this:

- Create a database of all ballots that were rejected for a non-matching signature ... maybe there were 1500 of these or something statewide.

- Run some algorithm to determine the likelihood of each of these 1500 ballots being a vote for Coleman as based on things like the precinct the ballot was cast in, any information Coleman has about the voter in his voter file, and perhaps even the voter's name (you can tell more than you'd think about someone based on their first and last name).

- All ballots determined by this algorithm to have a >50% likelihood of being Coleman votes were included on his list ... there turned out to be about 650 of these.

There's more beneath the fold:

Marc Ambinder has a must-read post about the Democrats' use of voter data this cycle.  The whole thing is worth reading, so I won't excerpt here, but you should do yourself a favor and go read it--the glossary at the beginning is worth the click alone.

Consumer data is one of the most frustrating areas of voter file design, especially if you're interested in transparency or an open-source methodology.  Large parts of the voter file process use publically available data, and can be duplicated by any group with relatively low levels of equipment and technical skill (for example, witness my tutorial effort, which I absolutely promise to get back to after the election).  There's no such ability for data appends.  After the jump I'll explain why.

Mark Blumenthal of Pollster.com sat in on a session that David Plouffe and other Obama staff had with some Atlantic/Natl. Journal reporters.  It's got some fascinating tidbits about how the Obama campaign uses polling, their definition of undecided voters, and Plouffe's opinion on the worst aspect of campaign coverage.  Check it out.

Patrick Ruffini annotates an article in Salon about Barack Obama's microtargeting efforts.

My colleague Dirty D makes a valuable point in the latest installment of his excellent series on building a poll.  I'd like to expand on that a little bit, if I might.  He writes:

[When designing a poll], you don't want to cut out any of the dependent variables you're testing, but at the same time, you don't want to lose that valuable contextualising information.  What are you to do?   This is where having data in which you have high confidence becomes invaluable.

In short, there are certain circumstances in which you can substitute information that's on the voter file already for information that you would ordinarily have to ask your respondents for.  This can be helpful in a number of ways.

First of all, information from the voter file is available for all voters--which might not be true for information directly asked of poll respondents.  For example, let's say that your poll reveals that African-Americans are unusually receptive to an anti-Bush message in a congressional race.  If race is on the voter file--and, crucially, if this information is high-quality, as Dirty D emphasizes--then you can use that to design a direct mail and field targeting program.  If, on the other hand, it's not available on the voter file, you'll be forced to resort to a much cruder approach--for instance, blast-mailing to everyone who lives in precincts that the census has identified as heavily black. 

Secondly, information on the voter file can be more static and binary than voters' attitudes, making it easier to interpret.  For a good example, see this discussion by Margie Omero of the difference between party registration and party ID.  When you're designing a field program, it's much more useful to target registered Democrats than self-identified Democrats--they're easier to find.

On the other hand, this can be a curse as well as a blessing.  For instance: if you were doing a poll that attempted to measure why certain groups have certain partisanship patterns, party ID might be more useful than party reg.  And more broadly, voter file data can be limited in either scope or accuracy.  A good voter file, along with quality polling, are complements, not competitors, and when used together can be much more powerful than if they're isolated from one another.

Once the data is (yes, is, prescriptivists--I went there) in a standardized format, we move from the realm of "interesting" into "faintly creepy".  The information from Secretaries of State or state parties is generally pretty innocuous--name, address, maybe phone number or age.  The appended consumer data, on the other hand, is more unsettling.  There's nothing on there that would do real damage if anyone knew it--no credit card numbers, nothing that people could use to steal your identity--but it can be kind of strange to think who realizes that you own two dogs and a cat.