Discussing Specific Changes to the Hugo Nomination Election: A Post Not By Bruce Schneier

from Making Light on 2015-05-12 06:36 (#8W62)

After a couple thousand posts, here's the current proposal, summarized in this comment by Keith "Kilo" Watt.

- o0o -

[Remaining Issues "To-Do" List:]

Name of the system
How to handle ties
How to handle withdrawn nominations
Best way to present at the business meeting
The final formal proposal language itself

[Plain-Language Explanation of SDV-LPE]

Least Popular Elimination (formally called "single divisible vote with least popular eliminated" or SDV-LPE for short) is very simple and straightforward.

- You have one nomination "vote", which we'll call one "point" to avoid confusion.

- You can distribute that nomination "vote" among as many works as you feel are Hugo-worthy, and it will get divided among them equally. So, if you nominate two works, each gets half a point, if you nominate three works, each gets one third of a point, etc.

- All the points for each work from all the ballots submitted are added together, and the two works that got the least number of points are compared with each other. One of these works is the least popular and will be eliminated.

- For those works that are eligible to be eliminated, we compare the total number of nominations they each received (that is, the total number of times that work appeared on anyone's nomination ballot). The work that received the fewest number of nominations is the least popular and now completely vanishes from the nomination process as though it never existed.

- We start over for the next round, and repeat the process, however, if one of your nominations was eliminated, you now have fewer works on your nomination ballot -- so each one gets more points since you aren't dividing your vote among as many works.

Ties (in Points):

- This isn't really anything different. Basically, if there is a tie for least or second-least number of points, all of those works are eligible to be eliminated.

Ties (in Number of Nominations):

- If two or more works are tied for appearing on the fewest number of ballots, the tied works with the lowest point total will be eliminated. (In other words, if we're tied for number of nominations, we'll go back to comparing points for this case.)

- If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, then all members of the tie will be eliminated, unless that leaves the final ballot with fewer than five works; in that case, keep all the members of the tie. (In other words, if we're tied in both comparisons, in general we'll just eliminate all the tied works.)

Why does this help? In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn't penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate -anything- you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don't, that's okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

How does this eliminate slates? It doesn't, not completely -- nor should it, as slate proponents have as much right to representation as any other voter. However, by nominating a full slate of works, they have weakened their votes by spreading their points out among the five works on the slate. Since the rest of fandom is slowly increasing the points given to their choices, it's not possible for slates to control the -entire- ballot. It is very likely, however, that they will get some of the final ballots slots -- and again, this is entirely fair and appropriate, since a large number of people are supporting those works. In the end, it is the final ballot that will determine the winner -- and the voting system for the final winner is completely unchanged by this system. With this system, a significant fraction of the final ballot will have been determined by independent members of fandom.

[FAQ's]

1. How does this system eliminate slate or bloc voting?

It doesn't, exactly, nor should a work be automatically eliminated just because it appears on a slate. On the other hand, any slate which nominates a full set of five works will find that each of its nominations only count 1/5 as much. With a large enough support behind the slate (five times as much), the slate may still sweep a category; however, if that many voters support the slate, they arguably deserve to win, and no fair and unbiased system of nomination will prevent that. The answer is, simply, to increase the general pool of voters. Regardless, with SDV-LPE, slates will never receive a disproportionate share of the final ballot, as occurred in the 2015 Hugos.

2. What if there are multiple slates (slate wars, "parties", etc.)?

As with a single slate, the more works that anyone nominates, the less their votes count for each work. The end result is that even multiple slates are unable to sweep the nominations.

3. What happens if a genuinely popular work is nominated by a group of unrelated people?

If it is genuinely popular, the system will still select that work for the final ballot.

4. What happens if a genuinely popular work also appears on a slate?

Even if it is on a slate, if the work garners support from individuals - particularly if they list it as their only nomination, or with just a few nominations - then the system will select that work for the final ballot.

5. Isn't it true that any voting system can be gamed (or strategized, etc.)?

Yes, there is a theorem which proves that all voting systems must have inherent flaws. The objective is to choose a system whose flaws are not in an area of concern to the electorate.

6. What are SDV-LPE's flaws?

In very rare cases, it is possible for eliminating both members of a tie to change the final ballot slightly from what it would be if the tie were broken so that only one member was eliminated. This situation was extremely rare using realistic data, so the statistical probability of how ties are handled affecting the results is extremely small.

7. What are SDV-LPE's benefits?

Simply put, it reduces the power of bloc voting without eliminating the chance that works appearing on slates will make it to the final ballot. Conversely, it makes it very difficult for slates to prevent non-slate works from appearing on the ballot.

8. Couldn't slates just recommend a single work for a candidate, and it will automatically appear on the final ballot?

Yes, that is certainly a viable possibility - it's also completely fair. It does not force any other works off of the ballot, and the final Hugo winner is determined by the same voting process we have always had. Just appearing on the ballot isn't a guarantee of winning a ballot. However, if a large section of fandom strongly believes that a work deserves a Hugo nomination, then it should, in fact, be represented on the ballot.

9. What happens with a large field with no stand-out favorites when a slate votes?

Even in this case, we were unable to find a simulation in which no non-slate works appeared on the final ballot. Slate works did receive a larger proportion of nomination slots than they did otherwise, however, again, this could be considered a fair and valid result. If there was no general favorite, then voters really had no collective preference.

10. Isn't this system too complicated for the average voter to understand?

No, it's actually quite simple and straightforward, both in terms of voter instructions and in how the system operates. Essentially, the total number of points for each work are totaled (and this will usually be the sum of fractional points). These points are used to determine our two candidates for elimination, since the voters felt the least strongly about them. We then look at the number times the two works appeared on any ballot. The work that appeared the least number of times must be the less popular of the two, so is eliminated. This process continues until the five finalists remain. Note that this is not that different from the STV process used in determining the final Hugo winner once the finalists have been selected.

11. I think we should just increase the number of nomination slots on the final ballot to (for example 6), and decrease the number of slots a voter can vote for to a smaller number (for example, 4). Wouldn't that be simpler and easier?

Unfortunately, this simply means that the largest slate will receive four of the nominations and the next largest will receive the remaining two. It doesn't solve the problem of forcing works off the ballot that had a chance to win the final election. Keep in mind that SDV-LPE will work with this (or most any other) change as well, so one does not preclude the other.

12. I think we should set up a committee to handle these situations as they occur. The committee would be empowered to add nomination slots or throw out slate-influenced ballots as required.

This could work. The problem is that now you have a small group of people who serve as literal gatekeepers to the Hugo Awards. In spite of the word on the Internet, this has never been the case in the past. Establishing it now means that those groups who believed it existed in the past will now be correct. Ultimately, human judgement is fallible. The fairness of a committee's decisions will forever be subject to opinion. The end result is that the prestige of the Hugo Awards will forever be tarnished.

13. I think we should use [insert other mathematical voting system].

We considered essentially every applicable type of voting system currently in the literature, guided by two experts in the field. It should be kept in mind, however, that the goals and requirements for choosing a set of representatives in a political situation are different from those for choosing a set of Hugo finalists. Some of these systems do, in fact, have positive properties that speak for them. None of them were as simple or as intuitive as SDV-LPE, yet SDV-LPE meets all of the stated goals for a Hugo nomination system.

14. Won't SDV-LPE be complicated to code and implement?

Actually, no. One of our non-experts coded a full simulator for the system in a matter of days. A full web-based app would not be much more difficult to handle.

15. Wasn't this system just designed by Social Justice Warriors to block the Good Stuff?

It is true that much of the discussion for this system occurred on Teresa and Patrick Nielsen Hayden's "Making Light" discussion board, and it is also true that groups such as the Sad Puppies and the Rabid Puppies consider TNH and PNH to be The Enemy, and therefore completely biased and not to be trusted. Other than serving as moderators, TNH and PNH had no real input in the discussions of the system, however. Those of us who worked on the system were very clear that our goal was not to keep the Sad/Rabid Puppies off of the Hugo ballot and that any system which specifically targets any type of work is inherently wrong and unfair. One of the members of the group is a retired US Naval officer, a combat veteran, a certified Navy marksman, a Christian, and considers Robert Heinlein to be the greatest science fiction author who has ever lived. In short, he is exactly the Puppies' demographic. But any slate, of any sort, be it a Sad Puppy or a Happy Kitten of Social Justice, breaks the Hugo Award because a small percentage of voters can effectively prevent any other work from appearing on the final ballot. This is a major flaw in the Hugo nomination system, and it is a flaw that must be fixed if the integrity of the award is to be maintained. Politics should play no role whatsoever in whether a work is Hugo-worthy or not.

[Current Draft of Proposal Language]

Short Title: Change Nominations to Single Divisible Vote, Least Popular Elimination

Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.

3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.

3.8.3: Any nominations for "No Award" shall be disregarded.

3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.

3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.

3.8.6: The Committee shall move a nomination from another category to the work's default category only if the member has made fewer than five (5) nominations in the default category.

3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.9 or 3.2.10, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category.

3.8.8 The final Award ballots shall list in each category the five eligible nominees as determined by successive rounds of elimination, where each nominator gets a single vote, divided equally among their nominations. For example, if only one work is nominated in the category, that nomination gets a full vote. If two works are nominated in a category, each nomination gets 1/2 of a vote. If three works are nominated in a category, each nomination gets 1/3 of a vote, etc.

3.8.8.1 In each round, the two works (or more, in the case of a tie) with the least total number of (fractional) votes will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots.

3.8.8.2 In the event that two or more works are tied for appearing on the fewest number of ballots, the tied works with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, then all members of that tie will be eliminated.

3.8.8.3 If (due to a tie) elimination would reduce the number of ballots to fewer than 5, then instead none of the works should be eliminated and all remaining works appear on the final ballot.

3.8.8.4 For the next round, votes are reassigned to each work as in 3.8.8. (Example: If a ballot contained five nominations originally and one of those nominations is eliminated, each of the four remaining nomination now gets a 1/4 of a vote)

Section 3.11 Tallying of Votes

3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, ... places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes. During the same period a record of at least the last fifteen rounds of the selection process for each category shall also be published.

Submitted by: (insert members here, ideally including at least one attending member who will be at all the relevant WSFS meetings. A proposing member gets the privilege of speaking first to the pro side of the proposal)

Commentary: In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

- o0o -

Jameson Quinn offers:

I've created a public Google doc to hammer out a proposal text and FAQ for SDV-LPE.

There have been numerous proposals in this thread which do not fall under the SDV-LPE umbrella. In my opinion, they have turned out to be either off-topic or dead ends. This is not a criticism; several of the proposals were my own, and I thought and still think that they would have been good ideas. But it seems the community is very reluctant to experiment with anything that goes much beyond fixing the immediate problem, and I can respect that.

So I think the remaining order of business is to wrap up the SDV-LPE proposal, either using the doc above, or through other means.

- o0o -

Joshua Kronengold adds the following:

Re name, if we want an acronym that means something, how about:

One Notional Equally-divided Vote, with Oppositional Test Eliminations?

(ie, ONEVOTE)

I've always favored "rerun the vote with declined candidates eliminated, but only fill as needed" when people declined as a rule, since it ends up with the fairest overall result (and withdrawing Hugo nominations is a can of worms we never need open). I'd be fine (5 vs 10) with just taking the next item on the list.

Re tiebreakers: I'm reasonably fine with no tiebreaker (as the system was originally conceived), but also entirely fine with the current "just score" version, or using rank as a not-really-random (but close enough while being verifiable) tiebreaker. It's probably best to just have "just score" as tiebreak despite potential issues, because it makes things much easier to independently verify and keeps things simple.

Source	RSS or Atom Feed
Feed Location	http://nielsenhayden.com/makinglight/atom.xml
Feed Title	Making Light
Feed Link	https://nielsenhayden.com/makinglight/