The Instant-Run-off Voting Rabbithole

Summary

This post is the result of my diving into the differences between different IRV implementations and comparing those to the implementation we are currently using.

As I looked into it more, it became necessary to communicate the context around the various implementation options and how this relates to our goals in governing MakerDAO.

The TL;DR of it is that the current implementation we are using is neither a ‘correct’ implementation nor is it the implementation that I think makes the most sense for Maker Governance.

There will be no more IRV Ranked Polls until we determine how to move forward.

Goals

Our Goals

Ensure executives pass
I’ve covered the reasons for this elsewhere, but I’ll reiterate them here.

Due to the executive voting system, we are forced to bundle the results of multiple polls into one executive. Having executives that do not pass can be extremely costly in terms of time and effort for Governance and the Mandated Actors.

If we allow changes without majority support into the executive vote, it weakens its chance of passing. Having an executive that doesn’t pass is not good because each week of governance costs time, effort, and gas.

Here is a brief layout of the usual costs of a week’s worth of polling and executive:

  • I spend between 2-4 hours each week writing, reviewing, and submitting polls.
  • 1 hour is spent by various people reviewing polls or executive text.
  • Smart Contracts spend time preparing the executive contents (new collateral) and creating the executive spell itself. (Not sure how long this takes, but it’s not a trivial process.)
  • Gas costs are paid for submitting polls and executive spells.

If an executive fails to pass, depending on the contents, we have to:

  • Attempt to figure out why it didn’t pass.
  • Attempt to determine what should be included in the next executive
    • Do we repeat failed contents?
    • Do we exclude potentially controversial things?
  • Potentially repeat polls with fewer / more options.
  • Repeat Smart Contracts Domain work re-writing an executive with similar contents.
  • Governance needs to vote again on a similar set of things.
  • Gas costs are paid again.

Ensure executives contain majority-desired changes
We want to make sure that if there is a majority of MKR Holders that want a specific change then that change is included in the executive.

Differences in Goals

IRV as generally described is written for use in the real world. Real-world systems of electing governments have a different set of requirements.

First, elections are expensive, nations cannot afford to have elections fail to resolve. Despite the above costs, Maker’s polling process is much cheaper to operate.

Second, elections are the final part of the governance process for most nations. Maker has approval voting (executives) subsequent to polling votes. Executives require more MKR voting for them than is voting for a previous executive. In other words, it requires a majority of active MKR to pass an executive.

What is IRV and how does it work?

I’m not going to spend time writing a worse explanation than already exists on the internet for the purposes of this post. Here are some links that explain it in depth. If you’re not terribly interested in the details, you can skip to the Commonalities section below:

Links

Description of Process
Worked Example

Commonalities

  • All Instant Run-off Vote implementations include the concept of rounds. If after a round, the leading option does not meet some criteria, an option is eliminated and votes are redistributed to those voters’ next-ranked candidates.
  • All Instant Run-off Vote implementations allow voters to rank their preferences.

Key Variations

Rank all vs. Rank any

Rank-All
In this variation, voters are forced to rank all the options in their order of preference.

However, this does mean that voters are forced to rank options that they would not accept when compared to the status quo.

Rank-Any
In this variation, voters are able to rank as many options as they want (including only 1.)

However, this means that this implementation can resolve without a Total-Majority.

Shrunk Majority vs Total Majority

Shrunk-Majority
In this variation, a majority of the remaining-non-discarded votes is used to determine whether the rounds have concluded and the winner decided.

Using a shrunk-majority an outcome is guaranteed, however, it is not guaranteed that the outcome is preferred to the status quo by a majority of the total voters.

Total-Majority
In this variation, a majority of the total votes is used to determine whether the rounds have concluded and the winner decided.

Using a total-majority, an outcome is not guaranteed. However, if there is an outcome, then it represents something the majority of voters favor.

Stop Criteria

Stop when x remain
If there are x options remaining after a round, the calculation is complete.

Stop-On-Total-Majority
If an option has a majority of the total votes after a round, the calculation is complete.

Stop-On-Shrunk-Majority
If an option has a majority of the remaining votes after a round, the calculation is complete.

Implementation

Our Current Implementation

Rank-Any, Total-Majority, Stop-On-Total-Majority, Stop-when-2-Remain

Advantages

  • Rank-Any allows users to explicitly express their opposition to an option.

Disadvantages

  • A majority outcome is not guaranteed when combining Total-Majority and Rank-Any.
  • Cannot require a majority outcome given that it isn’t guaranteed to find one if it exists.
  • Stop-When-2-Remains means that rounds will end when a majority could have been found (transfer from the lower of the last remaining options.)
  • Isn’t actually a ‘correct’ implementation of IRV.

‘Correct’ Implementation #1

Rank-All, Total-Majority, Stop-On-Total-Majority

Advantages

  • Always resolves with a majority.

Disadvantages

  • Forces users to rank options they do not want to vote for in an executive vote.
  • The majority it resolves with is manufactured from the inability of users to express opposition. It is guaranteed to provide the preferred option, but not that that option is preferred by a majority to the status quo.

‘Correct’ Implementation #2

Rank-Any, Shrunk-Majority, Stop-On-Shrunk-Majority

Advantages

  • Always resolves with a majority or plurality.
  • Allows voters to express opposition to options by not ranking them.

Disadvantages

  • Plurality resolution can lead to failed executive bundles as detailed above.

My Preferred Implementation of ‘IRV’ for Maker

Rank-Any, Total-Majority, Stop-On-Total-Majority, Stop-When-1-Remains

Advantages

  • Always identifies and resolves with a majority if one exists (due to Stop-When-1-Remains.)
  • Plurality changes will not proceed to an executive vote.

Disadvantages

  • Can fail to resolve.
  • Isn’t actually a ‘correct’ implementation of IRV.

My Desired Voting System: Approval Voting

Approval voting is the type of voting we use on the forum with multiple-choice polls. Voters vote for everything they are willing to accept. The winner is the option with the highest support and a majority. If there is no majority, the change does not proceed to the executive.

Advantages

  • Approval voting is less complex than IRV
  • The process is easy to understand.
  • The outcome is easy to understand.
  • It leads to the least-objectionable outcome winning the poll, avoiding controversy.

Disadvantages

  • It does not allow voters to rank preferences.

Next Steps

There will be no more IRV Ranked Polls until we determine how to move forward.

The current implementation is both not technically correct, and not optimal for our governance process. The current implementation of IRV is not guaranteed to find a majority if one exists, and including plurality options in an executive risks the executive failing to pass.

I’d like to hear people’s thoughts on the above suggestions. If people want to pursue any of these options, please create a signal request.

If a signal request isn’t created in the next week, we’ll start using my preferred implementation of IRV.

11 Likes

Um what? Didn’t the community signal that they want ranked voting and you’re gonna unilaterally decide we can’t use it anymore? Maybe add a signal request to postpone using it or changing the methodology? I’m not a fan of this authoritative decision making.

2 Likes

What about just keeping the current implementation until we have something “better”?

The current implementation does work. If people are not willing to compromise (see last poll), the are simply shooting themselves in their foot. No need to harm the voters that are willing to compromise

1 Like

Btw… rank all is ok from my point of view too. The voting frontend could enforce that (and the people who work around it by calling the smart contract correctly without ranking all… Don’t care)

1 Like

I think we should have one poll

Shall we stop doing IRV for now or just keeping it until we have something better.

Additionally to that: a debate (with a later poll) about what the “better” system can be

In practice I already have a huge amount of influence over which polls use which voting method. Though if someone is dead set on using a certain type then I’m usually fine with it.

The issue with using the current version is that I can no longer say that ‘This poll must resolve with a majority or it will not proceed to an executive’ when I’m aware that the frontend will stop looking for a majority when there are only two options remaining.

Technically it would be possible to do the calculation manually, but given what happened last week I don’t think it’s worth the confusion and potential conflict.

If governance does not want to micromanage the implementation then we can switch to the suggested fix and start using it next week. If governance does want to give further input they are able to do so, but I don’t think it makes sense to continue using the current version given that we now know it isn’t reliable in all cases.

I think I covered this above, but to be explicit, the current version is not guaranteed to find a majority if one exists. Any time we use this we’ll need to manually check and public the results separately from the frontend. It just seems like it will lead to more confusion.

In my view it isn’t, given that it hides what might be strong feelings against certain options.

2 Likes

And that is fine. No majority no change. Simple as that. I don’t get all the fuzz about this problematic poll from last week

1 Like

Does that mean that we are going just have to wait one week and then we get what you have been applying in the past? I am all in for that :slight_smile:

No, the point is that there might actually be a majority and the front end will show a plurality and the poll would not proceed when it should have done so (luckily, this was not the case with the poll last week).

Yep, I know you and others probably want to drop the matter completely, frankly, so do I. Unfortunately, it’s my job to mitigate the issue and make sure it doesn’t happen in the future. Investigating our implementation of IRV is one part of that.

Effectively yes, only now it will actually work correctly and identify a majority if one exists.

There was also a lot of discussion last week over whether the implementation was correct so I wanted to give people a chance to understand the differences in implementation and what they mean in practice.

2 Likes

A) of there is a bug in the front-end, it needs to get fixed. V1 did not have the problem right?
B) if there is a problem with perceived/intended majority Vs what comes out of voting, we need to educate the voters

V1 had it’s own problems as Derek described in the other post. The problem is that while this could be termed a bug, it’s not really clear in which direction it should be fixed. Governance did not vote on a specific implementation of IRV, we could move to any of the implementations I listed above.

I agree completely, and I’m in the process of tackling that as well. In fact that’s what led me to identify this as an issue. At this point I can’t write accurate documentation of the IRV process because it doesn’t always work correctly in all cases.

I think I have an alternative idea which allows us to secure a majority in most situations, without sacrificing the versatility of IRV…

If you feel strongly that the status quo needs to be changed, create a signal request prior to the on-chain poll which says that the poll should be binary and take a community vote on the one other option

This entire situation could have been avoided if someone said a week earlier “it’s very important we reach a majority on this poll, please vote on what the singular option versus the status quo should be.”

Would love to hear other’s feedback on this.

3 Likes

Yep, this is definitely a good idea in some circumstances. If the community can reach strong consensus on an option such that we can offer a binary poll that’s almost always better than having a ranked poll on-chain.

The downside to this is that larger MKR Holders may disagree with the wider community, but there are always trade-offs to any process.

Not entirely sure what you meant by this, but I do want to emphasize that I don’t think requiring a majority outcome makes IRV less versatile or useful in certain situations. It allows MKR Holders to rank choices which is a level of signal that we don’t get with other types of vote.

2 Likes

I’m saying that for things like stability fees, I think the current system actually works pretty well. I don’t want to see us change everything for an outlier situation.

2 Likes

This is the system I thought we were working under until last week, as this is the one I am most familiar with from state and national elections. I know we want to reduce the chance that executives are blocked because we didn’t seek a majority, that is a reasonable concern, and is also almost the entire reason we poll on-chain prior to an executive; however, I think it’s more important that polls resolve than we attempt to seek a majority in this case. That is, I think we have high constraints on our schedules, and while the work put into signaling for something isn’t as much work as a state or national election, it is highly disruptive to that process to not resolve one way or the other.

I think if this as having faith in the mechanism. We may not reach a majority supporting the overall option, but the non-winners can have faith that the process worked as defined. In the same way randomly rolling for gear in an MMO can produce unfair outcomes, the fairness in the process of randomly rolling leads to faith in the system.

Once we have DssGov, and can unbundle executives, we would no longer have the majority support constraint on the entire executive. Under this new regime, we are certainly better to go with Rank-Any, Shrunk-Majority, Stop-On-Shrunk-Majority. For now, so long as our polls always have a “Do Nothing” option, I would favor finality in the poll over a majority.

If someone would be so kind as to add the option for Rank-Any, Shrunk-Majority, Stop-On-Shrunk-Majority to any signal polling, I would be very appreciative. I understand that we would need to change our implementation, as it is currently Rank-Any, Total-Majority, Stop-On-Total-Majority, Stop-when-2-Remain. In that case, I think we can do what @g_dip said in the interim. When signaling, we should specify if something must resolve and make it a binary poll.

I won’t go into other options I’m not in favor of because of time, but I will say I would strongly discourage a system that forces MKR holders to rank all options. There is just no way I will ever choose the “print unbacked DAI” option, even if I made it my last choice. Joking aside, this is mostly because of the UX burden of making everyone rank everything in the poll.

Thanks for putting this together @LongForWisdom.

“Truth suffers from too much analysis." –Ancient Fremen Saying

10 Likes

I think it is clear that once executives can be unbundled, and as long as “no change” is always an option, then Shrunk-Majority or any combination of parameters which ensures that every vote results in some option being selected is the right call.

Assuming we do plan to move to Shrunk-Majority (or anything that ensures a winner) after DssGov, I’d rather have no more IRV polls in the meantime. Another debate like we just had would be a huge distraction. And under the current system, I do not think the excellent clarifications we had (especially from @LongForWisdom) prevent another one from coming up.

1 Like

Trying to tell Y’all–Americans ain’t down with RCV
image

One person = One Vote. It was written.

Thank you LFW.

Personally I am 100% with this solution due to its simplicity.

More sophisticated voting systems are cool, but realistically we can’t expect voters to spend 2 hours on wikipedia to read all the (changing over time?) details of the voting system.

So, SIMPLE + CLEAR = GOOD.

I agree with @Aaron_Bartsch on this.
We all understand this is important and that you have given a lot of thought on this topic. But It would still be good/better to ‘approve’ it officially, using the ‘regular procedure’.

Thanks @LongForWisdom keep up the excellent job!

3 Likes
  • imvho it does not really matter which of the (in)correct variations of IRV we use, as long as we all know which one it is.
  • just binary voting is not enough for the polls. MKR holders sometimes need multiple options to vote on. I really liked the switch to IRV because it allows to compromise. We can’t always go for multi-select signaling and then just take the most voted option from the signal to the on-chain polling. So far, @LongForWisdom did a great job on deciding which way to go for all the previous onchain votes (and to be really explicit: also for the problematic one!).

If we need to signal on IRV implementations (and I am totally fine with not doing that an defaulting to @LongForWisdom preferred implementation), I would go for:

  • our current implementation - it did work, good enough (yes, i don’t think that a poll always need to resolve. if voters are unwilling to compromise, nothing happens)
  • correct implementation 1 (because I like that voters need to rank all options. kind of guarantees that they make an educated decision). I still hope that “print unbacked dai” is never popping up as an option @cmooney :wink:
  • the preferred implementation
2 Likes

Strong +1 for approval voting for on-chain polls. As we’ve seen, ranked choice voting has a lot of complexity and unintuitive behavior. And while choice ranking is expressive, it doesn’t allow you to signal how much you prefer one option to another, which limits its added usefulness.

Approval voting remains more expressive compared to plurality voting, but it’s a lot simpler to understand and is more mathematically robust to vote gaming than IRV.

3 Likes