Colby's Research: Range Polling. Provide Community Feedback Please!

Introduction to Colby’s DAO Scaling Research #1

Hello! My name is Colby Anderson, and I have received a scholarship grant from the SES core unit to research scaling solutions for MakerDAO. I am a computer science major from Brown University, with a background in distributed systems and decentralized finance. If you would like to learn more about my background, you can view my LinkedIn here.

The purpose of this post is to introduce some of my research to the community in order to receive feedback.

TLDR:

  • My research attempts to explore solutions to general classes of problems within DAO governance and operations.

  • These solutions can be seen as building blocks (primitives) that can inform the community in its consideration of better governance automation and improvements.

  • The first governance primitive I’m investigating is an automated voting system that creates range polls, i.e. a poll in which token holders vote within a continuous number range without pre-defining a discrete number of options. This primitive can be used to vote on various parameters such as debt ceilings, compensation, risk parameter values, surplus buffer, etc.

  • The research will continue by exploring potential smart contract implementations. With this post, my goal is to introduce my research to you and start collecting feedback on the mechanisms that I’ve considered.

Motivation

With the increase in both the quantity and complexity of MIPs, MKR holders are required to invest more time to make consistent and competent votes. This is an essential scaling problem of MakerDAO. One way of dealing with this is to find decisions within these MIPs that can be grouped together and voted on at once. These kinds of decisions, if recurring, can then be standardized and use smart contracts to automate the polling process.

The purpose of my research is to investigate which types of polls could be automated and standardized in order to decrease the amount of MIPs. An essential aspect of this involves creating automated voting systems. As a result, it is good practice to try and build these automated voting systems so that they can be applied to many different kinds of elections.

Recently, I have been looking into an automated voting system that would change MakerDAO’s budgeting framework. The idea is to create a recurring vote on a budget ceiling that would give funds to MakerDAO for operations. Then, MKR holders could dynamically allocate these funds to elected officials who would be in charge of managing the budget for the core units.

Range Polling Primitive

One key part of this automation process involves creating an automated and transparent range poll. A range poll is a poll where there are a range of possible outcomes (usually continuous), and voters can somehow vote within this range. This idea of a range vote extends further than a budget ceiling poll, and can be implemented when automating MKR compensation decisions, doing system parameter polling, and much more.

There are many different aspects of implementing a range poll. The two biggest aspects are how votes will be submitted and how a consensus among these votes can be reached. Besides this, there are three important measures that a polling method can have: efficiency, voter advantages, and decentralization. Efficiency is a big concern since all computation in the polling method has to be paid for in gas fees. Voter advantages refers to some or all voters having an advantage in the election. For example, some polling methods may give an advantage to a voter who votes last. Decentralization refers to the amount each voter’s vote is worth in determining the final outcome.

There are three mechanisms I’d like to highlight as part of this post:

  1. Standard Statistical Metric Polling
  2. Increment Polling
  3. Clustered Polling

Standard Statistical Metric Polling

One polling strategy would consist of applying a common statistical metric (such as mean, median, or truncated mean) to all voter’s submitted amounts. This method is susceptible to tactical voting in which a competitive advantage is given to voters who are further back in the order. This feature can be eliminated in a commit-reveal strategy, but then the polling process would be much more inefficient. Furthermore, choosing a metric to reach consensus has many impacts. Some metrics are seen as better for certain distributions of data. These metrics also offer varying levels of decentralization.

Note: in the above picture, the black circles are governance tokens (MKR tokens for example) that correspond to the amount of weight each vote has.

Increment Polling

Another polling strategy would consist of some centralized party proposing a starting point for the poll as well as increment and decrement amounts. Then, instead of voters submitting their desired number, they would be allowed to increment/decrement for as many governance tokens as they have. Although this strategy is computationally efficient, it does create more centralization and is still susceptible to tactical voting (specifically by the last few voters).

Clustered Polling

Finally, another strategy would be to find the largest cluster of voters (that’s a majority) and take the mean of this cluster. In other words, this is essentially a majority rules method, except the winning number is reached by taking the mean of the majority. In order to find a majority cluster, a lot of computation and storage needs to be done, making this polling method very inefficient. The actual algorithm is called agglomerative hierarchical clustering and can be read more about here. This method is susceptible to tactical voting in which the last voter has the biggest comparative advantage. However, this can be prevented by turning the polling phase into a commit-reveal strategy. The commit-reveal strategy would make the computation more inefficient and could cause more user complexity.

In the above diagram, the format of votes can be seen as (the vote: # of votes for that number). For example, 80:2 means that 2 people voted for the value 80. This diagram shows calculation of the winning value which takes place after all votes have come in.

Conclusion

The next steps in my research are to collect the community’s feedback on these automated voting systems, particularly their pros/cons as well as what kind of applications each polling method best fits. Then, the goal would be to implement the best polling methods in smart contracts.

Presentation Video

Here is a presentation video that goes into a little more depth. Here are the slides for that presentation.

There is also some extra written documentation on the topic here.

Thanks in advance for your input!

12 Likes

All the methods you mention are susceptible to tactical voting, a competitive advantage is given to voters who are further back in the order. However, this doesn’t seem like such a big problem. It seems likely that many votes will be dominated by Recognized Delegates who will not want to be seen as unduly trying to influence the direction of the outcome by voting tactically.

2 Likes

Thank you for your feedback! @psychonaut

  1. All methods are susceptible to tactical voting (EXCEPT when a commit-reveal strategy is put on top of them). Also, some methods differ in the extent to which tactical voting is possible.

  2. This is true. Many votes will be dominated by Recognized Delegates. However, tactically voting is not necessarily seen as malicious. In fact, it is seen as rational. So, some Delegates and community numbers may not see it as “unduly”. This point is up for debate though, just playing devil’s advocate.

  3. Besides commit-reveal, there are probabilistic ways of incentivizing voters to not vote last. However, these strategies tend to still be susceptible to a smaller extent of tactical voting. Let me know if you would like me to expand on this idea.

  4. In the post, I mention that employing a commit-reveal strategy to completely eliminate tactical voting would be very inefficient. However, if the majority of voters are recognized delegates, there would actually not be many total voters, making the computation cost relatively low.

2 Likes

Hey, this is interesting.

Not sure if you’re aware, but 1inch implemented ranged voting for their governance.

Have you considered using median rather than mean for this? I’d be interested to hear the pros and cons of each.

My intuition is that clustering is not feasible on L1 due to the gas costs.

Incremental polling is not something I’ve considered previously. I feel like the decision on the initial amount could have an outsized affect on the outcome of the poll.

One thought re last-vote advantage is to apply a multiplier to voting power based on when the voter votes in the poll, maybe some exponentially decreasing curve segment starting in the last day or something.

If you’re interested, I’d like to chat with you about this at some point. My gut feeling is that range polling is not great for compensation, but is good for parameter setting.

4 Likes

This might be off-topic, but I want to ensure that you are aware of, Prioritization in MakerDAO

1 Like

Thank you for your feedback! @LongForWisdom

  1. I have not looked into 1inch’s range vote. But, I will definitely do so now!

  2. I have looked into median! I have researched the following polling methods: mean, truncated mean, normalized mean, median, clustered, and increment. I have also looked into adding commit-reveal to mean, truncated mean, normalized mean, median, and clustered methods. In my community post, I link to a presentation I did explaining all of these in more detail. I recommend watching if you have the time!

  3. The clustering approach might be feasible. It all depends on the expected number of voters. Furthermore, in the actual implementation, all of the calculation would not be happening at once. If you would like to see the algorithm, you could read more about here: What is Hierarchical Clustering? | Displayr.com

  4. Yes, in incremental polling the decision would be greatly affected by the initial amount and the increment/decrement amount.

  5. Yes. I have considered variations of this as well. In fact, it is what I was referring to in my previous reply to @pyschonaut. It was my bullet point number 3. I have considered both lowering the probability of your vote counting over time, lowering the amount of vote power over time, and lowering the power of your vote based on how far away it is from the mean (see normalized mean polling in my documentation linked to in the post). However, in each of these three situations, tactical voting is still possible, just to a lesser extent.

  6. Ultimately, the particular method of range voting chosen will both depend on how large the range is, how many voters will be participating, and what the distribution of votes is expected to look like.

  7. Sure, I would be happy to talk about it sometime. You can reach me at [email protected]

4 Likes

@pyschonaut
I was not aware of this post! Thank you for informing me about it. This seems to be on a slightly different kind of poll than the one I am referring to (correct me if I’m wrong). It’s purpose seems to be a kind of a poll that ranks proposals, instead of choosing one value in a possible range of values.

The linked post is definitely super interesting, and I will try to get more in depth into it as I think it will affect my own research and serve as some inspiration.

2 Likes

Yeah, that’s correct. That’s why I said that it might be off-topic, but who knows which direction your research will take :slight_smile:

1 Like

Things to consider.

  1. What is the goal of polling?
    Is it to get a unaltered sample of the participants or to achieve a form of consensus?
  2. What is the reasoning to have a continuous variable choice vs. a set of fixed choices? Why would this be better or worse? With any input method there is this issue of what is a statistically or practically significant change. Example changing a DC that is 100M. Doing this in 1M or less incements is kind of pointless. 10M’s maybe. Point here is what is the different result achieved by having a continuous variable input.
  3. I saw nothing in the resultant calculations that include a MKR weighting factor. Point here is whether a 1 vote 1 wallet has any sense or basically would be obfuscated by sybils, vs. using a linear MKR weighting when it comes to signals.

I know @LongForWisdom and GovAlpha generally has considered this, but the idea that we can do MKR polling on another cheaper chain (like xDAI) might improve participation. Using something like snapshot.org also may be a useful way to get better polling info. The end result here is something that ends up having to pass either on-chain final poll which is going to leave a number of voters out (due to L1 on-chain costs). Implications for polling on another chain and basically using a polling modifier for whether the MKR on the L1 chain will actually vote vs. offering their signal on a cheaper sidechain and conflicts this could create with respect to actually passing polls if such signals don’t have a L1 participation factor (0-1) applied based on L1 participation.

Conviction voting. This is a distinctly different kind of voting where the vote accumulates weight over time - this basically allows for people to change their votes, but in the end the conviction vote for any particular voter increases the longer they sit on a choice.

Also I don’t think I see any analysis of different voting formats.

Example. Ranked choice voting where each position gets a rank number that is flat in terms of weight and the winner processed by a form or rank elimination with votes moving to next candidate vs. a kind of weighted voting where basically the voter (with however much MKR) is given equally say 1000 points that they can then allocate to the ranked choice options to create a voting ranked choice weight distribution…

In the end I think the work is interesting. I also don’t understand why this is called an ‘automated voting system’.

2 Likes

Hum, you’re talking about counting strangely in some of this.

If votes are cast by locking up MKR tokens, there are many ways to count a single vote. For example, a vote could equal the minimum amount that MKR could be divided into. Or, one vote could be equivalent to 0.5 MKR token. Either way, the quantity of MKR for one vote should be standardized.

There isn’t really such a thing as ‘a vote’ in tokenized systems. The vote weight is continuous. This has implications with respect to this statement:

Median can easily be gamed by splitting votes into smaller pieces

This isn’t true. Maybe you meant mode here rather than median? Not sure. Calculating the median result in a token-weighted vote is pretty simple.

  1. Split results into option buckets.
  2. Order option buckets by option value.
  3. Take total vote-weight
  4. Divide by 2.
  5. Figure out which bucket the ‘middle’ of the vote-weight is in.
  6. Take that option as the result.

EG:

Vote A: 1000 MKR on 0.2
Vote B: 5000 MKR on 0.8
Vote C: 2000 MKR on 0.6

Into ordered buckets:
0.2: 1000 MKR
0.6: 2000 MKR
0.8: 5000 MKR

Total weight = 8000 MKR
Midpoint = 4000 MKR
(for convenience dividing these into 8 equal buckets of 1000, but they won’t be the same size in practice)

0.2 - 0.6 - 0.6 - 0.8 - 0.8 - 0.8 - 0.8 - 0.8

Mid point is in between the leftmost two 0.8 buckets.

Not sure how comprehensible that was (and you may already know.) Apologies, it’s late here.


Also, I still didn’t see a huge amount of info on median schemes versus mean schemes. The choice has interesting implications dependent on the parameter it’s applied to, I think. In some cases a mean average is worse than a median result at an extreme.

A contrived example of this:

At a 4% stability fee we know people will move out of Maker. Some group wants 8% to make as many fees as possible. Some group wants 3% to ensure retention of customers. In this case the 3% group needs more votes to achieve their objective (retain customers) than the 8% group does to achieve theirs (increase revenue).


This research is still great by the way, please don’t take the above as harsh criticism.

4 Likes

I compared these superficially on the thread that @psychonaut linked.

2 Likes

I must have missed this can you include it in your post I am linking to.
Edited: I remember that thread. This was also where you were talking about how conviction voting could be a better model for prioritizing DAO processes…

Fff, @psychonaut not Meoh.

I want to echo this. Anything that helps us look at things is really great. Gives me/us a chance to see someone else’s analytical processes and posit questions and some of our own ideas.

Posts here are simply meant to help focus your research and maybe give you some ideas…

2 Likes

@LongForWisdom

  1. Yes, I am referring to a more generalized way of counting votes in tokenized systems. In your example, the weights were approximately continuous. They were still discrete at the lowest level (the minimum amount that a MKR token could be divided by), which to my understanding is how most tokenized systems work. I just included the possibility of possibly setting a higher limit to a vote. Let me know if I have any incorrect interpretation here. I set up this framework because with other polling methods I presented, such as cluster. The motivation for this is that in a system with millions of voters and a gigantic range of values, it might be inefficient (when doing the clustered polling approach) to have this approximately continuous range, instead of a more discrete range of voting power. Let me know if I’m not making sense here.

  2. Regarding my statement “Median can easily be gamed by splitting votes into smaller pieces”. This is a false statement in the context of votes being weighted the same (in most tokenized systems, they are weighted the same). I should remove it. Thanks for spotting this.

  3. Regarding discussion of mean and median, the analysis I provide on comparing these two is quite small. In my presentation, I note that a median will be at the peak of a skewed distribution while a mean will not (basic statistics). So, in a skewed distribution, it depends on whether you want to barely satisfy the majority (mean) or greatly satisfy a smaller set of voters (median).

  4. Regarding your stability fee example: I think mean and median would both do poorly for the exact reasons you mentioned. That’s why I think a clustered polling method is the best for split distributions.

  5. All criticism is welcome!

4 Likes
  1. The goal of polling in a general sense is to get a sample of voters or actually reach a consensus. However, in my research it is meant to reach consensus and skip the sampling part. This would be the point of implementing the polling process through smart contracts.

  2. A set of fixed choices would not be considered a range poll, which is what I was researching. Regarding the motivation for a range poll, and not a poll with a set of fixed options: I think both can be good polling mechanisms for different contexts. The benefit of a range poll is decentralization. In a poll with fixed choices, someone (usually a centralized team, such as a governance team) will have to choose those fixed choices from the range. However, a downside of an approximately continuous range is that it could lead to uneducated voters choosing very “odd/bad” votes.

  3. In my written documentation (not the forum post, but its linked in the forum post), I do briefly touch on MKR weighting factor being something to consider, but do not do any analysis on it. In my research, I was trying to be as general as possible, and I think MKR weighting factor can only be argued when talking about a specific application of the polling methods (let me know if I’m wrong here).

  4. I think the polling methods here apply to all blockchains with turing complete smart contract languages. So implementing any of these polling methods on a different chain is definitely an option.

  5. Regarding conviction voting and ranked choice voting: I did not do any research into these because I was specifically looking at range polling (how to achieve consensus on an approximately continuous range of values).

  6. “I don’t think I see any analysis of different voting formats.” I’m not exactly sure what you mean by this. Can you expand please? If you mean specific applications, I did not do this because I was just researching range polling strategies while remaining agnostic to the application.

  7. I called it automated voting systems because it is a voting system that reaches a consensus in a much more automated fashion than sampling the voters and doing a binary vote. However, I agree that it is a loose connection and not a good title. I will change the title now.

2 Likes

Thank you for the feedback! I really appreciate it. If you would like to discuss it more in depth or provide me with more in depth criticism (or new ideas) please reach out to me at [email protected] I would be happy to talk!

Hey, congratulations on your research, I found it very interesting.

How about using Knowledge Extractable Voting as a tool to reward the options closest to the winning option? By this, you would mitigate the deviations that the votes too far away from the mean can generate without the need to normalize it with a mathematical formula, which would give more democratic legitimacy to the result.
With KEV, voters would be incentivized to make proposals that agree with the most likely outcome (according to a rational evaluation based on the systematic study of the issue in question) and therefore their vote would stay away from deviations.
Furthermore, it would allow you to sustain decentralization and would function as a disincentive for uneducated voters choosing bad votes.
Then you can weight votes by using the outcome of the poll as a multiplier. By doing this, we can establish that the weight of each voter is directly influenced by her previous behavior in past votes. If someone votes for the correct option according to what the majority thinks, he is rewarded with more voting power. This power is then used in future votes to increase the weight of the user’s vote and as a counterweight to the tactical vote, either allowing the voter with more voting weight to vote last or to give it a higher nominal value.
I don’t know if it would work, but it occurred to me that it could be a tool to think about options that counteract the downsides of range polls.

2 Likes

Thank you for your feedback! I really appreciate the link!

I read “Knowledge Extractable Voting”, and it seems like quadratic voting could be implemented on top of any of the strategies I presented (if it was defensible to sybil attacks somehow). Whether or not it should be included probably depends on the application (what is being voted on). In regards to the NFT tokens that are weighted based on prior elections, this could be implemented on top of any of the methods I layed out. However, when applying it, I am not sure that it fits into any application relevant to MakerDAO. In most elections, the majority vote might not be the rational vote. And, in some elections, the rational vote may be too hard to identify altogether.

The normalization strategy is different from KEV because normalization disincentives voters from voting far away from the mean, but won’t affect their voting power the next time (unless they vote far away from the mean again).

KEV is definitely an interesting strategy. In terms of limiting tactical voting, I think normalization is the way to go. In terms of limiting “bad votes”, I think it is a toss up between KEV and normalization; I see an argument for both.

Thanks again for bringing this idea to my consideration, because I did not know about it previously.

1 Like

Hey Colby!

Deniz here, I’m with the DUX (Development & UX) CU, working on the Governance Portal and thinking about Maker’s governance process as a whole. We have an intro call scheduled later today but wanted to reply here in public as well :innocent:

Thanks for your contribution! Very interesting to read an academic take on DAO voting methodologies and super relevant timing since the future of the Maker governance process seems to be a hot topic. As the proposals within the DAO become more complex, it will require our governance process to evolve with that. Introducing more advanced and bespoke governance primitives (such as the ones you introduce) will be an important puzzle piece of that renewed governance system.

I believe that GovAlpha has the mandate to decide on which particular primitives should be explored at which specific time, based on the nature of governance proposals being shared. I do want to share my perspective on the implementation of such new primitives below:

  • With regards to adding new primitives to the status quo: A large part of the Maker governance process currently exists on L1, which is expensive to interact with. Some of the considered primitives might prove unviable because of the gas fees associated with voting on such poll types.

  • With regards to the longer-term: As MakerDAO is currently defining its long-term purpose and mission, renewed MKR tokenomics and also a renewed governance process, I believe it would be super valuable to incorporate your research results into the requirements/design of that renewed governance process. It would be great to see your personal involvement in those discussions going forward—from my perspective I think that’s where you’ll be able to make real impact within the DAO as you’ll be less constrained by the (technical) status quo.

  • With regards to using Snapshot for polls: Temporarily using Snapshot for governance polls to reduce the cost of participation in Maker governance seems smart and presumably will increase the participation rate for polls. Calculating the MKR voting weight for governance polls on Snapshot will require the creation of a custom Snapshot strategy, since it’s more complicated than the usual ERC-20 address balance commonly used in Snapshot. We did a writeup on how the MKR voting weight is calculated for polls, find it here. Also note that we probably don’t want to rely on third-party tooling for critical infrastructure (such as governance polling), so I would suggest we frame this solution as temporary. In any case, we (DUX CU) are happy to help you in whatever way possible. You know where to find us; in our Discord.

Good to have you around @colby :slight_smile: I’m eager to see what impact you’ll make here over the upcoming months.

(PS: thanks @wouter for pointing me towards this thread—it can be a struggle to keep up with everything happening these days)

10 Likes