Maker SourceCred Trial

Intro

As some of you may be aware, myself and @rich.brown have been working with SourceCred to deliver a trial implementation of the SourceCred algorithm on the Maker forum. We hope that this will encourage high-quality participation and increased governance bandwidth.

The details of this trial and the timeline moving forward can be found below. But briefly, the way this works is that weekly cred scores will be computed based on recent and historical participation on this forum, including posts, replies, and likes. The intention is to reward individuals for making valuable contributions (those which spark good discussion and receive lots of likes).

The SourceCred team, primarily @s_ben (with support from @decentralion and @beanow) will be on hand to offer support and run the trial assuming it goes ahead. Given the potentially intrusive nature of this trial, it will only go ahead if confirmed via a forum poll.

Assuming the forum poll ends favourably the intention is to run this trial for 3 months, distributing at least $5k worth of DAI each month, and allowing monthly redemption for everyone with a balance of at least $10. If this trial goes well, there are additional features and capabilities that SourceCred offers that can be investigated.

The rest of this post has been written in collaboration with the SourceCred team.

Timeline

18/05/2020 - Trial Summary Forum Post
21/05/2020 - SourceCred Presentation in Governance Call
21/05/2020 - Confirmatory Forum Poll
31/05/2020 - Forum Poll Ends
01/06/2020 - Trial Starts (given poll success)
31/08/2020 - Trial Ends (given poll success)

Context

Back in January 2020, in response to ongoing discussions around Maker governance overhead and how to compensate workers post Foundation, @s_ben proposed a new data source for consideration. This data source uses SourceCred, a reputation protocol for open collaboration, to score contributions on the Maker forum. Several people saw potential in the SourceCred algorithm to help to solve problems of incentivisation and participation in the Maker community.

@LongForWisdom created a working group to explore a trial of SourceCred with the blessing of @rich.brown and reached out to members of the SourceCred community (@s_ben, @decentralion, @Beanow) who agreed to work with us on designing the trial. On Feb 23rd, a governance initiative was posted detailing these plans.

SourceCred worked with the working group to design a controlled three month pilot trial, wherein Maker rewards contributors to Maker Governance on the forums using SourceCred “cred” scores. SourceCred will be given a development grant to provide technical support for the trial.

Trial Summary

SourceCred believes it can incentivize governance work by providing an effective, decentralized and incentive-compatible way to value contributions. We will test this hypothesis by running a controlled pilot trial, wherein Maker rewards contributors to Maker Governance using cred scores.

SourceCred will setup a cred instance based on the Maker Discourse forum. We will then run a three-month trial, distributing DAI to governance participants based on their cred scores. From an implementation standpoint, the cred scores will be calculated and published off-chain, and SourceCred will manually distribute DAI to contributors, these tokens will be sourced from the Maker Foundation community development fund. Results will be measured with a combination of objective and subjective metrics.

Benefits to Maker

SourceCred hypothesizes that if cred scores can be used to pay contributors in a credibly neutral way, it should enable Maker to scale its contributor base without overwhelming current Foundation members, while also decentralizing decisions around valuing contributions.

If the trial is successful, SourceCred also expects increased DAI adoption, because it will give other DAOs the ability to permissionlessly pay contributors they currently cannot, due to issues similar to those Maker is facing. At the end of the trial, it will create a report to share findings and best practices.

Voting participation could be positively impacted, due to better content for MKR holders. Future experiments could also involve more directly incentivizing voting using SourceCred, for instance by providing an input to vote delegation mechanisms, or incentivizing a particular type of content.

Trial Implementation

Scores

SourceCred, as the name suggests, is all about attributing cred. Cred is a metric that scores every contribution and contributor in a project, giving a sense of how important they were. SourceCred calculates cred scores using a modified version of the PageRank algorithm (what Google uses to rank webpages).

Contributions are arranged in a graph, where contributors and contributions are nodes, and have edges indicating how they relate to other contributions. For example, if someone writes a reply to this post, there will be an edge connecting that reply to this post. Liking and referencing a post in other posts will also create edges. You can think of each edge in the graph as being a “thank you”.

In general, SourceCred is designed to flow cred to a contribution when there is a review of that contribution (likes in the case of forum posts), rather than when the contribution is created. This reduces the danger of spam and reinforces the idea that contributors dictate what is valuable.

See the Resources section below for more information on how cred scores are calculated.

Scores will be published weekly on the Maker forum, and the SourceCred team will be available to answer any questions and concerns from the Maker community.

Payouts

Payments will be distributed monthly, with the payout amount equal to the sum of the weekly published preview payouts.

Contributors that do not opt-in will be exempted from payouts. This is because not all contributors will want to take on risks and costs associated with receiving cryptocurrency income. Opting in will require the communication of an Ethereum address to the distributor before the communicated closing date. The closing date and method for opt-in will be communicated before the trial begins.

Contributors whose total payout is below a threshold of $10 will also be exempt. This will make the number of payouts more operationally manageable, and incentivize more meaningful commitments.

The Maker Foundation grants program will allocate $5,000 worth of DAI for distribution in the first month. If results are positive, it may increase this amount. The actual amount distributed by Maker will be less than the total monthly distribution allocated, as payments are opt-in and certain categories of contributors are exempted.

Exempt Parties

Those in the Maker-SourceCred trial working group are exempt for conflict of interest reasons. This includes the following users:

@LongForWisdom
@rich.brown
@s_ben
@decentralion
@Beanow

Foundation employees inside the community development department, or who otherwise have roles for which forum posts and community communication are considered ‘part of their responsibilities’ are exempt from this trial.

Fast and Slow Payout Components

For calculating payment amounts, SourceCred will use a method developed during its internal dogfooding of SourceCred, which has now paid its contributors ~$400,000 worth of its native token, Grain. This method splits payments into “fast” and “slow” components.

The fast component allocates directly based on each contributor’s cred in the latest full week. This ensures that anyone who made a meaningful contribution during the week is rewarded, keeping the system responsive.

The slow component is a function of lifetime cred scores and past payments, with an adaptive component that adjusts payments such that a contributor’s total earnings are in line with their share of total cred. If there are no changes to past contribution weights (e.g. because the weights were changed to mitigate gaming, or some older contributions were revalued due to reflect new information), the slow component is equivalent to distributing according to lifetime cred scores.

The amount distributed by fast and slow components is determined by a set percentage, with fast payments constituting 20% of the distribution, slow payments 80%.

The section below provides more detail on the adaptive component of the slow payout.

Adaptive Component

SourceCred uses a mechanism it calls “underpayment” to determine allocations, as described below.

The underpayment mechanism imagines a hypothetical world, where the sum of all DAI distributed during the trial was distributed according to the current lifetime scores. Based on this, we can calculate the “fair” lifetime earnings for each participant. Usually, some will be “underpaid” (they received less than this amount) and others are “overpaid”.

We can sum across all users who were underpaid to find the “total underpayment”. As an invariant, the total underpayment, before distribution, is equal to the total overpayment, as every token of overpayment to one contributor is underpayment for another contributor.

Once we have calculated each contributor’s underpayment, and the total underpayment, we divide the total tokens being distributed in proportion to their relative underpayment.

SourceCred Support

The SourceCred Team will support the trial in the following ways:

  • Weekly reports on cred scores, posted to the forums, where community members can ask questions and raise concerns.
  • Building moderation tools as needed to protect against attacks and gaming
  • Updating UI and documentation based on questions and feedback
  • Creating and processing subjective surveys
  • Producing a final report for the Maker community analyzing results

Metrics

Success Metrics

Success will be measured with a combination of objective engagement metrics provided by Discourse, and subjective feedback in the form of surveys for contributors and the Maker-SourceCred working group.

Cancellation Metrics

The trial will be cancelled if either of the following become true:

  • The system is being overtly gamed to an extent that the Maker-SourceCred working group feels the experiment is harming the project, and SourceCred cannot address gaming via changes to the algorithm or other measures.

  • The Maker-SourceCred working group determines there is strong community opposition in response to the trial in progress.

Resources

Below are resources that provide more information on the SourceCred algorithm:

21 Likes

As promised, we will be having a forum poll to confirm that the community wants to trial SourceCred on the Maker Forum. Please make your opinion known!

  • Yes, we should proceed with a trial of the SourceCred algorithm on the Maker Governance Forum
  • No, we should not proceed with a trial of the SourceCred algorithm on the Maker Governance Forum.
  • Abstain, I have no strong feelings either way.

0 voters

This poll has now closed. The trial of SourceCred on this discourse forum will commence on June 1st.

4 Likes

Hi all! I’m Dandelion, the lead / “Temporary Benevolent Dictator” of SourceCred. I’ve been fascinated by Maker for a long time (and hold MKR), and am excited that we may have the chance to support and work with y’all.

Here are the slides from my presentation at the governance meeting today. If you want to learn more about SourceCred, you can also check out my Ready Layer One presentation, and our website.

You’re all warmly invited to come to the SourceCred community call on Tuesday at 11am PT (calendar invite); it happens in the voice channel of our Discord server. Next Tuesday (5/26) we’re going to prioritize answering questions from anyone in the Maker community who attends.

Also, here’s a currently up-to-date cred instance for Maker.

12 Likes

Posting for discover-ability, here is Dandelion Mane’s presentation about SourceCred and Maker from the May 21st Governance and Risk call.

4 Likes

The confirmatory poll has now closed, with the vast majority either voting in favour, or abstaining. Given this result, the trial will begin on Monday.

Thank you to all who voted, I’m looking forward to seeing how this goes!

3 Likes