SourceCred Trial: Review of Month 1, Payout Increase

The SourceCred trial has reached one month! :partying_face: Speaking for the SourceCred team, I want to thank everyone that has opted in, provided feedback, asked good questions and not made this harder by attacking the system.

While the working group is generally satisfied with results so far, we have identified a couple areas for improvement. To improve score quality for newer contributors, we have scoped out a new feature that SourceCred is currently implementing. We have also decided to increase the amount of DAI distributed each month from $5,000 to $10,000. This should give us better insight into whether SourceCred is incentivizing greater participation. Below is a review of the first months’ results, rationale for the payout increase and an explanation of the new feature.

Results

As I shared on the June 18th Governance and Risk meeting, the trial has gone roughly as the SourceCred team expected. As the Maker governance community is relatively small, high trust and known for its civility, we did not expect overt gaming or drama. We were right :sweat_smile: We also suspected that $5,000/month worth of DAI may not be enough to financially incentivize many contributors. This also appears to be the case. As seen in the chart below of total Cred created since the beginning of the forum, Cred created has actually decreased in the past month.

While this downtrend is likely unrelated to SourceCred, and simply activity from Black Thursday leveling off, our impression is that contributors are not generally increasing their participation in response to the payouts. This mirrors SourceCred’s own experience dogfooding SourceCred to pay its own contributors. As seen in SourceCred’s payout history, it also started out paying conservatively, and needed to increase payouts before contributors responded with meaningfully higher engagement.

By increasing monthly payouts to Maker contributors from $5,000 to $10,000 a month worth of DAI, we hope to make the payouts large enough to see increased engagement (more quality content).

Algorithm Improvements

We did observe some light brigading in posts related to new collateral applications. This was not too concerning, as it did not appear to be intentional, and the amounts of Cred and DAI involved were relatively small and unlikely to be claimed. This did however lead the working group to collaborate on a new feature design that mitigates this issue and improves Sibyl resistance generally.

To mitigate brigading and other forms of low-quality posting by new contributors, SourceCred is implementing a feature that utilizes Discourse’s Trust Levels. Users will need to attain a certain trust level for their likes to create Cred. Once a user attains the required trust level, their likes will create Cred, creating a “bounty” of sorts to attain that level.

Note that Cred scores may change with algorithm changes, and that overt gaming may result in reduction of Cred score. DAI payouts, including those posted to the forum but not yet sent on-chain are immutable and cannot be taken away.

If you have any questions or concerns about the scores, payouts, data or presentation thereof, please don’t hesitate to ask!

9 Likes

I think this is so cool.

Another wierd question came to me when you were commenting about incentivizing increased engagement (more quality content). When you used this on your own systems did you have a measure for increased participation from new contributors with respect to engagement or was it just more posts that engaged others in the same community?

I ask because I know governance has repeatedly made calls for greater community engagement. @rich.brown favorite line “Please join us and help shape the future of the MakerDAO” a kind of call to people listening in CC’s or just out there to join in here and so I wonder if your term ‘increased engagement’ means more people engaging or just the same people being more active or a good mix of these two.

Thanks again @s_ben and just because I like to look at interfaces to test them out and see how they work. Is the actual June payout available via some interface? I was just curious to see what that might look like.

1 Like

If you look at SourceCred’s own instance, you can pretty clearly see the effect of turning on Discourse. The current UI makes looking at total Cred difficult for a specific platform though, so have created the below chart by running SourceCred just on SourceCred’s own Discourse (NOTE: this doesn’t take into account the rest of the SourceCred graph (e.g. references on GitHub to Discourse posts), but isolating Discourse I think gives us a better view into your question).

As you can see, we got a noticeable spike in activity right after turning the Discourse plugin on. Actually, at the time, the increase was more dramatic because we were originally minting Cred based solely on activity (posts, replies, references to posts) (more on this below). When we turned Discourse on (previously we were only minting Cred on GitHub), we got a large number of posts that were not getting much engagement. To mitigate this, we reduced the total amount of Cred flowing to Discourse.

We subsequently made another change in January inspired by early feedback from Maker actually – minting Cred on likes instead of activity. This improved the quality of scores considerably. As seen in the above plot, it also likely led to a temporary decrease in engagement, before Discourse Cred rebounded to ATHs a few weeks later :rocket: The subsequent reduction in activity I would speculate is migration, for better or worse, from Discourse to Discord. In February, we hacked up the original Discord plugin prototype at ETH Denver (in collaboration with MetaGame, got give them Cred :sweat_smile: ). Since then, Discord has seen more action than Discourse.

Most of this new engagement came from existing contributors upping their committment to the project. I hesitate to speculate on how this might apply to Maker however, as the SourceCred community was smaller at the time, and new community members have mostly come in through Discord.

@decentralion I believe will be making the on-chain payments and updating the observable notebooks we’re using to track Cred and payouts. I imagine something similar to SourceCred’s latest notebook, though don’t quote me. On-chain payments are scheduled to go out on Monday.

3 Likes