Maker Vault Activity Metrics

Hey all,

this is the first among a series of analyses coming from @Risk-Core-Unit which focuses on studying vault behavior. Besides benefitting risk analysis of the protocol, this is likely to also prove useful for other domains such as business development and growth.

The aim of this individual analysis is to present a complementary set of protocol-level activity indicators by getting a deeper understanding of vault behavior. These include both growth metrics and risk metrics as there is often no clear line between the two. For example, investigating Black Thursday’s and May 19 sell-off’s (unpredictable risk events) effect on vault churn (a growth indicator) is a question where both growth and risk hats need to be on to dive deeper into the dynamics.

Based on when a vault was opened in the MCD system’s lifetime, how likely is it to experience a liquidation? How well are Maker vaults retained over their lifetime, how has that changed over time? How has the number of active vaults changed over time? Or to dive even deeper into the foundations, how do we define an active vault? These are some of the questions that we would like to dive into in this analysis and continue further work together with the community to gain an even more holistic view of Maker protocol performance.

Chart 1: Monthly Active Vaults (MAV) - New / Retained

More than 25000 vaults have been opened so far in the Maker’s MCD system. More than 20000 (4/5) of those exceeded 100 borrowed DAI at some point. As vaults open and become inactive it makes sense to get a better idea of how the number of monthly active vaults (MAV) changes over time. Vaults can perform one of the following operations: open (vault), deposit (collateral), generate (DAI), payback (DAI), withdraw (collateral), liquidate (DAI), and move (vault).

Relevant to mention here is MAV’s relation to DAI supply as two distinct growth indicators which are complementary to each other. If we look at total DAI supply, there is a strong power law where a minority of vaults (whale vaults) contribute to the majority of DAI supply. This is especially true if we would aggregate vaults per a specific vault owner, whether an individual or an organization. This means that MAV is somewhat indicative but also likely not the strongest signal for growing DAI supply. Meanwhile, if the complementary aim of the protocol is permissionless access to finance, MAV matters a lot and it is a stronger indication of how many people (actively) use the protocol compared to looking only at DAI supply. That’s a question to open up to the community. Growing DAI supply and MAV is not a mutually exclusive task, they only present two different indications of fulfilling our mission in practice. Another risk-related view on MAV is that the more debt exposure is distributed across many vaults, the less likely there will be a sharp decrease in DAI supply if one or several of larger vault owners decide to close down their vaults. That way, looking at MAV provides another measure for Maker system’s resilience.

The definition of an active vault in our analysis has two conditions. Firstly, a vault needs to have at least 100 DAI borrowed in the last operation before the end of the period (month). Secondly, a vault made an operation (one of the 7 mentioned above) within the 3 months before the period of interest. If a vault had 110 DAI borrowed in January 2021 and made a vault operation in December 2020 we still count it as active in January 2021. If the vault had either less than 100 DAI borrowed in the period or made an operation before October 1st 2020, we don’t (before t-3 months). The reason for a more conservative inclusion of active vaults is to avoid counting inactive vaults where someone opened a vault in the past, borrowed a couple of DAI and left it there since. We believe this shouldn’t be counted as an active vault.
In the chart below we can see a healthy MAV development. A good indication is both whether MAV grows over time and also whether the metric is constructed of mostly new or retained vaults. If new vaults contribute to the overall metric less over time (as we can see below), that makes a strong indication of a good retention of old(er) vaults and not that vaults only open and withdraw their funds shortly after. That’s of course assuming a relatively constant inflow of new vaults (which we will confirm in a later chart).
We can also see that considering a more bearish market since May 19th, there was an understandable drop in MAV but not as severe as the drop in overall market sentiment.

Chart 2: Monthly Active Vaults (per Monthly Cohort)

Below we can see the MAV metric in the form of stacked monthly cohorts. A cohort is defined by the period (in this case a month) in which a specific vault was opened.

As expected, older cohorts contribute less to the overall MAV as the cohort’s size continues to shrink over time. This chart is a good visual representation of the importance of both vault inflow (new vaults being opened) and also vault retention (existing vaults continuing to satisfy the active vault condition) in growing MAV.

Chart 3: Vault Cohort Retention

The below chart shows the percentage of vaults per cohort that opened their vaults, became active in the same month and also continued to be active over their tenure (vault usage lifetime).

As an example, of all the vaults that were opened and became active in October 2020 (cohort date), 70.4% of vaults remained active in their third month (M3 retention).

As already indicated by MAV, the retention looks healthy with a relatively minor drop in M2 (contrary to the patterns seen in most software products). Yet as expected, each cohort’s retention slowly shrinks over time.

Two outliers that diverge from the overall pattern are February 2020 and April 2021 cohorts, one influenced by Black Thursday in March 2020 and the other by May 2021 sell-off. You can see that both of these cohorts’ M1 retention (31.9% and 56.6%, respectively) had a much larger drop compared to other cohorts. The latter had a smaller drop, most likely due to multiple reasons (better prepared vault owners to this kind of tail risks, improvements in the liquidation mechanism with the Liquidation 2.0 module, etc.) This speaks positively about Maker’s protocol resilience to market risk events.

Chart 4: Vault M3 Cohort Retention

The chart below is derived from Chart 3 and focuses on retention in the third month after a vault is activated (M3 retention).

We can see again the influence of both tail risk events (Black Thursday, May 19 2021 sell-off) in M3 retention in the few cohorts preceding these events. Overall, M3 retention above 60% is a good indication of protocol’s sustainable growth.

Chart 5: Vault Activation

In the chart below we can see funnel analysis which shows the percentage of vaults transitioning from one activation step to another. All percentages are relative to the number of vaults that were opened in a specific period (cohort). This is often used to identify bottlenecks in the product’s onboarding process to JTBD (job to be done) which is in this case borrowing DAI (and later doing something with it).

Activation steps:

  • Opened vault
  • Deposited collateral
  • Borrowed DAI
  • Borrowed 100 DAI (partial active vault requirement)

As an example, we can see that from all vaults that were opened in November 2019 (when MCD was launched), about 85% of them deposited collateral and borrowed DAI. 60% of all opened vaults exceeded 100 DAI threshold in borrowed DAI.

The conversion from an opened vault to exceeding 100 DAI borrowed has increased significantly over time. The conversion to both collateral deposited and DAI borrowed has remained stable and high over time. Currently, all vaults that borrow DAI also borrow more than 100 DAI because of dust restrictions which have been raised significantly over time. That makes the conversion from borrowed DAI more relevant to track before the large dust increases. One interesting insight is that the conversion to borrowing DAI after opening a vault has decreased in June relative to previous few periods. This might be caused by the current bear market where vault owners deposit collateral into their vaults (relatively low gas cost) but are more hesitant to also borrow DAI to potentially create a long position. This observation is of course based purely on speculation. We would like to invite the community’s collective wisdom to join us in assessing this dynamic. Overall, there is no significant drop-off from one step to another in the activation funnel which is another good indication of protocol’s health.

Chart 6: Vault Cohort Liquidations

The chart below shows both the number of opened vaults per month and percentage of cohort vaults that experienced a liquidation. An example calculation is: if 100 vaults were opened in a specific month and 10 of those experienced a liquidation during their lifetime, then the Percentage of Cohort Vaults Liquidated is 10%.

We can see a spike in the percentage of liquidated vaults that were opened before Black Thursday (especially pronounced is February 2020 cohort).

We can also see that the number of opened vaults has remained stable over time with a drop in last month.

Another observation is that the percentage of cohort liquidated vaults stabilized around 5%-10% after Black Thursday despite another tail event on May 19 2021.

Summary and next steps

The metrics presented above seem healthy and show an increasing resilience to market risk over time. As expected, we recently noticed a drop in some of the measures because of more bearish crypto markets.

MakerDAO’s aim to grow DAI Supply is aligned with the aim of growing the number of active vaults. Besides acquiring and activating new vaults, this also means the importance of retaining existing ones.

There is a clear pattern of vault churn during large price drop events. This can be explained by both vaults following the market sentiment, causing them to stop using MakerDAO’s services and also negative user experience of vault owners due to liquidation events. Only the latter is something that is actionable which raises questions on the topic of vault protection against liquidations.

The next step for us is to focus even more on risk-related vault dynamics. We already hinted at some of the topics for the analysis but we would still appreciate your input on what would be the most helpful for the community to dive into.

Finally, we would like to thank Token Flow (@tmierzwa, @piotr.klis and the team) for giving us access to well modeled data and some tips that enabled this analysis.

16 Likes

@josolnik Great piece of work. Really interesting perspective you bring here at the interface between risk and growth. As any good analysis, it usually leads to more questions. I think it would be really interesting and helpful to understand further the impact between MAV (chart 1) and cohort liquidation (chart 6). For example, if we base on the hypothesis that a below average lifetime cohort liquidation (< 10%) is good, do hibernated vaults (low MAV) explain above avg liquidations (extreme events excluded)? If yes, what is the contribution of those hibernated vaults (in volume of supply and quantity)? What is the relative importance of “explainers” to vault liquidation: is time/tenure/maturity more or less important than MAV in eventual liquidation?
Great piece. Keep this coming.

4 Likes

Thanks @williamr, great follow-up questions. Understanding “explainers” of vault liquidation is definitely an important point to explore further.

Love to see this. Very helpful. And good to know we’re beginning to build out our data.

1 Like

Are we missing chart 6 here?

Correct, added the chart. Thanks @LongForWisdom!

1 Like

No problem. This is awesome by the way. I don’t have time to dig in right now. But the cohort data is extremely interesting. Chart 2 especially.

This is awesome! And as a side note, the coloring of Chart 2 is just absurdly pleasing to look at.

@josolnik Thanks for the analysis! Great work.
I am working at our BI here at Oasis Apps and we have been talking about using the definition you have defined here for an active vault.
However, I am not 100% sure I understand your definition actually.

A vault has to have at least 100 DAI borrowed (generated) with in the last month AND had another operation done within the last 3 months? So two operations in all?

Thanks!
Kasper

1 Like

Thanks for the question @KasperB!

You got the second part right. The first part doesn’t have to do with a vault needing to perform any operation, only checking the state (borrowed amount) in that period. The ‘last operation’ part of definition only means that we check the DAI borrowed amount after the last operation that was performed before the end of the period. The operation can also be performed before the period (as long as it’s within last 3 months, otherwise the vault doesn’t count as active anymore).
Did a vault have borrowed DAI and recently performed an operation? That’s what we are interested in.

Hopefully this makes it clearer.