MakerDAO’s understanding of vault liquidation behavior is crucial to long-term economic resilience of its system of smart contracts. While a tail event, there is significant solvency risk involved in case of too much bad debt being liquidated at the same time during large price drops of ilk’s underlying collateral assets. While a lot of effort has been already invested across different teams to maximize robustness of the liquidation system, there is an external element of vault activity which can also aid in preventing insolvency from ever happening.
@Risk-Core-Unit team decided to look into this dynamic to derive useful insights as a part of a series of analyses on vault behavior.
We begin by exploring the high-level vault behavior, both looking at how many vaults are being protected over time and also how vault owners protect their vaults.
As expected, vaults tend to have the most liquidation protection activity during large drops in collateral asset prices. Besides that, we also look at how successful vaults are at protecting their borrowed funds, split over various dimensions.
We extend the research by comparing vault liquidation activity during two days of major drops in collateral asset prices: March 12th 2020 (Black Thursday) and May 19th 2021. Both days include the most data points to analyze liquidation activity in isolation and also in comparison.
Key component behind the analysis is how we identify liquidation protection activity. As we can see in the below taxonomy of liquidation protection action patterns, a liquidation protection action can be performed in either a single operation within a single transaction, multiple operations within a single transaction and multiple operations within multiple transactions. For this reason we decided not to describe activity in terms of transactions or operations and rather defined all three liquidation protection patterns as a single liquidation protection action.
The analysis includes all vault types except those collateralized by stablecoins.
The analysis looks at three main ways to increase collateralization ratio via a liquidation protection action type:
deposit collateral (DEPOSIT)
payback DAI (PAYBACK)
withdraw collateral, swap for DAI (possibly at a DEX, outside of Maker) followed by paying back DAI (WITHDRAW|PAYBACK)
Only the third action type (WITHDRAW|PAYBACK) can include two separate operations which can be executed in either 1 or 2 separate transactions/blocks. To interact with the vault, the vault owner could execute the action via their own bot, use a Maker vault management UI or through one of the specialized liquidation protection services (DeFi Saver, InstaDapp, etc.)
For actions performed in two separate blocks, we filter out only those that were executed during the same OSM price to minimize identified false positives.
Another requirement for a liquidation protection action is that it was performed on an unsafe vault position. Unsafe position is defined when the collateralization ratio is less than or equal to 20% above the liquidation ratio in relative terms (CR <= LR *1.2). In the example of ETH-A it would mean actions executed when the collateralization ratio was 180% or below. We are looking at the collateralization ratio before the liquidation protection action was executed.
Important to mention is that our detection methodology is not a ground truth for every liquidation protection action. It’s a heuristic to classify patterns that we’re interested in while trying to minimize both false positives and false negatives. Setting the 1.2 multiplier is a typical precision/recall trade-off problem where we can’t increase one without decreasing the other.
When looking at the liquidation protection success rate, we define an action as successful if the vault wasn’t liquidated within 24 hours after action execution.
Overall, we identified around 38000 liquidation protection actions performed by around 5000 unique vaults.
In the chart below we can see that the most monthly protected vaults align with the months with the two major price drops (March 2020 and May 2021). Meanwhile, it’s clear that the number of liquidations relative to the number of protected vaults is much lower in May 2021 compared to March 2020.
This reflects well on the responsiveness and success rate of protection activity of the vaults affected by the collateral price drops. Other than that, the usual number of monthly protected vaults range from 200 to 1200.
We can also aggregate monthly overview on debt size instead of the number of vaults. As expected there is a strong relationship between protected and liquidated amounts. There is a stronger influence of DAI supply growth which causes the recent months to be larger both in protected and liquidated amounts.
A possible extended metric is to look at the ratio between liquidated debt size and protected debt size. As it can be derived from the chart, this has been relatively constant since a major increase in the month of Black Thursday.
Distribution of liquidation protection action types over time shows that DEPOSIT, PAYBACK and WITHDRAW|PAYBACK_ST (ST = single transaction) make most of the total actions. Each of them currently contributes to a bit less than ⅓ of total actions while the WITHDRAW|PAYBACK_MT (MT = multiple transactions) action type contributes to less than 5%.
Meanwhile, DEPOSIT has lost some of its share since MCD’s launch to WITHDRAW|PAYBACK_ST. Similar applies to WITHDRAW|PAYBACK_MT.
When looking at protected debt size across different action types, we can see that PAYBACK tends to protect the highest average debt size. Meanwhile, the other three action types have a similar average value. Considering the impact of outliers, this indicates that larger vaults (whales) tend to use the PAYBACK action type more when entering the specified danger zone. That’s likely because it’s common for whale activity to include farming with DAI on other venues, which means that they have enough DAI at hand for repayment in case of an unsafe drop in vault collateralization ratio.
Next, we compare the contribution of protected debt size across different action types. It’s immediately clear that relative shares are less consistent over time than in Chart 3. Also, WITHDRAW|PAYBACK_ST action type loses in relative contribution to total protected debt size which is partly caused by a lower average protected debt size seen in Chart 4.
The most active liquidation protection days tend to occur during large market price drops which doesn’t come as a surprise. For this reason March 2020 and May 2021 again pop up as days with a high liquidation protection activity.
When we look at days with the most protected debt size we can see that all days happened in the last few months. This makes sense considering a rapidly growing DAI supply where, at a given amount of liquidation protection activity, later dates are more likely to protect larger debt sizes.
Liquidation protection actions distributed across vault types show that as debt exposure became distributed across different collateral assets, so did vault protection activity.
While ETH-A contributed to more than 90% of total action in the first months after MCD launch, the number currently stands at around 50% which mirrors its current DAI collateralization share.
Overall success rate of all liquidation protection actions is 95%. That means that 1 in 20 actions doesn’t save the vault from being liquidated within 24 hours after action execution.
There is no difference when looking at success rate across different action types so we decided not to include the chart in the analysis. Meanwhile, when looking at distribution by vault type, we can see that while there is a minor difference in their success rate, actions on Uniswap LP vault types tend to be the most successful (99%+) and actions on ETH-A tend to be the least successful (90%). Both make sense, the former because of lower asset volatility and the latter because only ETH-A and BAT-A vault types were available during Black Thursday market crash which had the highest number of liquidations (unsuccessful protection actions).
In the comparison of two downfall events, Black Thursday to May 19th, we can see that the latter had about twice as many active vaults at the time. An active vault is defined as a vault that has had a vault operation within 3 months before the chosen date and more than 100 DAI being borrowed at the time of the executed action.
Even with that, there were both more protected vaults on Black Thursday and also the vaults were being more actively protected (average protection actions per protected vault).
That also impacted a lower percentage of daily active protecting vaults during the May 19th event. On the other hand, the liquidation protection success rate was considerably higher in the price crash this year which sheds a positive light on the improved robustness of the system since Black Thursday.
One important factor to consider is that during Black Thursday, there was no flash loan feature available yet which increased the number of actions.
Splitting number of liquidation protection actions by action type shows us that while there was an overall higher vault protection activity during Black Thursday, there is the largest difference in the amount of executed WITHDRAW|PAYBACK_ST and WITHDRAW|PAYBACK_MT actions (3.5 BT actions for every M19 action).
Meanwhile, the difference is lower for DEPOSIT and PAYBACK action types (1.7 and 1.15 actions during BT for every 1 action during M19, respectively).
The last chart shows distribution of relative collateralization increase before and after the liquidation protection actions (e.g. 5% increase means increasing collateralization ratio from 160% to 168%).
Most increases were smaller, with 60% of them being lower than 10%. For the sake of clear visualization, we filtered out all increases that more than doubled their collateralization ratio (increase > 100% which constitute about 2% of total actions).
We can see that during Black Thursday, vault owners boosted their vault collateralization ratios on average lower than they did on May 19th.
This indicates that they performed more actions with lower amounts relative to the May 19th tail event.
There are a couple of key takeaways from our analysis.
When looking at the number of liquidation protection actions, there is a relatively equal distribution across between DEPOSIT, PAYBACK and single-transaction WITHDRAW|PAYBACK action types. As expected, there is not much multi-transaction WITHDRAW|PAYBACK activity on unsafe vault positions.
When looking at action type distribution normalized by debt size, we can see that whales tend to use PAYBACK actions more relative to other action types. The best explanation seems to be that they are just more likely to have DAI on hand as they’re extending their activity across other protocols in the ecosystem.
The number of liquidation protection actions per vault type maps well with DAI supply distribution. Vault types with longer MCD tenure tend to have a lower liquidation protection success rate.
When comparing Black Thursday with May 19th price drop, it was somewhat surprising to us that there was much more liquidation protection activity during the former event. This can be partially explained by lower average collateralization increases during Black Thursday (more actions with lower increases). Meanwhile, a higher success rate during the latter event is to be expected.
Next steps for us are to automate metrics that would make sense to monitor on a continuous basis and potentially add a vault-level risk score. This means moving towards quantifying potential liquidated debt also by taking into account vault behavior. This could include historical liquidation protection activity, DAI capital availability score and other factors.