Should BAT SF be reduced?

Hey @Andy_McCall thanks for that really great analysis. Certainly exploring betas and daily volatilities is one avenue towards assessing risk. As others mentioned, however, we are opting to take a slightly different approach.

While it is counterintuitive, a market risk approach (such as the one you have used using CAPM and such) is less appropriate given the CDP nature of MakerDAO. Instead, a credit risk analysis is (in our view) more applicable.

So daily volatility becomes much less of a factor as opposed to risk of a CDP defaulting, which occurs due to sudden and sharp drawdowns in a collateral asset. Daily volatility is less useful for anticipating these sudden drawdowns, versus some other approaches (such as just basic fundamental analysis). So while BAT may have a lower daily vol over the past 3 years, or a higher expected return, it is (likely) to have significantly higher risk of a sudden drawdown. (In a credit risk model, you might liken this drawdown to a probability of default and the loss given default).

A simple counterexample that I like to use a lot is Tether. USDT has incredibly low volatility, but that does not make it a safe collateral type for MakerDAO.

5 Likes

Hey @LongForWisdom - is there a reason why the protocol does not allow an asset SF to be less than the DSR? I can imagine a number of conditions where we would want a particular asset to have a SF lower than the DSR.

The protocol allows it (that’s my bad I need to be more clear in what I’m saying), but so far as I can tell it’s rarely a good idea.

It means that the asset is being subsidised by other assets or by MKR Holders. Fair enough, there might be a business reason to do this.

More problematic is that is allows Vault holders to lock their money, mint Dai, and then lock that Dai in the DSR to receive the balance of the DSR - SF Rate, this creates ‘parasitic’ Dai that costs the protocol income without providing us with anything in return. It doesn’t increase the Dai supply directly, because the new Dai would be locked in the DSR, and out of circulation. There is no reason not to do this when DSR - SF > the interest rate for the asset on other platforms.

Hypothetical example:

DSR at 5%, Eth SF at 2%

Users can:

  1. Lock Eth
  2. Mint Dai
  3. Lock Dai in the DSR.
  4. Gain 3%* on their Eth with the only downside being the risk of liquidation.

*It’s not actually 3% because you need to take into account the collateralization ratio, in this case I think it’s max 2%.

I think -0.25% DSR spread option should be allowed in polls today. MKR holders should decide if it is acceptable or not.

I believe the protocol should also have a safety switch which would revert to last (positive?) DSR if the current DSR causes MKR minting because of the negative spread AND high DSR utilization.

Why? -0.25% is not that different from 0% spread.

I think you mean DAI minting, not MKR minting. In any case, it would be hard to determine whether people were taking advantage of the risk free return. One address might own the CDP and another address might be earning DSR on the DAI. The only way to know for sure is to remove the possibility.

I’m back and I have more numbers!

ok so I’ve looked through @cyrus slides and done the bare minimum amount of research that i could into the merton model. Wanted to explain my methods and discuss my findings. Wanted to get peoples critiques.

Ok first the TLDR. My methods are probably too over-simplistic to be certain, but based on the numbers i came up with it may be the case that BAT_SF < 9% but it is also the case that BAT_SF > ETH_SF.

Ok so now the boring part! Numbers!

Let me talk through my methods for solving the Merton Model (there were probably plenty of the things i did were suspect).

  • just like in my CAPM model above i set the risk free rate == DSR.
  • i am basically treating the whole of the debt ceiling for each asset as 1 big CDP, and that the DC for each asset == K
  • V = CR * K
  • I also had a constant CR == 2 * LR == 3
  • You can find my work here. Like i said there is probably a few things i have gotten wrong. I encourage anyone who is interested to take a look.

Those simplifications noted you should probably just look at any of the numbers below as (at best) a ballpark estimate.

Ok so what did if get for my SF calculation? Well the first numbers I’ll give you is cyrus’s method where I calculate the ln(Risk free CDP / Risky CDP) = SF.

Using this method i found the SF_BAT should be ~ 5.58%. Hey its lower right!?!? Well news isn’t so good i also found that ETH have a SF ~4.04%.

Knowing that, maybe the argument is over BAT_SF > ETH_SF, and all of my numbers are measurably too low so who cares? Well I personally found the model used to calculate those numbers a little too simplistic bc it doesn’t take the market risk premium into acount.

However, i found a metric that i can derive from K, V that does take those two factors into account and seems more in touch with the current SF votes; Expected Return on Debt (RoD). Using that CAPM graph from earlier i found those numbers for each asset to be RoD_ETH ~ 18.08% and RoD_BAT ~ 10.61%. Maybe my original argument still stands?

Just doing a quick gut check both of these numbers seem like better candidates for SF given how the executive votes since MCD launch have been pushing the SF from ~ 4% -> 9%. So time for new questions!

  • Firstly, any criticism into my methods are always welcome.
  • Secondly, and most importantly in my opinion, how is the foundation trying to incorporate the “market risk premium” into their risk model? Obviously i know the “risk model” is a work in progress, but any pointers to reading i can do would be very appreciated.
6 Likes

Hey, awesome work! Kinda surprised you went back and looked at slides lol. So, we’ve come a bit of a ways longer since we presented those slides, but yeah that’s is the gist of it. Here are some additional thoughts/ considerations.

  • There’s really no good justification for what the crypto risk-free rate should be, although I am expecting some super-quants to come in and make some solid justifications for something or another. In the end, it doesn’t matter though as the drift ends up having minimal impact on the outputs.
  • we have broken up the CDP distribution into buckets instead of 1 giant CDP, which allows for some additional granularity.
  • Instead of using a constant CR, we incorporate a behavioral function that allows for users to adjust their CR over time (users tend to add collateral or wipe dai on selloffs, and vice versa on rallies).

There are also two additional critical features we have been incorporating.

  • liquidity risk means that there is additional slippage to be incurred upon liquidation
  • the merton model does not include jump risks, which we view as critical to the model. so we add an additional jump component to the basic stochastic model. The parameters of the jump component can be estimated using a fundamental analysis framework for the collateral asset.

Finally, there is a slight difference between CDP pricing and CDP risk management. While the model you describe is for an average or expected stability fee rate, we need to take a few extra steps for the scenario analysis. So we’re doing a value-at-risk analysis that helps inform us what the max theoretical debt ceiling can be. Would love to get your input on any of the above. Unfortunately we haven’t published too much stuff yet regarding what i’ve said, but hopefully in the next few weeks.

6 Likes

So is +0.25% not much different from 0 but it exists in the poll. Perhaps we could round all spreads to 0.5% or even 1%. I believe that small increases/decreases are not efficient at all at this DAI supply.

No, I mean MKR. When the income from SF is not enough to pay for all DSR - new MKR is minted and sold to keep the system afloat.

It’s not a risk free return. I don’t think it’s worth locking 150% collateral to get less than 0.25% profit a year but some people will probably try that.

Yeah i know, but the DSR seems to me to be about as good of starting place as anything else would be. Maybe you could come up with an index that incorporates the some yield numbers from other stablecoins? Coinbase is paying something for holding USDC iirc. Still might not be any more useful though

I do have some thoughts on jump risk. I assume what we are trying to do there is quantify the likelihood of some sort of black swan event and price that number in somehow. I could see how that would be a very useful number for overall VaR analysis, but I question how relevant it is to the stability fees charged for an asset.

Firstly, my intuition says that the premium we would end up charging for “jump risk” probably could be measured in the 10s of basis points if not single digits. Secondly, it seems like even if I could quantify the risk and decided the premium was significant enough to start charging for, the protocol doesn’t appear to have a great way of storing that value for a rainy day. Maybe you could assume that the MKR holders would take the extra money and set it aside, but it seems like we would need the facilities to earmark that premium for some sort of insurance fund. Maybe put it into some non-dai stablecoin? Maybe buy some off-chain asset like a 10y T-note?

Again this is only an opinion but is seems to me that the “best” risk management strategy as it pertains to black swan events in the crypto market is going to be bringing in off-chain assets as DAI collateral, but I’m sure that is going to be a fairly large effort that brings in a ton of questions concerning how to appropriately decentralize it or price in the counter-party risk.

For the time being (and probably for the next few years) the fate of DAI seems very intertwined with the fate of crypto generally. Meanwhile though MKR is trading at ~90 PE ratio (Ben Graham would not approve), so it seems to me that seems to me that we should be more interested in getting earnings up for the sake of the MKR holder.

Yes it does. MKR’s value is the insurance fund. In the event of a default, MKR is minted and sold to buy back any DAI that is not backed by locked value.

Absolutely, but BAT isn’t enough. I’m eagerly waiting to hear about onboarding more assets.

that’s what the surplus buffer is for :slight_smile:

In the medium term, there will also hopefully be ways to hold off-chain assets such as treasuries. Your intuition is spot on.

Regarding jump risks, currently they come out to be a super low charge, but that’s primarily because of just how overcollateralized the eth vaults are. When makerdao starts becoming more competitive, the liquidation ratio will fall, and the jump risks more pronounced, and thus the stability fee will be more distilled.

3 Likes

Hey guys still out there working on figuring out that expected return number up there. I have a new method drawn up but its gonna take me a little while to code up before I can be back with more numbers.

There have been some replies that i wanted to address while im working on that though.

Yeah i mean i guess. I think the idea of: storing dai to save dai/mkr in the event of the collapse of the dai system seems a little suspect. Also $500k seems like not be enough of a cash infusion to really keep us afloat in the event of a black swan happening. That said I’ve seen talks about the treasury happening around the forum. Im really intrigued by this idea of streaming money constantly to the treasury. Hopefully we can get some sort of a proof of concept up around that soon.

On that note:

I think with the kind of jump risk the foundation is trying to estimate it might be a little naive that such an event would not affect the MKR price as well. Think BTC < $100. We would be talking about a 2008 style financial crisis in the crypto markets. ETH price would plummet as well and likely so too would MKR.

This idea that jump risk will be more pronounced isn’t really tracking with me. Ok so if the default probability is main factor driving price of the SF and you accept my premise above that the CR of a CDP is some function of the LR it seems to me that lowering the LR is going to drive up default probability. If we are saying the jump risk premium ends up being “a super low charge” with the liquidation ratio set to 1.5 wont it just be more insignificant when the LR is ~1.01?

I don’t understand how you distinguish between the surplus buffer ($500k) and MKR’s value and “a treasury” (with or without streaming). As far as I can tell, all these are fungible forms of value storage.

Yeah, of course. That’s why we’re having a conversation about risk and conservative risk parameters. We have some backstop against black swan events, but it’s not infinite. And it will never be infinite. Risk management is the name of the game today and in the future.

1 Like

I don’t know how suspect it is, but I do think it’s a good idea to diversify the buffer outside of Dai. While Dai is the only thing you need to cover bad debt, it feels a bit weird for MakerDAO to hoard a chunk of the Dai supply. There have indeed been various proposals for some sort of treasury, and I think some people inside the Foundation are working on a plan as well (although I am not really involved with that initiative).

$500k is also certainly not enough. I was hoping we could put out some models to calculate more meaningful numbers before we hit that amount, but unfortunately it did not happen. As we scale, that surplus will need to go much, much higher.

Additionally, there are some technical concerns about the behavior of the surplus in regards to emergency shutdown, and we’re still working through how to remedy that. So for now, the surplus is just kind of up in the air.

The jump risks do affect MKR as well, of course, and we have built in a steep discount to the spot price as an input parameters (close to 80-90% IIRC). $500 mil market cap certainly doesn’t mean $500 mil of insurance.

The jump risk premium goes up as the LR falls, right? The SF is ultimately a function of losses. Lower LRs lead to more harmful jumps. Right now with a high LR, the jumps just aren’t large enough to trigger any losses (hence why we’ve never had an undercollateralized Vault). We’re more likely to see this with a lower LR. Maybe I’m misunderstanding your point.

5 Likes

As far as I’m aware in current setup surplus buffer should not be arbitrarily large because in case of emergency shutdown it goes to dai holders and that may screw incentives if it will be too large in comparition to dai supply.

But truth be told probably in an event of sudden market crash DAI buffer is probably better backstop against undercollateralisation than minting MKR and hoping there will be suffficient demand for them.

1 Like

Yes, Rune talked about it in ‘treasury topic’. I assume buffer size was sized for circa 100m DAI supply.
If we assume we don’t want to change the ratio, we could easily double to 1m if supply goes to 200m.

Just wanted to give an update since I’ve been silent for the past few days. I’m working on a program to get a better look at the debt that is backing dai so i can refine my previous answer to be a bit more reflective of reality. That said, as it turns out jumping into blockchain development is harder than i expected it to be, so progress has been slow.

Stay tuned for updates on that front. I’m hoping that soon i’ll be able to give my own closing thoughts on this topic.

In the meantime you can watch my progress and struggles here: https://github.com/tamccall/dai-vaults.

3 Likes

If You need any help with a blockchain part PM me, will be happy to help You. I do not know a lot about risk analysis, but if a challenge is to get data from MakerDAO smart contracts then I can easily help.

2 Likes

Ideally, you should not need to deal with intricacies of data retrieval from the blockchain.
I don’t feel confident enough about my understanding of data representation, but it seems you are dedicating a lot of effort on tasks not related to your core objective. Does anyone else feels this is significant barrier for attracting people interested in analysis and it’s worth exploring further?

1 Like

Hi guys, it’s been a bit since my last post and I wanted to give an update on my findings.

…So after calculating the expected return on debt (rD) number from before and finding that it seemed to approximate what we have been seeing in the executive votes i decided to refine my model a bit to hopefully reduce the difference between our calculated sf and the calculated actuals.

To do this I decided to not model each collateral type as one large cdp and instead find the weighted average rD for each individual CDP. You can find the source code of the program i made to do this here: https://github.com/tamccall/dai-vaults.

Looking at the CDPs individually did make a small difference for my rD calculation. Using the same CAPM parameters from before. I found that the rD_ETH to be ~14.7% and the rD_BAT to be ~9.04%.

Although, You can probably take those numbers with a grain of salt though as they are very dependent on how you determine the market risk premium for eth and bat respectively. As I said earlier between the time periods of Jun 1, 2017 - Jan 26 2020 ETH had an annualized return of about -10% that said if you look at ETH over the period of 2016 - 2020 you’ll come out with very different numbers, and of course all of these numbers are based on past returns which are not necessarily correlated with the future. A second criticism might be the usefulness of rD itself. Should a better metric be used to help inform stability fee votes? I’m unsure.

Ultimately having gone through this exercise I have come away with 2 main takeaways.

  1. Maker should probably consider making the underlying debt portfolio easier to access
  2. The way we vote on spread seems a bit backward to me.

First I’ll tackle what is probably the first of the most controversial of those two assertions. If you notice in both my models above to calculate an estimate for SF one of the first things that we need to know is the “risk free rate” (rf). Now, if you accept my premise that the rf in the crypto market is == DSR then it seems odd to me that we first decide on SF and then subtract a few basis points from there to determine DSR. Given a rD number in the model above along with the input parameters sans rf (K,V, etc). I would have a hard time arriving at rf that was used for calculation. It is much easier to calculate the former (rD) from the latter (rf). Likewise, it seems to me that maker voters should probably be setting stability fees based on the DSR and not the other way around.

My last point was touched on a bit earlier by @jernejml , but I’ll drive this home as my biggest take away from this exercise. Although it is public information, looking at the underlying debt portfolio of maker is certainly not a trivial task to do. I am a software engineer and a “money enthusiast,” so with a little help from @Adam_Skrodzki I was able to get the data I was looking for. For most people, however, I would imagine writing a javascript program to query the ether blockchain is just too much of a barrier of entry to cross. Could be keeping out people with more of an economics background from adding their input on the protocols risk. Perhaps the foundation might consider publishing certain datasets in easy to consume formats such that independent analysis need not be such an undertaking. I could imagine something like a weekly snapshot of the debt portfolio linked from makerdao.com.

Maybe I’ll put together a signaling request around the later at some point. That said, I may come back to this topic at some point in the future, but I think I’ve spent enough time trying to figure this out for now!

6 Likes