Other Presentations and Updates
Derek Flossman
DsChief v1.2 Migration Walkthrough
50:50
- What I’m going to do is basically refer to the forum post. I’ll iterate a little bit of what Chris said earlier. I will walk through the top 4 steps of voting on a proposal to authorize the new chief and remove the permissions from the old chief. That’s the executive proposal that’s on the portal now. If you haven’t voted for it, please do so. Then once that spell is scheduled, we’ll have a 72-hour delay and office hours. Expect that to be Monday or Tuesday; that’s when the new chief gets the authority. The permissions from the old chief are removed, but it’s not yet activated. This will occur when you vote on the new chief. To do that, you have to withdraw from the old chief and deposit in the new chief and vote as you usually do.
- The process that I’ll walk you through now is this process of withdrawing, deposit, and vote.
- In the portal, you’ll see a box at the top of the screen on the executive page. You’ll be able to withdraw MKR to your wallet, and you’ll approve that.
- You will then see a box that shows you the ability to deposit either to use a hot wallet or a hot wallet with a cold wallet. In this instance, let’s look at the hot wallet only.
- So that’s number one. Click here to deposit directly into the new chief. In this case, I have 3.5 MKR, so I will deposit that in. once you confirm the transaction, you’ll see in the voting contract how much you have. The note will tell you, please vote on the new chief, which is a separate transaction.
- The executive proposal will appear here as it usually does. However, the second scenario uses the hot/cold wallet setup. There is option two on the list to create a voting proxy if you wanted to do that. This set of screens will be very familiar if you’ve done this before. It walks you by selecting your voting wallet, typically the hot wallet, which you choose from a drop-down list, followed by choosing a cold wallet. In my case, I was using a ledger. Click the wallet address, and here you’ll see the 3.5 MKR I’ve withdrawn from the old chief. Confirm those transactions and link the wallets together. Confirm the locking of the MKR in the proxy. Now you’re able to go back to start voting, which will return you to this screen and invite you to vote on the activate proposal.
- A few things as I was walking through this that may help people make sure everything works. Make sure the contract data is enabled on ledger devices. Make sure you have enough ETH in your wallet to confirm the transaction. When setting up the voting proxy, pay attention to which wallet is automatically selected as your hot and cold wallet. When doing a lot of testing, it was a thing that came up where I had to make sure I’m looking at the right wallet. Otherwise, you should be good to go.
- Quick reminder, please vote. I think we’re about 12000 short. I expect that we’ll get through it relatively soon, which will set us up for next week. We’d like to get through this phase relatively quickly because we also have executives lined up for the following Friday. Bear in mind that the final vote to activate the new chief also has a 48-hour delay on that spell being activated. We’ve included the reduction from 72 hours to 48 hours in the current executive on the portal.
- Chris: Yeah, so after we migrate over to launch the new spell, there’s a 48-hour delay on the launch?
- Derek: Yes
- Chris: So the launch can happen. We can start migrating over right away, and the launch can happen once we hit 80,000 MKR. We don’t have to wait on the GSM delay to launch the new chief. Then that executive on the 11th will be subject to the 48-hour delay.
- Derek: Yes, perfect. That was all from me. Ill put this up on the forum post I initially mentioned. Any questions? Let me know if anything is confusing. I’ll do a short video and walk through if people feel that would be helpful.
- LongForWisdom: Great, thank you, Derek. Sorry if you were a little rushed.
Nik Kunkel
Oracle Team Update
57:09
- This week’s update is a little bit different than the other week. I need to talk to you guys about something rather serious. Last Wednesday, between the hours of 22:46 UTC and 02:23 UTC, we experienced an oracle outage. During that time, none of the oracles were updated. The cause was tracked down to be related to a bug in Scuttlebutt, one of the dependencies that we use in the Oracle stack. Scuttlebutt is the kind of P2P “gossip network” that we use for feeds and relayers to “gossip their price messages around.”
- We’re not aware of any liquidations that were affected by this at this time. There are no liquidations that were triggered or were failed to trigger due to the oracle outage. Stepping into the root cause analysis, Scuttlebutt’s works are almost similar to a blockchain in that messages are hash-linked. You can think of it like how each block has a hash pointing to the previous block. Every time you post a message, you include a hash to the previous message. This means that it has the same kind of immutable properties as blockchains because you can’t go and edit a message from the past. The hash would change, and you can’t delete a message from the past because there would be a break in this kind of chain of hashes linking back to the first message. One of the property results is that every node and every peer on the network needs to have this state locally. The state is relatively uniform, so everyone, for the most part, has the same local state size. It essentially keeps growing over time as more messages gossip on the network.
- So, what really happened here? Well, there’s actually a hard-coded limit in Scuttlebutt due to an offset codec that it uses. This hard-coded limit is about 4.2 gigabytes. After that limit is reached, it’s not a graceful failure. Basically, the Scuttlebutt client will just seize up, and it will never restart.
- This was not something new to us. We’ve actually been aware of this issue going a long while back, as far as September 6th of 2019. we extensively tested the oracle v2 code base for about a year before multi collateral DAI even launched. This was one of the issues that we ran into and documented. We calculated how often we would encounter this message. The factors are that there are a certain number of feeds. There’s a certain number of collateral types. There’s also a certain frequency at which you have to update the prices of assets. Therefore with the feed and the number of collateral types, we projected far enough into the future that we wouldn’t exceed the limits of those models. We saw that the frequency of price updates is actually much, much bigger than during our testing phase. We were led to believe that we had about 18 months to migrate to a new Scuttlebutt network with the original calculations before we would hit this issue. Nonetheless, as it stands, we’re earlier than 18 months. MC multi-collateral DAI just turned one a few weeks ago. We hit this threshold much, much quicker than we had initially anticipated.
- what are we doing about it? Since we launched the Maker oracle protocol, our goal has just been de-risking the oracles across the board. The aim is that we have a hundred percent uptime, and we want to be redundant on different levels. On the one hand, we want to be infrastructure redundant so that no one piece of infrastructure can go down that will affect us. On the other hand, we want to have implementation redundancy, so there’s no one tool that we rely on where a bug in the tool affects every person on the network that we don’t experience any downtime from that. We call this infrastructure redundancy and implementation redundancy. Regarding implementation redundancy, we’ve been quite busy working on it for the past year. This includes coming up with multiple price tools. Instead of just using a single price tool called Setzer, we have another price tool called Gofer. The idea is that you would run both in parallel. If one doesn’t work, the node and the feed would just flip over and use the other. This is on the pricing tool side. One of the things that we’re working on right now is the transport layer side. The transport layer is this network where price messages can be transmitted and distributed. Scuttlebutt right now is the only transport layer. But we’ve been working for the past quarter and a half on integrating lib p2p as another transport layer that would be in parallel. What’s frustrating about this is that had this event happen, if we were to hit this state size limit, say two months in the future, it wouldn’t have affected us in the slightest. We would have already had lib p2p integrated. In terms of mitigations, we’ve already been working on this for quite some time. It’s actually quite frustrating the timing of this how it happened.
- LongForWisdom: Sorry to interrupt, but we need to move on soon, so try and cover your last points quickly.
- Nick: I’ll just wrap up really quick.
- I’m just going to skim through a swift timeline. All of this will be posted in a forum post that the oracle domain team is coming up with, so you’ll get all the nitty-gritty details. Essentially, we found out about the issue when a community member notified us that they were having an issue with one of their relays. We quickly identified the cause as the known Scuttlebutt issue. We immediately started reaching out to feeds to assess the situation. After we figured out that this was going to be an issue that was critical and quite time-sensitive and that it could lead to a loss of quorum between the oracles, we decided we needed to execute an emergency migration. This entailed setting up a new Scuttlebutt network and getting everyone all the feeds and relayers migrated over. The goal was that we needed 13 feeds because the quorum for the oracle is 13, so we needed 13 of the feeds to migrate over as quickly as possible. We alerted all the customers of the oracles of the impending downtime. The oracles were not down yet, but we knew based on our calculations of the state size growing that they would be within an hour. We notified them ahead of time of the impending outage so they could take precautions on their end. The oracle domain team drafted together some documentation and tested it for release to the feeds. Then the feed started slowly coming online. As I said earlier, it took us three and a half hours to hit the required quorum level of 13. Afterward, the oracles started updating again. After that, over the next 12 hours or so, we filled out the rest of the feeds and got them on board. That’s it on my end for the event’s description, happy to answer more questions in the chat or on RocketChat later. We’ll have the post mortem out shortly in the forum.
SamM
PSM Implementation Update
01:09:34
- Last week, the vote to accelerate the PSM launch passed. As per the vote instructions, I set up a post that I mentioned in the chat to track the PSM launch progress. There’s sort of 4 key things that need to be done.
- Smart contract audits; I reached out to a few community members. Some have gotten back to me, and I’m waiting on others to go over the contract to make sure it’s all good and do some more informal audits. I’m also in discussions to get a more formal audit in place, and more on that to come.
- The second piece is risk analysis. We should have that next week so stay tuned for that.
- I’ve deployed the contracts to the Kovan. I’m just waiting on the Kovan deployment process to merge that in, and then we’ll have it on Kovan.
- The final step is deployment to the mainnet. Any questions, feel free to ask me on rocket chat
- LongForWisdom: Thank you, Sam.
Lucas
Open Discussion over the prioritization of Real-World Assets
01:11:12
- Lucas: I started a forum thread outlining what’s happening with real-world assets. I just want to give a brief timeline to get everyone in the same picture. Some of the first world real assets and MIPs were accepted or voted in sometime in May. Since then, we’ve been working. This was not even the beginning of this initiative of bringing real-world assets into Maker. Since then, we have had continuous discussions about collateral onboarding calls. We’ve started with a real-world assets working group that began assessing the risk. I’m super happy to see that we now have the first real-world asset going live sometime soon with the risk assessment and all the other domain work being done. Although there are others and for which New Silvers’ risk assessment has been out since Tuesday, I believe. ConsolFreight is hopefully to follow soon. Centrifuge is doing a lot of the leg work necessary to bring those on. In that work, we actually did a few things. We proposed a sub proposal for MakerDAO to declare the intent to onboard Centrifuge assets according to the framework we’ve explicitly outlined with the MIP13csp5. MIP22 also outlined liquidations and how Maker is to integrate with Centrifuge based assets. Both of those were posted on the forums in September and approved by community vote. It was then ratified, and we’ve now been working on it. At the same time, we’ve been getting mixed signals from domain teams and the Maker community concerning these assets’ priority. Centrifuge has brought a lot to the first world assets; we have New Silver now ready to start originating the first loans the minute the spell gets executed. They are already looking at assets that fit within the bounds that the Risk Team has agreed upon. We started working on a spell and have been working with the community. Nevertheless, we’ve had this apparent signal request that ended with a vote on-chain that was very heavily in favor of prioritizing real-world assets. But this is still taking much longer than we are hoping to. We’re doing everything that we can to get this ready into one of the following executive spells. I wanted to bring this up as a topic to discuss in general. One of the examples right now on the forum is that it seems like renBTC and a bunch of other coins is ranked higher than these assets that we have that are ready to diversify their collateral base to generate a significant amount of DAI and generate a significant amount of liquidity in the system are not seeing the attention that I believe they should get.
- My question to the domains team is, what could we do to speed these things up? I think the Maker community has often indicated that Centrifuge assets and real-world assets are something that we want to see in production.
- Chris: The domain teams are all feeding back into that collateral priority sheet that I linked in the sidebar. We’ve got several collaterals that the community has greenlit. There are several different teams. There’s the sort of greenlit score itself. I don’t know if Governance has some feedback, but we give it a gut check on each of these metrics. Then that rolls up into a score. One thing that we can do in the sheet to prioritize real-world assets, we could ask if this is a real-world asset or not? If it is, we could add additional weight to that column, which might adjust the score and move real-world assets above some of the other things on the list. That would be where I would kick the discussion off. What we should do is funnel all assets into the priority sheet. If it’s done well and we iterate on it, we should see the most bang for the buck for the protocol. Suppose we believe that we should diversify into real-world assets as that community poll had suggested. In that case, we certainly could add an additional rating score that could bump all real-world assets up the list a little bit more. That’s where I would start.
- Lev: What’s actually still required from the domain teams to get MIP22 onto a collateral onboarding poll similar to the 6S one that has just finished?
- Chris: What we need are domain team assessments. We would start with New Silver, and we need a domain team assessment from smart contracts and oracles. So we need those two assessments. I’m working with Lucas to see if we could review that, but it’s not just as easy as grabbing an ERC20 and knowing what the stack looks like. There’s certainly more involvement, and we don’t really know what we might find when we dig into it. The areas that Lucas and I have already talked about would be similar to MIP21; what we do in an emergency shutdown and make sure the code can handle an emergency shutdown and be able to yank tokens. In MIP21, we sort of wave hands, and liquidations happen via the trust. Still, liquidations are actually taking place on-chain using the Centrifuge contracts. I have not dug into that code yet, so we still have to dig in and ensure that liquidations are safe. I don’t know if the Risk Team has evaluated that. The same thing can be said about oracles. We get a standard oracle price from these audits that the rep does. For MIP21 and for MIP22, they’re sort of providing their own pricing. Therefore, we’re bypassing our oracles, so we’re getting both pricing and liquidation from Centrifuge contracts. that need some sort of heavy screening to make sure that’s correct.
- Nkunkel: It’s not that it needs scrutiny or that it’s correct. I think it’s incentive misalignment. I don’t see how you can have an issue with an asset having the ability to change the price on the fly. The Maker protocol is taking on a considerable amount of risk. I don’t really think that there’s no oracle here. There needs to be a layer in between whatever the Centrifuge price is and the price the system is using.
- Lucas: Can I address this point. Centrifuge and I are not the issuers. We are a team of people who have been working for more than 2 years on addressing this issue of how we allow issuers like New Silver to do this. I’m standing here asking and outlining the process and everything we’re doing to make this safely. I’m not asking for a 200 million dollar debt ceiling to retire in the Bahamas. We’re talking about slowly ramping up these assets with a scope that allows these asset originators to get the first experience for us to prove out this can be done. That this is something that the DAO can do. In the last 1.5 years, we haven’t been able to show this. We want to grow this ramp, right? Seb, Will, and Phil have worked over the last couple of weeks tirelessly assessing these businesses. They are assessing what’s going on. Are you insinuating that we’re doing something here that will lead to them walking away with the 4 million or 5 million dollars proposed as a debt ceiling for New Silver? Are you worried that this is something New Silver will do when their entire business and reputation are on the line?
- Nkunkel: I don’t think we should frame it like that. We have crypto where we can build trustless systems. We don’t actually have to trust anyone. Trying to fall back on those weaker guarantees is not something that’s really appealing to me. There is some oracle risk there. And when I say some, I mean significant oracle risk. I would like there to be some layer of obstruction in between that gives the system some protection to dampen that risk.
- Sebastian: In response to the price that is given from Centrifuge, that is something that we have discussed with Lucas. We might want to set the price, and the risk team decided that we wanted another price. If there’s an underlying collateral ratio, which is screened manually by the risk team. Suppose the pool assets go below this ratio. In that case, we’ll ask for the pool to be liquidated, thereby basically asking for drop prevention. We need to be sure that the pool doesn’t borrow too much DAI on the system. It’s actually a bit different from 6S. There is no price at all, and everything is on-chain. Centrifuge is more on-chain than 6S. However, I agree that it’s more complicated to integrate.
- Lucas: The risk team looked at the credit risk, businesses, and whether giving them 5 million DAI is a reasonable thing to do. We are talking about a price feed that the risk team has already looked at. Is this a lender that we can safely give 5 million provided that they spend it on the right stuff? There is a process in place to do this. This is not a collateral type where anyone in the world can open a vault and try to manipulate a price feed and profit by tricking Maker into minting DAI that wasn’t supposed to be minted. We’re talking about New Silver having a process to originate loans and how they agreed on with the real world assets working group. I reached out to the oracles domain team at least 5 times in the past 6 months explaining how we’re doing this and showing how we could address this. I don’t think this was a lack of trying. We still need to do this, yet we stand here suddenly saying, oh, this is not ready. The community said real-world assets should be prioritized.
- Nkunkel: Yeah, my team, in particular, is working with a limited number of resources. We have this constant collateral onboarding that goes on every month. It takes up a considerable amount of our time. We have the community telling us we want to do LP tokens. We’ve been working on that.
- Lucas: I think the community said they want to do real-world asset tokens and not LP tokens. I’m happy to put out a signal request about this, but that’s bullshit.
- Lev: We’ve already had a collateral prioritization signal request for real-world assets. It was pretty decisive.
- Nkunkel: So we did. We recently finished the oracle assessments, risk assessments, and technical assessments.
- Lucas: Everyone in the Maker community said that real-world assets are getting a special fast track outside of the normal onboarding process because they’re new and special. The community said we should prioritize them. I don’t think that saying we have one is an excuse for saying that Centrifuge shouldn’t get its work done.
- Nkunkel: That’s not the point I’m trying to argue at all.
- Lucas: So when is it getting done?
- Nkunkel: I don’t want to give you a date and then lapse on that date. I just don’t know. We have spoken in the past, but The problem is that is was too wishy-washy. There was nothing to really look at. At some point, we said a third party would be validating the price and giving us the price. Then we would query the price from the third party. This doesn’t seem to be the case anymore.
- Lucas: Risk has assessed this without external oracles. No external oracles were promised in this risk assessment.
- Nkunkel: That’s just from the risk side. Whether you said that I’m pulling the price from someone else or from you, there’s still an oracle. The oracle just happens to be you.
- Lucas: There is no off-chain oracle feeding in the price. We look at how much has been borrowed and make sure that the value of the collateral within Maker will get repaid. There’s no oracle here.
- Nkunkel: You provide a value that extends the amount of credit that someone can get against the token. Which formula you use on your side to determine this doesn’t matter. You have the ability to set that price.
- Lucas: Up to 5 million DAI. We do have that ability. The New Silver can say how much their assets are worth. The real-world assets working group and risk team monitor what they say and how they’re providing info. In a way, that is very similar to what 6s is doing. We have an entire process here, and in the end, if this value is assessed to be not correct, liquidation will be triggered. we have a process in place that will get Maker the DAI back specifically, which is MIP22.
- Sebastian: For the sake of getting a solution for the drop token, the price is not that difficult. It’s a constant price plus the accrued interest. Maybe the accrued interest is too complex to compute, but it starts with a price, say maybe $100.
- Nkunkel: It’s not about what formula you’re using to price it. I’m sure that’s all fine. It is from a risk perspective that the issuer has the power to set the price, and that, to me, is a huge conflict of incentives. There needs to be some abstraction between the issuer and the protocol that protects the protocol from this. This is a misalignment of incentives.
- Sebastian: If you can use a constant, it would be a bit of a loss for Centrifuge because they will not be able to borrow as much. We are speaking about 8% per year. It doesn’t change the world. Getting a constant is much easier, I guess.
- Juan: These are two different topics that we should discuss separately; oracle risk and prioritization. I think that Nik’s job, before anything else, are to protect the protocol. I believe that Nick’s concerns are entirely valid, and we should work towards mediating any type of risk. Protecting the protocol should be the first priority for anyone working for the protocol, even if the community wants to prioritize real-world assets. I think Chris said in the chat, it’s not that we’re rejecting the proposal. It’s just a matter of prioritization and bandwidth that each team has. I think there has been quite a lot of progress, especially since Seb started guiding the real world assets. It’s easier to achieve in the shorter term for scaling the protocol and making sure the Maker holders will have value. That’s what we’re looking for at the end of the day. I don’t think that we should mix both topics. One of the solutions was what external people can do to help speed things up. We mentioned for us, it’s much easier to review things than to create them from scratch. Any project that can already give us reports is always much easier than starting from scratch and doing all the heavy lifting. That’s one of the solutions that we’re proposing. Hopefully, we can see this happening because I think that’s something that everyone wants. We shouldn’t mix these topics up.
- Sebastian: Just to come back to the collateral prioritization sheet. One issue was that real-world assets were penalized for good reasons because they are a bit riskier. However, the collateral prioritization sheet didn’t consider the fact that the community and governance voted on making them a priority. I think the issue is on this point specifically. There was a senior request to make it a priority. On another side, there was a collateral prioritization sheet that didn’t include the senior request event. Therefore, it was two separate pieces of information, and everyone could merge this information differently.
- LongForWisdom: I’d like to make a quick point here if I can. The prioritization of real world assets like that happened in the past. As a result, the domain teams prioritized a real world asset to work on, 6S. 6S isn’t at the top of the collateral onboarding framework. That was picked explicitly because the community voted for onboarding real-world onboard assets. Whether that was a mismatch of expectations that the community expected the domain teams to drop everything and work on all of the real-world assets in exchange for the domain teams interpretation to get at least one real-world asset going and continue to do the things that we’re currently doing with the other crypto assets.
- Nkunkel: I’m not rejecting the Centrifuge collateral onboarding proposals in any way; I’m not saying no. I’m saying it needs work, just like everything else needs work. That’s the issue that we’re dealing with. It’s not that we’re choosing to ignore this. Everything is a priority, and there’s only so much that our teams can handle at once. It looks like right now, I’m the bottleneck here because the risk side has been completed, and it seems like the smart contracts side is a little farther along than I am. Going into the New Year, I will prioritize real-world assets more. It’s not a question of this doesn’t work, it’s this has a certain impact on the system that doesn’t look favorable and so we need to retool how that works. So Seb you mentioned that we could use a constant but that would affect the issuer, Centrifuge, negatively because they dont get to use the value accrual to improve their collateralization or to extend their credit so what if there was a tradeoff in between so for right now for stablecoins we use a smart contract called the dsvalue and the owner of this dsvalue can poke it with a value and that’s the value that gets set. What if instead of having this value controlled by the issuer we have the value controlled by Maker Governance and we commit Governance to once a quarter including in an executive to update the price according to the formula that risk uses or I think we recently passed instant access modules MIP, the risk team could be trusted to have an instant access module to change the price that’s in the dsvalue contract, either of those solutions are much more pallatable to me then having the issuer be able to control the price at will, because even though in the latter scenario you’re trusting the risk team it’s not as much of a incentive misalignment by trusting the risk team then by trusting the issuer of the token. That’s my only concern here.
- Sebastian: I fully understand you because when we started the risk assessment, we saw that the price of the pool should come from the formula, which maybe could be good or not, but we decided that it was not the work of the asset originator or the technical provider to set the price of the asset, you should do it on your side and what you suggest is just fine.
- LongForWisdom: Let’s wrap up the call here.
- Lucas: Nik, would you be able to stay on a little bit longer?
- Nkunkel: I have at least until the top of the hour
- LongForWisdom: We’ll do the wrap-up stuff then. Going into the holidays, we won’t have calls during the weeks of the 21st and 28th.
Links from Chat
Common Abbreviated Terms
MCD: The Multi-Collateral Dai system
CR: Collateralization Ratio
DC: Debt Ceiling
ES: Emergency Shutdown
EV: Executive Vote
GF: Governance Facilitator
GP: Governance Poll
SF: Stability Fee
DSR: Dai Savings Rate
MIP: Maker Improvement Proposal
OSM: Oracle Security Module
LR: Liquidation Ratio
RP: Risk Premium
RWA: Real-World Asset
Credits
- Anna Alexa K Produced this summary.
- Artem Gordon produced this summary.
- David Utrobin produced this summary.
- Denis Mitchell produced this summary.
- Jose Ferrari produced this summary.
- Everyone who spoke and presented on the call, listed in the headers.
