DAI Monitoring Dashboard

Hi All,

We presented our DAI risk monitoring dashboard at the weekly community call (https://www.monitor.simpleid.xyz/dai).

This is a tool requested by the risk team here: Wanted: Risk Domain Monitoring Tools

Would love to get the community feedback and thoughts on the information presented.

Thanks!

14 Likes

Niiiice! Great work

1 Like

Welcome to the forum Prabhaav, and thanks for coming on the community call today :grin:

3 Likes

This tool is amazing, maybe changing the DAI logo (it has the SAI one :sweat_smile:)

Thanks for create this !

2 Likes

Thank you! Fixed that icon :grin:

Oh damn this is nice

Are you able to add Compound?

Looks like they’re planning to by the side drop bar

2 Likes

This is very cool.

Not sure if this should be included or not but thinking here is to have in one place everything DAI.

I would add two catagories related to Maker vault use DAI minting, and burning. This basically is a monitor of where DAI comes from and where it ends up. Some of this stuff one is going to want as a graph over time.

Biggest problem here is the totality of information (which is good in one way, but bad in another). Looks like someone is going to have to think about how to present the information in a way that is more useful generally in an operational sense. A dashboard should be just that - something that on first glance tells me everything I need to know about something and on first glance it is difficult for me to get a global view without having to drill into each section. The data needs to be refined down into something that is a real dashboard (i.e. fits on a single screen vs. having to scroll) with the key data analytics that stand out (say a massive outflow from compound, into yearn or whatever), or large mint/burn events.

Other than the above the dashboard looks like a good start the key issues are going to be having events that are outside 2-3 sigma event bounds bubbling up to the top as those can signal key changes.

2 Likes

Thanks for the detailed feedback. We will be changing the tables into charts as you’ve noted to help grok the trends easier.

The massive inflow/outflow into other platforms should show up in the charts I mentioned above, but maybe we can highlight that in other ways.

What would you classify as a large mint/burn event over 100k dai, 1M dai, or something else? I can add a chart that indicates that information.

One other chart we’re working on is to show the monthly engagement of DAI users across the Eth universe.

Yep, as @Aaron_Bartsch noted, that’s in the works :raised_hands:

1 Like

Well UI/HMI for operational data is something I do daily but a good operational dashboard takes time. Really have to get your feed back from the actual users and I am not just talking people who glance at the dashboard, but people who are going to use it to carry out operational actions.

I simply glanced and felt while there was a lot of great information the presentation for a ‘dashboard’ was too busy - and I prefer lists to graphics. When I think about DAI in the Defi space I think about a pie and total outstanding, increase or decrease as one key metric (how much DAI is there, how much did it change in last 24Hrs) that is the first metric that determines the 100% mark. The second piece of the pie is how much as a % of the total moved from one cut of pie to another. What does the Pie look like (where is the lions share, and what does the ‘other’ total).

So when I think of DAI operations I am going to want to see a list of places DAI lives in order of % of total show nothing < 1% and no more than maybe 10-15 places. In the first column top show yesterdays total DAI and the %'s allocated to the places DAI lives for yesterday, and then second column showing Today and then +/- % change.

The following statement applies to all the catagories. 2-3 sigma is a data standard that can be used against all events in a catagory. Say DAI mints. If 95% of these are <1K we only care about these perhaps if the number of them in a day exceeded the 3 sigma threshold of the past 30-90 days of data. Same with burns. This also applies to sizes. If 3 sigma of all the last mints/burns is 100K then use that as the flag. The point here isn’t using a fixed value but using a statistical metric to define one value and then using another hard value to flag interesting events that are outside the ‘normal’ however you define this statistically or operationally. (this needs to be defined by users and the data generally)

When I think of operations dashboards, and being an operator of a machine/system, what I want to go into the background are things that are ‘normal’ and events that are ‘expected’ or within well defined statistical bounds. What should stand out are events that are outside of statistical bounds or as a magnitude often cases we would flag these with alarms that define various types of operational responses (alarm acknowledgement, cell phone mail/text, a possible predefined action - turn this down or off). Realize I am speaking from near approaching 20+ years of operational experience with an additional download of my friends 30+ years. We together know exactly what we want to see from operations dashboards, alarm handlers, UI/HMI screens because we are the ones that actually use them. Hence while I can make comments the real users are risk teams and in the end the drive for development should come from those who are going to use this information ‘for’ a purpose.

I honestly have no clue what ‘DAI engagement’ would look like. One idea you might want to include now that I think about this is the concept of moving DAI vs. static DAI. One important metric on top of DAI outstanding from yesterday to today and the pie 100% kind of display noted above would be to have another set of columns that show DAI moved in last 24hours and +/- % of outstanding in the same columns. I will try to do a spreadsheet mock up with wholly made up data based off of what you have up when I look and then you can see if this helps you get an idea of what to show.

The whole statistical measure of what is ‘abnormal’ vs. a value level is something that will require constant tweaking. I think it took us 2 years of normal operations tweaking settings for various alarms set/read mismatches and read only vs. active read only. (long story) Which then defined the events that should stand out in alarm handlers (or abnormal event reports).

Key goal in my mind on this dashboard is if there is a sudden shift in DAI either in activity (moving) vs. static we should see that immediately. The dashboard should also give us at a glance where this DAI moved from and to. If particular wallets stand out in terms of total activity (I’d say .1-1% or more of outstanding is probably something to watch).

Anway apologies for length. I simply don’t have time to edit this one down. I will see if I can generate a simple view of the rows/columns pie approach to a dashboard display based on what you are presenting and then you can try to make one - and we all can look and see what we think.

In the end I wanted to thank you for doing this work. It is something along with vault and liquidation activity analysis (as a kind of dashboard/report) I have been wanting to see for a long time and I can’t wait to see how this evolves.

Thank you again for doing the work, and listening.

Just a quick follow up as I was preparing to try to make a simpler dashboard. Looking at the data presented I am not entirely sure what data to pull forward as a display. Data binning by time vs. aggregated in total is not useful. More than 1/2 the outstanding DAI is outside of the top 500 wallets. I think one thing to look at is the number of wallets to get to 50%, 75, 85, 90, 95% of outstanding. Missing is aggregated DAI totals in the subpanels.

One thing I noticed is while it makes sense from a data gathering perspective to list all the contract addresses, I think it would be better to just list all of SUSHI contracts as ‘SUSHI’ etc. Breaking down the DAI dashboard by ‘protocol’, DEX, CEX, more useful than by wallet etc. So initial impression is the data needs to be aggregated better.

Some idea of how this data should be aggregated is going to take some looking at it, and some sophistication to make a dashboard that is useful. This takes a lot of effort and time and evolves based on what is available to be offered and more importantly what users need.

One thing I noticed was the sheer level of churn as a percentage of deposited assets in each protocol. This idea of DAI that is ‘static’ vs. ‘moving’ I think is an important metric and probably easier to find DAI that has been just sitting somewhere in a bin of time vs. trying to find all the moving DAI.

A few things I generally don’t like.
Having to mouse over and scroll or hit tabs on data.
Everything must fit in the display or be on some other page.

I think a csv export on the page data would be useful as I literally can’t deal with the data by having to click and menu bar slide around. Critically eliminate sums over entire time ranges and bin the data groups into days, weeks or months as a running set of data in a graph. It is hard or impossible for me to figure out what happened 3 months ago by taking the difference of the 1 month and 3 month data sets.

That is about all I have on this and trying to get down to a good dashboard would require iterative work both looking at the raw data and then various aggregations of that data from a ‘what can this tell me quickly’ kind of perspective.

Thanks for the feedback @MakerMan. I’ll follow up here when we release v2 of the dashboard.