My talk on Modeling Systemic Threat Using Mainnet Fork.
And so, next one, who am I? I'm Martinet Li, a Research Engineer
slash Security Auditor from QuantStamp. And since last year, I reviewed a lot of projects,
especially in the DeFi space, as this particular space is booming.
But I was also involved in some reviews on, you know, base layer chains.
Like Eth2, Prism Implementation, and AvaLabs as well.
And the girl beside me, that's my daughter, just by the way.
And next one. So, what this presentation is about,
I kind of want to be more like a light talk and perhaps interactive,
so feel free to just shoot me any questions in between.
But basically, what I wanted to share today is kind of the status quo on testing.
From an auditor's point of view, and mainly focusing on composability.
As projects integrate with others more closely,
how are people right now testing their projects? And what I believe would be beneficial.
I will also introduce my personal project on trying to improve testing
with two simple examples here. So, next one.
So, the main challenge here is the composability.
So, what is composability? It is a feature of design wherein
the various components of a system can be easily connected
to form any number of satisfying results. That's from MakerDAO.
And although blockchain projects have always been composable,
you know, like it's always open API, right?
Its power was really unleashed by the DeFi space,
and especially the standardization of these token interfaces like ERC-20, ERC-721.
In the beginning, we see most of projects focusing just building on their own ecosystem,
pretty much independent from other projects.
They issue these tokens just hoping other to use, maybe to trade.
You know, but then with the use of these standardized interfaces,
it allows like other projects to leverage like what,
well, it allows projects to leverage one another, right?
Kind of like Lego. So, that's the term money Lego.
And easily connecting them together to create like so many different kinds of functionalities.
Of course, exchanges are the first one, you know,
just aggregates just using the interface and then sell them.
They're just using the token, not really the internal functionality, right?
Another piece which is important to this composability,
which not necessarily fits into like this money Lego thing is the Oracle part.
Oracle feeds us like maybe real world data and then allows us to use them on chain.
So, it is also like an important piece in this composability, I would say.
So, next one.
So, composability is certainly a blessing.
So, we see that there are so many interesting projects
in the past few years that emerges from here, right?
Pulled together as one example,
it builds like no loss lottery on top of the compound protocol, right?
So, as time goes by, like projects tend to grow more complex.
One of the prime example recently would be the Yearn protocol, right?
That helps user yield farming by putting together like Curve, AVI, Compound, MStable,
and a lot of other protocols, right?
I would say it's one of the amazing projects that full of creativity
and demonstrate what can be done with this composability.
Now, next.
However, this composability is also a curse, right?
So, we have already witnessed several attacks and one systematic failure
that affected the whole DeFi community.
And I'll just like bring up a simple example here, which is the first BZX attack,
which involves a DYDX, Compound, BZX, Kyber, and Uniswap, right?
It uses the DYDX, a flagship ability, and then borrow WBTC from Compound,
uses BZX as a platform to short ETH in favor of WBTC.
And in turn, because internally like Kyber and Uniswap was used in the BZX,
it drove up the price of WBTC and Uniswap.
And then the hacker was able to just dump the WBTC and profit from there.
So, you see like this complexity is also opens a lot of like,
you know, different kinds of attack. That's for one.
And next.
And we also see like in the black Thursday, right?
Not sure if the graph is visible, but we probably all know,
they grew up for 30%.
At the same time, so MakerDAO price did not update for a while.
So they do have like Oracle.
But during that time, the gas price was extremely high.
And thus the price was not able to update in real time.
And when it updated, it triggered like a massive liquidation.
In turn, this is one of the reasons why like,
there are so many collateral were auctioned off for zero.
So, part of this is because of the Oracle issue here.
So, next.
So, I would say composability is almost like a nightmare for developers.
Repeatedly see this issue, like causes a lot of troubles for developers,
especially when it comes to testing the project, right?
Like, how should the developer ensure the project
that they interface with are working as expected?
And then how, like, this is already hard, right?
And so, I don't really see anyone like testing extreme scenarios,
like the ones that we would mention above.
So, let's go on to the next.
So, the status quo on testing, kind of doing a review.
What we see as auditors, typically, you know,
if they have tests, they have unit tests,
they have functional tests, like for internal protocol.
But when it comes to testing the integration points with other projects,
what we see, you know, some variety there.
For the simple ones, if the external project is really simple,
then you can deploy the external projects locally, you know, just in the ganache.
This is possible when it's small, right?
But it's impossible if the external project is complicated.
Just imagine, like, you're deploying the whole MakerDAO system.
Well, that's going to be a nightmare.
Next.
And so, there's the second option, which is mocking the external protocol, right?
A developer has to understand what the external contract would do
and mock the behavior accordingly.
Oftentimes, developers will simplify these logic to, you know,
to not really implement everything in the protocol, but just the behavior.
And so, there are possible discrepancies between the mocked contract and the real behavior.
And that actually presents a potential risk.
Now, next on, we have testing on testnet, right?
If you're interfacing with just one project, it kind of works.
Although, it's still tedious to get all the test tokens, testnet tokens, and, like, Ether.
You need the faucet for that.
But if you're building a project that interfaces with different projects,
then you can see on the right-hand side, like, this is how, you know,
projects on the testnet would look like.
Like, MakerDAO, they probably have their own testnet.
But for sure, I know Aave and Compound, they use, like, different dymocks on the testnet.
And so, if you're testing, if you are writing a project
that interfaces with both Aave and Compound and want to use dye in between,
this is simply different, you know, from the structure in mainnet.
Now, next.
Ah, well, here we have testing on mainnet.
And needless to say, this is super costly.
Typically, it only allows functional and high-level manual testing, you know.
And so, among all these, I would say the most common ones are mocking.
But testing on testnet do exist, but typically done, like, not very extensively.
Because developers, like, oftentimes find, you know,
fetching those resources very, very troublesome, right?
And so, I have personally seen projects that did both the mock and testing on testnet.
So that's, you know, they are, you can see that they're trying their best,
but still missed to test the profit sharing.
Because that requires some understanding of the underlying protocol
and how they would behave.
So they missed that in the mock.
And then on the testnet, because it was kind of like one-shot testing,
it doesn't really wait for the time and wait for the profit to accrue.
So, like, they missed the profit sharing part.
And we were able to locate that issue, luckily.
After that, I sort of see this as this lack of testing facility
that helps, you know, testing, like, composability thing
is a huge security issue for the whole ecosystem.
Next.
And so, with the standard tool, actually, I do,
there are one option that I would say it's quite nice.
I do recommend, like, testing under this mainnet fork.
And Ganache provides this fork option,
which, surprisingly, I haven't seen any project uses this to test external library.
And that's probably because of my side note right there.
The fork does have some bug.
You know, when you use Uniswap, you probably,
you'll face some weird re-entrancy error, like reverts,
but you never did a re-entrancy there.
So, but there is a pull request, but I don't think a lot of people know that.
I think that's probably one of the reasons that this is not used.
But if this is used and fixed, then I see this is actually pretty nice.
Projects interconnect with each other the same way as mainnet,
because it's the mainnet fork.
There's, of course, no cost, because it's a local Ganache.
And then huge amount of ethers when starting up the local blockchain, as always.
And, you know, Ganache also allows you to sign transaction as other accounts.
So you can be the whale, you know, anytime.
So this is already pretty nice.
Next.
So with this, we can already start to model a systemic threat, which is the whale, right?
For example, to be a die whale here, well, you can use an unlock account.
You find some die whale on the mainnet and, you know, just write his address down,
unlock his address on your local Ganache and use that to test.
So that's good.
The downside is that, well, if the whale just transfers his die to other places,
then, you know, this test doesn't work anymore.
You need to find another die whale.
So, you know, not really reliable as test, I would say.
The other two paths that you can do is that you can, of course, mint die on MakerDAO.
However, that requires you to, you know, you already have that huge amount of ethers,
so you can definitely do that, right?
But you have to know how to open vault, lock up collateral and use that to mint die.
But in the end, what you really want is just increasing your balance in the die.
You know, it's pretty tedious and going through all that.
And same for buying die on DEX.
If you want to do that, you need to know Uniswap's interface, perhaps, and how to interact with them.
So it's not as convenient.
So, next.
Yeah, and still, like, some risk remains very hard to model.
Like, if modeling a risk involves changing the price feed on MakerDAO or Uniswap v2,
that may be a problem.
And then, let's say, if you want to model what would happen on your platform or your smart
contract when you experience an oracle attack, like, for example, you're using Uniswap and
you're also using Chainlink as your oracle.
What would happen if there is a mismatch on my protocol, right?
Can I model that?
And then, you know, seeing the BZX attack, that's pretty sophisticated.
It might be very hard to come up with that sort of scenarios.
And so, one thing that you might want to do is just assume that there could be a mismatch in
the oracle price, then what would happen, right?
That's much easier.
But in this mainnet fork, you cannot do that because you still have to follow all the blockchain
rules, right?
And in general, if modeling requires, like, changing a variable that is not directly settable
by an externally owned account, this is challenging.
So, the question, next one, is what if we could ignore code as law, you know, on the local
blockchain?
And what if, like, can we change the storage directly without any permission?
Next one.
So, I kind of went on this direction and digged into the ganache because since this is the
testing environment that most of the people are using, I decided to dig into this one and
see if this can be accomplished.
And so, ganache itself, it kind of looks like this, you know, like the related part.
It has WS server, HTTP server, and it connects to this get API module that it has internally.
This get API module controls the state manager, and it's supported by a VM, which is the
Ethereum JS VM.
So, next.
And so, within this ganache, this state manager, you know, within this VM, then I found that
in the VM state manager, there's actually a free function that is very useful for us, which is
the put account, put contract code, put contract storage.
By using these, essentially, you can directly modify the storage and the state of the local
ganache.
So, what typically was done is that you still go through ganache and send transaction to ganache.
So, what I tried to do was to open up a backdoor, if you may, and say that, hey, I don't want
to verify my transaction.
I just directly want to modify my storage right there.
So, that's the root of my project.
And so, next one.
So, after some development, like, this is my project, which I call Godmode for Test.
It basically consists of three parts.
The project itself is still in, like, very early stage and still needs a lot of work.
It already works, and I personally use it for development, especially when it's come to
DeFi projects.
So, what I built consists of three pieces.
The first one is this ganache core.
So, this modified ganache core allows us to ignore the blockchain log and just directly
modify the storage, as previously stated.
And then, on top of that, I created a wrapper JavaScript library that you can integrate
into the Truffle framework.
So, you know, all the familiar tools that we use and leverage the Solidity contract as
sort of an API.
I think it's easier to understand how to modify the storage of the blockchain.
And there's some set of precompiled contracts.
So, what the precompiled contracts would do...
next, please...
is, you know, I want to provide some easy-to-set-up scenarios for the developers.
For example, to mint Dai, you don't have to think about finding that whale and, you know,
worry about the whale will transfer it away and the test won't work anymore, or worry
about all the vault, you know, all the original mint Dai process.
Now, what you need to do is just call this function call and then it will give Bob, you
know, 10,000 Dai, which is great.
Next.
And in a more general sense...
So, this is a general API that I provide that you can basically craft...
use this to modify, like, arbitrary contract storage.
And so, it might not be very clear just by seeing this.
And so, I'll demonstrate a few examples.
I'm happy that I have had the backup of, like, kind of all the screenshots.
And so, yeah, we'll...
I kind of...
I will kind of walk you through, like, all the screenshots on doing these things.
So, yeah.
So, next one.
So, we'll go through, like, the first example, which is using that general API and use it
to mint Dai, you know, like, the one that I already had in the library.
But right now, I just wanted to show you how this could be done.
And so, the first step would be you go and find Dai's source code and mainnet address.
And, of course, both are available on Etherscan.
So, you go there, copy it down.
I'm assuming that we're using, like, Truffle project framework here.
So, we just store the Dai source code under the contracts folder.
Next one.
And so, the following step is we clone that Dai contract, you know, rename it.
So, now we would have, like, let's say two contracts.
One is Dai.
Let's say the other one is called gmDai.
And we modify the mint function.
So, how we modify it, it's really simple.
The reason that we cannot mint Dai is it was authorized.
Like, it is controlled.
There's a requirement statement that prevents us from going in, which is this modifier auth, right?
So, what we can do is just to remove that auth.
And that allows anyone to call this mint.
So, you know, that's essentially how it's done.
Next.
And so, in the normal test, and this is how we would write it.
So, we have, we would require both artifacts, include both artifacts into the test,
which is the Dai and gmDai.
So, you recognize gmDai is the one that we just modified and removed the modifier, right?
And, you know, and like all other stuff are pretty much just set up, basic setup,
like, you know, talking to the Godmode Ganache and all that.
But the real essential part is the part that I, you know, had with the red square or rectangle,
whatever, that's the executeAs.
So, what you do here is actually you're executing the Dai contract.
So, you see this Dai contract is just like two lines above.
It's, it was like a way to Dai at this address.
And that's the address of the main net Dai address, right?
And so, I execute this Dai contract as if it is the modified contract.
So, I'm actually just executing, you can think of it that this way.
So, I am modifying the storage of Dai contract.
But according to the rules set by this gmDai artifact or contract.
And so, here, essentially, I'm just calling mint from the gmDai contract.
Then I say, give Alice this amount of Dai.
And the next line is assertion.
And so, this is, yeah, this is basically how it works.
Next one.
Now, so, that's the, that's just minting Dai.
And so, as a second example, I'll show you how to modify to make a DAO price feed.
Now, this is an essential step for modeling this systemic threat, like, you know,
oracle attacks and all of that.
Because what you want to do is to have, like, maybe two oracles report different values, right?
So, you want to read and perhaps change to make one of the oracles price feed.
Here, I chose my DAO.
And, of course, you just Google and find out where the maker DAO is getting their price.
And so, this is the address that I found on their page.
And then you just go to that ether scan the same way we did as Dai.
And then copy the source code.
Next one.
Yeah, and, you know, this is still a little bit technical.
You have to understand kind of how it works.
But luckily, most of the oracles are pretty simple.
They're basically reporting what they have.
Maybe they have something else.
Like, in this maker DAO price feed, this OSM module is their oracle.
I think it's oracle security module or something like that.
So, OSM module basically has an example.
Hey, OSM peak is getting the current value.
Or OSM read is getting the current value or fail.
And so, you go and look at the source code of OSM.
And you find out that, ah, it is actually returning the value from this curr.val, right?
So, this is where the current value is storing.
And so, what we want to do is basically just read this one.
And, you know, and perhaps modify it.
And so, the reason I say read is that if you later take a look at the OSM contract,
you see, well, you see right here, actually.
There's a modifier called tall.
And that prevents any user, like, external contracts from using, from reading the value here.
But if we want to just use, just inspect this value, we can actually do that.
But, okay.
So, next one.
And so, yeah, kind of same way we did on the die to GM die.
So, we modify the clone OSM module, add these two functions and event.
The reason I do event is that, unfortunately, right now, it doesn't read.
So, this execute as cannot just read a function.
You know, like, it will always do a state change.
So, if you do a view function, then it will revert on that.
So, this is kind of a workaround.
But nevertheless, you can still get the value this way.
And then, the next one is pretty simple, which is just setting that current value.
Right?
So, set curve val and curve dot val equals to the new value that we have.
Next one.
And so, with this, you know, we can just write test to check the behavior.
Similar way as we would say that this OSM contract is the OSM at, you know,
the original address that this contract were placed.
And then, the next two lines, this transaction, weird transaction thing is reading from the logs.
Basically, reading from the events.
And then, reading the current value out of that event.
So, in the middle, there's this await got node execute.
So, here, again, it's the same thing.
We're executing this OSM contract as our modified contract.
And now, you see that this set curve val is a function that we added.
It's not anywhere in the original contract.
And we just added that part.
But now, this modified ganache will be able to run this thing.
And then, it will just set the curve that val into this new function.
Oh, sorry, value.
And I just printed out, print the value again to see what's there.
Next one.
So, now, yeah, this is kind of the result.
You'll see that the original value is this, basically, 387 USD, right?
And then, I just set it to some number that I just thought set there.
So, now, you are able to change the make and dial price feed with this thing.
Yeah, so, next one.
So, some downside of this tool is that mainnet fork can be buggy for the ganache.
So, the fix that I have there in the side node is not official.
So, you know, sometimes, there are some bugs there.
And the second one, you kind of already saw that we cannot create view function and return
properly yet.
So, I'm still working on that.
And, of course, this is a local ganache.
So, you cannot simulate how others would behave.
You know, it's not like a simulation that you can see all, like, with this new values,
how are people going to behave with the new incentive and stuff, right?
So, you're alone in this local blockchain.
That's kind of an inherent limitation here.
And fourth is this arbitrary modify storage is sometimes too powerful.
Because, you know, it can get you to state that are just not possible to reach.
So, you have to be careful about and thinking about what kind of scenarios you want to craft.
And if that is a reasonable scenario.
Because before this tool, you have to think of ways to get to a state.
And then you can start testing.
Now, with this tool, you can get to any state directly.
But whether that state makes sense or not, that's another question.
So, that's another problem that we have to take care and think about.
Yeah.
And so, next one.
Kind of a conclusion is that DeFi composability is currently both a blessing and a curse.
I hope that with more testing facilities, we can turn it into a pure bliss.
And as of right now, as I've said, I don't see a lot of testing on even the integration side.
Let alone testing extreme scenarios to prevent systemic threats.
Like, you know, I've really never seen people testing how systems would react
if an Oracle attack is to happen, you know, modeling these different threats.
Right?
But this is mainly because there's no...
Well, there are not a lot of tools that support this.
Right?
And so, I believe there's still a lot of space to improve in testing.
Especially in terms of this composability thing.
And I think we need...
This presentation is almost like...
I would treat it like an open invitation for everyone to kind of work on this area.
I believe we need to work together to help secure all these blockchain projects
and make this, you know, make the testing facilities better
and make the composability not a headache anymore.
Yeah.
And that's it.
Thank you for your time.
Great presentation.
All right.
Any questions?
Ah, I see.
Are there limitations between forking mainnet from full versus archive nodes?
Ah, it depends.
So, I actually just used Infura, not even the full node.
Oh, Infura is the full node, right?
Yeah, but I've just been using Infura.
And if you just care about how it's going to react at its current stage, I think that's fine.
After like 30 minutes, then your test would start failing,
and it will say that, hey, you know, you don't have access to archive nodes.
Then that would start failing.
But, you know, just for the purpose of doing unit tests on mainnet fork,
especially for the integration part,
I think you don't really need an archive node, to be honest.
Yeah, no problem.
My pleasure.
Ah, yes, let me paste my GitHub repository into the chat.
I have to warn everyone that this is like a very, very, you know, initial work.
It works, but it's very ugly.
I'm not particularly proud about the code quality,
but I am pretty happy about the result that it can get me, though.
So I'm happy to share.
All right, any other questions?
Thank you, Miko.
Thank you.
Thank you.
Do we have other questions, like in Twitch?