This link which is mentioned in FRP-11 isn't working. Can this be updated?

MEV in 2021: A Year In Review

Date: Thurs 13th January 2022
Time: 3-5pm UTC
Link: Youtube stream
Recording: https://youtu.be/V_wlCeVWMgk

In 2021, Maximal Extractable Value (MEV) grew from a niche concept to one regularly talked about. MEV extraction is now a $1b+ industry that has grown beyond the walls of Ethereum.
As we enter 2022, Flashbots invites you to a public virtual event to look back at MEV and its extraction in 2021 on Ethereum, Avalanche, Polygon and other chains.

Subscribe to the Flashbots calendar

Agenda

Section 1: Exploring the Data (1h)

• A brief Survey of MEV on Ethereum, BSC, Avalanche and Polygon in 2021 by Alex Obadia
  A comparative survey of MEV numbers and activity on multiple chains throughout 2021.

• MEV on Ethereum in 2021 by Luke Van Seters
  On the negative externalities of MEV on Ethereum and the mechanics by which is is extracted, using numbers from the newly released mev-inspect-py.

  • Resources: mev-inspect-py repo, mev-inspect-py codebase walkthrough, mev-inspect-py docs

• MEV on Polygon in 2021: A Case-Study of MEV on Low-Fee Chains by Supragya Raj
  On MEV extraction on Polygon in 2021, including a focus on adversarial and non-adversarial spamming activity and how such activity exists on other low-fee chains.

• Interplay Between Protocol Design and MEV Strategies on Avalanche by Ward Bradt
  On MEV extraction on Avalanche in 2021, in particular on how changes to the Avalanche protocol affect MEV extraction strategies and vice versa.

Section 2: Introducing a Framework & Discussion (45m)

• Blockspace market design.. and its consequences by Hasu
  We've seen different market structures that lead to different outcomes in terms of MEV numbers and negative externalities to the network. How does one think about how these mechanisms define market structure?

• Q&A

Hi @sarahalle, I saw during your FRP Unconference intro, a topic that has yet to be added to the topics.md: Crypto X AI! It should be a great addition there. What do you think?

Date: Dec 16th
Time: 6pm UTC
Description: This call is scheduled to answer questions on the recently released paper from Flashbots Research titled Unity is Strength: A Formalization of Cross-Domain Maximal Extractable Value. Feel free to ask any questions on the session in the comments of this issue.

Call Link
Subscribe to Flashbots Calendar

Summary:

Data collection and analysis of Ethereum data in order to quantify the negative externalities caused by current techniques used for MEV-extraction as part of the effort to answer Flashbot Research Question 1.

Research Question: Flashbot Research Question 1.

Proposed deliverable:

• Graph of gas wasted over time
• Breakdown various protocols by gas/arb usage
• Gas/arb usage vs. MEV profits
• Upwards pressure on gas price?
• From bot perspective: wasted efficiency for each bot (how often failing / burning gas vs succeeding)
• Gas wasted conditioned on various events? e.g. certain auctions / rebases

I am opening this issue to give visibility to an idea that someone might want to explore.

The topic is to use a probabilistic framework to better assess Realized Extractable Value :

In short

The goal is to find a more accurate measure of than is, and therefore a more accurate measure of the REV lower bound. In the spirit of a more theoretical approach, I believe a probabilistic approach is a way of bypassing the ephemeral property of transactions in the mempool, essentially using a posteriori distribution.
Someone could then say: "Here is the block that would have been mined if it had been "untouched", essentially if transactions had been selected by time arrival I suppose, and here is the actual block that was mined. The gas difference between both makes the REV."

Background and Problem Statement

The recent article of Alejo Salles [1] provides a lower bound for that primarily depends on . While holds, it is much too imprecise as the term is not accurate of the real gas fees mempool distribution. Arguably, , implying REV is even greater than current estimates. In the spirit of a more theoretical approach, the goal of this project is to provide a probabilistic approach to better assess , and thus the lower bound. Going further, this would introduce a model that allows to correctly model a block that would not have been reordered. This would drastically improve the accuracy of MEV measures as until now research has not been able to look back to know what a block without permutations in the mempool would have given.

The goal is to find a distribution, noted , of transactions' order in the mempool such that better accounts for than does (on average). There are two different approaches coming to mind, essentially two different ways of viewing the problem.

1. Ensemble approach: The search for such distribution eventually boils down to finding a realistic permutation of the transactions, that in that order would be a realistic estimate of what a block not reordered would be. The work is thus to equip the permutation group of transactions in the mempool with the right probabilistic measure. A natural first question is: does work well enough ?

2. Statistic approach: monitor the mempool to infer the gas distribution it follows, potentially with non-parametric statistics. Therefore it would be possible to draw a sample block, which would arguably be a better estimate of than .

References

[1]https://hackmd.io/@flashbots/quantifying-REV#fn4

Merge ongoing FRPs, taking care of renumbering/formatting.

Summary:
By understanding how much non-essential bot logic can be brought off-chain without any security tradeoffs, we can design our bundle processing/switching logic with it in mind. Bringing such logic off-chain would alleviate network congestion.

Research Question: Flashbot Research Question 1.

Proposed deliverable:

• Study of how much of non-essential bot activity can be moved off-chain
• Formalize trade-off between on-chain / off-chain arb computation

Working doc with open questions and resources here.

FRP-1 is being partitioned into more manageable chunks #22. FRP-3 will tackle comparing the Layer 1 and Layer 2 scenarios. A proposal needs to be written for this.

Note #36 might be tackling this soon @Mikerah.

Prior notes taken with @sbaks0820:
Layer 1 vs Layer 2 tx ordering auctions
TBD: discussion with Karl Floersch regarding MEV auctions in layer 2 (rollup, etc...)
TODO: update after meeting with Karl

Summary:

By understanding how much non-essential bot logic can be brought off-chain without any security tradeoffs, we can design our bundle processing/switching logic with it in mind. Bringing such logic off-chain would alleviate network congestion.

Research Question: Flashbot Research Question 1.

Proposed deliverable:

• Survey of the logic bots are doing on-chain that doesn't have to do with direct execution.

FRP-1 is being partitioned into more manageable chunks #22. FRP-5 will tackle optimizing MEV extraction after the actor based model tackled by FRP-4 #42. A proposal needs to be written for this.

Note there's a proposed FRP to tackle this problem using Reinforcement Learning #38. While this would only cover one approach to the problem we might consider merging both FRPs.

Prior notes taken with @sbaks0820:
Finding solutions to objective functions delivered from FRP-4
be computationally efficient and optimal strategies should should be computable re:bidding on permutations and other NP-hard results --> for NP-hard results are there heuristic functions that approximate good/strong strategies)

I'm in the process of putting together an FRP to evaluate and compare how well different protocol design patterns can protect users from MEV.

I've been advised by @fiiiu to sound out the general idea here and gather feedback from the community on whether it's worth fleshing this out to a formal proposal. What follows is a brief informal outline:

Summary

Perhaps the most notorious type of MEV extraction activity in terms of its negative effects on users is frontrunning. Frontrunning has become well-known enough that a slew of novel protocols have appeared that make various claims to solve the issue or "protect users from MEV," whether as an additional layer over Ethereum (e.g. Flashbots Protect) or integrated at the smart contract level (e.g. COW protocol).

While it is fairly easy to see that new designs (using, for example, private order books) prevent certain kinds of attacks, as far as I am aware there have been no systematic studies to quantify the effects on users' utilities and hence evaluate and compare different approaches to "MEV-aware" protocol design. What is missing is a mathematical framework to analyse the UX effects of frontrunning, or more generally, of unpredictable execution state.

Approach

The starting point for this project is the observation that frontrunning is an instance of a more general problem in concurrent computation: shared state may be subject to an unknown set of updates between commitment and execution of a transaction that reads it. That is, from the caller's perspective, shared state effectively constitutes a random input to a message call. Frontrunning is simply the case where the random input distribution models the "intentional" behaviour of an external actor — game theoretically, it can be modelled as the "bid" distribution of a Bayes Nash equlibrium of the searcher game.

The random input leads to a utility distribution for the execution, that is, a distribution over the utility line of the caller. The shape of this distribution can be influenced by the design of the function according to preferences that may depend on user type and application.

Very briefly, in this work I propose to:

• Formalise the above discussion with precise definitions
• Develop models for utility distributions of functions in common approaches to dapp and privacy layer design and hence obtain formulas for distribution invariants (such as range and variance) that measure the "UX" impact of randomness; in particular, "frontrunnability."
• Examples will be drawn from the following classes:
  • AMM swaps, e.g. Uniswap v2/3
  • Order book based exchanges (COW, 0x, traditional CEXes)
  • Overcollateralised lending
  • Private mempools, e.g. Flashbots Protect ("fast" and "slow" mode)
  • NFT auction
  • Some non-financial (e.g. networking) applications, if I can think of any.

This project would fall under the protocol design research topic. The primary deliverable would be a formal research article targetting publication in a peer-reviewed academic journal or conference. Secondary deliverables are blog posts and conference presentations.

FRP-1 is being partitioned into more manageable chunks #22. FRP-4 will tackle defining the "MEV problem" taking into account the different actors in the system. A proposal needs to be written for this.

Note that this might indeed end up being part of the taxonomy defined in FRP-1, in which case this FRP will end up being merged with that.

Prior notes taken with @sbaks0820:
"Good" should apply to different types of actors (bidders, miners, external projects that are impacted by it)
Define "good" for each of the actors involved in the auction --> for some if may mean gas cost/failure rate, for other may mean negative externalities of MEV extraction, (what about for miners, what do they care about here --> auction can be run preferably several blocks before the block being bid for

I want to discuss what optimal MEV looks like, and how it can be approximated.
See here for (bad) details: https://hackmd.io/gjpgyASHSy2gNp_r62fjkw?view

FRP-1 is being partitioned into more manageable chunks #22. FRP-6 will assess how far the current Flashbots Alpha is along the path to optimal extraction. A proposal needs to be written.

Prior notes taken with @sbaks0820:
FRP-6: Where is the current POC and what is the "ideal" implementation we are going towards?
Are the differences between POC and "ideal" large enough that they require different auction designs? Why and does what auction they need determine which implementation is the better one? --> is one of them more "good" (define!) than the other
Right now(?): sells the ordering of the entire blocks
Ideal (?): to be able to sell block space before/after oracle updates and small tx bundles (2,3,4 txs in a specific ordering)
IMPORTANT add to MEV research the current spec/auction mechanism in use in the POC.

In the context of "privacy to minimize MEV" there are two encryption based solutions :

1. threshold decryption
2. timelock encryption

A problem: If the transaction is encrypted in full : fee payment data is encrypted (Gas related fields are unknown at time of mining) and therefore miners cannot optimise to take the highest paying transactions.
on the other hand, if the meta data fields are kept as plaintext they leak information.
Ideally, we would like to have the transaction fully encrypted but still allow the miner to order the ciphertexts that go in a block based on gas alone and ensure max gas is not exceeded.

Lets say that we have such cryptographic black box that does this magic (maybe based on functional encryption): the box lets a miner learn only the order of transactions based on fee without learning the actual fee.

An immediate problem is that a miner can abuse the black box to learn the exact gas of a transaction tx as follows: make a dummy block with X-c gas and add the tx to the block. If the gas limit is reached the miner learns that the tx requires > c gas. A binary search will reveal the exact gas.

We claim this issue can be bypassed in (1) and (2). In the threshold decryption case, we already assume a quorum, by allowing an extra round/s of coordination, we can use the threshold to manipulate ciphertexts and to limit the number of ordering attempts done by a miner.
In the time lock encryption case - we can make running the black box an operation which takes a specific fraction of the time it takes to open the timelock.

We hope this information is useful and would love to work on it.
Comments / questions are welcome.
credit goes to Dev.

Date: Thursday, Mar 18th, 2021
Time: 9am PDT / 12pm EDT / 4pm GMT

We are resuming regular research workshops with a quick checkpoint on research projects, and inviting prospective FRP authors to pitch their ideas and get feedback for the community before the new round of research grants. Timing is lax, and will be fine tuned as required, but please be ready for a 5 min presentation (no slides needed).

• Intro and announcement to new FRP round of grants (@fiiiu, 5 min).

• Status updates for all current FRPs (~5 min each + ~5 min questions/feedback):
  @fiiiu on MEV Quantification Framework
  @sbaks0820 on Auction Literature Survey
  @sunnya97 on Flashbots on Alternative Architectures

• Introduction of new potential FRPs (~5 min each + ~5 min questions feedback):
  @Mikerah on Mempool Privacy
  @mevrbebr on Reinforcement Learning for Searcher Strategies
  Other proposals

The discord link says it's expired. Is there a new link, or is this just my Discord being funny? :)

Date: Thursday, Feb 4, 2021
Time: 11:30am PST / 2:30pm EST
Join Call | Calendar

A discussion on auction mechanism designs for MEV auction:

1. Survey of trade-offs between auction mechanism design
2. Share update on the auction mechanism design FRP
3. Updates on Flashbots FRP process and new batch of MEV Fellowship

Starting this issue to get a conversation going on MEV in an Eth2 world.

Afaik in Eth2 a set of validators is chosen by epoch using some unbiasable source of randomness, where a validator is defined as a holder of a 32 ETH stake. Each validator has the right, not the obligation, to produce a block for their pre-assigned slot. There are about 32 slots per epoch, so ~12s periods where each validator can propose a block. The epoch can be thought of as a meta-block, where finality is only achieved after each epoch has passed (is that true?).

I'm wondering how does block production work in this world. If I'm in slot 25 and I formed a great block but then some of the txs I picked up have already been proposed by slot 31, I need to dynamically adjust the content of the block I propose based on the block proposals that went before me?

• Is the 6-min epoch a vector of attack where 6mins in advance, everyone knows who will mine which slot and can reach out to them in some way for some bribez?
• Putting my Flashbots hat on, what do miner tips look like in this world?
• How does tx replacement work in such a system? If your tx got picked up in slot 20, finality is not achieved yet until next epoch and you want to replace your tx?
• How does tx replacement interact with the fork choice rule? (h/t @lsankar4033)

While validators are chosen from an ideally unbiasable source of randomness, if someone controls a bunch of 32 ETH stakes they are more likely to have one of their stakes selected as a validator akin to how hashrate works (the randomness of solving PoW is replaced by another source of randomness I guess).

• Say I am a large validator and I get multiple slots per epoch, what kind of surface for MEV extraction does that imply?

FRP-1 is being partitioned into more manageable chunks #22. FRP-2 will tackle measuring MEV extraction. A proposal needs to be written.

Prior notes taken with @sbaks0820:
Measure PGAs, other MEV extraction strategies, define and concretize measurement infrastructure (reliability and assumptions on data)
Use current inspect tools to find PGA's and measure:

• current gas cost of a single arbitrage (average) taking part in PGAs --> expect this to be a long tail distribution --> try to determine the entire distribution of bots being used --> validate long tail from measurement FIRST
• for each extraction opportunity how many bots are showing up? (which ones are easier to find and whether some relationship can be determined based on the profitability of these opportunities)
  General: evaluate assumption whether we are seeing lower/upper bounds with MEV inspect or something else entirely --> validate MEV inspect opportunities (there is an issue in MEV inspect of arbitrage found when there isn't any) --> IMPORTANT: evaluate what assumptions an be made from MEV inspect --> perhaps manual validation? (i think we have enough people involved to put them to work) --> bring with Phil and Stephane
  in what way is MEV inspect different from frontrun.me? --> cross validation with MEV inspect

I noticed that the http://www.gasnow.org link routes to https://bitcoin.org/ and figured I would open up an issue just in case this was not known.

FRP-1 at the moment is a very broad proposal that attempts to tackle many questions regarding the auction mechanism design without a clear breakdown of what needs to be done and what theoretical questions need to be answered. This issue proposes a refactoring and breakdown of FRP-1 into approximately 4-5 FRPs that can each be managed by a research fellow or multiple ones managed by a research fellow team.

Date: Thursday, Jan 21 (tomorrow)
Time: 11:30am-1pm EST
Join Call | Calendar
Our first MEV research workshop of 2021 will go into a deep dive into the latest research community effort in quantifying and analyzing realized MEV:

1. Discussion and feedback on the latest research paper by Kaihua Qin, Liyi Zhou, Arthur Gervais: Quantifying Blockchain Extractable Value: How dark is the forest?
2. Early insights on the state of MEV: pre-release MEV Explore V0 (public dashboard prototype) metrics
3. Side by side review of MEV Inspect data & metrics vs. FRP-1 research side metrics requirement
4. Updates on FRP process and new timeline

Hi folks, we have been working on block-building, and have recently started to add support for sending larger bundles we find along the way to flashbots.

In debugging the auctions we lost, it quickly became clear that the bundle pricing used today is preferring small bundles w/ high miner payouts, instead of the highest overall payout. This is problematic because it leaves significant MEV on the table for miners/validators.

Here's a real life example that happened yesterday afternoon.

• Tx 1 traded USDT --> ETH and created a ~0.8 ETH MEV opportunity
• Tx 2 also traded USDT --> ETH and created a ~0.003 ETH MEV opportunity.

However, if you put Tx 2 in front of Tx 1 and backrun them as one big bundle it's 0.95 ETH MEV opportunity (due to AMM slippage). N.B. this is the order it landed on chain anyways (but not as one bundle) - so the DeFi user isn't getting a worse exchange rate.

Now let's look at the bundle pricing, simplified to: mevPerGas = minerPayout / totalGasUsed. And for further simplification, let's say we paid entire 100% as a payout to the miner.

0.95 ETH / (150000 + 116331 + 170353) = 2175 GWEI
0.80 ETH / (150000 + 116331)          = 3004 GWEI

The 0.80 ETH opportunity w/ its significantly higher mevPerGas (but lower total payout) will win the auction (and they did). However, the miner is losing out on 0.15 ETH here.

With bundles ranked according to their mevPerGas, adding more transactions to a bundle quickly increases the denominator on the pricing formula and pushes them down the ranking.

We've been discussing some other approaches over here that could help maximize miner payout, but I wanted to first check with you folks if this was something of interest to improve, or if you have thoughts yourselves on it.

FRP-1 is being partitioned into more manageable chunks #22. FRP-1 will tackle defining the MEV taxonomy. The proposal needs to be updated accordingly.

Prior notes taken with @sbaks0820:

• enumerate and classify MEV extraction strategies
• how many are currently covered by MEV inspect/frontrun.me?
• relates to evaluation MEV inspect in FRP-2

Problem

Dead link to Research Roadmap in grants.md.

It seems research_roadmap.md used to exist in previous commit, but not anymore. It would be interesting to see new roadmap.

Summary:

Data collection and analysis of Ethereum data in order to quantify the negative externalities caused by current techniques used for MEV-extraction as part of the effort to answer Flashbot Research Question 1.

Research Question: Flashbot Research Question 1.

Proposed deliverable:

• Percent of blockspace wasted over time
• What is the overhead of a single arb / PGA transaction?
• What is the overhead for a miner? What is the overhead for a full node? How do we measure waste?
• What happens to the p2p network / full nodes with much more priority auctions?
• How many (arb) txs broadcast first time in a block?
• How much worse is this in a stateless world? Or other proof-required-for-tx schemes?
• How many deletions? How many messages came in we already saw (duplication)?
• How spammy are various bots compared to other users? How much more duplication?
• Competition: # of bots per opportunity

Defining a taxonomy for MEV is important at this stage of our research, in particular a systematic framework, and consistent language and definitions for the sake of clarity and rigor.
Related FRP: FRP-1

We adopted the following definition for MEV in our blog post:
Miner extractable value (MEV) is a measure devised to study consensus security by modeling the profit a miner (or validator, sequencer, or other privileged protocol actor) can make through their ability to arbitrarily include, exclude, or re-order transactions from the blocks they produce. MEV includes both ‘conventional’ profits from transaction fees and block rewards, and ‘unconventional’ profits from transaction reordering, transaction insertion, and transaction censorship within the block a miner is producing.

A formal definition of Miner Extractable Value in the context of Extractable Value in the soon to be published paper by Daian et. al "Clockwork Finance" is as follows:

Please post your questions you would like to address in our discussion of MEV taxonomy, or your suggested approach in defining MEV taxonomy, as comments under this issue.