Brontes Database

Brontes uses a local libmdbx database to store off-chain data for its analysis pipeline. The data comes from a Clickhouse database managed by Sorella Labs. It includes centralize exchange quotes and trade data, mempool and relay data, address metadata, and more.

For details on the specific tables and their schemas, see the Tables Schema page.

Database Sync

On startup, Brontes syncs its local database by downloading the necessary data from Clickhouse.

Figure 1: Data download from Clickhouse to Brontes local storage

Snapshot Sync

To manage cloud egress costs, we don't currently provide api access to our clickhouse database for historical sync. Instead, users must download the latest db snapshot made available every Monday and Thursday. See the Installation Guide for detailed instructions.

Figure 2: User db snapshot download process.

Data Flow

Brontes adapts its data retrieval method based on its operational mode: for historical block analysis, it accesses the stored data locally; when operating at chain tip, it retrieves data through the Brontes API.

Figure 3: Querying methods for historical blocks and chain tip.

Note Users that want to run Brontes at chain tip, must request API access to query the data at chain tip. Configuration details for API access can be found in the Installation Guide.

Data and Usage

The data stored by Brontes can be categorized into three main types.

1. Block-Specific Data

Each value is mapped to a specific block. This data is fetched at each block before the inspectors are run.

Figure 4: Data usage.

The Metadata struct aggregates the essential block specific data, used by all Inspectors.

pub struct Metadata {
    pub block_metadata: BlockMetadata,
    pub cex_quotes:     CexPriceMap,
    pub dex_quotes:     Option<DexQuotes>,
    pub builder_info:   Option<BuilderInfo>,
    pub cex_trades:     Option<Arc<Mutex<CexTradeMap>>>,
}

BlockInfo: P2P transaction, block and mev-boost data.
DexPrice: DEX pricing with transaction level granularity for all active tokens in the block.
CexPrice and CexTrades: Centralized exchange quotes and trade data.
BuilderInfo: Metadata on the Block builder.

2. Range-Agnostic Data

Valid across the full block range. This includes:

Data for Decoding & Normalization:

TokenDecimals: Token decimals and symbols.
AddressToProtocolInfo: Maps contract addresses to their protocol, used during transaction decoding and normalization.

Metadata used by the Inspectors:

This data is used by the inspectors for filtering and analysis. It is queried ad-hoc via the database handle provided by the inspectors' SharedInspectorsUtils. See Fig 4.

BuilderInfo: Information on ethereum block builders, including aggregate pnl & block count.
SearcherInfo: Information on searchers eoas and contracts, including summary statistics on mev bundle count and pnl by mev type.
AddressMetadata: Detailed address metadata.

3. Analysis Output Data

Stores the output of the analysis pipeline in the MevBlocks table.

Brontes Book