NoteThe goal of this page is to help you understand the filtering capabilities, data flows, and impact filtering has on the different
dfuse for EOSIOcomponents.
Filtering is applied through the
--common-exclude-filter-expr parameters. They are both CEL programs.
Filtering in the dfuse platform uses the Common Expression Language, developed by Google, and used in high throughput environments (like their ACL checking routers).
Important: They filter what gets processed by different components across dfuse for EOSIO stack. They are not a language you can use to query dfuse Search indexes.
CEL is Google’s Common Expression Language:
Filtering is applied on certain components, while other components are happy to accept filtered data and process it.
relayer is the first component to filter in a deployment.
It will filter blocks it receives (from either another
relayer or a
mindreader), and serve filtered blocks through the same
BlockStream gRPC interface. Each block within will be marked as
FilteringApplied == true (one of the values in the EOSIO-specific protobuf
It uses primitives available to all components, but when the
relayer does the filtering, it cuts on much of the network traffic that follows, thereby shrinks much of the RAM needs of all components downstream (like
search will only index actions that matched the inclusion filter and did not match the exclusion one.
indexer and the
live components of
search will all potentially need the filtering criterias, as they might pick up merged block logs from Object Storage, and index things on-the-fly. In that case, they will first apply the filtering criterias.
trxdb-loader component will only save transaction traces in the database that contains at least 1 matching action.
This means that, once filtered, the resulting
trxdb will not contain all transactions. Will not be able to resolve transaction traces requested by
dgraphql or pointers reported by
search unless they are also in this database.
fluxdb will be affected by filtered data coming from a
relayer that relays filtered blocks.
You need to be diligent in what you filter, to not miss table changes that would otherwise make your view of tables inconsistent.
You can use
db.scope in your filtering criteria to make sure transactions that mutate certain tables always make it through filtering.
An similar identifiers available for searching in dfuse Search is available for filtering but with more capabilities because any action’s data field can be search for whereas in dfuse Search, only a subset of all fields is possible.
For example, a dfuse Search of:
would be filtered as:
receiver == 'eosio.token' && data.from == 'bob'
See the Search Terms page for all EOSIO terms that can be filtered.
Showcase examples here are given as examples, mainly for syntax purposes, so you can see the full power of the CEL filtering language
There might be new stuff to add to certain examples, like spam coins or new system contracts, this is not a fully accurate document for those stuff, you are invited to make your own research to ensure completeness based on your use case.
EOS Mainnet Spam
Here an example to filter out spam transactions on the EOS Mainnet:
account == 'eidosonecoin' || receiver == 'eidosonecoin' || (account == 'eosio.token' && (data.to == 'eidosonecoin' || data.from == 'eidosonecoin'))