The Nanotick | CME | Raw data artefacts contain data captured directly from network cards (currently ExaNIC cards) in binary presentation. Every packet capture file is a channel specific .erf file.
By way of example: Raw data is provided in a file named "channel310-AB.erf.gz" that contains a compressed "channel310-AB.erf" file.
This is the lowest level data artefact available on the Nanotick platform.
As the file naming implies with the "AB" naming, the Raw file contains a blended copy of both the A and B feeds.
We perform post capture integrity checks to ensure that the Raw file is "gapless" from data that we capture on A and B feeds spread over 2 machines (i.e. 4 points of capture). Any sequence gaps are filled to ensure the highest quality sequence of data.
We provide Security Definitions (secdef) from the beginning of the weekly session with the Raw data being Incremental Refreshes from Market Feeds.
By design, the CME | Raw data files are not self-sufficient as they contain only incremental updates. For example: full Order Books for Wednesday can be obtained only by processing data for Sunday open, Monday and Tuesday.
The CME may add new instruments during trading week. We include the CME secdef.tgz files which include the most recent trading symbols, settlement prices and so on.
The template.xml file is also provided that contains fields description and used during MDP 3.0 SBE decoding, or MDP 2 FAST decoding for the older pre-MDP 3.0 data.
File naming convention