When a company, contract or instrument becomes a tradable entity on a stock exchange the amount of data associated with that entity grows exponentially. This data can be grouped into different categories of data, such as reference data, fundamental data and market data.
Market data in particular generally refers to pricing information, such the last trade price or the current bid and ask price of an instrument, however it also includes numerous other data points:
All of this information is highly valuable for market participants seeking to understand and analyze market dynamics. Market data is different from, but complemented by, fundamental data which provides information about an underlying entity’s performance like company revenues or from macroeconomic data.
Market data can be classified in a number of different ways based on the content, the source of the data and the delivery of the data.
Generally market data is classified into tiers of data based on the order book. The order book is a list of all the current orders for an instrument, for both buying and selling, currently active at a venue waiting to be fulfilled. They are organized based on the “best price” i.e., the highest bidding/buying price and the lowest selling price appear at the top of the order book.
The simplest and most basic type of market data is called Level 1, which constitutes the best bids and offers that are in the order book at that time - the "Top of Book". Level 1 data is generally used for basic chart-based trading and strategies that require price discovery across venues so that they can hit the best price.
The remainder of price levels in the order books, below level 1, are known as Level 2 data or "depth of book", i.e., the venue’s entire trading book.
Market data is also classified based on where the data may have originated from. Most market data is generated by regulated venues such as stock exchanges or other trading platforms, in so-called lit venues, which means the data is viewable by the wider market. There are also over-the-counter (OTC) trading which generates market data based on bilateral (one-to-one) or multi-lateral trading relationships in dark pools.
In Europe for example, trades are grouped into one of three categories under the Markets in Financial Instruments Directive II (MiFID II):
A single instrument may trade on multiple of these venues making it difficult for users to find the best price or source of liquidity for an instrument as it could be trading at a different price on different venues.
To help overcome this problem of disparate sources of data, in the United States, there is an electronic service known as the Consolidated Tape, which captures all equity trading across different venues and includes OTC trades. This is done via the Securities Information Processor, which is managed by the Consolidated Tape Association. There have been calls for a similar service to be provided in Europe, however due to many more different disparate venues and differing opinions on what should be included, it has proven difficult to determine what should be included in a European Consolidated Tape.
The following are terms you may hear a market data vendor uses:
Exchanges are the most established venues for trading securities and other assets. In the past two decades, a variety of other venues have become popular. These include:
In addition, there is over-the-counter (OTC) trading, which is based on transactions directly between parties.
An asset class is a group of instruments or securities that behave similarly and are traded in a similar way on the same venue. Common types of asset classes include:
Some market participants consider derivatives for these markets as forming part of the same asset class, while others treat derivatives as an entirely different asset class. Types of derivatives include:
There are numerous alternative asset classes involving lesser-traded markets, hybrid instruments or synthetic instruments.
Each venue will have an order book for each instrument that features all of the orders placed at the venue at any given time, ranked according to various criteria such as time received, bid/offer levels, or quote amounts.
The most common type of order book is a Central Limit Order Book (CLOB), where Level 1 data refers to the latest or best bids and offers in the order book, sometimes referred to as the “top of the book.” Level 2 data typically refers to all of the other data in a book.
Upward and downward moves in prices constitute “ticks.” Tick data is generally used by quantitative analysts in models, such as when a trading strategy is back tested. Market data vendors may provide level 1 market data, level 2 market data, or both.
Market Data Fields
Market data feeds include various data fields specifying different types of information. This includes identification and settlement information such as:
Fields will vary according to the asset class. Cash fixed income securities may involve more fields than others due to the complexity of the assets.
Algorithmic Trading and Black Boxes
Algorithmic trading systems use algorithms to generate automated trades or trading recommendations, often based on quantitative models. In equities markets, algorithmic trading, which depends on real-time stock data feeds, generates a substantial portion of equity trading around the world and is common in a number of other asset classes. Algorithmic trading systems, many of which incorporate dedicated market data software, are sometimes referred to as “black boxes.”
Market data is generally used for price discovery, in order to determine a market’s current levels and facilitate immediate decision-making, or to analyze market conditions with the ultimate aim of understanding market dynamics and/or making predictions.
Financial market data providers help a wide range of market participants, each of whom may have different end-uses depending on the functions they are performing, source the market data that they require. For instance, sell-side brokers servicing their clients may use market data to provide real-time quotes to their customers. Or, they may use market data for analysis that will form the basis of research recommendations that are contained in the research reports they issue. Similarly, buy-side clients may use market data for price discovery or to conduct their own analysis.
Analysis performed on market data may be done automatically or manually, or via some combination of the two. Algorithmic traders such as high-frequency traders (HFTs) or quantitative analysis-based funds (known as “quant funds”) feed market data directly into models that can automatically generate trading decisions or scenarios.
Market data is also required for best execution, whereby market participants will seek to get the best price available via all venues to which they have access. This may involve the use of smart order routers which scour the market to identify and interact with the best available bids or offers available.
Sell-side participants, or buy-side participants with direct market access, often use algorithms that “slice and dice” orders, based on feeds of real-time market data, in order to minimize price disruption and optimize trading strategies.
Technological advances and financial innovation over time have led to increased trading volumes in major markets around the world, both in cash markets and in derivatives. This has been the case across the asset class spectrum and it has resulted in much higher levels of market data being generated, requiring ever-more sophisticated market data solutions for capturing, bundling, distributing, processing and analyzing it.
Market data delivery can be categorized into real-time market data or historical market data. In both cases, there may be delays or conventions involved that affect the nature of the data. For instance, a live stock data feed delivered with a short delay may be considered semi-real-time or near-real-time. Similarly, historical market data providers may include intraday data or only end-of-day information in their feeds. In the case of the latter, there is also the question of what time of day an end-of-day level is recorded based on a given venue’s conventions.
As well as using market data for real-time decision making, or for performing analysis that will determine short- or longer-term trading strategies, quant funds rely on historical market data providers when back-testing their strategies. This involves large quantities of historical market data being fed into models that will simulate market conditions in order to understand how the different trading strategies will perform during live scenarios. Analytical methods can range from simple trend analysis or various forms of technical analysis to hyper- sophisticated methodologies such as Monte Carlo simulations involving hundreds of thousands of data points to generate thousands of different scenarios.
Exchanges and other trading platforms that generate market data make it available to their members or customers via streaming data feeds or APIs. There are a variety of protocols, for instance, the London Stock Exchange uses GTP protocol, Eurex uses EMDI. Others include the widely-used FIX protocol or ITCH.
Clients that take data “raw” from these venues need to use the exchange’s protocol. This allows them to make full use of the data, but it requires development at the outset to decode, understand and react to the data. There is then ongoing maintenance as venues make changes over time to improve performance and add new functionality. These changes are known as Exchange Driven Changes (EDCs) and announced via specification updates from the exchanges in advance to allow users to prepare for them. Each venue typically has a single major EDC a year with multiple minor ones throughout the year leading to a large ongoing maintenance burden and effort.
For users needing to access many different venues across many different protocols this effort to manage the data quickly extrapolates. The hardware, software, connectivity and or components used for managing and distributing market data are often called Market data systems, market data platforms or ticker plants and come with a high total cost of ownership.
Recent trends in capital markets has led to an increase in EDCs globally. The number of trading venues is on the rise and increased competition and changing business strategies is driving more regular upgrade cycles, creating more EDCs.
Check out the key stats on 2019 EDCs >>
Alternatively, consumers of market data may take data that has been normalized into a proprietary or third-party protocol different from the one used by the original exchange or venue. Market data vendors provide products and services for this by taking the various raw feeds and normalizing the data using feed handlers so that a customer can receive one feed using one streaming protocol.
This means the client does not need to develop individual protocols for multiple feeds of data, making operations simpler, less costly and easier to manage as it abstracts the ongoing EDC changes from the client. This also rapidly decreases consumers’ time to access the market and add new feeds as they do not need to understand bespoke market nuances.
Taking a normalized feed of data from a vendor, however, can add latency, which may make it less suitable for latency-sensitive applications such as those performed by high-frequency traders (HFTs). Using a vendor requires connecting to their host infrastructure, wherever it may be located.
However, using a feed directly from an exchange may allow the customer to host its own infrastructure wherever it wants, including in a co-location center near an exchange’s matching engine. In these cases vendors often provide consumers with the software to normalize the feeds, providing them the benefits of normalization with the flexibility of deploying wherever they require it.
Most exchanges will provide a market data vendor list with details of the various market data products those vendors offer.
Market data is generally purchased either for direct usage or for redistributing to clients. For instance, a buy-side participant may purchase a feed of data for its own use. Alternatively, a sell-side participant, such as a broker, may purchase a feed of market data and broadcast it to all or some of its clients. Market data that is made available to all users or large groups of users at a given firm is known as enterprise market data.
Exchanges will have different data packages based on the content and each will come with rules about what can be redistributed, to what clients, in which geographies, and in what formats. Some license agreements may stipulate that data that is redistributed must be delayed, or must exclude certain geographies. A firm that has the appropriate licenses to redistribute data is known as a Vendor of Record.
Data vendors need to be able to prove that they have control over who receives the data and in what formats, requiring an audit trail of all data that has been redistributed. Market data entitlement systems allow vendors to restrict access and generate reports based on these restrictions to ensure compliance of any data license requirements or regulations.
Some market data solutions may restrict redistribution entirely and require that the market data is only used for internal purposes. This is known as non-display market data.
Data purchased directly from an exchange, depending on the purchase agreement, will generally have fewer restrictions than bundled data packages that are purchased from a data vendor, which itself may be subject to license restrictions.
Market data software costs will vary according to numerous factors. These include:
In the case of data exclusivity, feeds of major exchanges such as the London Stock Exchange or the New York Stock Exchange are available either directly from the exchanges or from numerous vendors. But some types of data, such as certain bond market data or data from more esoteric asset classes, may be rarer. This may be because the data is based on quote contributions from specific market participants, such as sell-side participants that specialize in a market.
Such harder-to-source market data may be more costly than readily-available data such as that which comes from high-volume lit markets.
In addition to the cost of the underlying data packages, there are associated costs such as IT development and infrastructure spend. Receiving a data feed requires an engineering team to decode it and engineer a solution based on the information. An infrastructure team will also be needed to actually get the data and manage it, and firms may need an administration team that is charged with ensuring data agreements are in place and are adhered to.
There may be costs associated not only with development for receiving and processing a data feed, but also for any data redistribution. Firms that redistribute data may need to feed hundreds of applications that are relaying the data to thousands of end users. That can involve internal infrastructure which needs to be developed and maintained, as well as specialized software for distribution and for managing entitlement systems.
Choosing a market data solution requires consideration not only of a firm’s needs but also of its core competencies. What data are required, what format it should come in and how it will be used are all key criteria. More tech-savvy firms, particularly those that are highly latency sensitive, may focus on different factors than other firms. For example, they may consider feed handling a core competency and, based on their own in-house resources, may opt for raw feeds in order to ensure the lowest latency.
On the other hand, taking a normalized feed from a data vendor can reduce costs and simplify operations. Firms in such cases will only have to work with a single API, a single data model and a single vendor. Less infrastructure is required and maintenance costs are minimized.
In conclusion, market data can be considered the lifeblood of electronic financial markets, and constantly evolves, so please check back regularly for updates to this guide.
Vela is a leading independent provider of trading and market access technology for global multi-asset electronic trading. Our software enables clients to rapidly access global liquidity, markets, and data sources for superior execution. We help firms successfully differentiate and innovate in an ever-changing, increasingly-regulated and fiercely-competitive landscape, while also reducing total cost of ownership.
The Vela Ticker Plant is a software-based feed handling solution providing ultra-low latency, normalized access to over 250 venues, with global coverage and support for all major asset classes. Delivered via a single, high-performance, flexible and unified API, all content is normalized to Vela's Market Data Model and optimized to deliver data to latency-sensitive trading applications.
Vela's Market Data Feed, SuperFeed, delivers high quality, low-latency normalized market data to access major markets across the globe and is delivered as-a-Service. We provide access to low-latency market data feeds, delivering time-critical streaming market data while lowering operational and capital expenditure and removing the need for infrastructure investment.
Vela provides global coverage across all major asset classes. Clients are supported by an award-winning team of technical and business experts available 24x7 from our multiple offices in the US, Europe, and Asia. Vela’s clients include traders, market makers, brokers, banks, investment firms, exchanges, and other market participants.
The aim of Markets in Financial Instruments Directive II (MiFID II) to make market data more accessible requires market data providers to offer unbundled pre and post-trade market data feeds that can be delivered as direct feeds or part of a consolidated service.
High performance algorithmic trading involves sophisticated software and hardware components operating in harmony to effectively accomplish market operations. Traders are continually seeking faster market data and order execution services with lower slippage to more accurately qualify their orders.