By Melinda Wilson, Sybase, An SAP Company
www.sybase.com
“The last few years have been the most volatile for all of recorded history ... 10 of the biggest 20 daily up-swings and 11 of the largest 20 daily drops since the beginning of 1980 to the end of last month have occurred in just the last three years.” - Andrew Lo, professor of finance at the MIT Sloan School of Management, The New York Times, Sept. 11, 2011
Trading firms are charting a course toward better handling the financial industry’s growing volumes of market data, messages and unstructured data, all the while keeping watch for new opportunities and market manipulation. The right technology can help firms monitor and correlate data streams in real time, but the wrong technology - or doing nothing at all - could run them aground.
Today’s advanced market surveillance solutions ideally consist of a three-layer architecture: a columnar database for historical analytics; a complex event processing (CEP) engine for real-time data streaming; and a visualisation front-end that graphically renders the real-time data streaming through the pipes.
Three Tiers for Surveillance
Market participants who have their own three-layer surveillance solution garner at least three advantages over their non-tiered competitors. First, analysing volatile message and trade activity in real time facilitates low-latency responsiveness to irregularities - and makes it easier to leverage new opportunities. This means advantageous situation detection and response for real-time portfolio management.
Second, these participants are compliant with recent regulations mandate that firms implement more robust and holistic monitoring and surveillance solutions. Third, having a single integrated architecture in a modern in-house system enables the performance of complex holistic market analytics across multiple asset classes, venues and geographies. This architecture also allows more flexible and proactive responses to future regulations.
Proper surveillance requires real-time visibility and management of trading activities. Firms have to closely and proactively monitor market participation. Robust surveillance enables firms to search market conditions and positions for indications of manipulative activity that could disrupt or adversely affect the integrity, performance or orderly expiration of contracts.
Unstructured Data
Unstructured data is an increasingly significant part of that surveillance. We see more and more firms utilise unstructured data in search of correlations, not only for negative patterns, but the next cutting-edge gold rush in analytics.
Some data forms, such as unconventional and unstructured data, include electronic news and sentiment, weather information, and even traffic patterns. But the ability to gather such wide varieties of information and the capacity to store such high volumes mean little without the technology to collect the most relevant, high-quality content that will allow traders to take strategic action.
Some solutions make unstructured and unconventional data objects accessible within the database environment, enabling access to unstructured and structured data using the same application and interface. For example, an option on one particular product can store and retrieve unstructured data objects as part of the same repository as transactional or analytical data, managing individual objects containing terabytes or even petabytes of data as necessary.
Taking Charge of Big Volumes
Capital markets firms and exchanges are improving their systems with advanced technology to handle growing volumes of market data. Increasing message volumes as well as the rise of unstructured data make it ever more important to have surveillance solutions for analysis across multiple venues to detect events that may indicate a market opportunity - or a red flag.
Beyond responding to abuse, participants can employ surveillance to stay ahead of the markets. New pattern detection and analytics tools can help firms handle volatility and unstructured data by monitoring and correlating data streams in real time using technology such as CEP, which allows for review of potential violations and opportunities.
Market surveillance solutions based on CEP technology offer a smooth course ahead for capital markets firms looking to both actively hunt for opportunities and quickly respond to threats in a trading environment where high volatility is the new normal.
Comments