Big Data for Finance

BNY Mellon Report Explores Big Data in Finance, in the 21st Century

A report recently published by BNY Mellon - it's available to anyone who request it from them - takes a look at big data, its definition and its potential, and "sketches its transformational influence on the 21st century global financial system, mainly from an asset management perspective."

IT Innovation Pays Dividends for Financial Leaders

By Thomas Kejser, Fusion-io
www.fusion-io.com 

Time is money.  In the finance industry, the truth of this adage has driven leaders to the forefront of the technology adoption curve.  When microseconds and milliseconds mean the difference between profits and losses, financial organisations soon build up teams of phenomenal IT experts to ensure the infrastructure powering the company and its services runs at top speeds.

Many of these teams focus entirely on managing huge datasets, and when it comes to processing massive amounts of data, time is often the first casualty.  For this reason, financial companies, who use big data to better understand their customers, their habits and preferences, are constantly striving to build faster, more powerful systems to meet the needs of their businesses.

Managing Big Data Woes: A How-to Guide for Hedge Funds

By Mark Kenney, Abacus Group
www.abacusgroupllc.com

The explosion of data over the last several years, coupled with dynamic compliance requirements, has put excessive stress on many firms’ storage capabilities.  With the need to juggle multiple strategies, multiple funds and multiple prime brokers, it’s not uncommon for today’s hedge funds to deal with many terabytes of data.

In the past, most firms didn’t have the infrastructure or capital to properly store, back up and monitor "big data."  They could keep it on site by implementing or adding capacity to an in-house storage solution - absorbing the cost of not only the high-ticket hardware and software, but also the staff with the knowledge to keep things running.

Big Data in Financial Services – Opportunity or Cost?

By John Bantleman, RainStor
www.rainstor.com

We recently hosted a dinner in New York City with 20 technology executives focused on big data in banking and financial services.  I found the event insightful, so I thought it would be interesting to share some of the perspectives from those who attended.

The first (and close to my heart) is the separation of the big data business problem and the available technology and solution architectures.  I am not trying to detract from the applicability and relevance of the Hadoop technology stack and the attractive, low cost scalability platform being central to making to solving big data challenges for today’s enterprise.  Yes, there is an elephant charging straight towards us but I do feel as an industry, we spend a lot more time focused on the technology itself and less on the business problems it solves.

Winning the Big Data Race for Trading Communication

By Paul Metcalfe, Orange Business Services
www.orange-business.com

Big data is increasingly becoming a key concern for technologists, especially with the recent introduction of recording for voice and SMS on mobile phones for trading activities in the UK.  As additional voice recording requirements contribute to the growing repository of data firms have to hold to comply, the pressure is on to manage and house bigger data volumes more than ever before.

As well as meeting regulatory requirements for greater transparency and a more comprehensive record of past trading activities, this data can also serve a purpose in other areas such as dispute resolution.

In-Memory Data Grids - A Step Change for Reconciliation and Matching

By Neil Vernon, Gresham Computing
www.gresham-computing.com

Right now in financial services, big data is a really big topic and if you aren’t talking about big data, then you simply aren’t talking.  So, why the excitement and what could it mean for the wider financial services market?

Much of the big data discussion currently centres on big data analytics; rapidly analysing huge volumes of data to improve understanding of customers and take informed actions to better serve them.  The excitement is the new ability big data analytics brings to fundamentally affect the way business gets done for retention and growth.

The Benefits of Going Big for Eikon Users

By Philip Brittan, Thomson Reuters
www.thomsonreuters.com

The scale of data in financial services is, although significant, dramatically smaller than it is for online consumer companies like Google or Amazon.  For financial services, petabytes are huge, but for those firms they are a rounding error.  Nevertheless the need for real-time throughput makes financial services data a unique challenge and there are tremendous learnings and benefits to be had from the big data world not only in terms of how we manage data at enterprise level but also in how we are able to present and manipulate that data through next generation desktops, such as Eikon.

Clearing the Big Data Hurdle: The Open Source Advantage

By Christopher M. Carter, Cloudsoft Corporation
www.cloudsoftcorp.com

In today’s world there is a new understanding, the emergence of a new “reality” that is much, much different than what we had even a decade ago.  This new reality of big data that exists within today’s enterprises cannot be underestimated.  Big data is becoming more important in all industries, but none more so than in the finance arena, both in enterprises and big finance in Wall Street firms.  Most businesses aren't ready to manage this flood of data, much less do anything interesting with it.

Learning From Mistakes: Applying Big Data to Realtime Risk Analysis

by Daniel Wakefield, IMGROUP
www.imgroup.com

As the financial sector once again reels under yet more allegations of huge losses (this time at JPMorgan Chase) there is a more than mild sense of déjà vu here at IMGROUP.  Yet again, questions are being asked about how one desk, perhaps even one trader, can lose such a significant amount of money without anybody appearing to notice.  The jury is still out as to whether we are talking about illegal activity and losses being deliberately hidden or whether it is a case of ignorance caused by the complexity of the system.  In some ways however this is irrelevant.  The fact remains that huge losses still seem to be easy to hide – wilfully or not.

Defeating Market Manipulation with Three-Layer Surveillance

By Melinda Wilson, Sybase, An SAP Company
www.sybase.com

“The last few years have been the most volatile for all of recorded history ... 10 of the biggest 20 daily up-swings and 11 of the largest 20 daily drops since the beginning of 1980 to the end of last month have occurred in just the last three years.” - Andrew Lo, professor of finance at the MIT Sloan School of Management, The New York Times, Sept. 11, 2011

Trading firms are charting a course toward better handling the financial industry’s growing volumes of market data, messages and unstructured data, all the while keeping watch for new opportunities and market manipulation.  The right technology can help firms monitor and correlate data streams in real time, but the wrong technology - or doing nothing at all - could run them aground.