Solace Systems

The reasons HFT is shifting asset classes may surprise you

2012-02-19T15:24:10Z

At Solace, the shift to electronic trading has been at the core of our business in capital markets for many years. We’ve seen a variety of recurring use cases that are now mainstream, specifically for equities. For example:

Buy-side firms engaging in high-frequency trading (HFT), either onsite or at a co-location facility.
Sell-side firms offering smart order routing or internal order crossing to increase revenue or save on costs of order processing.
Exchanges accelerating and enhancing their services to compete with ever more sources of liquidity.
All kinds of capital markets participants improving how they handle real-time risk across trading systems, and/or automate middle-office settlement and record keeping.

Reg NMS and the Explosion in HFT

The path to these systems began with decimilization of equities in 2001 which dramatically increased electronic trading. Then in 2007, Reg NMS opened the door for many more competitors to supply liquidity, which created a n-way set of very-short lived arbitrage opportunities across trading venues. Reg NMS included rules that regulated order fills to assure they were at the best price (across all available liquidity) within a time window (National Best Bid and Offer — NBBO). That led all sell-side participants to require very low latency technology, because they had to fall within the NBBO window to play at all. Mix in a bunch of clever quants to develop models and programmers to automate them and you have the HFT explosion in equities.

Dodd-Frank Sets the Stage for HFT Across More Assets

Over the past 12-18 months we’ve had more and more conversations with people looking to improve capacity of systems that support electronic trading of other asset classes such as foreign exchange, fixed-income and derivatives. I didn’t think too much about why until I read something Patrick Whalen, head of trading for AllianceBerstein, said in this article about the growth of HFT and electronic trading. In a nutshell, Patrick points out that the Volcker Rule will prevent banks from holding assets like bonds or currency positions on their books — they’ll need to move ‘em out quickly and efficiently, which means more electronic trading. As part of the new Dodd-Frank rules about the clearing of swaps, regulators are also pushing for an end to the over-the-phone transactions in favor of more transparent electronic trading. That will change the dynamics of those markets in terms of the need for efficiency, and the mechanics of acquiring those assets.

Thinking about his observations makes the future of electronic trading far more clear. Dodd-Frank will drive more asset classes to electronic trading for transparency and increased monitoring of adherence to rules. Most of these asset classes trade across many venues, which means there will be HFT-style arbitrage opportunities between them. Most of the trading firms have already made investments in low-latency technology, so the implementation times can be very short as the markets increase in liquidity. Therefore the barriers to HFT in asset classes like derivatives, FX or fixed income should be far lower than when equities underwent the shift.

Settle in and Watch the Trading Landscape Shift

Every significant new financial regulation has its own batch of winners and losers — the winners are those who best understand the secondary implications of the new rules, individually and collectively, and position themselves to capitalize on change. “Skating to where the puck is going to be,” as the Wayne Gretzky line goes. As Reg NMS gave birth to accelerated HFT for equities, the changes driven by Dodd-Frank will leave at least as big a change in the markets across many more asset classes, and will accelerate the spread of electronic trading across capital markets. Grab a big bowl of popcorn and settle in to see which firms ride the wave to new levels of success. Like the best movies, there will be lots of plot twists as well as plenty of heroes and villains.

The Journal on three world-changing tech trends

2012-01-31T00:36:29Z

There is an excellent article in today’s Wall Street Journal that details the technology-driven, macro shifts that are happening right now in the world of business. The three mega-trends they cite:

Big data – for anyone in IT this is a “no kidding Sherlock” inclusion, but it is remarkable to step back and appreciate how quickly big data is changing the landscape and its long term implications.
Smart manufacturing – The next wave in manufacturing – engineering from the molecular level to manufacture products using methods more like the ones we use to print documents. The emphasis shifts to design and IP, and away from cheap labor and physical factories.
The wireless revolution – The extension of internet access to more than a billion of the world’s people, paired with a new generation of applications and services, is fundamentally changing the way the world connects, socializes and engages in commerce.

The author does a little flag waving around America being an epicenter for these three trends, but the more important point is that all three are underway now and are good bets to literally change the world.

These trends line up very nicely with the big picture principles of Solace — to unshackle information and make it available wherever it is needed in ways not currently possible. Whether loading up a big data repository, collaborating on a design for smart manufacturing, or giving the world’s mobile user base real-time access to…well, anything…we’re in step with the vision outlined in this article.

If you haven’t read it, I recommend you check it out: The Coming Tech-led Boom (Wall Street Journal)

Ground control to major storm…

2012-01-10T15:04:41Z

Today, we announced a new customer relationship with communications and information technology giant Harris Corporation detailing that they have selected Solace to power the on-the-ground portion of a satellite weather system jointly developed by NASA (the space guys) and NOAA (the weather guys).

This is part of a project called GOES-R (Geostationary Operational Environmental Satellite R-Series). Here is its stated mission as described on the GOES-R website:

The advanced spacecraft and instrument technology used on the GOES-R series will result in more timely and accurate weather forecasts. It will improve support for the detection and observations of meteorological phenomena and directly affect public safety, protection of property, and ultimately, economic health and development.

GOES-R provides essential information related to air quality, coastal and marine monitoring, fire monitoring, hurricane forecasts, precipitation and floods, land cover observations, volcanoes, lightning detection, severe thunderstorms, tornado warnings and more. If you’re interested in learning more about this project and the advances it will deliver, I highly recommend checking out the GOES-R site. Here are some fun facts I learned in just a few minutes of browsing:

The instruments on the GOES-R series will produce more than 50 times the information provided by the current GOES system and will offer unique observations of the environment, with emphasis on hazardous weather in the western hemisphere and space weather impacts.
Data provided by the GOES-R instruments will be used to create 65 different products that meteorologists can use to monitor the weather.
Information from the GOES-R satellite will improve short-term weather forecasts such as severe thunderstorms and tornadoes, as well as long-term climatological trends.
GOES-R will be the first of a new fleet of GOES satellites (R/S), marking the first significant upgrade in GOES weather monitoring capabilities since the start of the GOES I/M series in 1994.

The instruments on the GOES-R satellite are particularly cool:

The GOES-R Advanced Baseline Imager (ABI), a sixteen channel imager with two visible channels, four near-infrared channels, and ten infrared channels, will provide three times more spectral information, four times the spatial resolution, and more than five times faster temporal coverage, than the current system.
The Geostationary Lightning Mapper (GLM) is an optical transient detector and imager that provides early indication of storm intensification and severe weather events, tornado warning lead time of 20 minutes or more and data for long-term climate variability studies.
Solar UV Imager (SUVI) locates coronal holes, flares and coronal mass ejection source regions. SUVI data characterizes active region complexity, enabling improved forecasting of space weather and early warnings of possible impacts to the Earth environment.
Space Environmental In-Situ Suite (SEISS) is an ensemble of electron, proton and heavy ion detecting sensors. SEISS data drives the Solar Radiation Storm portion of NOAA’s Space Weather Scales and other NOAA operational Alerts and Warnings.
Extreme UV/X-ray Irradiance Sensor (EXIS) detects solar soft X-ray irradiance (XRS) and solar Extreme UltraViolet (EUVS) spectral irradiance in the 5-127 nm range. XRS monitors solar flares (and helps predict proton events) that can disrupt communications and degrade navigational accuracy. EUVS monitors solar variations that directly affect satellite drag/tracking and ionospheric changes, which impact communication and navigation operations.
The magnetometer measures the time-varying field in the magnetosphere. It provides the only operational measure of the impact of geomagnetic storms at geosynchronous orbit, and it is key for interpreting solar radiation storm measurements by SEISS.

The GOES-R satellite is scheduled to launch in 2015. We are very proud to to be working with Harris to make the GOES-R vision a reality.

Capturing data streams at “big data” scale

2011-12-14T17:39:42Z

Before you can break into a cold sweat about tackling the design of a system that analyzes big data volumes, you first need to be able to capture the data. More often than not, the design parameters feel like a traffic engineering problem — there are simply too many cars and not enough road.

Certainly, LAN and WAN network technology introduces many limits and the largest commercial databases (e.g. Netezza, Teradata) or open source big data stores (e.g. Hadoop, Splunk) can only store data so fast. Even in memory data grids are limited by how many in-memory writes can be performed per second. Managing the distributed information is usually some kind of middleware, once again, usually a commercial product (e.g. JMS or MQ) or open source code (e.g. Kafka or Qpid).

Even at full speed, a single instance of the middleware layer runs at far less capacity than the network, in-memory grid, or data store can process, making it the weakest link. This means to keep up, the software middleware traffic has to be scaled horizontally across many middleware brokers or servers. Each application becomes a fragile layered mess of servers and any disruption can lead to significant cascading problems of volume and backlog.

An increasing number of our customers with big data projects (e.g. in capital markets, internet infrastructure and transportation) have thrown in the towel on attempting to use traditional JMS, MQ, or open source for this scale of data capture. Instead, they’re opting for Solace’s hardware messaging to feed their big data stores. Where software messaging peaks at a few thousand messages per second, Solace’s failsafe queuing solution exceeds 150,000 messages per appliance. That means you would need to horizontally scale a typical JMS, MQ or open source alternative to 30 or more servers (assuming it could sustain 5,000 msgs per JMS server) to match the throughput of one Solace appliance. It just makes everything easier if the layers and moving parts in your scaling architecture stay light and lean. Fewer servers, less datacenter space, fewer outages = cheaper and less headaches.

Many customers initially think a commercial solution like Solace’s has to be more expensive than open source, after all open source is free and Solace costs money. But it is easy to show that when you factor in server costs, rack space, power, and management it’s far cheaper to pay for an appliance that replaces 30 or more servers.

Big data is right in the sweet spot of (one of the many) use cases that this company was built to address. If you are struggling with these problems, we’d like the opportunity to talk to you about solving them.

Handling data growth as digital information doubles every 18-24 months

2011-11-03T22:41:50Z

Various studies have claimed that the amount of digital information in the world doubles every 18 to 24 months. Using the conservative 2 year estimate, that means there is 32 times as much digital data today as there was in 2001. Some of that growth is from all those cute cat videos and embarrassing Facebook photos, but much of it is valuable business data generated by automated processes within and between businesses.

Today we announced a new partnership integrating our message routers with the B2B and Managed File Transfer (MFT) components of SEEBURGER’s Business Integration Server platform. The big “why” behind this partnership is captured in the “growth of global data” factoid above. SEEBURGER is a powerhouse in the B2B and MFT spaces, and does a lot more application integration (A2A/EAI) than people think.

Today, the growth in data generated by these systems puts stress on the enterprise services bus underneath their software, but with Solace integrated into BIS and MFT, SEEBURGER customers get worry-free capacity for their distributed information requirements in an easy-to-use appliance form factor.

As the volume of information grows, so does the need for smarter distribution technology. Remember, most popular middleware technologies were invented more than a decade ago, when information volumes were 1/32^nd what they are today. At Solace, we’ve changed the middleware game by boosting throughput by orders of magnitude to help companies deal with the flood of data without suffering from the cost and complexity of datacenter sprawl.

De-risking Excel use in front-office trading

2011-11-16T21:43:08Z

Microsoft Excel has been a staple of productivity on Wall Street for years because it puts the power to calculate market data models directly in the hands of quants and traders. But Excel use has been difficult for the bank’s IT staff to manage and auditors to oversee since the traders’ formulas live on their PC and are in constant flux. Wouldn’t it be great if trading firms could let their traders have the tool they want (Excel) and still be able to track and audit formulas and changes centrally?

That’s just what we recently announced with our partner, BCC Group of Germany. BCC Group introduced a solution called CALCNODE that can run thousands of very complex Excel spreadsheets with better performance and accountability than ever before.

CALCNODE offloads complex calculations to a high-performance computing environment, and uses our appliance to supply the spreadsheets with real-time market data and keep real-time data flowing between spreadsheets and the powerful calculation engine which lives in a shared back end.

This lets companies boost the productivity of their employees by making their existing Excel spreadsheets run faster, while simultaneously giving them unprecedented oversight and accountability so they can better comply with increasingly stringent government regulations.

It’s an ingenious use of high-speed messaging and high performance computing to solve a long term challenge – reducing the risk of using of Excel in trading.

Solace Recognized with Deloitte Fast50 Award

2011-11-16T21:41:23Z

Deloitte released their new Fast50 rankings today, and we’re excited to be on it for the second year in a row. Launched nearly 15 years ago, the program celebrates the achievements and evolution of the Canadian technology sector.

We’re especially proud of a leadership award we were given, described by Deloitte’s Mark Noonan in an Ottawa Business Journal article:

“Solace won a special leadership award in the category of hardware for demonstrating their ability to create a distinct competitive advantage in a high-growth market, which allows them to dominate their sector.”

We’re proud of our #25 spot on the list, and as Noonan further explained we would have cracked the top 10 if our fiscal year mapped to the calendar year. “If we look at their current year revenue model, they have achieved accelerated revenue in 2011. If we were to annualize that, Solace’s growth rate would almost be 2,600% percent, which would have been top 10,” he said.

And since real growth and results are what matter, we’re OK with that.

High frequency trading back in the political crosshairs

2011-10-12T04:58:26Z

This week at the Futures Industry Association (FIA) conference, the hot topic is whether government regulators should force the registration and regulation of high frequency trading firms. As this Wall Street Journal article highlights, government regulators know they don’t fully understand the impact of HFT on global markets, especially how they help or hurt non-HFT traders.

There is fairly clear evidence that HFT has lowered spreads by providing increased liquidity when markets are running smoothly, but there have also been incidents like the “Flash Crash” where HFT appeared to play a role in the panic as automated trading systems exited the market when volatility spiked. The press loves controversy, so most of what you will read about HFT is negative—highlighting what might or could go wrong.

If efforts regarding regulations proceed, I hope HFT gets a fair trial. In the current economic climate and election cycle, there’s a real risk that media punditry, political grandstanding, and whatever voice the Occupy Wall Street movement ends up contributing might skew an outcome before the data and facts are understood.

It seems to me that the genie is out of the bottle where electronic trading is concerned. We need to make sure that we understand and tweak the model for maximum benefit, not initiate change based on fear of the unknown.

bet365: Middleware innovation is safe bet for future growth

2011-09-19T19:56:56Z

Sports betting has changed quite a bit in the last decade. Online innovations have meant that players now have more choice than ever in the way they bet and what they bet on. In-Play betting, where players can place bets while a sport is in progress, e.g. the scorer of the next goal in a football match or the winner of the next point in a tennis game, is probably one of the most important developments and also one of the most popular. Leading the way is eGaming company bet365. Headquartered in Stoke-on-Trent, England, they are a global business that employs 1,700 people and is live in 17 languages.

Their site includes:

In-Play sports betting
A wide range of popular casino, poker, lottery and bingo games.
The ability to bet on financial markets, such as the closing prices of currency pairs and market indices.

It’s a remarkably rich and diverse offering, and if you take a look at their site, www.bet365.com, you’ll quickly realize how hard the company works to offer the best experience they can for their customers.

That’s all great, but why am I writing about gambling innovation in a middleware blog? Let’s take a closer look at what’s going on behind the scenes. Each day bet365 can serve over one hundred thousand concurrent users on their main In-Play system and over a million on their partner banner system. Odds information is updated every few seconds and around a million transactions can be processed in a day. That all adds up to the need for a seriously secure, robust and efficient data distribution infrastructure that gets the odds data out and can efficiently process inbound activity.

For bet365, Solace gives the confidence that as the business continues to evolve, they can reliably meet the demand for their services far into the future. We’re proud to be of service to such an innovative and respected market leader.

Baseball, the cloud, and big data

2011-09-09T14:13:38Z

As the release of the upcoming movie Moneyball approaches, it’s inevitable that we’ll be hearing much more about baseball’s sabermetrics. ReadWriteWeb is out front of the topic with a nice article today on How Big Data and the iPad have Fundamentally Changed Baseball.

This article ties together three of my favorite topics – big data analytics, cloud computing infrastructure and baseball. Take a look at the iPad dashboard at right and think about a starting pitcher and catcher sitting together on a flight using this kind of highly-visual tool to decide how to pitch each hitter. Now think about the rudimentary paper-based systems from five or ten years ago. Which pitcher has the edge? I’m sure that comparable data exists for batters on pitcher’s tendencies and release points, but it does appear that the overwhelming advantage of this technology favors the pitcher/catcher game plan.

But do the results back up the theory? Take a look at the aggregate statistics across major league baseball the last 5 years. Since 2006 both league batting average and runs per game have fallen every single year. Batting averages are down from .269 to .255 and runs per game are down from 4.86 to 4.28 (per team). Only one offensive stat is up in each of those years – strikeouts, from 6.52 per team per game to 7.06. MLB is marketing this as a pitching renaissance. Maybe it’s a data/knowledge renaissance? Pitchers simply have better tools than hitters and it’s showing up as a reduction in offense.

This phenomenon is not at all unique to baseball. What Moneyball and sabermetrics did for baseball happened a decade before in financial services. Human traders that specialized in arbitrage and technical chart reading have been overrun by automated systems that perform split-second algorithmic trading resulting in billions in profits. Online giants like Google and Amazon have applied similar big data analytics techniques to know what you want to buy before you do. I would be willing to bet that a similar explanation is behind recent record big company profits, despite the luke warm global economy. The past decade has seen an explosion in the number and accuracy of tracked company metrics, along the lines of the baseball dashboard above. It’s the same story – interactive dashboards and process automation are buoying decision making accuracy, resulting in fewer mistakes and greater corporate efficiency. This is a classic example of what Greenspan would have called “worker productivity improvements enabled by technology”.

We are marching inexorably towards a interconnected world of huge volumes of ever changing data and anywhere, anytime access which will make today’s incredible improvements look quaint when we look back ten years from now. Whether you work in a baseball front office, at a leading internet company, in corporate IT or supply companies like these with the world’s most scalable messaging middleware it’s an exciting time to be building applications that connect the web and mobile worlds with the processing power of the cloud in real time.