In his new book, Flash Boys, Michael Lewis directs his gleeful analytic spotlight on flash trading. True to form, Lewis tells a heroic story, in this case about how a team from Royal Bank of Canada found a defense for fair play against a mysterious group of traders who seemed to be gaming the system.
But Lewis gives short shrift to one of the real stars of the story: the data center and the people who run it. The forces that the flash traders have taken advantage of are creating a new type of data center, one in which many companies seek to cluster together in one location so they can communicate faster. The phenomenon is called proximity co-location.
It is not just in finance that this is happening. The digital advertising industry in which massive amount of advertising dollars are spend through auctions that take milliseconds also has clusters of companies hosting together in one data center. So do companies who uses lots of cloud resources.
Equinix, the global data center and computing infrastructure supplier, is making a business out of productizing this architecture. So far, Equinix has become important hub for finance and advertising, providing companies with the ability to host their computers, connect with each other, and take advantage of other services depending on their need.
This seems like the beginning of a network effect. As Equinix accelerates the development of real time infrastructure in new industries that are seeking to imitate what happens in finance and advertising, it will become the default choice It is time for business leaders who are waking up to the power of real time information to transform business processes to take a closer look at what is happening. Your industry may be the next to employ proximity co-location to support an innovation like flash trading or advertising exchanges.
Physics is the Law
One of the fascinating aspects of the tale Lewis tells is the role that the architecture of the data center and network plays in altering the speed of trading. A transaction that starts on a trading floor in mid-town Manhattan, may travel to Brooklyn, New Jersey, and many other destinations on its way to being delivered to an exchange.
While the virtual world of cyberspace is a set of beautiful abstractions, the implementation is a birds nest of hardware and network connections. Seemingly outmoded concepts like the physical distance matter a lot. Even with lots of high-speed connections and ultra high-powered computers, if you want really fast performance when two computers are going to talk to each other, they need to be physically near each other.
“Physics isn’t just a good idea, it’s the law!” said Mark Uhrmacher, founding CTO of ideeli, the pioneer in flash sales that required a hyper-scalable server infrastructure, who is currently a consultant specializing in fixing both architecture and technology leadership challenges.
The effects of this law are that many of the companies that engage in program trading and flash trading and advertising auctions attempt to locate their servers physically near the exchanges and the other companies they deal with. If you can shave milliseconds off of your response time, it can be a big deal.
“As long as there are marketplaces, like advertising exchanges, that have financial incentives to minimize latency these hubs will make sense,” said Uhrmacher. “By reducing distances and switching points the same software/hardware will be able to perform better enabling more transactions and better business performance. In fact, I’m surprised someone isn’t using these hubs for competitive differentiation in the IaaS/PaaS space against AWS.”
Finance and Advertising are First
In the first two massive real time data processing infrastructures, program trading and advertising technology, Equinix has become the dominant hub for proximate co-location of the infrastructure, raising three interesting questions:
- Why does proximity co-location matter for such infrastructure?
- How long will it matter?
- Does Equinix have an insurmountable lead for hosting future proximity co-location hubs as more industries adopt real time computing for high value business processes?
Why does Promixity Co-location Matter?
As John Battelle, search guru and opiner on technology, points out in his November article the creation of real time technology for delivering advertisements is one of the greatest achievements of the modern era. The broad story of the development of this infrastructure is fascinating and there are many victories that have been achieved including creation of:
- A liquid market for on-line advertising inventory.
- Online advertising auctions take place in less than 100 milliseconds and are conducted whilst the reader is waiting for her web or mobile page to load in her browser.
- A incredibly effective data supply chain that combines usage data, demographics, and many other forms of data that is processed both in batch and real time to provide a huge amount of information for evaluating the value of a page view, and, importantly the fleeting value of the reader.
- Massively scalable databases for accessing huge repositories in real time.
- Advanced products such as re-targeting so that ads follow high value buyers around the web.
Proximity co-location is needed because once performance under 100 milliseconds is required, the system must be optimized from end to end. Google, Facebook, Amazon, Twitter, Netflix all have Byzantine distributed architectures that combine massive centralized data centers, distributed systems for processing, caching, highly optimized networks, into a massive systems that deliver high performance to users around the globe.
The proximity co-location hubs Equinix has created are a different animal from Google, Facebook and the others because they are open systems. Both ad tech and program trading follow the same pattern. There are many players who each are offering a service such as analytics, settlement/clearing, hosting, storage, compute, etc. that must be as fast as possible. Each player may provide a service that supplies or use services from another player and also must communicate at high speed with people requesting ads from all over the Internet. In addition, the services are sometimes competing with each other, and sometimes cooperation. Usually, a few big players are enforcing rules, usually informally, to make sure everyone gets along. For example, in the ad exhanges, Google has a lot to say about how things work but there is no strong governance framework like in the finance industry.
In addition, each player has different needs for hosting. Some are building incredibly advanced technology and just want data center space so they can do everything themselves. Others want more managed services. Almost all of them want hyper-fast connections to cloud providers like Amazon and Rackspace so they can run parts of what they do in the public cloud.
Proximity co-location works best in these scenarios because of the variety it supports. Each participant can do as little or as much as desired on its own and can use a partner for the rest. Proximity colocation is cheap by comparison and only as complex as the members of the ecosystem want it to be. In a way, proximity co-location is an open innovation model, like an open source project, but one that is powered by hardware, networking, and a physical location.
What Equinix provides is optimized communication between all the players, between the players and public clouds, and between the players and the rest of the Internet where the requests for ads will be delivered. In addition, Equinix provides access to whatever sorts of hosting that the players want. Finally, Equinix supports an ecosystem of partners who are providing high value services to the players. Such partners include consultants on design and architecture, providers of high speed databases.
Proximity co-location for ad tech is more complex than the model created for program trading several years earlier. The ad tech cloud hub is a better model for other industries than the one created for program trading because ad tech requires connections to public cloud providers and to consumers on the Internet.
Equinix is also creating another type of proximity co-location for enterprises who are using lots of cloud applications. In this cloud hubbing set up, companies who host at Equinix are offered high speed connections to the cloud providers they use most. (See “Why You Need a Cloud Hubbing Strategy” and “Five Ways Cloud Hubbing Can Change Your Business” for more.)
How Long will Proximity Co-location Matter?
The massive Content Delivery Networks like Akamai, Limelight, NetApp, MirrorImage, and others from the big telecom providers, were created by necessity. Proximity co-location emerged from the same need, not by design, but because Equinix was the obvious choice for the first few providers and then the network effect kicked in.
In my view, proximity co-location will matter for a long time because networking speed is still governed by physical distance and the hub model is natural, not only from an architectural perspective, but because a such a hub is a cluster of knowledge and expertise.
In addition, it is likely that more specialized services will be available as public cloud offerings and will need optimized connections. Proximity co-location is a natural way to minimize this expense across all players.
Can Equinix Defend its Lead?
As I pointed out in “Why Hasn’t Open Source Taken Over Storage?” when a product requires lots of engineering it is far more defensible than when it is more abstract, in other words, more a computer science problem. Proximity co-location requires lots of engineering to make the connections between all the players work well, and that puts Equinix in a strong position.
In addition, proximity co-location is a small part of the value created by program trading and ad tech, but the absence of an optimized co-location infrastructure can damage an emerging offering for real-time computing. For that reason, even when the players are super sophisticated, they would rather just have the co-location problem solved for them.
So, in my view the important question is not: Will Equinix keep dominating the world of proximity co-location? Rather, I am curious to see what is the next industry to create a real-time system to increase efficiency, create new offers, and change the way business is done.
Follow Dan Woods on Twitter:
Dan Woods is CTO and editor of CITO Research, a publication where early adopters find technology that matters. For more stories like this one visit www.CITOResearch.com. Dan has performed research for Equinix, Rackspace, and other hosting and infrastructure companies.