Amazon recently announced that Alexa, the smart personal voice assistant is coming out of a small desktop gadget (pick your flavor: echo dot, spot, hub, show, etc.) and getting into the very fabric of our life. Anything and everything is getting an Alexa boost from things like microwave, home security, car infotainment and even your wall clock. Soon, smart devices are going to get even smarter – they can even hear you whisper (e.g. when your kids are napping in the living room) and answer you back with a whisper (not to disturb the sleep of your kids). That is so smart!! And thus, a smart fabric of Artificial Intelligence and Machine Learning is being built around us to make our lives easier.
But Smart is not only limited to the smart personal assistants. Today, there are a variety of smart personal assistants from Amazon, Google, Apple and others in the market, but the key thing to remember is that “Smart” is an evolution of life. We have seen many innovations already that are all happening around us, that are termed “Smart”. Over the last decade, we have happily adopted anything and everything that made our life easier. Be it the SmartPhone that came out from Apple in 2007 that gave us a new and unique human-compute interface, or Smart cars that are environmentally friendly, fuel efficient and easier to park in large cities. Or, smart credit cards with embed microprocessors that have the ability to securely process payment transactions, protecting consumers from fraud and identity theft. Life is getting Smarter all around us. I call this living in a Smart Era!
While the world around us has evolved to be “Smarter”, certainly consumer technologies have been leading the charge. However, the backend infrastructure to support Smart devices was still struck in a dinosaur era. For decades, the “backbone” supporting these devices was built using proprietary and purpose built hardware appliances such as Routers, Switches, Load Balancers, Firewall, CDN, Wan Accelerators, Gateways, etc. The list can go on and on. All these purpose built and proprietary hardware appliances are great at what they do, but they are not flexible, hinder innovation and are locked down. I call the hardware defined everything world ‘a dinosaur era’ because it is hard to sustain.
Agility, Effectiveness and Efficiency are the key drivers for the long term success of any business. In 2007, we saw a movement to disrupt this age old way of building monolithic and vendor locked networking infrastructure. Software took over hardware and Open Source technologies such as kernel based virtual machine (KVM), open virtual switch (OVS) and OpenStack became the building blocks of cloud infrastructure and revolutionized the way next generation networks would be built. Agility, Automation, and Community-driven Standards were at the heart of this software defined transformation. I called this a software defined everything world. The pendulum swung all the way to the other end and hardware became commodity for compute, network and storage infrastructure. Software became the brain and the most value-added asset in the software defined everything world. We entered an evolutionary phase with a radically different way of building software defined infrastructure with server virtualization, network virtualization, software defined networks, network function virtualization and software defined storage. So many innovations, all happening pretty quickly. Great news for the industry right? Well, not quite.
Nothing in life is free. As we know, there is a hidden cost to every transformation. This is true for the software defined world as well. As we shifted network functions from purpose-built hardware appliances to software-driven virtual appliances, we saw that general purpose CPUs took the center stage. While these CPUs are great for general application tasks, they aren’t specialized in handling network and storage workloads.
Running Virtual Network Function (VNFs) on the CPU impacts networking performance in terms of throughput, packet rate, latency, and I/O operations for storage workloads. Further, disaggregation of software and hardware, along with the virtualization of servers and networks, consumes even more CPU due to software overloads. Think of the layers of abstractions that software virtualization like hypervisors or virtual routers or overlay networking creates in order to run workloads in a hardware-agnostic manner. The result of all of this is decreased efficiency that looks something like below.
We achieved flexibility at the expense of efficiency. As seen above, we started with a clean slate, high performance, bare metal, commodity server and due to SDX such as virtualization, disaggregation and Security overheads, ended up with only 20-25% compute power available for running business applications, or for offering that power to tenants for infrastructure as a service (IaaS). Imagine a server costing $20,000 with only $5000 worth of available compute capacity. Essentially, due to SDX penalties, we need to pay up 400% on the capital expense to match to a bare metal server’s compute power. This is a great news for server CPU vendors as it increases the server footprint but it defeats the very essence of software defined everything. In term of capital savings, we are at the same place as before or even worse when building the next generation software defined infrastructure with commodity servers. In addition to the CapEx increase, the purely software defined world performs inferiorly compared to a hardware defined world, due to CPU-bound network and storage processing.
At this point, you would obviously ask – “But what does this all have to do with the Smart Era?” In the next blog, I will explain how Smart NICs are revolutionizing the software defined world to achieve smartness in building next generation cloud scale Infrastructure. Indeed, SmartNICs are a key innovation to make this era truly a Smart Era.