Microsoft Azure hardware innovation and research projects

Having attended the recent Microsoft Azure Virtual Datacenter Tour webinar, a series of highly interesting announcements were made around upcoming Microsoft Azure cloud technologies. Many of these technologies stem from Microsoft Research projects and are very promising in terms of changing the way we perceive the daily functions of the fundamental pillars of an IT infrastructure, namely compute, networking and storage. This blog post outlines some of the most important aspects of technologies already used in Azure and promising technologies soon to be introduced.

If you need to find out more about the Azure Global Infrastructure technologies and ongoing research projects, please navigate to: https://msdcinfo.azurewebsites.net/.

Azure Modular Datacenter (MDC)

The Azure Modular Datacenter (MDC) is a tailored-built mobile datacenter for customers who need cloud computing capabilities in hybrid or challenging environments, including remote areas. The MDC can give customers a path to migrate apps to Azure while still running these workloads on-premises with low-latency connections to their own datacenter. This provides a stepping stone for transforming workloads to the Azure API with the option of continuing to run these apps on-premises, or in public or sovereign clouds.

Azure Space

Azure Space was created to be the platform and ecosystem of choice for the mission needs of the space community. It's designed to make connectivity and compute increasingly attainable across industries including agriculture, energy, telecommunications, and government. Microsoft has announced two key partnerships which will enable the interconnection of Azure Space with the Azure Modular Datacenters:

- A partnership with SpaceX Starlink will provide high-speed, low-latency satellite broadband for the new Azure Modular Datacenter (MDC)

- Building on existing Azure Orbital partnership with SES, Microsoft will support its O3B Medium Earth Orbit (MEO) constellation O3b MEO, to extend connectivity between Azure cloud datacenter regions and cloud edge devices.

Satellite diagram

Project Natick

Similarly to running Azure in space, project Natick implements Azure datacenters under water. Project Natick seeks to understand the benefits and difficulties in deploying subsea datacenters worldwide. Phase two extends the research we accomplished in phase one by deploying a full-scale datacenter module in the North Sea, powered by renewable energy.

Azure Sonic

SONiC is an open source network operating system based on Linux that runs on switches from multiple vendors and ASICs. SONiC offers a full-suite of network functionality, like BGP and RDMA, that has been production-hardened in the data centers of some of the largest cloud-service providers. It offers teams the flexibility to create the network solutions they need while leveraging the collective strength of a large ecosystem and community.

Open Compute Project

The Open Compute Project Foundation (OCP) was initiated in 2011 with a mission to apply the benefits of open source and open collaboration to hardware and rapidly increase the pace of innovation in, near and around the data center. Now celebrating our 10th anniversary, just wait till you see what we have planned for the next ten years!

Project Olympus

Project Olympus enables IT and datacenter operators to take advantage of community-developed innovation and to scale proven hardware designs for their specific usage models.

Project Cerberus

This new open-sourced industry standard for platform security is collaboratively developed by the OCP community. Cerberus provides a critical component for security protection that’s been missing from server hardware: adding protection, detection, and recovery from attacks on platform firmware. Cerberus enables a more secure firmware implementation on all platform types across the industry, from datacenter to IoT devices. The Cerberus specification also supports hierarchal root of trust, so platform security can be extended to all I/O peripherals using the same architectural principles.

Project Denali

Denali is a streamlined solid state drive (SSD) architecture that also sets a new industry standard—unlocking innovation, improving reliability, and speeding time to market. The firmware is simplified and the interface is standardized, reducing the complexity of these components while lowering virtual machine cost and improving performance. Denali enables closer control of drive behavior and tailoring of the drive to specific workloads, as well as faster validation and deployment of new technologies.

Azure Stack HCI

Hyper-converged infrastructure is at the core of Azure Stack. HCI is in turn based on software-defined networking and software-defined storage technologies.

Microsoft Optics for the Cloud

Microsoft Optics for the Cloud is a series of research projects by Microsoft Research which introduce optical networking in public clouds. The two most prominent projects comprising Optics for the Cloud are project Sirius, Silica, HSD and project Iris.

Project Sirius

Project Sirius is investigating whether ultra-fast optical switching within data centers could allow us to sidestep these disruptions. It aims to develop an all-optical, data-center-wide network that is completely flat, in contrast to the hierarchy of electrical switches used today. By eliminating the inefficiencies of hierarchy and leveraging the strengths of optics, such a network could provide better and more predictable performance with higher reliability and at lower cost. Sirius builds upon recent advancements in nanosecond-granularity optical switching and increasing maturity of the optical fabrication ecosystem. It opens up the exciting opportunity of completely rethinking the cloud network stack from the ground up. This, we believe, can enable new cloud applications and scenarios that are difficult to support today.

Project Silica

Project Silica is developing the first-ever storage technology designed and built from the media up, for the cloud. We are leveraging recent discoveries in ultrafast laser optics to store data in quartz glass by using femtosecond lasers, and building a completely new storage system designed from scratch around this technology. This opens up an incredibly exciting opportunity to challenge and completely re-think traditional storage system design, and to co-design the future hardware and software infrastructure for the cloud.

Project HSD

Project HSD is a collaboration between Microsoft Research Cambridge and Microsoft Azure to re-imagine an old idea – holographic storage – as a cloud-first design. We are capitalizing on the recent exponential improvement and commoditization in optical technologies such as smartphone cameras, as well as the unique opportunity to design at cloud scale.

Project Iris

Project Iris explores novel designs of regional and Wide Area (WAN) cloud networks from the ground-up motivated by the rapid growth of cloud network traffic across data centers.

Shoal

Microsoft Research introduces the Shoal disaggregated data center design principle. Disaggregated racks comprise a dense cluster of separate pools of compute, memory and storage blades, all inter-connected through an internal network within a single rack. However, their density poses a unique challenge for the rack’s network: it needs to connect an order of magnitude more nodes than today’s racks without exceeding the rack’s fixed power budget and without compromising on performance. We present Shoal, a power-efficient yet performant intra-rack network fabric built using fast circuit switches. Such switches consume less power as they have no buffers and no packet inspection mechanism, yet can be reconfigured in nanoseconds. Rack nodes transmit according to a static schedule such that there is no in-network contention without requiring a centralized controller. Shoal’s congestion control leverages the physical fabric to achieve fairness and both bounded worst-case network throughput and queuing. We use an FPGA-based prototype, testbed experiments, and simulations to illustrate Shoal’s mechanisms are practical, and can simultaneously achieve high density and high performance: 71% lower power and comparable or higher performance than today’s network designs.