Run:AI Creates First Fractional GPU That Effectively Creates Virtualized Logical GPUs

Run:AI | May 07, 2020

  • Run:AI's fractional GPU system effectively creates virtualized logical GPUs, with their own memory and computing space that containers can use and access as if they were self-contained processors.

  • Run:AI also solved the problem of memory isolation, so each virtual GPU can run securely without memory clashes.

  • The addition of fractional GPU sharing is a key component in Run:AI's mission to create a true virtualized AI infrastructure, combining.


Run:AI, a company virtualizing AI infrastructure, today released the first fractional GPU sharing system for deep learning workloads on Kubernetes. Especially suited for lightweight AI tasks at scale such as inference, the fractional GPU system transparently gives data science and AI engineering teams the ability to run multiple workloads simultaneously on a single GPU, enabling companies to run more workloads such as computer vision, voice recognition and natural language processing on the same hardware, lowering costs.

Today's de facto standard for deep learning workloads is to run them in containers orchestrated by Kubernetes. However, Kubernetes is only able to allocate whole physical GPUs to containers, lacking the isolation and virtualization capabilities needed to allow GPU resources to be shared without memory overflows or processing clashes.

Read More: IT Management Solution Announces Updated Range of Virtualization Services for Businesses

Run:AI's fractional GPU system effectively creates virtualized logical GPUs, with their own memory and computing space that containers can use and access as if they were self-contained processors. This enables several deep learning workloads to run in containers side-by-side on the same GPU without interfering with each other. The solution is transparent, simple and portable; it requires no changes to the containers themselves.

 

To create the fractional GPUs, Run:AI had to modify how Kubernetes handled them. "In Kubernetes, a GPU is handled as an integer. You either have one or you don't. We had to turn GPUs into floats, allowing for fractions of GPUs to be assigned to containers. Run:AI also solved the problem of memory isolation, so each virtual GPU can run securely without memory clashes,

Dr. Ronen Dar, co-founder and CTO of Run:AI.



A typical use-case could see 2-4 jobs running on the same GPU, meaning companies could do four times the work with the same hardware. For some lightweight workloads, such as inference, more than 8 jobs running in containers can comfortably share the same physical chip.

The addition of fractional GPU sharing is a key component in Run:AI's mission to create a true virtualized AI infrastructure, combining with Run:AI's existing technology that elastically stretches workloads over multiple GPUs and enables resource pooling and sharing.

 

Some tasks, such as inference tasks, often don't need a whole GPU, but all those unused processor cycles and RAM go to waste because containers don't know how to take only part of a resource. Run:AI's fractional GPU system lets companies unleash the full capacity of their hardware so they can scale up their deep learning more quickly and efficiently,

Run:AI co-founder and CEO Omri Geller.



Read More: Synamedia Advances Video Network Portfolio with HPE ProLiant Servers for Network Virtualization

About Run:AI

Run:AI has built the world's first virtualization layer for AI workloads. By abstracting workloads from underlying infrastructure, Run:AI creates a shared pool of resources that can be dynamically provisioned, enabling full utilization of expensive GPU compute. IT teams retain control and gain real-time visibility – including seeing and provisioning run-time, queueing and GPU utilization – from a single web-based UI. This virtual pool of resources enables IT leaders to view and allocate compute resources across multiple sites - whether on premises or in the cloud. The Run:AI platform is built on top of Kubernetes, enabling simple integration with existing IT and data science workflows.

Spotlight

Josh Simons, Chief Technologist for HPC at VMware, chats how AI, machine learning, and datascience are changing with GPU virtualization with NVIDIA vComputeServer.

Spotlight

Josh Simons, Chief Technologist for HPC at VMware, chats how AI, machine learning, and datascience are changing with GPU virtualization with NVIDIA vComputeServer.

Related News

SERVER VIRTUALIZATION

Napatech accelerates infrastructure services processing for data center applications

Napatech | July 06, 2022

Napatech™ the leading provider of programmable Smart Network Interface Cards (SmartNICs) used for Data Processing Unit (DPU) and Infrastructure Processing Unit (IPU) services in telecom, cloud, enterprise, cybersecurity and financial applications worldwide, today announced a set of new SmartNIC capabilities that enable standard, unmodified applications in edge and core data centers to benefit from offloaded and accelerated compute and networking functions. As enterprises, communications service providers and cloud data center operators deploy virtualized applications and services in edge and core data centers, they increasingly leverage workload-specific coprocessors to offload functions such as Artificial Intelligence (AI), Machine Learning (ML), storage, networking and infrastructure services from general-purpose server CPUs. This architectural approach not only maximizes the availability of server compute resources for running applications and services but also improves system-level performance and energy efficiency by running the offloaded workloads on devices optimized for those specific tasks such as programmable SmartNICs, also known as Data Processing Units (DPUs) or Infrastructure Processing Units (IPUs). Thanks to this offload trend as well as an acceleration in global data center deployments, programmable SmartNICs represent the fastest-growing segment of the NIC market, with a Total Available Market (TAM) forecasted to reach $3.8B/year by 2026 according to Omdia. To maximize the portability of their software and to accelerate their time-to-market, developers of cloud applications and services incorporate industry-standard Application Programming Interfaces (APIs) and drivers within their software. Data center operators therefore need to be able to select offload solutions that are compatible with the relevant standards, to avoid having to create custom, vendor-specific versions of their software. The latest upgrade to Napatech's Link-Virtualization™ software, release 4.4, addresses this challenge by incorporating networking and virtual switching features that implement full support for the relevant open standards, while delivering best-in-class performance and functionality. Specifically, Link-Virtualization now supports a fully hardware-offloaded implementation of the Virtio 1.1 Input/Output (I/O) virtualization framework for Linux, including the standard kernel NIC interface, which means that guest Virtual Machines (VMs) do not require a custom or proprietary driver. Link-Virtualization also supports the open-standard Data Plane Development Kit (DPDK) fast-path running in guest VMs to maximize the performance of functions such as Open Virtual Switch (OVS). Link-Virtualization is also fully compatible with OpenStack, allowing a seamless integration into cloud data center environments worldwide. Other new features incorporated in Link-Virtualization include IPv6 VxLAN tunneling, RPM-based setup for OpenStack Packstack, configurable Maximum Transmission Unit (MTU), live migration on packed ring, port-based Quality of Service (QoS) egress policing and more. The software is available on Napatech's portfolio of SmartNICs, powered by AMD (Xilinx) and Intel FPGAs, that span 1 Gbps, 10 Gbps, 25 Gbps, 40 Gbps, 50 Gbps and 100 Gbps port speeds. As one example of the industry-leading performance delivered by Link-Virtualization, the complete offload of the OVS data path onto the SmartNIC means that only a single host CPU core is required to run the OVS control plane while delivering industry-leading throughput of 55 million packets per second for Port-to-VM-to-Port (PVP) traffic and 130 million packets per second for Port-to-Port (PTP) traffic. Reclaiming host CPU cores previously required to run OVS and making them available to run applications and services leads to a significant reduction in the number of servers required to support a given workload or user base. This in turn drives significant reductions in overall data center CAPEX and OPEX. It also results in lower system-level power consumption and improved energy efficiency for the edge or cloud data center. To aid in the estimation of cost and energy savings for specific use cases, Napatech provides an online ROI calculator, which data center operators can use to analyze their projected savings. "Napatech's Link-Virtualization software enables data center operators to optimize the performance of their networking infrastructure in a completely standards-compatible environment, which maximizes their flexibility in selecting applications, Besides full support for standard APIs, the solution also incorporates critical operational features such as Receive Side Scaling (RSS) for efficiently distributing network traffic to multiple VMs and Virtual Data Path Acceleration (vDPA), which enables the live migration of running workloads to and from any host, whether or not a SmartNIC is present." Napatech CMO Jarrod J.S. Siket About Napatech Napatech is the leading supplier of programmable FPGA-based SmartNIC solutions used in telecom, cloud, enterprise, cybersecurity and financial applications worldwide. Through commercial-grade software suites integrated with robust, high-performance hardware, Napatech accelerates telecom, networking and security workloads to deliver best-in-class system-level performance while maximizing the availability of server compute resources for running applications and services.

Read More

VIRTUAL DESKTOP TOOLS

Lulea University of Technology Transforms Delivery of Education with Citrix

Citrix | August 22, 2022

Just like work, education today can happen anywhere. And students want the freedom and flexibility to choose where and how they learn best. Luleå University of Technology (LTU) understands this, and is using solutions from Citrix Systems, Inc. to tear down the walls of the traditional classroom and deliver a modern experience that empowers its student to learn from anywhere and perform at their best. “Teaching has become space independent, We are a university of technology and we are expected to be at the forefront in using IT to provide a superior educational experience in the classroom, the dorm room and anywhere in between.” Daniel Ström CIO, LTU Borderless Classrooms When the pandemic hit, LTU did just this, leveraging Citrix DaaS to create cloud-based workplaces through which teachers and students could access everything they needed to stay connected and continue learning. LTU’s students use 300 different applications, some of which require significant computing power traditionally provided by powerful workstations available in on-campus computer labs. And for students to continue learning remote - often using low-powered, personal laptops and other devices - LTU needed to provide access to these applications. It was a tall order. But LTU was able to fill it using Citrix DaaS to provide students with secure access to their desktops and applications from anywhere and Microsoft Azure to allocate the processing power and computing resources required to support them. “With Citrix, it was very easy for us to step up and make this available for all the students through their own computers,” said Lennart Isaksson, Head of Student Computer Lab Administration, LTU. And it was able to do it quickly, enabling full remote availability for 19,000 students in a matter of days. “When it comes to remote desktop use cases, nothing comes close to Citrix,” Ström said. Simplified Management One of the biggest advantages of Citrix, according to Isaksson, is the time saved in application management. “With the traditional PC model, it would take several hours to install all the necessary applications on each PC. And we have over 800 student computer lab PCs,” he said. “With Citrix, it’s much better. We have a single, base image with all the applications and we just replicate it to all the virtual machines that we are spinning up.” Enhanced Security Another key benefit of Citrix DaaS is the enhanced security it provides. “It’s a much safer approach than the traditional PC model,” Isaksson said. “All the apps are up to date and always available from any endpoint device.” Modern Education And of importance to Ström, Citrix makes it easy for LTU to keep pace with technology as it evolves and deliver the IT performance and experience that its students expect. “Young people today are raising the bar,” he said. “When you build a new computer lab under the traditional model, you get only 12 to 24 months when it is modern. After that, the students get less and less satisfied. With the Citrix model, we can add new machines in the background as the technology evolves, so the computer labs are always new and fresh.” LTU joins hundreds of educational institutions around the world who are using Citrix to transform learning. Click here to read the stories of their success. About Citrix Citrix builds the secure, unified digital workspace technology that helps organizations unlock human potential and deliver a consistent workspace experience wherever work needs to get done. With Citrix, users get a seamless work experience and IT has a unified platform to secure, manage, and monitor diverse technologies in complex cloud environments.

Read More

VIRTUAL DESKTOP TOOLS

Options Announces VMware Cloud Verified Status in Park Royal Data Center

Options | August 01, 2022

Options Technology, the leading Capital Markets services provider, today announced the achievement of VMware Cloud Verified status in their Park Royal Data Center. The accomplishment follows the VMware Cloud Provider Principal Partner Status awarded to the firm last year, along with Cloud Verified status at LHC, LD4 and NY5 sites. Cloud Verification assures that a partner is validated to expertly deliver cloud Infrastructure-as-a-Service across VMware’s best-in-class network, storage, and compute solutions to meet individual client needs specifically. VMware Cloud Verified partner services enable customers to achieve unmatched levels of consistency, performance, and interoperability for traditional and containerised enterprise applications with the confidence the service received is based on the most advanced VMware cloud technologies. Options President and CEO Danny Moore said, “We are delighted to announce the addition of Park Royal to our growing portfolio of cloud solutions and Options’ third VMware Cloud Verified status in recent months. Today’s news not only validates the rapid expansion of services across our global backbone but Options’ unique position as the partner of choice in the delivery of holistic, flexible, and resilient cloud strategy for clients.” Today’s news comes as the latest in a series of strategic announcements for Options, including its seventh Microsoft Gold Partner status, hosting capabilities in Aruba IT3 Bergamo, and the launch of its Quantify and Data Store products. In 2019, Options received investment from Boston-based Private Equity Firm Abry Partners. This investment has enabled Options to accelerate its growth strategy and develop its technology platform while expanding its global reach in key financial centres. About Options Options Technology is the No. 1 provider of IT infrastructure to global Capital Markets firms, supporting their operations and ecosystems. Founded in 1993, the firm began life as a hedge fund technology services provider. Today, the company provides high-performance managed trading infrastructure and cloud-enabled managed services to over 550 firms globally, providing an agile, scalable platform in an Investment Bank-grade Cybersecurity wrapper. Options clients include the leading global investment banks, hedge funds, funds of funds, proprietary trading firms, market makers, broker/dealers, private equity houses, and exchanges. With offices in 8 key cities; New York, Toronto, Chicago, London, Belfast, Hong Kong, Singapore, and Auckland, Options are well placed to service its customers on-site and remotely. In 2019, Options secured a significant growth investment from Abry Partners, a Boston-based sector-focused private equity firm. This investment has enabled Options to considerably accelerate its growth strategy to invest further in its technology platform and expand its reach in key financial centres globally. Options has been named among the UK’s leading growth companies in the 2021, 2020, 2019, 2018, and 2017 Sunday Times HSBC International Track 200 league table. About Abry Partners Abry is one of the most experienced and successful sector-focused private equity investment firms in North America. Since its founding in 1989, the firm has completed over $82 billion of leveraged transactions and other private equity or preferred equity placements. Currently, the firm manages over $5.0 billion of capital across their active funds.

Read More