Run:AI Creates First Fractional GPU That Effectively Creates Virtualized Logical GPUs

Run:AI | May 07, 2020

  • Run:AI's fractional GPU system effectively creates virtualized logical GPUs, with their own memory and computing space that containers can use and access as if they were self-contained processors.

  • Run:AI also solved the problem of memory isolation, so each virtual GPU can run securely without memory clashes.

  • The addition of fractional GPU sharing is a key component in Run:AI's mission to create a true virtualized AI infrastructure, combining.


Run:AI, a company virtualizing AI infrastructure, today released the first fractional GPU sharing system for deep learning workloads on Kubernetes. Especially suited for lightweight AI tasks at scale such as inference, the fractional GPU system transparently gives data science and AI engineering teams the ability to run multiple workloads simultaneously on a single GPU, enabling companies to run more workloads such as computer vision, voice recognition and natural language processing on the same hardware, lowering costs.

Today's de facto standard for deep learning workloads is to run them in containers orchestrated by Kubernetes. However, Kubernetes is only able to allocate whole physical GPUs to containers, lacking the isolation and virtualization capabilities needed to allow GPU resources to be shared without memory overflows or processing clashes.

Read More: IT Management Solution Announces Updated Range of Virtualization Services for Businesses

Run:AI's fractional GPU system effectively creates virtualized logical GPUs, with their own memory and computing space that containers can use and access as if they were self-contained processors. This enables several deep learning workloads to run in containers side-by-side on the same GPU without interfering with each other. The solution is transparent, simple and portable; it requires no changes to the containers themselves.

 

To create the fractional GPUs, Run:AI had to modify how Kubernetes handled them. "In Kubernetes, a GPU is handled as an integer. You either have one or you don't. We had to turn GPUs into floats, allowing for fractions of GPUs to be assigned to containers. Run:AI also solved the problem of memory isolation, so each virtual GPU can run securely without memory clashes,

Dr. Ronen Dar, co-founder and CTO of Run:AI.



A typical use-case could see 2-4 jobs running on the same GPU, meaning companies could do four times the work with the same hardware. For some lightweight workloads, such as inference, more than 8 jobs running in containers can comfortably share the same physical chip.

The addition of fractional GPU sharing is a key component in Run:AI's mission to create a true virtualized AI infrastructure, combining with Run:AI's existing technology that elastically stretches workloads over multiple GPUs and enables resource pooling and sharing.

 

Some tasks, such as inference tasks, often don't need a whole GPU, but all those unused processor cycles and RAM go to waste because containers don't know how to take only part of a resource. Run:AI's fractional GPU system lets companies unleash the full capacity of their hardware so they can scale up their deep learning more quickly and efficiently,

Run:AI co-founder and CEO Omri Geller.



Read More: Synamedia Advances Video Network Portfolio with HPE ProLiant Servers for Network Virtualization

About Run:AI

Run:AI has built the world's first virtualization layer for AI workloads. By abstracting workloads from underlying infrastructure, Run:AI creates a shared pool of resources that can be dynamically provisioned, enabling full utilization of expensive GPU compute. IT teams retain control and gain real-time visibility – including seeing and provisioning run-time, queueing and GPU utilization – from a single web-based UI. This virtual pool of resources enables IT leaders to view and allocate compute resources across multiple sites - whether on premises or in the cloud. The Run:AI platform is built on top of Kubernetes, enabling simple integration with existing IT and data science workflows.

Spotlight

Learn how Palo Alto Networks’ cloud-native security solution helps organizations protect cloud, hybrid cloud and on-premises infrastructure — and what Insight is doing to partner with Palo Alto Networks to help clients experience even stronger success.

Spotlight

Learn how Palo Alto Networks’ cloud-native security solution helps organizations protect cloud, hybrid cloud and on-premises infrastructure — and what Insight is doing to partner with Palo Alto Networks to help clients experience even stronger success.

Related News

VPN

Turbo VPN Allows Access to Worldwide Content and Endless Entertainment for Global Users

Innovative Connecting Pte Ltd | November 25, 2022

Turbo VPN, developed by Innovative Connecting, a Singapore-based VPN provider, is providing worldwide access to content and endless entertainment for global users. Reputable for its super-fast and secure connection, this VPN service was established in 2018 and is trusted by 300 million users. Featuring a new tagline of "VPN for Everyone," Turbo VPN has quickly expanded its reach and is ready to serve internet users in more than 150 different countries. Turbo VPN uses an ultra-fast VPN to facilitate live streaming and gaming in high definition. Using secure, military-grade encryption and a no-logs policy, Turbo VPN is able to protect online data and minimize risks. Users can watch worldwide entertainment with just one click with Turbo VPN, including social media, streaming, and gaming. With Turbo VPN, users can hide their address and location while being able to access streaming platforms cross-regionally. The VPN service also offers data security when connecting to mobile hotspots and public WiFi sources. By featuring a strict no-log policy and high-level encryption protocols, Turbo VPN offers a service that users can trust. Additionally, users will find this application simple to use, offering a one-tap connection to the desired VPN server. Headquartered in Singapore, the company that developed Turbo VPN, Innovative Connecting, is a fast-growing technology company, trusted by over 300 million users globally. Innovative Connecting makes its customers the top priority, offering 24/7 customer care service and the utmost privacy. Innovative Connecting was founded with the vision to create a high-quality network that is accessible to everyone. It wanted to create a safe place for entertainment of all genres to be easily available to people of all ages. Founded in Singapore, a global hub for innovation and technology, Innovative Connecting focuses on connecting the world with technologies and believes in the power of the open web. When using Turbo VPN, users can protect up to 5 devices at once while also freely streaming their favorite shows, regardless of whether they are abroad or at home. Users can access websites, apps, social media networks, and other online services without any slowdown. Turbo VPN is also practical for gamers who wish to minimize a game's latency and accelerate mobile game speeds. Turbo VPN can improve the gaming experience for users of many gaming apps and websites. A Chrome-compatible version of Turbo VPN was released in June 2022. Turbo VPN is also compatible with iOS, Mac, Android, and Windows. This user-friendly application makes stable and fast VPN servers accessible to internet users worldwide. Turbo VPN is available for purchase for as little as $4.17 per month. One-month plans are available for $11.99. Subscribed Turbo VPN users receive unlimited access to over 21,000 fast and secure servers from over 45 countries without having to pay any extra download or usage fees.

Read More

VIRTUAL DESKTOP STRATEGIES

Sophos Firewall Unleashes Xstream Power for Distributed and Enterprise Edge Computing

Sophos | November 22, 2022

Sophos, a global leader in innovating and delivering cybersecurity as a service, today introduced new Sophos Firewall capabilities to better meet the complex and demanding needs of distributed and enterprise edge computing. Sophos Firewall now delivers performance enhancements that accelerate encrypted traffic inspection, dynamic traffic routing for Internet Protocol version 6 (IPv6), added resiliency with software-defined wide area network (SD-WAN) load balancing and high-availability enhancements, and seamless integration with Microsoft Azure Active Directory. “One of the key benefits of the Xstream architecture and Flow Processors is that they are programmable. This means that while other firewalls get slower over time, we can increase performance, even when we add new features and capabilities, Our design ensures customers’ investment in Sophos Firewall is future-proofed and enables seamless transition to a cloud-enabled world. SD-WAN and Secure Access Service Edge (SASE) demand a more efficient platform, which is not only resilient but also makes day-to-day management easier and faster than ever.” Daniel Cole, vice president of network security product management at Sophos The effective and reliable assessment of network traffic is critical in protecting against threat actors, as evidenced in Sophos’ 2023 Threat Report that published today. Distributed offices, remote workforces, cloud workloads, custom-built legacy apps, and a growing reliance on global software-as-a-service providers create a configuration and risk management headache for network security managers. Sophos Firewall now provides the performance, protection and resiliency that distributed enterprises require, while simplifying the management of complex networks. Benefits include: Advanced performance and protection: A new high-performance dynamic routing engine and Xstream Transport Layer Security (TLS) FastPath acceleration improves encrypted traffic inspection while also adding headroom for traffic that requires deep-packet inspection; the asymmetric cryptographic capabilities within Xstream Flow Processors – included in every XGS Series appliance – enable TLS inspection on even the most demanding networks Added resiliency and peace of mind: New SD-WAN load balancing for performance and reliability in the event of an internet service providers’ (ISP) outage along with enhancements to high-availability clusters ensure maximum business continuity and uptime for mission critical networks Improved ease of management: Managing network security is easier than ever with new Microsoft Azure Active Directory integration for seamless administrator single sign-on and new host and service object search Sophos Firewall integrates with Sophos ZTNA (zero trust network access) under one unified management plane and is a key pillar of Sophos’ SASE strategy, providing a more simplified, scalable and secure solution over traditional remote-access virtual private networks (VPN). The network solution is also part of the Sophos Adaptive Cybersecurity Ecosystem, which integrates Sophos’ entire portfolio of products, services and Sophos X-Ops threat intelligence for faster and more contextual and synchronized detection, protection and response. Availability Sophos Firewall is available for immediate purchase exclusively through Sophos’ global channel of partners and Managed Service Providers (MSPs). It is easily managed in the cloud-native Sophos Central platform alongside other solutions, where users can oversee installations, respond to alerts and track licenses and upcoming renewal dates via a single, intuitive interface, or by Sophos Managed Detection and Response (MDR). What Analysts, Channel Partners and Customers Say “With their latest firewall release, Sophos has leveraged the flexibility of their Xstream architecture to deliver improvements in performance for VPN throughput and more efficient handling of TLS encrypted traffic, which is vitally important in today’s encrypted world,” said Christopher Rodriguez, research director of IDC’s Security & Trust practice. “They’ve also completed the build-out of their networking feature set to provide an integrated SD-WAN solution. Combined with other recent enhancements to their secure access portfolio like ZTNA as a Service, they’re positioning to provide the features and capabilities that larger distributed enterprise organizations demand, while also building out a SASE strategy that will appeal to organizations of all sizes.” “As a multinational technology consulting firm that’s relied on Sophos Firewall since it was first incepted, Seidor Networks intimately knows the offering is a must-have solution for protecting against malware and other unwanted network traffic,” said Sean Hancock, ISP manager at Seidor Networks. “Sophos is continuously innovating and adding new features that are industry best. This new version of Sophos Firewall raises the bar even higher with unrivaled network protection and performance; and when you pair Sophos Firewall with Sophos XDR, the results are truly next level as endpoints and firewalls share real-time threat intelligence for further improved network visibility and lateral movement control.” “The new Sophos Firewall software has multiple advantages for all of our customers,” said Marc Hurrelmann, chief executive officer at Midland IT. “Many of the features added have been designed to address the challenges that larger organizations are facing with implementing SD-WAN, optimizing performance, scaling their network, improving resiliency and up-time, and enhancing management efficiency. Smaller organizations will benefit from all the added value packed into Sophos Firewall with better performance, protection, networking, and management.” About Sophos Sophos is a worldwide leader and innovator of advanced cybersecurity solutions, including Managed Detection and Response (MDR) and incident response services and a broad portfolio of endpoint, network, email, and cloud security technologies that help organizations defeat cyberattacks. As one of the largest pure-play cybersecurity providers, Sophos defends more than 500,000 organizations and more than 100 million users globally from active adversaries, ransomware, phishing, malware, and more. Sophos’ services and products connect through its cloud-based Sophos Central management console and are powered by Sophos X-Ops, the company’s cross-domain threat intelligence unit. Sophos X-Ops intelligence optimizes the entire Sophos Adaptive Cybersecurity Ecosystem, which includes a centralized data lake that leverages a rich set of open APIs available to customers, partners, developers, and other cybersecurity and information technology vendors. Sophos provides cybersecurity-as-a-service to organizations needing fully-managed, turnkey security solutions. Customers can also manage their cybersecurity directly with Sophos’ security operations platform or use a hybrid approach by supplementing their in-house teams with Sophos’ services, including threat hunting and remediation. Sophos sells through reseller partners and managed service providers (MSPs) worldwide. Sophos is headquartered in Oxford, U.K.

Read More

VIRTUAL DESKTOP TOOLS,SERVER VIRTUALIZATION

Options Technology Achieves VMware Cloud Verified Status in Singapore, Expands Availability in Asia

Options Technology | January 23, 2023

Options Technology, a capital markets services provider, announced that it has achieved VMware Cloud Verified status in SG1. This follows the company's achievement of VMware Cloud Provider Principal Partner Status and Cloud Verified Attainment at other sites. These accomplishments demonstrate Options Technology's commitment to giving its customers access to the latest virtualization technology. The Cloud Verification assures that the partner is validated to expertly deliver cloud infrastructure-as-a-service to meet specific client needs using VMware's network, storage, and compute solutions. With the Cloud Verified Status at SG1, Options Technology can now offer VMware's cloud infrastructure-as-a-service to customers in the Singapore area. The CEO of Options, Danny Moore, said that their partnership with VMware continues to provide service excellence, capability, and choice for their customers across the capital markets. The VMware Cloud Verified status means that Options Technology has met the strict requirements set by VMware and is able to provide customer-focused cloud infrastructure solutions that meet the needs of the capital markets sector. With the Cloud Verified Status, Options Technology is now well positioned to provide customized cloud infrastructure solutions to Singapore customers. About Options Technology Options Technology is a leading provider of IT infrastructure to global capital markets firms. Founded in 1993, the company originally provided technology services to hedge funds, but now provides high-performance managed trading infrastructure and cloud-enabled managed services to over 550 firms globally, including investment banks, hedge funds, and exchanges. With offices in 9 key cities around the world, Options is well-positioned to serve its customers both on-site and remotely. In 2019, the company secured a significant growth investment from Abry Partners, a private equity firm, which has enabled it to accelerate its growth strategy and expand its reach in key financial centers globally. Options has been named among the UK’s leading growth companies in the Sunday Times HSBC International Track 200 league table for five consecutive years.

Read More