Red Hat, the open source software firm owned by IBM, is acquiring Neural Magic, a startup that optimizes AI models to run faster on commodity processors and GPUs.

Terms were not disclosed.

MIT research scientist Alex Matveev and professor Nir Shavit founded Somerville, Massachusetts-based Neural Magic in 2018 inspired by their work in high performance execution engines for AI.

Neural Magic's software enables AI workloads on processors and GPUs to run at speeds similar to those of specialized AI chips, such as TPUs. The reason Neural Magic is able to achieve this is because it can run models on off-the-shelf processors, which often have more memory than those traditionally used to deploy AI and deep learning.

Many big tech companies, from AMD to a barrage of other start-ups like NeuReality, Deci, CoCoPie, OctoML, and DeepCube, offer some kind of AI optimization software. But Neural Magic stands out as one of the few with a free platform, along with an assortment of open source tools to complement it.

Neural Magic had raised up to $50 million in venture capital so far from backers including Andreessen Horowitz, New Enterprise Associations, Amdocs, Comcast Ventures, Pillar VC, and Ridgeline Ventures.

Especially interesting is that their work on vLLM, an open source project for model serving," said Red Hat CEO Matt Hicks talking about the company's work with Neural Magic. "With Neural Magic, the open source company is certainly enjoying a vLLM-based "enterprise-grade" stack that will allow customers to optimize and deploy models across cloud environments, all under full control over the infrastructure and security, said Hicks.

For instance, Red Hat is already involved in the vLLM project and uses this work to run models in products such as Red Hat Enterprise Linux AI and Red Hat OpenShift AI. "Through this combined work between Red Hat and Neural Magic, we believe that its infrastructure partners will be better equipped to scale AI across platforms, and its integrated service provider partners will be able to add more robust inference and performance to their offerings," Hick added.

"AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardized and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs," Hicks said in a statement. "We are excited to add Neural Magic's transformative AI innovation to our hybrid cloud-focused portfolio, furthering our drive not just to be the 'Red Hat' of open source, but the 'Red Hat' of AI as well."

Red Hat's acquisition of Neural Magic comes as the company makes a number of AI-related announcements at KubeCon, the annual computing conference, in Salt Lake City this week. Red Hat also unveiled Climatik, a tool developed in partnership with Intel and together with the efforts of Bloomberg and IBM to optimize data center energy efficiency, and new releases of its OpenShift AI and Device Edge development platforms.

With escalating power demands and deployment costs for making AI, big tech providers are getting desperate to snatch firms that could assist in optimizing AI algorithms. In April, for instance, Nvidia got its hands on Run:ai, which wants to make life easier for developers in terms of managing and optimizing their AI infrastructure. And in October 2020, Intel scooped up SigOpt, specializing in optimization modeling.