Big-data and machine learning software provider Databricks Inc. said today that it will acquire the generative artificial intelligence startup MosaicML Inc. in a deal worth $1.3 billion, with a goal to make open-source AI models more accessible to enterprise businesses.

The acquisition comes at a time when generative AI is capturing the attention of every industry and enterprise businesses are beginning to integrate it into their workflows. Generative AI applications can produce original text, images and computer code based on users’ natural language prompts. These capabilities have put the technology on the map since the release of OpenAI LP’s ChatGPT chatbot, which can answer questions conversationally.

MosaicML provides its own generative AI large language models that companies can use to train and fine-tune on their own data with high quality and low cost. The company is best known for its own family of the MPT family of LLMs, with more than 3.3 billion downloads of the MPT-7B model. It recently released the MPT-30B model that has only 30 billion parameters and quality equal to OpenAI’s GPT-3 model, which has more than 175 billion parameters.

Databricks said that through the acquisition, MosaicML’s technology would be delivered on its data and AI infrastructure through the Lakehouse Platform, which would open up new opportunities for customers to access AI training, fine-tuning and deployment. Notably, it would provide customers a way to serve and customize generative AI models and retain control, security and ownership of their own data without spending a lot of money.

“Every organization should be able to benefit from the AI revolution with more control over how their data is used,” said Ali Ghodsi, co-founder and chief executive of Databricks. “Databricks and MosaicML have an incredible opportunity to democratize AI and make the Lakehouse the best place to build generative AI and LLMs.”

MosaicML’s training, fine-tuning and inference deployment service is already used by a number of customers such as the nonprofit research institute Allen Institute for AI, Generally Intelligent, Scatter Labs and the browser-based coding tool Replit. Hippocratic AI, a company building a chatbot for the healthcare industry is also using MosaicML’s capabilities to train its generative AI model.

“We started MosaicML to solve the hard engineering and research problems necessary to make large scale training more accessible to everyone,” said Naveen Rao, co-founder and chief executive of MosaicML. “With the recent generative AI wave, this mission has taken center stage. Together with Databricks, we will tip the scales in the favor of many.”

With the ability to host their own models in Databricks Lakehouse, companies will be able to specialize and tailor their models on specific company data and securely deploy it. That’s important because foundation models can only answer generic questions and do general tasks, but don’t have specialized contexts for what law firms or the healthcare industry or other specialized businesses may be doing.

At the same time, many businesses are concerned about sharing their proprietary data with AI providers because of privacy and safety concerns. Using served models such as OpenAI’s could lead to the potential of data leaks or other dangers that could open up the company to other risks. That’s especially important for highly regulated industries where data control and needs to stay within the confines of secure clouds where the model and the data need to remain together.

SiliconANGLE CEO and analyst John Furrier said that in the future the AI will be the value to enterprises not so much just the software – the data is the intellectual property.  “Enterprises want their own models to build, tune and run,” Furrier said. “Enterprises don’t want to share it or have it uploaded into a public LLM. This is the trend that Databricks is getting in front of and it’s all about the new ‘data developer.’”

Databricks can now extend its platform for building, training and hosting traditional machine learning models to large language models, says Justin DeBrabant, senior vice president of product at enterprise customer data platform provider ActionIQ Inc. “That means Databricks offers products and services on the Lakehouse that extend from ETL [data management’ tools to SQL analytics to custom machine learning and now to hosted LLMs,” he says. “That’s pretty compelling.”

Databricks competes in a similar market to cloud-based data warehouse provider Snowflake Inc., which recently acquired Neeva Inc., a startup developing an AI-based search engine using a large language model. Both companies are holding their annual user conferences this week.

Image: MosaicML