Celestial AI Inc., a startup developing optical chip technology to speed up artificial intelligence models, today announced that it has raised $100 million in funding.

The round included contributions from the venture capital arms of Samsung Electronics Co. Ltd. and Porsche AG. IMEC, one of the world’s largest nanoelectronics research labs, participated as well along with about a half-dozen other backers.

Advanced AI models deployed in data centers don’t always run as fast as they could. The reason is a performance issue commonly referred to as the memory wall challenge. Celestial AI has developed an optical interconnect technology, dubbed Photonic Fabric, that it says addresses the issue.

The memory wall challenge relates to the fact that the speed of processors, and in particular graphics processing units, has been increasing at a faster rate than memory chips’ performance. As a result, memory chips are struggling to keep up with data center requirements. That phenomenon is starting to negatively impact the speed at which AI models process information. 

If a graphics card can theoretically crunch 2 gigabytes of data per second, but can only retrieve 800 megabytes of data from memory in that time frame, it will process just those 800 megabytes. In other words, the GPU’s processing capacity can’t be fully used. That prevents AI models from optimally utilizing GPUs’ performance and thereby limits processing speeds.

Celestial AI’s answer to the challenge is to increase the rate at which processors can retrieve data from memory and send information back. The startup’s Photonic Fabric interconnect transfers data between memory chips and processors not in the form of electricity, which is the usual approach, but rather using light. Because light travels faster than electricity, network speeds increase.

Celestial AI claims Photonic Fabric can provide compute-to-memory connectivity speeds of several terabits per second. Moreover, it’s promising latencies in the nanosecond range. It’s not the only company using optical methods to speed up compute-to-memory data movement, but it claims that its technology provides up to 25 times more bandwidth than competing approaches.

Some optical interconnects transfer data to processors via the processors’ so-called beachfronts, or sides. Celestial AI’s Photonic Fabric doesn’t connect to a chip’s beachfronts, but is rather meant to be deployed below the chip as a kind of base layer. That increases the amount of chip surface area through which information can be moved, which is part of the reason the startup’s technology promises faster performance.

Making AI models faster is not the only benefit the startup promises to provide. According to Celestial AI, Photonic Fabric can also help data center operators reduce hardware costs. 

Because of certain technical constraints, adding more memory to an AI cluster often requires adding more processors. According to Celestial AI, Photonic Fabric enables companies to provision additional memory without necessarily buying new processors. As a result, hardware requirements decrease, which lowers data center costs. 

“This next wave of data center infrastructure is being architected to deliver tremendous advancements in AI workload efficiencies, resulting from disaggregation of memory and compute resources which is enabled by optical interconnectivity,” said AI founder and Chief Executive Officer Dave Lazovsky. 

The startup says its technology supports other use cases as well. Celestial AI claims that Photonic Fabric can be used to move data not only between memory chips and processors, but also between processors. Furthermore, it claims the technology can manage the flow of data within each processor.

Celestial AI plans to license its technology to companies such as chipmakers. According to the company, Photonic Fabric can be integrated into products such as GPUs and central processing units. Its plans to monetize its technology also involve an internally-developed AI accelerator chip, Orion, that uses Photonic Fabric to manage data movement.

Image: Unsplash