RUUZEN
AI

Q.ANT teaches photonic chip to accelerate language model performance

The German company combined an optical coprocessor with an AMD accelerator to speed up AI computations and reduce video memory load.

19 Nov, 12:35Read in Russian
habr.com

Q.ANT has introduced an experimental hybrid inference system in which part of the language model computations is performed using light. At the SC26 conference in Chicago, developers demonstrated the joint operation of the photonic processor NPU Gen 2+ and the AMD Instinct MI210 accelerator.

The optical chip performs a discrete Fourier transform, which is used instead of traditional self-attention mechanism operations in certain neural network layers. According to the developers' concept, this approach will accelerate query processing and reduce the volume of KV-cache, which occupies a significant portion of video memory when working with long contexts.

The experiment used a language model with approximately 1 billion parameters. Meanwhile, the remaining transformer computations were still performed on the GPU.

However, Q.ANT has not yet disclosed key test metrics: token generation speed, actual memory savings, and the quality of the model's responses. Therefore, it is currently impossible to assess the real efficiency of the technology.

The company plans to bring commercial systems to market in the second half of 2027. Developers have also released an SDK with C and Python interfaces, allowing experimentation with the technology without access to specialized equipment.

At the same time, the idea of replacing the attention mechanism with a Fourier transform is not new: Google presented a similar approach in the FNet architecture back in 2021. The main question now is whether Q.ANT will manage to maintain generation quality while ensuring a noticeable performance boost.