はじめに
Groq が The Language Processing Unit なるものを発表しました。
Groqのプレスリリースは、こちら
The Language Processing Unit (LPU)
上記の記事によると、
- LLM Llama-2 70B を１秒あたり100トークン以上の実行ができるようです。そして、4,000後のエッセイを1分強で生成できるようです。
ちなみに、GPUでは、
ということなので、それよりも早いと。。。
で、The Language Processing Unit についての詳細はありません。これって、現在のTSPではなく、新しいTSPを開発したのでしょうか？ということは分かりません。
おわりに
2年前の7月(2021年7月)に、HBM付のチップを開発しているというようなブログを書いています。もしかしたら、これが上がってきて、評価したのでは？と思っています。
この時、どこのFoundry使うのと、書いていますが、上記の記事によると、Samsung SF4X (4nm) を使うとあります。HBM付のチップがこのプロセスかどうかまでは分かりません。
P.S 上記の記事の著者である The Futurum Group の CEO、Daniel Newman は Groq の投資家です。ということで。。。
追記)、2024.02.07
groqchat にて、 Llama 2 70B-4K での "What is Groq LPU ?" の回答
GROQ LPU (Learning to Perform Unsupervised) is a type of deep learning model that is trained using an unsupervised learning approach. The main goal of GROQ LPU is to learn a representation of the input data that can be used for various tasks, such as classification, clustering, dimensionality reduction, and anomaly detection. GROQ LPU models are trained using a large amount of unlabeled data, and they are designed to learn a robust and generalizable representation of the input data. This is achieved by using a contrastive learning objective, which encourages the model to learn a representation that captures the underlying structure of the data. GROQ LPU has been shown to be effective in various computer vision tasks such as image classification, object detection, and segmentation. It has also been applied to natural language processing tasks such as text classification and language modeling. One of the key benefits of GROQ LPU is that it does not require labeled data, which can be time-consuming and expensive to obtain. Additionally, GROQ LPU models can be easily adapted to new tasks, making them a versatile tool for a wide range of applications. Overall, GROQ LPU is a powerful tool for unsupervised learning that has the potential to revolutionize the field of machine learning. Its ability to learn robust and generalizable representations of data without the need for labeled data makes it an exciting area of research with many potential applications.
Mixtral 8x7B-32K での "What is Groq LPU ?" の回答
Groq is a company that has developed a new type of processor architecture specifically designed for artificial intelligence (AI) and machine learning (ML) workloads. The Groq Processing Unit (GPU) is the company's flagship product, and it is a programmable processor that can execute up to 1 exaops (1 billion billion operations per second) of compute performance. The Groq Tensor Streaming Processor (TSP) is the core of the Groq LPU (Logical Processing Unit), which is a building block for creating scalable and efficient AI/ML systems. The Groq TSP is a massively parallel processor that can execute large numbers of operations in a single clock cycle, making it well-suited for AI/ML workloads that require high levels of computational throughput. The Groq LPU is designed to be programmed using a high-level programming model called TensorStream, which allows developers to express their AI/ML models as a series of tensor operations. This programming model is designed to be easy to use and efficient, allowing developers to quickly and easily create high-performance AI/ML applications. Overall, the Groq LPU is a powerful and flexible solution for AI/ML workloads, offering high performance, scalability, and ease of programming. It is well-suited for a wide range of applications, including image and speech recognition, natural language processing, and other AI/ML tasks.
