Apr 24, 2024 13:25 JST

Source: NEC Corporation

NEC Develops High-speed Generative AI Large Language Models (LLM) with World-class Performance
"NEC cotomi Pro" and "NEC cotomi Light" achieve the same quality as global models at more than 10 times the speed

TOKYO, Apr 24, 2024 - (JCN Newswire) - NEC Corporation (NEC; TSE: 6701) has expanded its "NEC cotomi" generative AI services with the development of "NEC cotomi Pro" and "NEC cotomi Light," two new high-speed generative AI Large Language Models (LLM) featuring updated training data and architectures.

With the rapid development of generative AI in recent years, a wide range of organizations have been considering and verifying business transformation using LLMs. As specific application scenarios emerge, there is a need to provide models and formats that meet customer needs in terms of response time, business data coordination, information protection and other security aspects during implementation and operation.

NEC’s newly developed NEC cotomi Pro and NEC cotomi Light are high-speed, high-performance models that deliver the same high performance as global LLMs, but at more than ten times the speed.

Generally, to improve the performance of an LLM, a model needs to be made larger, but this slows down the operating speed. However, NEC has succeeded in improving both speed and performance with the development of an advanced new training method and architecture.

"NEC cotomi Pro" achieves performance comparable to top-level global models such as "GPT-4" and "Claude 2," with a response time that is approximately 87% faster than GPT-4 using an infrastructure of two graphics processing units (GPU). In addition, the even faster "NEC cotomi Light" has the same level of performance as global models such as "GPT-3.5-Turbo," but can process a large number of requests at high speed with an infrastructure of about 1 to 2 GPU, providing sufficient performance for many tasks.

Specifically, in an in-house document retrieval system using a technique called RAG, the system achieved a correct response rate higher than GPT-3.5 without fine-tuning and a correct response rate higher than GPT-4 after fine-tuning, with a response time that is approximately 93% faster.

Features of NEC cotomi Pro and NEC cotomi Light1. High throughput for various tasks

Both "NEC cotomi Pro" and "NEC cotomi Light" have high levels of processing power that are at the top level globally in terms of knowledge and ability to handle various tasks, such as document summarization, logical reasoning, and question answering.

"ELYZA Tasks 100" (top 2 graphs) and the "Japanese MT-Bench" (bottom graph) benchmark comparisons

Specifically, NEC confirmed top-level performance, not only in Japan but also globally, in two benchmarks commonly used to measure overall LLM performance: "ELYZA Tasks 100" and "Japanese MT-Bench." NEC cotomi Pro provided results more than five times faster than GPT-4 (*) on a standard server with two GPUs. Moreover, it outperformed the speed of models such as "Gemini 1.0 Pro" and showed performance that is comparable to Claude 2 and GPT-4. At the same time, "NEC cotomi Light" provided results that are more than 15 times faster than GPT-4 while outperforming large models such as "LLaMA2-70B" and demonstrating performance that is comparable to GPT-3.5-Turbo.

2. High speed

In addition to high performance during inference, the time (speed) between sending a request and receiving a response is also important for the practical application of LLM. NEC cotomi Pro and NEC cotomi Light have achieved high speed processing that is 87% to 93% faster than GPT-4 with two standard GPUs. This has been accomplished thanks to architectural innovations that have enhanced both performance and speed, as well as a large Japanese dictionary (for tokenization) in the model. These innovations not only increase the speed of inference, but also increase the number of simultaneous accesses and reduce the training time required for fine-tuning. Moreover, additional speed improvements can be made by increasing the number of GPUs to 4 or 8, allowing for flexible construction to meet the needs of each application.

By utilizing a model that achieves high processing power with high speed and mass access, it is possible to significantly shorten the response time of business applications that utilize generative AI and improve user experience. In addition, high processing power can significantly improve performance after fine-tuning of individual data for each company.Going forward, NEC will continue to strengthen cooperation with partners and provide safe, secure and reliable AI services based on the expanded NEC cotomi lineup, with the goal of helping customers to solve a wide range of complex challenges.

(*)Experiment provided 16-bit calculation precision in a GPU environment equipped with two L40sShare

About NEC Corporation

NEC Corporation has established itself as a leader in the integration of IT and network technologies while promoting the brand statement of “Orchestrating a brighter world.” NEC enables businesses and communities to adapt to rapid changes taking place in both society and the market as it provides for the social values of safety, security, fairness and efficiency to promote a more sustainable world where everyone has the chance to reach their full potential. For more information, visit NEC at https://www.nec.com.

Source: NEC Corporation
Sectors: Artificial Intel [AI]

Copyright ©2024 JCN Newswire. All rights reserved. A division of Japan Corporate News Network.

Related Press Release

NEC and Ursa Space collaborate on satellite image analysis services using one of the world's largest virtual constellations
June 20 2024 12:20 JST
NEC hosts New Zealand Prime Minister Christopher Luxon
June 19 2024 18:00 JST
NEC develops compact millimeter-wave distributed antenna for Beyond 5G/6G through the development of a radio-over-fiber system enabling low power consumption
June 17 2024 15:06 JST
NEC announces external appointment of executive personnel
June 05 2024 16:43 JST
Successful demonstration of a superconducting circuit for qubit control within large-scale quantum computer systems
June 03 2024 17:58 JST
Celonis and NEC begin Proof of Concept Integrating Generative AI and Process Intelligence
May 28 2024 19:20 JST
NEC X Invests in Agtech Startup Verdi, Integrates AI-Powered Platform with NEC's Cropscope Smart Farming Initiative
May 23 2024 10:20 JST
NEC Leverages RISE with SAP and AI to Build Business AI in Move to the Cloud
May 17 2024 10:19 JST
NEC X Invests in Public Safety Startup Multitude Insights, Scaling AI-Powered Collaboration and Information Sharing for Law Enforcement
May 08 2024 09:24 JST
NEC Strengthens Commitment to Space Industry with Investment in Seraphim Space Venture Fund II
April 22 2024 14:09 JST
More Press release >>

Latest Press Release

More Latest Release >>