Guohai Securities: Big model technology drives AI valuation to reshape and maintain the computer industry's “recommended” rating

The Zhitong Finance App learned that Guohai Securities released a research report saying that big model technology is undergoing accelerated changes, from structural innovation to training paradigm upgrades, driving the advent of the AGI era at an accelerated pace. The integration of the model architecture MoE and Transformer became mainstream, and synthetic data became the “new type of oil”. The amount of RL calculation and inference time in the post-training stage became the key, and DeepSeek drove a new paradigm of reinforcement learning. Through low-rank decomposition technology such as MLA, only a consumer-grade video card is required for local deployment of the 32B model, and the launch of the big model ushered in the true first year. Big model technology has been steadily improving, driving the advent of the AGI era at an accelerated pace. Technological iterations based on big models may continue to drive domestic AI valuations to usher in reshaping and maintain the computer industry's “recommended” rating.

Guohai Securities's main views are as follows:

Big Model Development Review: Based on Transformers, Scaling Law Throughout

In 2017, the Google team proposed the Transformer architecture to creatively promote the development of the attention layer and feedforward neural network layer and accelerate model performance. 2018-2020 was an era of pre-trained Transformer models. GPT-3 broke through the limits of possibilities for large-scale pre-training with 175 billion parameters, and technologies such as SFT and RLHF helped the model to align human values faster. Since then, as the power law relationship described by the Scaling Law on the training side declined, and high-quality text data was superimposed or gradually exhausted by AI, the inference model began to enter people's eyes; the Aime 2024 model response accuracy rate was raised from 13.4% of GPT4O to 56.7% with OpenAI, and the model maintained accelerated iterative updates.

Domestic model progress: full competition in the industry, cost reduction and efficiency improvement as the main theme

With limited resources, it is expected that low cost and high performance will rival overseas SOTA as the theme for the big domestic model in 2025. The bank takes DeepSeek, Doubao, and Ali Qianwen as examples. 1) DeepSeek- R1/V3 relies on innovative cost reduction and efficiency methods. The core aim is to greatly improve the utilization rate of GPUs in computing/communication under conditions of limited resources. 2) The Doubao Big Model came into force in the second half of 2024, with Moon Activity Data rushing to second place in the world and number one in the country; it also relied on sparse MoE architectures to achieve high performance with small parameters in the cost reduction and efficiency paradigm; 3) While Ali Qwen led the domestic open source model, QWQ-32b, which relied on the reinforcement learning paradigm, has reached the top of the world's strongest open source model, and the 32b parameter model continues to be the main theme.

Overseas big model progress: resource head concentration, betting on AGI

Under conditions of abundant computing power, resources are inclined to bet on AGI. 1) OpenAI: The inference model O1 and the multi-modal model Sora have all achieved industry leadership. CEO Altman has mentioned many times that OpenAI's first agent will be released since 2025, and 2025 will also be the first year of the agent outbreak; 2) Google: Forward-looking layout of native multi-modal Gemini will release multiple agent products at the end of 2024, while laying out the lightweight model Gemma to seize the end-side ecosystem; 3) Meta: Ll3.3ama achieved 70B parameters in December 2024 Llama 3.1405B's performance; based on Meta Live, real-time voice interaction and cross-device collaboration capabilities have been achieved to boost general intelligence; 4) The Claude3.5 Sonnet upgrade in October 2024 added computer use capabilities to allow Claude to use computers like a human; in addition, the hybrid inference model Claude-3.7-sonnet was first released in 2025.

Model future research and judgment: post-betting training+algorithm optimization, low-cost landing+achieving AGI as the ultimate goal

The model has ushered in accelerated transformation in terms of architecture and pre-training—post-training—implementation. 1) At the model architecture level, the integration of MoE and Transformer has now gradually become the mainstream architecture, and the number of large global MoE models is exploding in 2024; 2) At the pre-training level, in the context of high-quality data or gradual exhaustion, synthetic data has become a “new type of oil” in the digital economy era, continuing to support model training iterations; 3) In terms of post-training, the key to leapfrogging inference model performance is also gradually shifting to RL computation volume at this stage and thinking time in the test inference stage. DeepSeek has driven a new paradigm of pure reinforcement learning; 4) At the model implementation level, DeepSeek has driven the model to accelerate the trend of low-cost deployment, drastically reducing video memory usage through low-rank decomposition methods such as MLA, and only consumer-grade graphics cards are required for localized deployment of DeepSEEK-R1-32B and below models, and the launch of the big model ushered in the first year in the true sense of the word.