Journal of Modern Power Systems and Clean Energy

ISSN 2196-5625 CN 32-1884/TK

Model Fusion for Scalable and Sustainable Artificial Intelligence: A Review and Outlook
CSTR:
Author:
Affiliation:

1.Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China;2.InfiX.ai, Hong Kong, China;3.Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, China

Clc Number:

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Large language models (LLMs) have achieved remarkable progress in recent years. Nevertheless, the prevailing centralized paradigm for training generative artificial intelligence (AI) is increasingly approaching its structural limits. First, the concentration of large-scale graphics processing unit (GPU) clusters restricts the access to the pre-training stage, confining the fundamental model development to a small number of resource-rich institutions. Second, the economic and energy costs associated with operating massive data centers render this paradigm progressively less sustainable. Third, the hardware gatekeeping narrows the participation to computer science specialists, limiting the involvement of domain experts who are essential for high-impact applications. Finally, small- and medium-sized enterprises remain dependent on expensive application programming interface (APIs) or shallow fine-tuning methods that are insufficient to modify the core knowledge of a model. Together, these constraints impede innovation and hinder equitable access to next-generation AI systems. Model fusion offers a scalable alternative by integrating multiple specialized models without retraining from scratch. This paper analyzes the current landscape of model fusion, outlining the strengths and limitations of existing methods and discussing future directions. We highlight recent advances such as InfiFusion, InfiGFusion, and InfiFPO, which improve the alignment and scalability through techniques like top-K logit selection, graph-based distillation, and preference optimization. These techniques demonstrate substantial efficiency and reasoning gains, pointing toward a more accessible and resource-aware paradigm for large-scale model development. Finally, we discuss the practical applicability of model fusion, using the energy domain as an illustrative example.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:October 10,2025
  • Revised:December 16,2025
  • Adopted:
  • Online: January 30,2026
  • Published:
Article QR Code