(untitled) (Journal of Computational Intelligence and Robotics, 2024)
Distributed computing plays a pivotal role in the training of large-scale AI models, enabling the parallelization of computations across multiple nodes within a cluster. This paper explores the integration of distributed computing techniques within .NET clusters for efficient and scalable training of AI models. The .NET ecosystem, with its versatile and extensible framework, provides a robust foundation for developing distributed computing solutions. The paper begins by outlining the challenges associated with training large-scale AI models and the need for distributed computing solutions to address computational bottlenecks. It then delves into the architectural considerations for implementing distributed computing in .NET clusters, emphasizing the utilization of technologies such as Microsoft’s Azure Service Fabric or third-party frameworks like Akka. NET. The proposed solution leverages the inherent capabilities of .NET for building distributed systems, allowing seamless communication and coordination among cluster nodes. Key aspects such as data parallelism, model parallelism, and asynchronous communication are explored to harness the full potential of distributed computing for AI model training. A case study is presented to demonstrate the practical implementation of the proposed solution in a real-world scenario. Performance metrics, scalability analysis, and comparisons with traditional single-node training are provided to showcase the advantages of employing distributed computing for large-scale AI model training in .NET clusters.