Emerging trends in machine learning: balancing accuracy and energy efficiency

Emerging Trends in Machine Learning: Balancing Accuracy and Energy Efficiency

Barcelona, Spain – With technology advancing at an unprecedented rate, machine learning (ML) has demonstrated remarkable proficiency in tasks like image classification and language processing. However, the increasing computational demands of ML have raised concerns about energy consumption, necessitating a shift towards more sustainable practices. Recognising this emerging issue, software engineering is increasingly focused on energy efficiency—an area identified by Gartner as a key trend for 2024.

A recent study conducted by researchers at the Universitat Politècnica de Catalunya has taken significant strides in this direction. The research team evaluated various optimisation techniques within the PyTorch ML framework to gauge their impact on energy usage, accuracy, and economic costs. The study is particularly pertinent given that ML projects have historically valued accuracy over energy efficiency, which has contributed to rising energy demands.

The researchers focused their efforts on optimising pre-trained models used for image classification tasks, relying on datasets such as ImageNet and CIFAR-10. They examined several techniques including dynamic quantization, torch.compile, and pruning, analysing their effects on 42 models from the Hugging Face library. This rigorous analysis deployed the Green Software Measurement Model (GSMM) to measure key metrics like energy consumption, computational complexity, and the economic impact.

The results of the study indicated that dynamic quantization was particularly effective in reducing inference time and energy consumption without significantly compromising accuracy. In contrast, torch.compile offered a balanced trade-off, improving energy efficiency while maintaining reasonable accuracy levels. However, the pruning techniques produced mixed results; global pruning at a 25% threshold resulted in increased costs due to extended optimisation times, while local pruning did not enhance model accuracy.

The study’s implications are far-reaching, especially for ML engineers tasked with implementing efficient models. It provides a decision-making framework to guide the selection of optimisation techniques based on priorities such as inference time, accuracy, and economic and energy costs. The findings also propose improvements for stakeholders across the ML ecosystem. Recommendations include better documentation from Hugging Face to enhance model reliability and updates to PyTorch libraries to make parameter pruning more effective.

This study not only sheds light on the importance of balancing accuracy with energy efficiency in ML but also calls for further research. The researchers suggest that future studies should explore optimisations for natural language processing (NLP) models, multimodal applications, and the TensorFlow framework. Additionally, the concept of energy labels for models based on performance metrics is proposed as a way to inform decision-making within the ML community.

The significance of this research lies in its potential to guide ML development towards more sustainable practices, addressing the urgent need to mitigate the environmental impact of rapidly advancing technology.

Source: Noah Wire Services