In the realm of deep learning, the challenge of developing efficient deep neural network (DNN) models that combine high performance with minimal latency across a variety of devices remains. The existing approach involves hardware-aware neural architecture search (NAS) to automate model design for specific hardware setups, including a predefined search space and search algorithm. However, this approach tends to overlook optimizing the search space itself.
In response to this, a research team has introduced a novel method called “SpaceEvo” to automatically create specialized search spaces tailored for efficient INT8 inference on specific hardware platforms. What sets SpaceEvo apart is its ability to perform this design process automatically, leading to hardware-specific, quantization-friendly NAS search spaces.
SpaceEvo’s lightweight design makes it practical, requiring only 25 GPU hours to create hardware-specific solutions, which is cost-effective. This specialized search space, with hardware-preferred operators and configurations, enables the exploration of more efficient models with low INT8 latency, consistently outperforming existing alternatives.
The researchers conducted an in-depth analysis of INT8 quantized latency factors on two widely used devices, revealing that the choice of operator type and configurations significantly affects INT8 latency. SpaceEvo takes these findings into account, creating a diverse population of accurate and INT8 latency-friendly architectures within the search space. It incorporates an evolutionary search algorithm, the Q-T score as a metric, redesigned search algorithms, and a block-wise search space quantization scheme.
The two-stage NAS process ensures that candidate models can achieve comparable quantized accuracy without individual fine-tuning or quantization. Extensive experiments on real-world edge devices and ImageNet demonstrate that SpaceEvo consistently outperforms manually designed search spaces, setting new benchmarks for INT8 quantized accuracy-latency tradeoffs.
In conclusion, SpaceEvo represents a significant advancement in the quest for efficient deep-learning models for diverse real-world edge devices. Its automatic design of quantization-friendly search spaces has the potential to enhance the sustainability of edge computing solutions. The researchers plan to adapt these methods for various model architectures like transformers, further expanding their role in deep learning model design and efficient deployment.
Check out the Paper and Reference Article. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 31k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.
If you like our work, you will love our newsletter..
We are also on WhatsApp. Join our AI Channel on Whatsapp..
Pragati Jhunjhunwala is a consulting intern at MarktechPost. She is currently pursuing her B.Tech from the Indian Institute of Technology(IIT), Kharagpur. She is a tech enthusiast and has a keen interest in the scope of software and data science applications. She is always reading about the developments in different field of AI and ML.
