A Comparative Study of Small Language Models for Resource?Constrained AI Systems

Abstract

The study focuses on the development of a Rotating Platform with Built-In Power Extension (Modern Lazy Susan) to address common problems associated with multiple electronic devices on tables, such as wire entanglement, limited access to power sources, workspace clutter, and safety hazards. The proposed design combines a rotating platform with a stationary power extension, allowing users to access devices conveniently while preventing cord twisting and tangling. The project aims to improve functionality, safety, efficiency, ergonomics, and workspace organization in homes and offices. The research is grounded in several theories, including the Technology Acceptance Model (TAM), which emphasizes perceived usefulness and ease of use; Affordance Theory, which promotes intuitive interaction; and Fitts’ Law, which guides the placement of sockets and controls for efficient access. These frameworks support the development of a user-friendly, practical, and widely acceptable product. The study investigates respondents’ perceptions of the product\'s usefulness and ease of use, considering demographic factors such as age, course, and year level. It also seeks recommendations for future prototype improvements. The project is significant for users, furniture designers, manufacturers, students, and future researchers because it introduces an innovative furniture solution that integrates technology with traditional design while enhancing convenience and safety. A review of related literature highlights the importance of usability, ergonomics, user-centered design, and innovation adoption. Concepts from researchers such as Donald Norman, Jacob Nielsen, Fred D. Davis, and Everett Rogers support the idea that products are more likely to be accepted when they are useful, easy to use, intuitive, and provide clear advantages over existing solutions. The study employed a descriptive survey research design involving 30 furniture-related students from Cebu Technological University. Data were collected through researcher-made questionnaires measuring demographic characteristics, perceived usefulness, and ease of use. Ethical standards, including informed consent, privacy protection, and voluntary participation, were observed throughout the research process. Findings revealed that most respondents were aged 18–20 years, primarily enrolled in Furniture and Cabinet Making programs. Results showed strong acceptance of the product: 73% rated it as useful or very useful, while 77% considered it easy or very easy to use. Respondents appreciated the rotating mechanism, built-in power outlets, reduced clutter, and modern design. Suggested improvements included increasing durability, using stronger materials, and adding more outlets or USB ports.

Introduction

This study addresses the challenge of managing large volumes of multilingual, inconsistent, and unstructured voter data generated by modern Voter Management Systems (VMS). While Large Language Models (LLMs) have shown strong capabilities in processing such data, their high computational costs, latency, energy consumption, and reliance on cloud infrastructure make them unsuitable for real-time and resource-constrained environments. To overcome these limitations, the study proposes an adaptive framework that integrates Small Language Models (SLMs) into voter data preprocessing and analysis.

SLMs are lightweight language models designed to provide efficient language understanding with significantly fewer parameters than LLMs. Their smaller size enables deployment on mobile devices, edge systems, and IoT platforms while maintaining acceptable performance. The proposed framework leverages SLMs to perform context-aware data cleaning, multilingual processing, sentiment analysis, issue classification, and real-time voter data intelligence with low resource consumption.

The study is based on modern NLP technologies, particularly the Transformer architecture, which uses self-attention mechanisms to capture contextual relationships in text more effectively than traditional RNN and LSTM models. It also discusses LLMs and SLMs, highlighting their strengths and limitations. Various model optimization techniques, including quantization, pruning, and knowledge distillation, are explored as methods to reduce model size and computational requirements while preserving performance.

A comprehensive literature survey reviews major developments in NLP, including transformer-based models such as GPT-3, BERT, LLaMA, Phi-3, Gemma, TinyLlama, StableLM, and SmolLM. Existing research demonstrates significant advances in language understanding but often prioritizes accuracy over deployment efficiency. The survey identifies a gap in comparative evaluations of SLMs under consistent conditions and their practical integration into real-world edge applications.

The proposed system extends the Adaptive Edge-Aware Processing Framework (AEAPF) by incorporating SLMs into the complete voter data processing pipeline. Unlike cloud-based approaches, the framework performs on-device preprocessing and analysis, reducing latency, bandwidth usage, and computational costs. It is specifically designed for handling large-scale voter data in resource-constrained environments while maintaining scalability and responsiveness.

The research methodology involves selecting popular open-source SLMs, evaluating their performance on NLP tasks such as text classification and question answering, analyzing memory usage, latency, and energy consumption, and conducting a comparative study to determine the best balance between efficiency and accuracy.

The comparative analysis focuses on models such as Phi-3 Mini, Gemma, TinyLlama, Qwen 2, and StableLM Zephyr. Results indicate that SLMs, particularly Phi-3 Mini and TinyLlama, offer strong performance with significantly lower computational requirements, making them well-suited for mobile and edge deployments.

In conclusion, the study demonstrates that Small Language Models can effectively support real-time voter data preprocessing and analysis while reducing resource consumption. By integrating SLMs with model compression techniques such as quantization, pruning, and knowledge distillation, the proposed framework enhances the efficiency, scalability, and practicality of Voter Management Systems. Future work will focus on improving reasoning capabilities and developing multimodal SLMs capable of processing text, images, and speech simultaneously.

Conclusion

This research presented a comparative study of Small Language Models for deployment in resource?constrained AI systems. The analysis demonstrated that modern SLMs provide strong performance while significantly reducing computational cost compared to large language models. Models such as Phi?3 Mini and TinyLlama demonstrate promising efficiency for mobile and edge deployments. Techniques including quantization, pruning, and knowledge distillation further improve performance on limited hardware. Future research will focus on improving reasoning capabilities of small models and developing multimodal SLMs capable of handling text, images, and speech simultaneously.

References

[1] A. Vaswani et al., “Attention Is All You Need,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5998–6008. [2] T. B. Brown et al., “Language Models are Few-Shot Learners,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2020, pp. 1877–1901. [3] H. Touvron et al., “LLaMA: Open and Efficient Foundation Language Models,” arXiv preprint arXiv:2302.13971, 2023. [4] Microsoft Research, “Phi-3 Technical Report: Efficient Small Language Models,” Microsoft AI Research, 2024. [5] Google Research, “Gemma: Open Models Based on Gemini Research and Technology,” Google DeepMind Technical Report, 2024. [6] Y. Bengio, A. Courville, and P. Vincent, “Representation Learning: A Review and New Perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, 2013. [7] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186. [8] T. Wolf et al., “Transformers: State-of-the-Art Natural Language Processing,” in Proc. EMNLP System Demonstrations, 2020, pp. 38–45. [9] N. Shazeer et al., “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer,” in Proc. ICLR, 2017. [10] J. Hoffmann et al., “Training Compute-Optimal Large Language Models,” DeepMind Chinchilla Paper, arXiv:2203.15556, 2022. [11] A. Radford et al., “Improving Language Understanding by Generative Pre-Training,” OpenAI Technical Report, 2018. [12] OpenAI, “GPT-4 Technical Report,” arXiv preprint arXiv:2303.08774, 2023. [13] Hugging Face Research, “SmolLM: Efficient Small Language Models for Edge AI,” Hugging Face Technical Report, 2024. [14] A. Paszke et al., “PyTorch: An Imperative Style High-Performance Deep Learning Library,” in Proc. NeurIPS, 2019. [15] T. Dettmers et al., “QLoRA: Efficient Finetuning of Quantized LLMs,” in Proc. NeurIPS, 2023. [16] E. Frantar and D. Alistarh, “GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers,” arXiv:2210.17323, 2022. [17] S. Han, J. Pool, J. Tran, and W. Dally, “Learning Both Weights and Connections for Efficient Neural Networks,” in Proc. NeurIPS, 2015. [18] G. Hinton, O. Vinyals, and J. Dean, “Distilling the Knowledge in a Neural Network,” arXiv:1503.02531, 2015. [19] M. Tan and Q. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” in Proc. ICML, 2019, pp. 6105–6114. [20] S. Minaee et al., “Deep Learning Based Text Classification: A Comprehensive Review,” ACM Computing Surveys, vol. 54, no. 3, pp. 1–40, 2021. [21] IBM Research, “Small Language Models for Efficient Edge AI Deployment,” IBM AI Research Whitepaper, 2024.

Copyright

Copyright © 2026 Omkar Sanjay Pomendkar, Dr. Rashmi Thakur. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.