AI Human Fitness Tracker using Computer Vision with MediaPipe

Abstract

In recent years, the integration of artificial intelligence (AI) into health and fitness domains has significantly enhanced personal training and physical wellness monitoring. This research introduces an AI-powered Fitness Tracker system that utilizes computer vision and pose estimation techniques to detect human body posture and accurately count repetitions or steps for various physical exercises. The system leverages MediaPipe for real-time human pose detection, computing joint angles to analyse movements and classify exercises such as push-ups, pull-ups, squats, sit-ups, and walking. It incorporates audio feedback for correct posture recognition and rep completion, enhancing user engagement and form correction. The architecture is modular, consisting of key components: pose detection, angle calculation, exercise classification, and feedback generation. This approach minimizes the need for external sensors or wearable devices, offering a non-intrusive, camera-based solution that is both accessible and scalable. The proposed system aims to assist users in performing workouts with improved accuracy and consistency, promoting a more effective and injury-free fitness routine.

Introduction

I. Background

With growing global interest in fitness, there's demand for cost-effective, real-time exercise monitoring tools that don't require wearables or personal trainers. Traditional trackers (like smartwatches) are often expensive, uncomfortable, or inaccurate for complex movements. Computer vision, especially pose estimation with MediaPipe, enables camera-only fitness tracking, delivering feedback on posture, repetitions, and form correction in real time.

II. Problem Statement

Existing systems suffer from:

• Dependence on wearables,

• Lack of real-time corrective feedback,

• Poor personalization.

Goal: Create a real-time, camera-based AI fitness tracker that:

• Accurately counts exercise reps,

• Detects incorrect postures using joint angles,

• Provides instant feedback via sound,

• Runs efficiently on low-resource devices without GPU.

III. Literature Survey

• Vision-based systems using OpenPose and CNNs showed promise but struggled with resource demands and outdoor lighting.

• MediaPipe is ideal for real-time, lightweight pose detection on edge devices.

• Joint-angle logic is effective for posture correction and repetition counting.

• Prior systems also explored audio-visual feedback and yoga pose correction.

IV. Methodology

The system uses MediaPipe + custom Python modules built into a Streamlit dashboard for real-time video processing.

Key Components:

1. Body_part_angle.py

   • Calculates joint angles (arms, legs, neck, abdomen) using landmarks.

   • Essential for posture analysis.

2. Type_of_exercise.py

   • Inherits from BodyPartAngle to analyze exercises:

     • Push-up, Pull-up, Squat, Walk, Sit-up

   • Detects proper execution, counts reps, and plays a beep on success.

3. Utils.py

   • Initializes pose detection and calculates angles.

   • Draws counters/status on video frames.

4. Main_exercise_tracker.py

   • Handles real-time webcam or uploaded video analysis.

   • Processes each frame, performs pose detection, and overlays feedback.

   • Uses Streamlit for interactive UI.

Popular Libraries Used:

• MediaPipe – pose tracking

• OpenCV – image/video processing

• Streamlit – web app interface

• PyGame – sound feedback

• NumPy & Pandas – data handling & calculations

V. Results

A. Uploaded Video Accuracy

B. Real-Time Webcam Accuracy

C. Observations

• Minor 3-second delay in real-time analysis.

• Walking was most consistent due to continuous motion.

• Pull-ups had lower accuracy due to vertical pose challenges.

• Uploaded video processing was more stable than live analysis.

VI. Dashboard & Visual Feedback

• Users can select exercise type, input mode (video or webcam), and start/pause analysis.

• System displays video with overlaid counters and posture status.

Conclusion

The system effectively demonstrates a low-cost, accurate, real-time fitness tracker using only a webcam and open-source tools. It:

• Eliminates the need for wearables,

• Provides immediate feedback for better form,

• Performs well across common exercises,

• Is deployable on low-resource devices.

Conclusion

This project successfully implements a real-time AI-powered fitness tracking system capable of accurately counting exercise repetitions through video analysis. Using MediaPipe for landmark detection and custom repetition logic for each exercise, the system provides reliable feedback to users in both offline and real-time scenarios. With an average accuracy of over 90%, the system proves effective for practical use cases such as home workouts, physiotherapy monitoring, and gym supervision. Minor latency issues in real-time processing suggest opportunities for optimization, especially in reducing buffering time and improving model inference speed. In conclusion, this system lays the groundwork for future enhancements such as: 1) Integration with mobile devices. 2) Personalized fitness feedback using ML models. 3) Support for complex or dynamic exercise routines. 4) Real-time audio/visual feedback mechanisms. This AI-based exercise tracker represents a promising step toward automated fitness monitoring, encouraging consistent workout habits while reducing the dependency on human supervision.

References

[1] Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, “OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, pp. 172–186, Jan. 2021. [2] J. M. Wang, D. J. Fleet, and A. Hertzmann, “Gaussian Process Dynamical Models for Human Motion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 283–298, Feb. 2008. [3] F. Zhang, X. Zhu, and H. Ye, “MediaPipe Hands: On-Device Real-Time Hand Tracking,” arXiv preprint arXiv:2006.10214, 2020. [4] R. Pandey and S. Shukla, “Human Pose Estimation for Fitness Tracking Using Joint Angles and Deep Learning,” in Proc. of IEEE Int. Conf. on Smart Technologies and Management, 2021, pp. 225–230. [5] P. N. Singh and A. Ghosh, “Yoga Posture Correction Using Machine Learning and Pose Estimation,” in Proc. of 2021 International Conference on Computer Vision and Image Processing (CVIP), pp. 1–7. [6] A. Parate, M. Ganesan, M. Marlin, and J. A. Landay, “Detecting Repetitive Activities Using Accelerometers,” IEEE Pervasive Computing, vol. 12, no. 2, pp. 32–39, Apr. 2013. [7] Y. Kim and H. Choi, “A Real-Time Exercise Repetition Counting System Using CNN-LSTM,” in 2020 IEEE Int. Conf. on Consumer Electronics (ICCE), Las Vegas, NV, USA, pp. 1–2. [8] T. Lee and K. Lee, “Smart Trainer: A Real-Time Repetition Counting and Feedback System Using Pose Estimation,” in 2020 IEEE Global Conference on Consumer Electronics (GCCE), pp. 22–25. [9] Y. Mishra, A. Jaiswal, and G. Soni, \"A comparative study on human activity recognition using smartphone dataset through machine learning approach,\" Int. Res. J. Eng. Technol. (IRJET), vol. 11, no. 7, pp. 813–818, Jul. 2024. [Online]. Available: https://www.researchgate.net/publication/390667823 [10] Google MediaPipe, Available: https://ai.google.dev/edge/mediapipe/solutions/guide

Copyright

Copyright © 2025 Yashraj Mishra, Ankita Jaiswal, Anany Shukla, Abhishek Verma, Humesh Verma, Dr. Goldi Soni. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.