Overview
Flex is an AI-powered iOS application designed to help users work out anytime, anywhere, without the need for a personal trainer. By leveraging computer vision and machine learning, Flex provides real-time feedback on exercise form, helping users improve their workouts without human scrutiny or expensive coaching.
Watch the trailer here!
Features
🎯 AI-Powered Exercise Guidance
- Uses computer vision to extract the user’s pose in real time.
- Recognizes exercises with a classification model.
- Provides instant feedback using a deterministic algorithm.
- Provide quick answers with an AI chatbot.
🖥️ Smart Mirror Interface
- Displays a live video feed of the user.
- Overlays real-time hints to guide proper form.
- Offers customizable feedback (visual pop-ups and voice guidance).
- Provides a summary and common mistakes at the end of a workout.
- Offers encouragement and rep counting.
- Detects your exercises automatically.
📚 Exercise Library & Search
- Includes a searchable exercise database with video demonstrations.
- Provides common form corrections, videos, and tips for each exercise.
📊 Workout Tracking & Personalization
- Logs workout history and personal preferences in Firebase.
- Adjust feedback according to customized parameters such as strictness.
- Provides personalized exercise recommendations based on past sessions.
- Displays user statistics to track progress over time.
- Connect with friends, share your workout progress, and encourage each other.
Technologies
📱 Frontend (iOS)
- Swift & SwiftUI for the user interface.
- AVFoundation for video processing.
- CoreML for on-device machine learning.
🧠 AI & Computer Vision
- Pose estimation to track user movements (MediaPipe).
- Exercise classification model for identifying exercises (PyTorch).
- Deterministic algorithm for real-time feedback.
- Google Gemini API for chat bot.
☁️ Backend & Deployment
- Firebase for workout history and user data storage.
- Python-based backend for AI inference.
- AWS EC2 (FRP tunneling) to reduce hosting costs while maintaining accessibility.
- Flask for deployment.
Future Work
FrontEnd
- Enhance authentication, privacy, and security for logins and friends.
- Enable messaging between friends.
- provide a tutorial for the app.
- Polish and bug fixes.
BackEnd
- Polish hint generation and rep tracking.
- Deploy the backend entirely on an AWS EC2 instance and make it scalable.
- Arnold, our AI chatbot, should be able to pull information from the user’s database and offer customized feedback.
- Mitigate delays and traffic with load-balancers and rate-limiters.
Authors
Carter Kruse, Selena Han, Kevin Cao, Bradley Vogt, Ethan Hodess
Acknowledgments
Thank you to Professor Alberto Quattrini Li for guidance with this project, alongside classmates who provided feedback.
Check out the Medium article here.
Check out our final presentation.