JARVIS – Gesture-Controlled Smart Assistant (LLama Powered)

A powerful, lightweight AI assistant that runs on your machine at startup and listens to your gestures, voice, or commands — just like J.A.R.V.I.S from Iron Man! Built with LLama LLM integration and real-time gesture recognition, it offers hands-free control and intelligent responses without an internet connection.

Features

• Gesture Recognition using OpenCV + MediaPipe
• LLama-based local LLM (via llama.cpp or llama-cpp-python)
• Optional Voice Command Interface
• Auto-starts on system boot (Windows/Linux)
• Performs tasks like:
• Opening apps
• Responding to questions
• Performing calculations
• Reading system info
• Offline, private, and secure

Tech Stack

• Python 3.8+
• LLama (local LLM using llama.cpp or llama-cpp-python)
• MediaPipe for gesture recognition
• OpenCV for hand tracking
• Tkinter / PyQt / CLI for interaction (if UI is added)
• Pyttsx3 for text-to-speech
• SpeechRecognition for voice input

Architecture

            +-------------------------+
            |     System Startup      |
            +------------+------------+
                         |
                         v
               +-------------------+
               |  Gesture Listener |
               +-------------------+
                         |
        +----------------+----------------+
        |                                 |
+---------------+               +------------------+
|  Predefined   |               |   LLama LLM      |
|  Gesture Map  |               | (Response Engine)|
+---------------+               +------------------+
        |                                 |
        +----------->  Execute Task  <----+
                         or Respond