↓ Ir para o conteúdo principal
Scholion
Scholion

← todas as notas

📎 Webclip

Build your own personal SIRI with LLAMA-3 like a PRO! 🧙‍♂️ 🪄

The tutorial shows how to build a multi-modal voice assistant in Python with LLAMA-3. It sets up speech recognition, chat responses through Groq, image analysis with Google Generative AI, and three text-to-speech options: OpenAI, gTTS, and pyttsx3.

It also adds support for daily chat logs, screenshot capture, webcam capture, clipboard extraction, and a shell script to automate setup and execution. The project uses a .env file for API keys and stores responses and images in local folders.

Reading notes
#

  • The article teaches how to build a voice assistant in Python with LLAMA-3.
  • The initial setup creates folders for code, logs, chat history, and response files.
  • The project uses SpeechRecognition, OpenCV, Groq, Google Generative AI, Whisper, PyAudio, and other libraries.
  • The utils.py file brings together functions to get the day’s log, exit the program, capture a screenshot, remove the last screenshot, and read the clipboard.
  • The setup.py file loads the credentials from .env and requires the Groq and Google Generative AI keys.
  • Webcam support looks for an available camera and saves the image in a user folder.
  • The Siri class initializes the Groq, OpenAI, and Google Generative AI clients, as well as the Whisper model for transcription.
  • The assistant keeps a conversation history and listens for the wake word “siri”.
  • Audio transcription uses Faster Whisper to convert the recorded file into text.
  • Command extraction removes the wake word and captures the user’s request.
  • The chat response is generated by Groq with the llama-3.1-8b-instant model.
  • The text can be converted to speech by pyttsx3, OpenAI TTS, or gTTS.
  • The image analysis method uses Gemini to extract context from a screenshot or webcam before the response.
  • The action selection method chooses between extracting the clipboard, taking a screenshot, deleting a screenshot, capturing the webcam, or proceeding as generic.
  • The main processing saves the audio, transcribes it, identifies the action, and, when necessary, generates the voice response.
  • main.py imports the modules from src, loads the credentials, and starts the listen() method.
  • The shell script automates virtual environment creation, dependency installation, and running the program.