Top 5 AI Models You Can Run Locally on Your Device

The article argues that running AI models on your own device can improve security, stability, and privacy compared with using AI APIs. It recommends GPT4All as an easy cross-platform way to do this and lists five open source models with different sizes and hardware requirements.

Reading notes

#

• Running AI models on your own device avoids depending on external servers and an internet connection to access AI.
• The text says that many AI APIs send messages in plain text and that some companies collect conversation data or conduct human review of chats.
• For local use, the author chooses GPT4All because it is easy to use and has cross-platform support, instead of a CLI like Ollama.
• The article warns that the models can take up from 2 to 6 GB and that it is necessary to check disk space before downloading.
• Nous Hermes 2 Mistral DPO is presented as a general, fast, fine-tuned model with about 7 billion parameters.
• Qwen2-1.5B-Instruct is recommended for weak devices, but the text warns that it can hallucinate easily.
• Llama 3 8B Instruct is described as an open source model, with good quality and a friendly personality.
• Mini Orca (Small) is suggested as a small and fast model, more useful for explanations than for conversation, and also prone to hallucinations, especially in mathematics.
• Mistral Instruct is pointed out as a general model with a license for commercial use and without ethical limitations.
• The text teaches how to open the models tab, download a model and then go to the chats tab, create a new conversation and load a model to test.