How to install an AI chat on your computer and use it on a plane?

This article is a supplement to the video published on the Beyond AI channel, which demonstrates how to install a local version of an AI Chat on your computer without the need for an internet connection. To learn more, visit the Beyond AI channel and discover other interesting content related to artificial intelligence.

Watch this material on YouTube:

‍Introduction

Hi, I'm Michał from Beyond AI. Today, I will show you how to install an AI Chat locally on your computer that will work without any internet access.

I have checked all the available options and will show you the simplest one, which requires neither programming skills nor specialized technical knowledge.

Why Install Locally?

When we use services like ChatGPT from OpenAI or Gemini from Google, we must be connected to the internet. Naturally, all our communication—every document we upload and every photo—is sent to OpenAI or Google servers.

If you do not have a paid account and haven't disabled the training options, the data you send may be used to train new versions of their models. This is not always a desirable situation. We don't always want our work to be sent away and potentially become part of a new model that might someday be subject to a leak.

If these are your concerns, or if you simply want to have access to Chat in situations where you have no internet (for example, while flying on a plane or dealing with a weak, unreliable connection), we will show you step-by-step what needs to be done to run a local version of an LLM.

Hardware Requirements

To do this, you primarily need a computer with a dedicated graphics card or a Mac with an M1 processor or better. While it is possible on other computers, the performance will be too slow for any reasonable daily work. However, if you have a computer with a decent GPU or a Mac with M1/better, you can configure these models so that the speed is sufficient for everyday tasks and genuinely helpful.

Required Software

To fully utilize the available possibilities, we will need to install two applications:

1. LM Studio – A fantastic program that allows you to easily download and run any model locally. LM Studio also provides a simple Chat version where you can talk to these models. However, it lacks features like document uploading.
2. AnythingLLM – This is a Chat interface that can utilize models hosted by LM Studio, allowing for easy document management. You can create multiple "Workspaces" where you can load documents from a specific project, work-related files, or other areas of your life, and then use the Chat to query and process those documents.

The primary advantage is that nothing is sent to the internet. Everything happens locally on your computer, ensuring full security and privacy for your data processing.

Installing LM Studio

1. Go to the LM Studio website nd choose the installer for your system (available for Mac, Windows, and Linux).

2. Once downloaded, drag it to your Applications folder and launch it.
3. In the LM Studio interface, go to the "Search" tab to see all models available for local download.
4. Select a model (e.g., Bielik – the latest Polish model prepared by the SpeakLeash group).

5. LM Studio uses colored squares to indicate which models will fit in your GPU's memory.
6. Choose the largest model that fits your memory and click "Download."

Using LM Studio

1. After the download is complete, go to the "AI Chat" tab.
2. Select one of your downloaded models from the top menu.
3. The model loads into memory, and you can begin testing.

Installing AnythingLLM

This program allows us to use an AI chat with the ability to operate on files.

1. Go to the AnythingLLM website.
2. Click on "Download AnythingLLM for Desktop."
3. Choose the version for your operating system.
4. After installation, drag the application to the Applications folder.

Configuring AnythingLLM

1. Launch AnythingLLM, and follow the setup Wizard.
2. In LM Studio(!), go to the "Local Server" tab and start the local server.
3. Copy the server address and paste it into AnythingLLM.
4. Select the model (e.g., Bielik).
5. Select the embedder (e.g., AnythingLLM embedder).
6. Choose the vector database.
7. Create a new Workspace.

Using AnythingLLM

1. You now have a Chat interface utilizing AnythingLLM.
2. You can add various documents to your workspace:
   • Local files
   • Websites
   • Code repositories from GitHub
   • Video transcriptions
3. After adding your documents, click "Save & Embed."
4. Now you can ask questions regarding the added documents.

Summary

With this solution, you can locally install and run any model, add documents, websites, or YouTube videos, and then search and answer questions based on that data in offline mode. The instance works fast enough, although searching may take longer with a high volume of documents.

This is particularly useful when you lack internet access or do not want to send documents to external servers. I recommend testing different models—some are dedicated to specific tasks like coding or the Polish language, while others like Mistral are good general-purpose models.

Thank you for your attention, and I invite you to watch or read our other episodes!

—

FAQ

1. Do I need an internet connection to use the local version of AI Chat?

No, after installing and configuring tools like LM Studio and AnythingLLM, you can use AI Chat without any internet connection.

2. What are the hardware requirements for running local AI models?

A computer with a dedicated graphics card or a Mac with an M1 processor or newer is recommended.

3. Can I process local documents with the AI Chat?

Yes, using AnythingLLM, you can manage local documents and use AI models to process them.

4. Is the local version of AI Chat secure?‍

Yes, all operations take place locally on your computer, ensuring total data security and privacy.

Glossary

• ‍Chat AI – A computer program that uses artificial intelligence to conduct conversations with users.
• LLM (Large Language Model) – An advanced language model that processes and generates text based on massive datasets.
• Quantization – A process of optimizing an AI model that reduces its size and increases efficiency at the cost of a slight loss in quality.
• Inference – The process of the AI model processing data to provide answers to user queries.
• Workspace – In AnythingLLM, a specific area where documents and other data processed by the AI model are stored.

Visit our YouTube channel!

Subscribe to the Beyond AI channel, your guide to the dynamic world of artificial intelligence. You will find more practical tips, interesting facts, and the latest news related to AI. See you there!