Running AI locally within your IDE/editor using Ollama

At a Glance
Ollama offers you to run open source models locally and is adding more and more very useful features to their API and ecosystem, which we know from the big cloud providers. Parts of that are also integrations into IDE and editors.

I guess a lot of you are using GitHub Copilot. They have the best integrations across a wide variety of editors and IDEs, including JetBrains PhpStorm/PyCharm, VS Code, and Zed. Copilot ships with a handful of models nowadays. It’s not limited to OpenAI’s models and also supports Claude, Gemini, or Grok. While that’s my absolute go-to on a normal workday, there are also reasons against it. For example, for privacy reasons or none or poor internet connections. Whatever the reason, there are alternatives.

Many of you probably heard of Ollama. They let you run many different open-source models on your device. Many new and exciting features have been added to their ecosystem, which is challenging the big cloud providers. They, for example, are building a cloud service to run larger open-source models, adding IDE integrations, tools/functions, and search to their API.

Ollama lets you run open-source models such as Meta’s Illama, OpenAI’s GPT OSS, Google’s Gemini, Alibaba’s Qwen, Mistral or DeepSeek.

The challenges of local models

While running models locally is excellent for privacy, since the data never leaves your device, models can be huge and therefore very power-intensive. I couldn’t imagine saying that at some point after CPU on our laptops and phones got crazier and crazier, but with a usual working device nowadays, running such a huge models will take forever and isn’t an option to work with. That’s why I’m usually opting for cloud services like Github Copilot. The upcoming Ollama cloud service also sounds very interesting.

Running models locally is excellent for privacy, since the data never leaves your device. Models with a lot of training parameters are huge and need a lot of resources to run.

But let’s say you’re on a flight and you really would like intelligent autocompletion or help from an AI while you code. I sometimes don’t need much —just some simple instructions to clean up code or handle simple tasks. Sometimes, some feedback about ideas. While it’s better to have a more powerful model, occasionally, I’m also happy with simpler things than nothing at all. Ollama comes in here with their local models.

Integrate Ollama into your IDE

Ollama shows in their documentation how to integrate with a variety of editors, and it’s getting better and simpler.

For example, in Zed, you don’t even have to do much other than install Ollama on your computer. It detects it by itself and shows you the downloaded models right away.

PhpStorm

Let’s take a look at a JetBrains IDE, since I use PhpStorm most of the time.

There, the Ollama integration is relatively new, and some parts are still in beta. JetBrains is offering a paid AI integration with different cloud models. The same as Copilot, I think (haven’t used it by myself). There’s also a free offering that can be used to run local models, like we want in this case. So subscribe to the free package and download the “JetBrains AI Assistant” plugin to get up and running.

Decide wether to use Copilot or the AI Integration

One thing to note is that Copilot and the AI Integration don’t work together well, so choose one or the other. I mostly use Copilot and turn off JetBrains AI integration. When I’m on bad internet, I switch to the local models.

Give it a try. Be aware of alternatives to open source and running models locally. In the near future, we see more and more hardware coming to market that is optimised for running neural processes, like Apple’s M5 chip. Models will be more usable locally, more cost-efficient, and protect privacy.

Happy coding. Online, offline, or with privacy concerns. Use AI to improve your workflow and build better code.