Configuring Continue.dev with Ollama for Local Large Language Model Integration in VSCode Development Environment

Continue.dev + Ollama: The Complete Guide to Local AI Coding in VS Code

Build a local AI coding assistant with Ollama and Continue. Configure chat, code completion, editing, and agent workflows while keeping inference on hardware you control.

Last reviewed: October 28, 2026

Introduction

AI coding assistants are becoming a standard part of software development. However, many rely on hosted inference APIs, usage limits, recurring costs, or sending source code to an external service.

Continue and Ollama provide an alternative. Ollama runs language models locally and exposes an API. Continue integrates models into your development environment, providing chat, code completion, editing, and other AI-assisted development capabilities.

Together, they let you build a local AI coding environment that can operate without a hosted inference provider for supported workflows.

The important distinction is that a local coding assistant is not just a chatbot inside an editor. Different development tasks benefit from different models. Inline autocomplete prioritizes low latency, while repository analysis and multi-step code changes require stronger reasoning, context handling, and tool support.

This guide covers the complete setup, model selection, configuration, troubleshooting, and practical limitations of running Continue with Ollama.

1. How Continue and Ollama work together

The architecture has three primary components:

• Visual Studio Code: The editor where you work with your project.
• Continue: The extension that manages model interactions and integrates AI into your development workflow.
• Ollama: The local model runtime that loads and executes your selected models.

The request path is straightforward:

1. You ask a question, request an edit, or trigger autocomplete in VS Code.
2. Continue prepares a request using the relevant code and context.
3. The request is sent to Ollama's API.
4. Ollama executes inference with the selected model.
5. Continue presents the response or proposed changes in the editor.

The default Ollama API address is http://localhost:11434.

This architecture separates the editor integration from the inference engine. You can change models without rebuilding your application, and you can use Ollama from other local tools that support its API.

Official references:

• Continue documentation
• Continue's Ollama integration guide
• Ollama

2. Prerequisites and hardware planning

You need:

• Visual Studio Code.
• The Continue extension.
• Ollama installed and running.
• Enough available memory for your selected model and context.
• A working internet connection for initial downloads, unless the required software and models are already available locally.

The amount of memory required depends on the model, its quantization, context length, and the number of active requests. A small model can be useful for autocomplete, while a larger coding model may be preferable for complex edits and repository-level reasoning.

Do not select a model based only on its parameter count. Consider the model's actual download size, runtime memory requirements, supported context, and speed on your hardware.

On Apple silicon, CPU and GPU share unified memory. On systems with discrete GPUs, model execution may use VRAM, system RAM, or a combination of both.

A practical starting point is to choose the smallest model that performs your intended task adequately, then evaluate whether a larger model materially improves results.

3. Install Ollama

Download Ollama from the official installation page.

After installation, verify that the command-line interface is available:

ollama --version

Check whether Ollama is responding:

curl http://localhost:11434

A normal response indicates that the service is reachable.

If the service is not running, start it using the appropriate method for your operating system. Depending on your installation, Ollama may already run as a background service. Where necessary, use:

ollama serve

Do not start a second server process if an existing service is already listening on the port.

4. Download and test a coding model

Before configuring Continue, verify that the model works directly through Ollama.

For a small autocomplete-oriented starting point:

ollama pull qwen2.5-coder:1.5b

Run a basic test:

ollama run qwen2.5-coder:1.5b

Enter a programming question and confirm that the model responds. Exit the interactive session when finished.

This example uses a small model from the Qwen2.5-Coder family. It is useful for demonstrating the setup, but it should not be interpreted as a universal recommendation for the best coding model in 2026.

For more demanding chat and editing tasks, investigate current coding models available through Ollama, including supported variants of Qwen Coder, Devstral, and other instruction-tuned models. Check each model's current hardware requirements, license, context capabilities, and tool support before downloading it.

List installed models:

ollama list

Inspect active model workloads:

ollama ps

The exact model identifier shown by ollama list is the identifier you should use in your Continue configuration.

5. Install Continue in Visual Studio Code

Open VS Code and install Continue from the Visual Studio Marketplace.

Open the Continue sidebar and locate its configuration controls. Continue's current configuration format is YAML, normally stored in:

• macOS and Linux: ~/.continue/config.yaml
• Windows: %USERPROFILE%\.continue\config.yaml

You can also open the configuration through the Continue interface. The available UI may vary by extension version.

Older tutorials frequently use config.json. That format is deprecated in favor of config.yaml. For a new installation, use YAML rather than copying an older JSON example.

6. Configure Continue to use Ollama

Start with a minimal configuration for local chat and editing:

name: Local Ollama Coding
version: 0.0.1
schema: v1

models:
  - name: Local Coding Model
    provider: ollama
    model: qwen2.5-coder:1.5b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit

Save this as ~/.continue/config.yaml, or use the corresponding Windows path.

The important fields are:

• name: A label for the configuration.
• version: The configuration version.
• schema: The Continue configuration schema.
• provider: Identifies Ollama as the model provider.
• model: The exact installed model identifier.
• apiBase: The Ollama service address.
• roles: The development tasks the model is configured to perform.

The apiBase points to the Ollama service root. Do not append /api/generate or invent a field named apiBase/v1 for this configuration.

If the model is not installed, download the exact identifier before using it.

After saving the configuration, reload it through Continue's configuration controls. Select the local model and test a simple programming question.

For the full configuration reference, see Continue's Ollama provider documentation.

7. Use separate models for different coding tasks

A single model can support several roles, but a more effective local setup often assigns different models to different tasks.

Continue supports roles including:

Not every model supports every role equally well. The configured roles should reflect real model capabilities, not just the features you want to enable.

See Continue's model roles documentation.

A practical two-model configuration

Use a small model for autocomplete and a separate model for chat and editing:

name: Local Coding Workflow
version: 0.0.1
schema: v1

models:
  - name: Local Chat and Edit
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply

  - name: Local Autocomplete
    provider: ollama
    model: qwen2.5-coder:1.5b
    apiBase: http://localhost:11434
    roles:
      - autocomplete

This is an illustrative configuration. Download both models before using it, and replace the identifiers with models available in your environment.

The larger model may provide better results on difficult coding tasks, while the smaller model can reduce latency for inline completion. Whether the split is worthwhile depends on available memory and performance.

Why autocomplete deserves its own model

Autocomplete is different from conversational code generation. It needs to predict what belongs at the cursor using the surrounding code, often under tight latency constraints.

Models trained for fill-in-the-middle completion can perform better for this role than a general chat model, even when the chat model has more parameters.

A smaller autocomplete model can therefore improve the overall experience by returning suggestions quickly without requiring every keystroke to invoke a larger model.

See Continue's autocomplete guide.

8. Configure code autocomplete

After adding a model with the autocomplete role, select it as the active autocomplete model in Continue's model controls.

If suggestions do not appear, check the following:

1. Confirm that inline autocomplete is enabled in Continue.
2. Verify that the selected model is assigned the autocomplete role.
3. Confirm that the exact model is installed in Ollama.
4. Check that VS Code's editor.inlineSuggest.enabled setting is enabled.
5. Disable competing autocomplete extensions temporarily if they interfere.
6. Inspect Continue's logs and the VS Code developer console for errors.

Autocomplete models are sensitive to latency. A model that performs well in a long chat interaction may feel frustrating when it takes too long to suggest the next line.

Tune the model based on real editing behavior rather than relying exclusively on code-generation benchmarks.

9. Configure chat, editing, and apply workflows

The chat role supports conversational assistance, such as explaining functions, identifying likely bugs, or discussing architectural trade-offs.

The edit role is used for code transformations based on an instruction. The apply role helps produce changes that can be applied more precisely to existing code.

For example, you might ask Continue to:

• Explain the data flow through a module.
• Refactor a function while preserving its public interface.
• Add input validation to an API handler.
• Write unit tests for an existing implementation.
• Compare two approaches to a concurrency problem.
• Identify missing error handling.

For consequential changes, review the diff before accepting it and run the relevant tests. A model can generate plausible code that does not compile, violates project conventions, or misses important edge cases.

A local model should be evaluated by its actual contribution to development quality, not by how convincing its explanations sound.

10. Agent mode and tool support

Agent workflows can involve more than generating text. Depending on the configured capabilities and the extension version, Continue may use tools to inspect code, search files, or perform other development actions.

This makes model selection especially important.

Some models can produce useful code but struggle with structured tool calls. Others may advertise tool support but fail on specific tool schemas or multi-step workflows.

If a model reports that Agent mode is unsupported:

1. Check the model's documented tool-calling support.
2. Update Ollama and Continue if an older version is involved.
3. Verify that Continue correctly detects the model's capabilities.
4. Only set capabilities: [tool_use] when the underlying model genuinely supports tool calling.
5. Test with a simple, low-risk tool workflow before relying on more complex operations.

For example:

models:
  - name: Local Tool-Capable Model
    provider: ollama
    model: your-tool-capable-model:tag
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply
    capabilities:
      - tool_use

Replace the example model identifier with a real installed model that supports the required tool-calling behavior.

Declaring a capability in configuration does not create that capability in the model. It tells Continue how the model should be treated. Actual support must still be verified.

For further details, consult Continue's model capabilities documentation.

11. Using a local GGUF model

You may already have a GGUF file that you want to run through Ollama.

Ollama supports importing compatible GGUF models through a Modelfile. You do not need to manually construct a YAML file in Ollama's internal model directory.

Create a directory for the model and place the GGUF file inside it. Create a file named Modelfile:

FROM ./my-model.gguf

From that directory, create the Ollama model:

ollama create my-local-coder -f Modelfile

Test it:

ollama run my-local-coder

If the model requires a particular prompt template or other configuration, consult the model publisher's instructions and Ollama's Modelfile documentation.

Once the model works, configure Continue with the name created above:

name: Local GGUF Coding Model
version: 0.0.1
schema: v1

models:
  - name: My Local GGUF Coder
    provider: ollama
    model: my-local-coder
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit

This method is useful when the model is distributed as GGUF rather than through a ready-made Ollama model tag.

Compatibility still depends on the model architecture, GGUF metadata, tokenizer, and template. Importing a file successfully does not guarantee that every feature will work correctly.

Official reference: Ollama Modelfile documentation.

12. Tune context length and memory usage

A common problem is that a model works through ollama run but fails or performs poorly when Continue uses it.

One possible reason is context length. Continue may send more source code and instructions than a basic interactive prompt. A larger context can increase memory use substantially.

You can configure model defaults in Continue:

models:
  - name: Local Coding Model
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
    defaultCompletionOptions:
      contextLength: 8192
      temperature: 0.2

The context setting is a starting point, not a universal optimum. Ensure the selected model supports the requested context and your hardware can accommodate the resulting memory requirements.

If memory errors occur, try a smaller context or a smaller model before making other changes.

Remember that model weights are not the only memory consumer. The KV cache, runtime allocations, concurrent requests, and other applications also need resources.

For a deeper understanding of Ollama's model runtime, see the Ollama documentation.

13. Connect to Ollama on another machine

A useful configuration is to run inference on a dedicated machine and use Continue from another computer.

In this arrangement, the development workstation sends requests to an Ollama server over the local network. This can be useful when the inference machine has more RAM or a more capable GPU.

Update the model configuration:

models:
  - name: Remote Local Coding Model
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://192.168.1.50:11434
    roles:
      - chat
      - edit

Replace the example address with the actual address of the Ollama server.

Ollama commonly binds to loopback by default. Remote access requires deliberate network configuration so that the service listens on an appropriate interface.

Do not expose the API publicly without an explicit security design. The standard Ollama API should not be treated as a fully authenticated public endpoint.

Prefer a private network, firewall restrictions, or an authenticated tunnel. Avoid opening port 11434 directly to the public internet.

14. Troubleshooting

Continue cannot connect to Ollama

First, test the service directly:

curl http://localhost:11434

If the service is unreachable, check that Ollama is running and listening on the expected address and port.

If Ollama works but Continue fails, verify the apiBase field, reload the Continue configuration, and inspect the extension logs.

The model is not found

List installed models:

ollama list

Compare the exact identifier with the model field in config.yaml.

A model name and tag must match the installed model. Download the missing model if necessary.

The model runs out of memory

Try these changes in order:

1. Reduce the context length.
2. Close other memory-intensive applications.
3. Use a smaller quantization or model.
4. Reduce the number of simultaneous workloads.
5. Reassess whether the model is suitable for the available hardware.

Autocomplete is missing

Check the model role, Continue's autocomplete settings, VS Code's inline suggestion setting, and any competing extensions.

Then verify that the model generates reasonable code completions when tested independently.

Agent mode does not work

Check actual tool-calling support rather than assuming that any coding model supports agents. Review capability detection, update the relevant software, and test a simpler workflow.

Responses are too slow

Measure the behavior of the actual task. Inline completion, long code edits, and repository analysis have different latency requirements.

Try a smaller model for autocomplete, reduce unnecessary context, and inspect ollama ps to understand the active model and resource allocation.

Code quality is inconsistent

Use task-specific evaluation prompts and a repeatable set of coding tasks. Review compilation, test results, and diffs. Consider a stronger model for complex edits while retaining a smaller one for autocomplete.

15. Privacy and security considerations

Running Ollama locally can keep inference on hardware you control, but local inference alone does not guarantee that all development data stays local.

Check the complete workflow, including:

• Whether Continue is configured with any remote model providers.
• Whether extensions or other tools send telemetry or diagnostics.
• Whether the application uses external search, documentation, or retrieval services.
• Whether your source code contains secrets that should not be included in prompts.
• Whether a remote Ollama endpoint is exposed beyond the intended network.

Treat model-generated code as untrusted until reviewed and tested. If an agent can execute commands or modify files, keep permissions constrained and review consequential operations.

Local execution changes where inference occurs. It does not eliminate the need for access control, code review, dependency management, or secure development practices.

16. A practical workflow for local AI development

A useful setup is not necessarily one that runs the largest available model. It is one that allocates the right model to each task.

A reasonable workflow is:

1. Use a small, responsive model for inline autocomplete.
2. Use a stronger coding model for chat, debugging, and edits.
3. Use a model with verified tool support for agent workflows.
4. Keep context sizes within the available memory budget.
5. Review generated diffs and run tests.
6. Record model versions and configuration so the setup can be reproduced.

As you become more familiar with the tools, evaluate models on the languages, frameworks, and tasks that matter to your own projects.

For example, a model that performs well on isolated Python functions may behave differently when asked to modify a multi-package TypeScript repository. A useful evaluation should include the surrounding code, project conventions, and the tests that determine whether the changes work.

Conclusion

Continue and Ollama provide a flexible foundation for local AI-assisted software development. Ollama manages local model execution, while Continue connects that inference capability to the editor and its development workflows.

The most effective configuration is task-aware. Use a model suited to fast autocomplete, a capable model for more complex editing, and only rely on agent functionality when tool support has been verified.

Start with a working local model, configure Continue using YAML, test each role independently, and measure the results on real development tasks. As models and integrations evolve, keep the configuration and evaluation process reproducible.

Further reading

• Continue documentation
• Continue's Ollama integration guide
• Continue configuration reference
• Continue model roles
• Ollama documentation
• Ollama Modelfile reference