Building a Custom AI Agent Framework with Next.js and Ollama

In today's rapidly evolving AI landscape, agent-based systems have emerged as powerful tools for task automation and complex problem-solving. This blog post will guide you through creating a sophisticated Next.js application with a custom AI agent framework powered by Ollama, an open-source local LLM runner.

What We're Building

We'll develop an application where users can submit goals like "Create a content calendar for social media" and watch as an AI agent systematically works through the problem, documenting its reasoning and delivering high-quality results. The beauty of this approach is that everything runs locally on your machine using Ollama, providing privacy benefits and eliminating API costs.

Key Concepts in Our Agent Framework

Before diving into the code, let's understand the core concepts that make our custom agent framework powerful:

1. Step-Based Task Decomposition

Complex tasks become manageable when broken down into smaller steps. Our agent takes a user's goal and automatically divides it into logical steps, similar to how a human would approach a complex problem:

typescriptCopy
1// Sample task decomposition
2const steps = [
3  "Analyze target audience and choose platforms",
4  "Establish content themes and post types",
5  "Create first half of weekly content calendar",
6  "Create second half of weekly content calendar",
7  "Add engagement strategies and hashtag recommendations"
8];

2. Reasoning Before Action

For each step, our agent first explains its reasoning before taking action. This creates transparency and allows users to understand the agent's thought process:

typescriptCopy
1// Sample reasoning for a step
2const reasoning = "Before creating content, I need to understand who we're targeting and which platforms would be most effective for a coffee shop. Typically, Instagram and Facebook work well for food/beverage businesses.";

3. Streaming Progress Updates

Users receive real-time updates as the agent works through each step, maintaining engagement and giving visibility into the process:

typescriptCopy
1// Sending a real-time update to the client
2await sendUpdate({
3  type: 'log',
4  message: `📝 Step ${step.number}: ${step.description}`
5});

4. Contextual Memory

Each step builds upon previous steps, maintaining context throughout the execution:

typescriptCopy
1const stepPrompt = `
2  Task: "${this.goal}"
3  Step ${stepNumber}/${Math.min(steps.length, this.maxSteps)}: ${stepDescription}
4  Previous steps: ${this.steps.map(s => `Step ${s.number}: ${s.description} -> ${s.output?.substring(0, 100)}...`).join('\n')}
5  Execute this step and provide the output. Be thorough but focused on just this step.
6`;

Setting Up the Project

Let's begin by creating a Next.js project and installing dependencies:

bashCopy
1npx create-next-app@latest next-ollama-agent
2cd next-ollama-agent
3npm install dotenv react-markdown

Next, download and install Ollama, then pull the Mistral model:

bashCopy
1ollama pull mistral

Building the Custom Agent Class

The heart of our application is the Agent class, which handles the execution of tasks:

typescriptCopy
1// src/lib/agent.ts
2export interface Step {
3  number: number;
4  description: string;
5  reasoning?: string;
6  output?: string;
7}
8
9export interface AgentResult {
10  goal: string;
11  steps: Step[];
12  output: string;
13}
14
15export type StepCallback = (step: Step) => Promise<void> | void;
16
17export class Agent {
18  private goal: string;
19  private maxSteps: number;
20  private onStepComplete?: StepCallback;
21  private steps: Step[] = [];
22
23  constructor(options: {
24    goal: string;
25    maxSteps?: number;
26    onStepComplete?: StepCallback;
27  }) {
28    this.goal = options.goal;
29    this.maxSteps = options.maxSteps || 5;
30    this.onStepComplete = options.onStepComplete;
31  }
32
33  async execute(): Promise<AgentResult> {
34    // Step 1: Task analysis
35    const taskAnalysis = await this.callOllama(
36      `Analyze this task: "${this.goal}". Break it down into ${this.maxSteps} clear steps that would lead to a high-quality result. Return a JSON array of step descriptions only, no additional text.`
37    );
38
39    // Parse steps from the model response
40    let steps: string[] = [];
41    try {
42      const parsed = JSON.parse(this.extractJSON(taskAnalysis));
43      steps = Array.isArray(parsed) ? parsed : [];
44    } catch (e) {
45      // Fallback extraction with regex if JSON parsing fails
46      const stepRegex = /\d+\.\s*(.*?)(?=\d+\.|$)/gs;
47      const matches = [...taskAnalysis.matchAll(stepRegex)];
48      steps = matches.map(match => match[1].trim());
49    }
50
51    // Default steps if extraction fails
52    if (steps.length === 0) {
53      steps = ["Analyze the problem", "Generate solution", "Refine the output"];
54    }
55
56    // Execute each step
57    for (let i = 0; i < Math.min(steps.length, this.maxSteps); i++) {
58      const stepNumber = i + 1;
59      const stepDescription = steps[i];
60
61      // Generate reasoning for this step
62      const reasoning = await this.callOllama(
63        `For the task: "${this.goal}", I am on step ${stepNumber}: "${stepDescription}". Explain your reasoning for how you'll approach this step. Keep it clear and concise.`
64      );
65
66      // Execute the step with context from previous steps
67      const stepPrompt = `
68        Task: "${this.goal}"
69        Step ${stepNumber}/${Math.min(steps.length, this.maxSteps)}: ${stepDescription}
70        Previous steps: ${this.steps.map(s => `Step ${s.number}: ${s.description} -> ${s.output?.substring(0, 100)}...`).join('\n')}
71        Execute this step and provide the output. Be thorough but focused on just this step.
72      `;
73      const stepOutput = await this.callOllama(stepPrompt);
74
75      // Record the step
76      const step: Step = {
77        number: stepNumber,
78        description: stepDescription,
79        reasoning,
80        output: stepOutput
81      };
82      this.steps.push(step);
83
84      // Notify via callback if provided
85      if (this.onStepComplete) {
86        await this.onStepComplete(step);
87      }
88    }
89
90    // Generate final comprehensive output
91    const finalPrompt = `
92      You've been working on: "${this.goal}"
93      You've completed the following steps:
94      ${this.steps.map(s => `Step ${s.number}: ${s.description}`).join('\n')}
95      Now, compile all of your work into a comprehensive final output that achieves the original goal.
96      Format your response using Markdown for readability.
97    `;
98    const finalOutput = await this.callOllama(finalPrompt);
99
100    return {
101      goal: this.goal,
102      steps: this.steps,
103      output: finalOutput
104    };
105  }
106
107  private async callOllama(prompt: string): Promise<string> {
108    try {
109      const response = await fetch('http://localhost:11434/api/generate', {
110        method: 'POST',
111        headers: {
112          'Content-Type': 'application/json',
113        },
114        body: JSON.stringify({
115          model: 'mistral',
116          prompt: prompt,
117          stream: false,
118        }),
119      });
120
121      if (!response.ok) {
122        throw new Error(`Ollama API error: ${response.statusText}`);
123      }
124
125      const data = await response.json();
126      return data.response;
127    } catch (error) {
128      console.error('Error calling Ollama:', error);
129      return `Error: ${error instanceof Error ? error.message : 'Unknown error'}`;
130    }
131  }
132
133  private extractJSON(text: string): string {
134    // Try to extract JSON from the text
135    const jsonRegex = /(\[.*\]|\{.*\})/s;
136    const match = text.match(jsonRegex);
137    return match ? match[0] : '[]';
138  }
139}

Building the Frontend

Our frontend uses React and Next.js to create a clean, responsive interface:

tsxCopy
1// src/app/page.tsx
2"use client";
3
4import { useState, useRef, useEffect } from "react";
5import ReactMarkdown from "react-markdown";
6
7export default function Home() {
8  const [goal, setGoal] = useState<string>("");
9  const [logs, setLogs] = useState<string[]>([]);
10  const [isRunning, setIsRunning] = useState<boolean>(false);
11  const [result, setResult] = useState<string>("");
12  const logsEndRef = useRef<HTMLDivElement>(null);
13
14  // Auto-scroll to the bottom of logs
15  useEffect(() => {
16    if (logsEndRef.current) {
17      logsEndRef.current.scrollIntoView({ behavior: "smooth" });
18    }
19  }, [logs]);
20
21  const handleRunAgent = async () => {
22    if (!goal.trim() || isRunning) return;
23    setIsRunning(true);
24    setLogs(["🤖 Initializing custom agent powered by Ollama..."]);
25    setResult("");
26
27    try {
28      const response = await fetch("/api/run-agent", {
29        method: "POST",
30        headers: { "Content-Type": "application/json" },
31        body: JSON.stringify({ goal }),
32      });
33
34      if (!response.ok) {
35        const errorData = await response.json();
36        throw new Error(errorData.error || "Failed to run agent");
37      }
38
39      // Use streaming for real-time updates
40      const reader = response.body?.getReader();
41      const decoder = new TextDecoder();
42      
43      if (reader) {
44        while (true) {
45          const { done, value } = await reader.read();
46          if (done) break;
47          
48          const text = decoder.decode(value);
49          try {
50            // Handle multiple JSON objects in the same chunk
51            const jsonObjects = text.split('\n').filter(line => line.trim());
52            
53            for (const jsonStr of jsonObjects) {
54              if (!jsonStr.trim()) continue;
55              const data = JSON.parse(jsonStr);
56              
57              if (data.type === "log") {
58                setLogs(logs => [...logs, data.message]);
59              } else if (data.type === "result") {
60                setResult(data.content);
61              }
62            }
63          } catch (error) {
64            console.error("Error parsing stream data:", error);
65          }
66        }
67      }
68    } catch (error: any) {
69      setLogs(logs => [...logs, `❌ Error: ${error.message}`]);
70    } finally {
71      setIsRunning(false);
72      setLogs(logs => [...logs, "✅ Agent execution completed"]);
73    }
74  };
75
76  return (
77    <main className="flex min-h-screen flex-col items-center p-8 max-w-5xl mx-auto">
78      <h1 className="text-4xl font-bold mb-3">AI Agent Workspace</h1>
79      <h2 className="text-xl text-gray-600 mb-8">Powered by Custom Agent Framework + Ollama</h2>
80      
81      <div className="w-full space-y-8">
82        {/* Goal Input Section */}
83        <div className="bg-white p-6 rounded-lg shadow-md">
84          <h3 className="text-lg font-semibold mb-3">What would you like the agent to accomplish?</h3>
85          <div className="flex gap-3">
86            <input
87              type="text"
88              placeholder="e.g., Create a marketing plan for a new product launch"
89              value={goal}
90              onChange={(e) => setGoal(e.target.value)}
91              className="flex-1 p-3 border rounded-md text-gray-800 focus:ring-2 focus:ring-blue-500"
92              disabled={isRunning}
93            />
94            <button
95              onClick={handleRunAgent}
96              disabled={isRunning || !goal.trim()}
97              className={`px-6 py-3 rounded-md font-medium transition ${
98                isRunning ?
99                "bg-gray-300 text-gray-600" :
100                "bg-blue-600 text-white hover:bg-blue-700"
101              }`}
102            >
103              {isRunning ? "Working..." : "Run Agent"}
104            </button>
105          </div>
106        </div>
107
108        {/* Agent Logs Section */}
109        <div className="bg-gray-50 rounded-lg shadow-md">
110          <div className="bg-gray-100 p-4 rounded-t-lg border-b">
111            <h3 className="text-lg font-semibold">Agent Thinking Process</h3>
112          </div>
113          <div className="p-4 max-h-80 overflow-y-auto">
114            {logs.length === 0 ? (
115              <p className="text-gray-500 italic">Agent logs will appear here...</p>
116            ) : (
117              <div className="space-y-2">
118                {logs.map((log, index) => (
119                  <div key={index} className="p-3 bg-white rounded border">
120                    {log}
121                  </div>
122                ))}
123                <div ref={logsEndRef} />
124              </div>
125            )}
126          </div>
127        </div>
128
129        {/* Result Section */}
130        {result && (
131          <div className="bg-white rounded-lg shadow-md">
132            <div className="bg-green-100 p-4 rounded-t-lg border-b">
133              <h3 className="text-lg font-semibold text-green-800">Agent Result</h3>
134            </div>
135            <div className="p-6 prose max-w-none">
136              <ReactMarkdown>{result}</ReactMarkdown>
137            </div>
138          </div>
139        )}
140      </div>
141    </main>
142  );
143}

Creating the API Endpoint

Next, let's create the serverless API endpoint that will run our agent and stream results back to the client:

typescriptCopy
1// src/app/api/run-agent/route.ts
2import { NextResponse } from 'next/server';
3import { Agent, Step } from '@/lib/agent';
4
5export const runtime = 'nodejs';
6
7export async function POST(request: Request) {
8  // Initialize the response encoder for streaming
9  const encoder = new TextEncoder();
10  const stream = new TransformStream();
11  const writer = stream.writable.getWriter();
12
13  // Function to send updates to the client
14  const sendUpdate = async (data: any) => {
15    await writer.write(encoder.encode(JSON.stringify(data) + '\n'));
16  };
17
18  // Process the request in the background while streaming updates
19  const processRequest = async () => {
20    try {
21      // Parse the request body
22      const { goal } = await request.json();
23      
24      if (!goal || typeof goal !== 'string') {
25        await sendUpdate({
26          type: 'log',
27          message: '❌ Error: Please provide a valid goal'
28        });
29        writer.close();
30        return;
31      }
32
33      // Initialize the custom agent
34      const agent = new Agent({
35        goal: goal,
36        maxSteps: 5,
37        onStepComplete: async (step: Step) => {
38          await sendUpdate({
39            type: 'log',
40            message: `📝 Step ${step.number}: ${step.description}`
41          });
42          
43          if (step.reasoning) {
44            await sendUpdate({
45              type: 'log',
46              message: `🤔 Reasoning: ${step.reasoning}`
47            });
48          }
49        }
50      });
51
52      // Log initialization
53      await sendUpdate({
54        type: 'log',
55        message: `🧠 Analyzing task: "${goal}"`
56      });
57
58      // Execute the agent
59      const result = await agent.execute();
60
61      // Send the final result
62      await sendUpdate({
63        type: 'result',
64        content: result.output
65      });
66
67      // Close the stream
68      writer.close();
69    } catch (error: any) {
70      console.error('Agent execution error:', error);
71      await sendUpdate({
72        type: 'log',
73        message: `❌ Error: ${error.message || 'Unknown error occurred'}`
74      });
75      writer.close();
76    }
77  };
78
79  // Start processing in the background
80  processRequest();
81
82  // Return the stream response immediately
83  return new NextResponse(stream.readable, {
84    headers: {
85      'Content-Type': 'application/json',
86      'Transfer-Encoding': 'chunked',
87    },
88  });
89}

Advanced Enhancements

After getting the basic application working, we can enhance our agent framework with more advanced capabilities:

1. Adding Specialized Tools

Let's enhance our agent with tools that can perform specific functions:

typescriptCopy
1// src/lib/tools.ts
2export interface Tool {
3  name: string;
4  description: string;
5  execute: (input: string) => Promise<string>;
6}
7
8// Sample search tool
9export const searchTool: Tool = {
10  name: 'search',
11  description: 'Search the web for information',
12  async execute(query: string): Promise<string> {
13    // This is a mock implementation - in a real app, you'd integrate with a search API
14    return `Simulated search results for: ${query}\n\n1. First relevant result\n2. Second relevant result\n3. Third relevant result`;
15  }
16};

2. Multi-Agent Workflows

For complex tasks, we can create workflows with multiple specialized agents:

typescriptCopy
1// src/lib/workflow.ts
2import { Agent, AgentResult } from './agent';
3
4export async function runResearchAndSynthesisWorkflow(topic: string, updateCallback: (message: string) => Promise<void>) {
5  await updateCallback(`Starting research workflow on: ${topic}`);
6  
7  // Research agent gathers information
8  const researchAgent = new Agent({
9    goal: `Research key facts about: ${topic}`,
10    maxSteps: 3,
11    onStepComplete: async (step) => {
12      await updateCallback(`Research step ${step.number}: ${step.description}`);
13    }
14  });
15  
16  await updateCallback("Starting research phase...");
17  const researchResult = await researchAgent.execute();
18  
19  // Analysis agent evaluates the research
20  const analysisAgent = new Agent({
21    goal: `Analyze these research findings and identify key insights: ${researchResult.output}`,
22    maxSteps: 2,
23    onStepComplete: async (step) => {
24      await updateCallback(`Analysis step ${step.number}: ${step.description}`);
25    }
26  });
27  
28  await updateCallback("Starting analysis phase...");
29  const analysisResult = await analysisAgent.execute();
30  
31  // Synthesis agent creates final output
32  const synthesisAgent = new Agent({
33    goal: `Create a comprehensive report on ${topic} using this research and analysis:
34    Research: ${researchResult.output}
35    Analysis: ${analysisResult.output}`,
36    maxSteps: 3,
37    onStepComplete: async (step) => {
38      await updateCallback(`Synthesis step ${step.number}: ${step.description}`);
39    }
40  });
41  
42  await updateCallback("Starting synthesis phase...");
43  const finalResult = await synthesisAgent.execute();
44  
45  await updateCallback("Workflow complete!");
46  
47  return {
48    topic,
49    research: researchResult.output,
50    analysis: analysisResult.output,
51    synthesis: finalResult.output
52  };
53}

3. Memory and Database Integration

For persistence between sessions, we can integrate a database:

typescriptCopy
1// Using Prisma with SQLite for simplicity
2import { PrismaClient } from '@prisma/client';
3
4const prisma = new PrismaClient();
5
6export async function saveSession(goal: string, output: string, logs: string[]) {
7  return await prisma.session.create({
8    data: {
9      goal,
10      output,
11      logs: JSON.stringify(logs),
12    },
13  });
14}
15
16export async function getSessions() {
17  return await prisma.session.findMany({
18    orderBy: {
19      createdAt: 'desc',
20    },
21  });
22}

Testing and Evaluation

When testing your agent, try these diverse goals to evaluate its capabilities:

• Business Tasks: "Create a marketing strategy for a new fitness app"
• Creative Tasks: "Write a short story about time travel with a twist ending"
• Analytical Problems: "Analyze the pros and cons of remote work for a small business"

Conclusion

We've built a sophisticated AI agent framework that leverages Next.js and Ollama to create a powerful task automation system. This combination offers several key advantages:

1. Local Privacy: By running models through Ollama, you maintain control of your data without sending it to external API services.
2. Cost Efficiency: Eliminate per-token or per-request charges by running inference locally.
3. Architectural Flexibility: Our custom agent implementation provides a structured framework that can be extended as needed.
4. Realtime Feedback: The streaming architecture keeps users informed of progress throughout the execution.

The step-based approach with explicit reasoning creates transparency that builds user trust in the AI system. By mastering these technologies, you're well-positioned to build intelligent applications that blend the best of human creativity with AI capabilities.

What tasks would you automate with your custom agent framework?

---

Want to explore more advanced agent patterns and LLM applications? Follow our blog for more tutorials and insights into the world of AI development.