Complete Guide: Building an AI-Powered Next.js Application with Custom Agent Framework and Ollama Integration
A comprehensive guide to building a Next.js application with a custom agent framework that integrates with Ollama for local AI task automation, featuring real-time streaming, task decomposition, and interactive user interfaces.
Daniel Kliewer
Author, Sovereign AI


Building an AI-Powered Next.js Application with Custom Agent Framework, and Ollama
1. Introduction
What is Ollama? Ollama allows you to run large language models (LLMs) locally on your machine rather than relying on cloud APIs. This approach provides privacy benefits, reduces costs, and eliminates API latency issues—making it ideal for development and privacy-sensitive applications.
By the end of this tutorial, you'll have created a web application where users can submit goals like "Create a content calendar for social media" or "Analyze quarterly sales data," and watch as an AI agent systematically works through the problem, documenting its reasoning and producing high-quality results.
2. Setting Up the Project
2.1 Prerequisites
Before starting, ensure you have:
- Node.js 18+ installed
- Basic knowledge of React and Next.js
- Ollama installed (we'll cover this in detail)
2.2 Creating a Next.js Application
Let's begin by creating a fresh Next.js project:
bash1npx create-next-app@latest next-ollama-app23cd next-ollama-app
During the setup, select the following options:
- Would you like to use TypeScript? → Yes (for type safety)
- Would you like to use ESLint? → Yes
- Would you like to use Tailwind CSS? → Yes (for styling)
- Would you like to use the src/ directory? → Yes (for organization)
- Would you like to use App Router? → Yes (for modern routing)
- Would you like to customize the default import alias? → No
2.3 Installing Dependencies
Install the necessary packages:
bash1npm install dotenv react-markdown
2.4 Setting Up Ollama
- Visit Ollama's official website and download the installer for your operating system.
- Install Ollama following the on-screen instructions.
- Open a terminal and pull the Mistral model (a powerful open-source LLM):
bash1ollama pull mistral
This will download the model, which may take several minutes depending on your internet connection.
2.5 Creating a Custom Agent Framework
Let's create our own lightweight agent framework:
Create a file at src/lib/agent.ts:
typescript1// src/lib/agent.ts2export interface Step {3 number: number;4 description: string;5 reasoning?: string;6 output?: string;7}89export interface AgentResult {10 goal: string;11 steps: Step[];12 output: string;13}1415export type StepCallback = (step: Step) => Promise<void> | void;1617export class Agent {18 private goal: string;19 private maxSteps: number;20 private onStepComplete?: StepCallback;21 private steps: Step[] = [];2223 constructor(options: {24 goal: string;25 maxSteps?: number;26 onStepComplete?: StepCallback;27 }) {28 this.goal = options.goal;29 this.maxSteps = options.maxSteps || 5;30 this.onStepComplete = options.onStepComplete;31 }3233 async execute(): Promise<AgentResult> {34 // Step 1: Task analysis35 const taskAnalysis = await this.callOllama(36 `Analyze this task: "${this.goal}". Break it down into ${this.maxSteps} clear steps that would lead to a high-quality result. Return a JSON array of step descriptions only, no additional text.`37 );3839 let steps: string[] = [];40 try {41 const parsed = JSON.parse(this.extractJSON(taskAnalysis));42 steps = Array.isArray(parsed) ? parsed : [];43 } catch (e) {44 // If parsing fails, try to extract steps using regex45 const stepRegex = /\d+\.\s*(.*?)(?=\d+\.|$)/gs;46 const matches = [...taskAnalysis.matchAll(stepRegex)];47 steps = matches.map(match => match[1].trim());48 }4950 // Ensure we have steps51 if (steps.length === 0) {52 steps = ["Analyze the problem", "Generate solution", "Refine the output"];53 }5455 // Execute each step56 for (let i = 0; i < Math.min(steps.length, this.maxSteps); i++) {57 const stepNumber = i + 1;58 const stepDescription = steps[i];5960 // Generate reasoning for this step61 const reasoning = await this.callOllama(62 `For the task: "${this.goal}", I am on step ${stepNumber}: "${stepDescription}". Explain your reasoning for how you'll approach this step. Keep it clear and concise.`63 );6465 // Execute the step66 const stepPrompt = `67Task: "${this.goal}"68Step ${stepNumber}/${Math.min(steps.length, this.maxSteps)}: ${stepDescription}69Previous steps: ${this.steps.map(s => `Step ${s.number}: ${s.description} -> ${s.output?.substring(0, 100)}...`).join('\n')}7071Execute this step and provide the output. Be thorough but focused on just this step.72`;7374 const stepOutput = await this.callOllama(stepPrompt);7576 // Record the step77 const step: Step = {78 number: stepNumber,79 description: stepDescription,80 reasoning,81 output: stepOutput82 };8384 this.steps.push(step);8586 // Notify via callback if provided87 if (this.onStepComplete) {88 await this.onStepComplete(step);89 }90 }9192 // Generate final comprehensive output93 const finalPrompt = `94You've been working on: "${this.goal}"9596You've completed the following steps:97${this.steps.map(s => `Step ${s.number}: ${s.description}`).join('\n')}9899Now, compile all of your work into a comprehensive final output that achieves the original goal.100Format your response using Markdown for readability. Include headings, bullet points, and other formatting as appropriate.101Ensure your response is complete, well-structured, and directly addresses the original goal.102`;103104 const finalOutput = await this.callOllama(finalPrompt);105106 return {107 goal: this.goal,108 steps: this.steps,109 output: finalOutput110 };111 }112113 private async callOllama(prompt: string): Promise<string> {114 try {115 const response = await fetch('http://localhost:11434/api/generate', {116 method: 'POST',117 headers: {118 'Content-Type': 'application/json',119 },120 body: JSON.stringify({121 model: 'mistral',122 prompt: prompt,123 stream: false,124 }),125 });126127 if (!response.ok) {128 throw new Error(`Ollama API error: ${response.statusText}`);129 }130131 const data = await response.json();132 return data.response;133 } catch (error) {134 console.error('Error calling Ollama:', error);135 return `Error: ${error instanceof Error ? error.message : 'Unknown error'}`;136 }137 }138139 private extractJSON(text: string): string {140 // Try to extract JSON from the text141 const jsonRegex = /(\[.*\]|\{.*\})/s;142 const match = text.match(jsonRegex);143 return match ? match[0] : '[]';144 }145}
This custom agent implementation provides similar functionality to what we'd expect from Mastra:
- Breaking down a task into logical steps
- Reasoning about each step before execution
- Executing steps sequentially
- Providing step-by-step progress updates
- Generating a comprehensive final output
3. Understanding the Frontend (React + Next.js)
Now, let's build a responsive, user-friendly interface for our agent application.
3.1 Creating the Home Page Component
Create or replace the file at src/app/page.tsx with:
tsx1"use client";2import { useState, useRef, useEffect } from "react";3import ReactMarkdown from "react-markdown";45export default function Home() {6 const [goal, setGoal] = useState<string>("");7 const [logs, setLogs] = useState<string[]>([]);8 const [isRunning, setIsRunning] = useState<boolean>(false);9 const [result, setResult] = useState<string>("");10 const logsEndRef = useRef<HTMLDivElement>(null);1112 // Auto-scroll to the bottom of logs13 useEffect(() => {14 if (logsEndRef.current) {15 logsEndRef.current.scrollIntoView({ behavior: "smooth" });16 }17 }, [logs]);1819 const handleRunAgent = async () => {20 if (!goal.trim() || isRunning) return;2122 setIsRunning(true);23 setLogs(["🤖 Initializing Mastra-inspired agent powered by Ollama..."]);24 setResult("");2526 try {27 const response = await fetch("/api/run-agent", {28 method: "POST",29 headers: { "Content-Type": "application/json" },30 body: JSON.stringify({ goal }),31 });3233 if (!response.ok) {34 const errorData = await response.json();35 throw new Error(errorData.error || "Failed to run agent");36 }3738 // Use streaming for real-time updates39 const reader = response.body?.getReader();40 const decoder = new TextDecoder();4142 if (reader) {43 while (true) {44 const { done, value } = await reader.read();45 if (done) break;4647 const text = decoder.decode(value);48 try {49 // Handle multiple JSON objects in the same chunk50 const jsonObjects = text.split('\n').filter(line => line.trim());5152 for (const jsonStr of jsonObjects) {53 if (!jsonStr.trim()) continue;5455 const data = JSON.parse(jsonStr);5657 if (data.type === "log") {58 setLogs(logs => [...logs, data.message]);59 } else if (data.type === "result") {60 setResult(data.content);61 }62 }63 } catch (error) {64 console.error("Error parsing stream data:", error);65 }66 }67 }68 } catch (error: any) {69 setLogs(logs => [...logs, `❌ Error: ${error.message}`]);70 } finally {71 setIsRunning(false);72 setLogs(logs => [...logs, "✅ Agent execution completed"]);73 }74 };7576 return (77 <main className="flex min-h-screen flex-col items-center p-8 max-w-5xl mx-auto">78 <h1 className="text-4xl font-bold mb-3">AI Agent Workspace</h1>79 <h2 className="text-xl text-gray-600 mb-8">Powered by Mastra-inspired Architecture + Ollama</h2>8081 <div className="w-full space-y-8">82 {/* Goal Input Section */}83 <div className="bg-white p-6 rounded-lg shadow-md">84 <h3 className="text-lg font-semibold mb-3">What would you like the agent to accomplish?</h3>85 <div className="flex gap-3">86 <input87 type="text"88 placeholder="e.g., Create a marketing plan for a new product launch"89 value={goal}90 onChange={(e) => setGoal(e.target.value)}91 className="flex-1 p-3 border rounded-md text-gray-800 focus:ring-2 focus:ring-blue-500"92 disabled={isRunning}93 />94 <button95 onClick={handleRunAgent}96 disabled={isRunning || !goal.trim()}97 className={`px-6 py-3 rounded-md font-medium transition ${98 isRunning ?99 "bg-gray-300 text-gray-600" :100 "bg-blue-600 text-white hover:bg-blue-700"101 }`}102 >103 {isRunning ? "Working..." : "Run Agent"}104 </button>105 </div>106 </div>107108 {/* Agent Logs Section */}109 <div className="bg-gray-50 rounded-lg shadow-md">110 <div className="bg-gray-100 p-4 rounded-t-lg border-b">111 <h3 className="text-lg font-semibold">Agent Thinking Process</h3>112 </div>113 <div className="p-4 max-h-80 overflow-y-auto">114 {logs.length === 0 ? (115 <p className="text-gray-500 italic">Agent logs will appear here...</p>116 ) : (117 <div className="space-y-2">118 {logs.map((log, index) => (119 <div key={index} className="p-3 bg-white rounded border">120 {log}121 </div>122 ))}123 <div ref={logsEndRef} />124 </div>125 )}126 </div>127 </div>128129 {/* Result Section */}130 {result && (131 <div className="bg-white rounded-lg shadow-md">132 <div className="bg-green-100 p-4 rounded-t-lg border-b">133 <h3 className="text-lg font-semibold text-green-800">Agent Result</h3>134 </div>135 <div className="p-6 prose max-w-none">136 <ReactMarkdown>{result}</ReactMarkdown>137 </div>138 </div>139 )}140 </div>141 </main>142 );143}
3.2 Understanding the Frontend Components
The frontend is built with several key features:
-
State Management:
goal: Stores the user's input tasklogs: Maintains an array of execution logsisRunning: Tracks the agent's execution stateresult: Stores the final output from the agent
-
Streaming Response Handling:
- Uses the Fetch API with a reader/decoder to process streamed updates
- Separates log updates from final results
-
UI Components:
- A clean input section for submitting tasks
- A scrollable log window showing the agent's reasoning process
- A formatted result section using React Markdown for rich text display
-
User Experience Enhancements:
- Auto-scrolling logs to keep the latest updates visible
- Disabled inputs during processing
- Visual feedback for running state
4. Building the Backend (API Route with Custom Agent + Ollama)
Now, let's create the serverless API endpoint that will run our custom agent with Ollama.
4.1 Creating the API Route
Create a file at src/app/api/run-agent/route.ts:
typescript1import { NextResponse } from 'next/server';2import { Agent, Step } from '@/lib/agent';34export const runtime = 'nodejs';56export async function POST(request: Request) {7 // Initialize the response encoder for streaming8 const encoder = new TextEncoder();9 const stream = new TransformStream();10 const writer = stream.writable.getWriter();1112 // Function to send updates to the client13 const sendUpdate = async (data: any) => {14 await writer.write(encoder.encode(JSON.stringify(data) + '\n'));15 };1617 // Process the request in the background while streaming updates18 const processRequest = async () => {19 try {20 // Parse the request body21 const { goal } = await request.json();2223 if (!goal || typeof goal !== 'string') {24 await sendUpdate({25 type: 'log',26 message: '❌ Error: Please provide a valid goal'27 });28 writer.close();29 return;30 }3132 // Initialize the custom agent33 const agent = new Agent({34 goal: goal,35 maxSteps: 5,36 onStepComplete: async (step: Step) => {37 await sendUpdate({38 type: 'log',39 message: `📝 Step ${step.number}: ${step.description}`40 });4142 if (step.reasoning) {43 await sendUpdate({44 type: 'log',45 message: `🤔 Reasoning: ${step.reasoning}`46 });47 }48 }49 });5051 // Log initialization52 await sendUpdate({53 type: 'log',54 message: `🧠 Analyzing task: "${goal}"`55 });5657 // Execute the agent58 const result = await agent.execute();5960 // Send the final result61 await sendUpdate({62 type: 'result',63 content: result.output64 });6566 // Close the stream67 writer.close();68 } catch (error: any) {69 console.error('Agent execution error:', error);70 await sendUpdate({71 type: 'log',72 message: `❌ Error: ${error.message || 'Unknown error occurred'}`73 });74 writer.close();75 }76 };7778 // Start processing in the background79 processRequest();8081 // Return the stream response immediately82 return new NextResponse(stream.readable, {83 headers: {84 'Content-Type': 'application/json',85 'Transfer-Encoding': 'chunked',86 },87 });88}
4.2 Understanding the Backend Architecture
Our API route implements several advanced features:
-
Streaming Response:
- Uses the Web Streams API to send real-time updates to the frontend
- Maintains a single connection instead of polling
-
Custom Agent Integration:
- Initializes our custom Agent class with the user's goal
- Configures step limits and callback functions
- Streams progress updates in real-time
-
Progress Tracking:
- Uses the
onStepCompletecallback to report each step's progress - Separates reasoning logs from final results
- Uses the
-
Error Handling:
- Robust error catching and reporting
- Ensures the stream is properly closed even on errors
5. Running and Testing the Application
Now let's run our application and test its capabilities:
5.1 Starting the Development Server
Ensure Ollama is running, then start your Next.js application:
bash1npm run dev
Open your browser to http://localhost:3000.
5.2 Testing with Different Goals
Try entering various goals to test the agent's capabilities:
Business Planning Examples:
- "Create a marketing strategy for a new fitness app"
- "Develop a 30-day content calendar for a tech startup"
- "Draft a project plan for website redesign"
Creative Tasks:
- "Write a short story about time travel with a twist ending"
- "Create a detailed character profile for a fantasy novel"
- "Develop three unique logo concepts for a sustainable fashion brand"
Analytical Problems:
- "Analyze the pros and cons of remote work for a small business"
- "Compare three different pricing strategies for a SaaS product"
- "Create a SWOT analysis for entering the electric vehicle market"
5.3 Sample Interaction
Here's an example of how the agent might process a goal to "Create a 7-day social media plan for a coffee shop":
Agent Logs:
text1🤖 Initializing Mastra-inspired agent powered by Ollama...2🧠 Analyzing task: "Create a 7-day social media plan for a coffee shop"3📝 Step 1: Define the target audience and social media platforms4🤔 Reasoning: Before creating content, I need to understand who we're targeting and which platforms would be most effective for a coffee shop. Typically, Instagram and Facebook work well for food/beverage businesses.5📝 Step 2: Establish content themes and post types6🤔 Reasoning: Coffee shops can benefit from diverse content including product highlights, behind-the-scenes, customer features, and educational content about coffee.7📝 Step 3: Create a content calendar for Monday through Wednesday8🤔 Reasoning: I'll start with the first half of the week, focusing on driving early-week traffic when coffee shops might be slower.9📝 Step 4: Create a content calendar for Thursday through Sunday10🤔 Reasoning: For the latter half of the week, I'll focus on weekend promotions and creating content that encourages longer visits and higher purchases.11📝 Step 5: Add engagement strategies and hashtag recommendations12🤔 Reasoning: Social media success requires engagement beyond just posting. I'll add strategies for responding to comments and effective hashtags.13✅ Agent execution completed
Agent Result: The final output would be a comprehensive, day-by-day social media plan formatted in Markdown, including specific post ideas, optimal posting times, hashtag recommendations, and engagement strategies.
6. Expanding the Project
Once you have the basic application working, consider these enhancements to create a more powerful agent system:
6.1 Adding Specialized Agent Tools
Extend your custom agent with specialized tools for different tasks:
typescript1// src/lib/tools.ts2export interface Tool {3 name: string;4 description: string;5 execute: (input: string) => Promise<string>;6}78// Sample search tool9export const searchTool: Tool = {10 name: 'search',11 description: 'Search the web for information',12 async execute(query: string): Promise<string> {13 // This is a mock implementation - in a real app, you'd integrate with a search API14 return `Simulated search results for: ${query}\n\n1. First relevant result\n2. Second relevant result\n3. Third relevant result`;15 }16};1718// Then enhance your Agent class to use tools19// In src/lib/agent.ts, modify the execute method:2021async execute(): Promise<AgentResult> {22 // ... existing code2324 // Add tool usage where appropriate25 if (this.tools && this.tools.length > 0) {26 const toolDescriptions = this.tools.map(t => `${t.name}: ${t.description}`).join('\n');2728 // Let the agent decide whether to use a tool29 const toolUsage = await this.callOllama(`30 For the task: "${this.goal}", I'm on step ${currentStep.number}: "${currentStep.description}".31 I have these tools available:32 ${toolDescriptions}3334 Should I use a tool for this step? If yes, specify which tool and the exact input to provide to the tool.35 Return your answer in JSON format: { "useTool": boolean, "toolName": string, "toolInput": string }36 `);3738 try {39 const toolDecision = JSON.parse(this.extractJSON(toolUsage));40 if (toolDecision.useTool) {41 const tool = this.tools.find(t => t.name === toolDecision.toolName);42 if (tool) {43 const toolResult = await tool.execute(toolDecision.toolInput);44 // Use the tool result in further processing45 currentStep.output = `Used ${tool.name} with input: ${toolDecision.toolInput}\n\nResult: ${toolResult}`;46 }47 }48 } catch (e) {49 // If JSON parsing fails, continue without tool usage50 }51 }5253 // ... rest of execute method54}
6.2 Implementing a Database for Conversation History
Add persistence to your application with a database:
typescript1// Using Prisma with SQLite for simplicity2// 1. Install Prisma: npm install prisma @prisma/client3// 2. Initialize Prisma: npx prisma init --datasource-provider sqlite45// 3. Create schema.prisma:6// prisma/schema.prisma7/*8generator client {9 provider = "prisma-client-js"10}1112datasource db {13 provider = "sqlite"14 url = "file:./dev.db"15}1617model Session {18 id String @id @default(uuid())19 goal String20 output String @default("")21 logs String @default("[]") // JSON string of logs22 createdAt DateTime @default(now())23}24*/2526// 4. Run migration: npx prisma migrate dev --name init27// 5. Generate client: npx prisma generate2829// 6. Create a database service30// src/lib/db.ts31import { PrismaClient } from '@prisma/client';3233const prisma = new PrismaClient();3435export async function saveSession(goal: string, output: string, logs: string[]) {36 return await prisma.session.create({37 data: {38 goal,39 output,40 logs: JSON.stringify(logs),41 },42 });43}4445export async function getSessions() {46 return await prisma.session.findMany({47 orderBy: {48 createdAt: 'desc',49 },50 });51}5253export async function getSession(id: string) {54 return await prisma.session.findUnique({55 where: { id },56 });57}
6.3 Implementing Multi-Agent Workflows
Create complex workflows with multiple specialized agents:
typescript1// src/lib/workflow.ts2import { Agent, AgentResult } from './agent';34export async function runResearchAndSynthesisWorkflow(topic: string, updateCallback: (message: string) => Promise<void>) {5 await updateCallback(`Starting research workflow on: ${topic}`);67 // Research agent gathers information8 const researchAgent = new Agent({9 goal: `Research key facts about: ${topic}`,10 maxSteps: 3,11 onStepComplete: async (step) => {12 await updateCallback(`Research step ${step.number}: ${step.description}`);13 }14 });1516 await updateCallback("Starting research phase...");17 const researchResult = await researchAgent.execute();1819 // Analysis agent evaluates the research20 const analysisAgent = new Agent({21 goal: `Analyze these research findings and identify key insights: ${researchResult.output}`,22 maxSteps: 2,23 onStepComplete: async (step) => {24 await updateCallback(`Analysis step ${step.number}: ${step.description}`);25 }26 });2728 await updateCallback("Starting analysis phase...");29 const analysisResult = await analysisAgent.execute();3031 // Synthesis agent creates final output32 const synthesisAgent = new Agent({33 goal: `Create a comprehensive report on ${topic} using this research and analysis:34 Research: ${researchResult.output}35 Analysis: ${analysisResult.output}`,36 maxSteps: 3,37 onStepComplete: async (step) => {38 await updateCallback(`Synthesis step ${step.number}: ${step.description}`);39 }40 });4142 await updateCallback("Starting synthesis phase...");43 const finalResult = await synthesisAgent.execute();4445 await updateCallback("Workflow complete!");4647 return {48 topic,49 research: researchResult.output,50 analysis: analysisResult.output,51 synthesis: finalResult.output52 };53}
6.4 Optimizing Model Selection and Configuration
Allow users to customize the LLM parameters:
typescript1// src/lib/modelConfig.ts2export interface ModelConfig {3 modelName: string;4 temperature: number;5 maxTokens: number;6}78export const availableModels = [9 { name: 'mistral', label: 'Mistral (Balanced)' },10 { name: 'llama3', label: 'Llama 3 (Creative)' },11 { name: 'codellama', label: 'CodeLlama (Technical)' },12];1314// Then update your Agent class to use these configurations:15private async callOllama(prompt: string): Promise<string> {16 try {17 const response = await fetch('http://localhost:11434/api/generate', {18 method: 'POST',19 headers: {20 'Content-Type': 'application/json',21 },22 body: JSON.stringify({23 model: this.modelConfig.modelName || 'mistral',24 prompt: prompt,25 temperature: this.modelConfig.temperature || 0.7,26 max_tokens: this.modelConfig.maxTokens || 2048,27 stream: false,28 }),29 });3031 if (!response.ok) {32 throw new Error(`Ollama API error: ${response.statusText}`);33 }3435 const data = await response.json();36 return data.response;37 } catch (error) {38 console.error('Error calling Ollama:', error);39 return `Error: ${error instanceof Error ? error.message : 'Unknown error'}`;40 }41}
7. Conclusion
In this comprehensive tutorial, we've built a sophisticated Next.js application that leverages a custom agent framework and Ollama to create an AI-powered task automation system. This combination offers several key advantages:
-
Local Privacy: By running models through Ollama, you maintain control of your data without sending it to external API services.
-
Cost Efficiency: Eliminate per-token or per-request charges by running inference locally.
-
Architectural Flexibility: Our custom agent implementation provides a structured framework that can be extended as needed.
-
Realtime Feedback: The streaming architecture keeps users informed of progress throughout the execution.
While we couldn't use the actual Mastra package due to availability issues, our custom implementation follows similar architectural principles, delivering a comparable experience. In a production environment, you might choose to use a commercially available agent framework like Mastra when it becomes publicly available, or continue to evolve your custom solution to meet your specific needs.
The agent-based approach we've implemented demonstrates how complex goals can be broken down into manageable steps with explicit reasoning at each stage. This not only produces better results but also creates transparency that builds user trust in the AI system.
Additional Resources
- Ollama GitHub Repository
- Next.js Documentation
- LangChain.js (complementary framework)
- Prompt Engineering Guide
By mastering these technologies, you're well-positioned to build the next generation of intelligent applications that blend the best of human creativity with AI capabilities.

Sovereign AI: Building Local-First Intelligent Systems
by Daniel Kliewer · Paperback · 72 pages
The hands-on guide to building AI that runs on your hardware, keeps your data private, and eliminates cloud dependence. Working code included.