mirrors/archon
1
0
Fork 0
mirror of https://github.com/coleam00/Archon.git synced 2025-12-24 02:39:17 -05:00
Code Issues Packages Projects Releases Wiki Activity

Remove original_archon folder from main branch

Browse Source 
- Removed the original_archon/ directory containing the legacy Archon v1-v6 iterations
- This was the original AI agent builder system before the pivot to the current architecture
- The folder has been preserved in the 'preserve-original-archon' branch for historical reference
- Reduces repository size by ~5.2MB and removes confusion about which codebase is active
This commit is contained in:
Rasmus Widing
2025-08-25 10:11:39 +03:00
committed by Wirasm
parent 85f5f2ac93
commit 51a8c74525
219 changed files with 0 additions and 25863 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
original_archon/.dockerignore
View File

@@ -1,38 +0,0 @@
# Ignore specified folders
iterations/
venv/
.langgraph_api/
.github/
__pycache__/
.env
# Git related
.git/
.gitignore
.gitattributes
# Python cache
*.pyc
*.pyo
*.pyd
.Python
*.so
.pytest_cache/
# Environment files
.env.local
.env.development.local
.env.test.local
.env.production.local
# Logs
*.log
# IDE specific files
.idea/
.vscode/
*.swp
*.swo
# Keep the example env file for reference
!.env.example

43
original_archon/.env.example
View File

@@ -1,43 +0,0 @@
# Base URL for the OpenAI instance (default is https://api.openai.com/v1)
# OpenAI: https://api.openai.com/v1
# Ollama (example): http://localhost:11434/v1
# OpenRouter: https://openrouter.ai/api/v1
# Anthropic: https://api.anthropic.com/v1
BASE_URL=
# For OpenAI: https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# For Anthropic: https://console.anthropic.com/account/keys
# For OpenRouter: https://openrouter.ai/keys
# For Ollama, no need to set this unless you specifically configured an API key
LLM_API_KEY=
# Get your Open AI API Key by following these instructions -
# https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# Even if using Anthropic or OpenRouter, you still need to set this for the embedding model.
# No need to set this if using Ollama.
OPENAI_API_KEY=
# For the Supabase version (sample_supabase_agent.py), set your Supabase URL and Service Key.
# Get your SUPABASE_URL from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
SUPABASE_URL=
# Get your SUPABASE_SERVICE_KEY from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
# On this page it is called the service_role secret.
SUPABASE_SERVICE_KEY=
# The LLM you want to use for the reasoner (o3-mini, R1, QwQ, etc.).
# Example: o3-mini
# Example: deepseek-r1:7b-8k
REASONER_MODEL=
# The LLM you want to use for the primary agent/coder.
# Example: gpt-4o-mini
# Example: qwen2.5:14b-instruct-8k
PRIMARY_MODEL=
# Embedding model you want to use
# Example for Ollama: nomic-embed-text
# Example for OpenAI: text-embedding-3-small
EMBEDDING_MODEL=

2
original_archon/.gitattributes vendored
View File

@@ -1,2 +0,0 @@
# Auto detect text files and perform LF normalization
* text=auto

39
original_archon/.github/ISSUE_TEMPLATE/bug_report.md vendored
View File

@@ -1,39 +0,0 @@
---
name: Bug Report
about: Create a report to help improve Archon
title: '[BUG] '
labels: bug
assignees: ''
---
## Description
A clear and concise description of the issue.
## Steps to Reproduce
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
## Expected Behavior
A clear and concise description of what you expected to happen.
## Actual Behavior
A clear and concise description of what actually happened.
## Screenshots
If applicable, add screenshots to help explain your problem.
## Environment
- OS: [e.g. Windows 10, macOS Monterey, Ubuntu 22.04]
- Python Version: [e.g. Python 3.13, Python 3.12]
- Using MCP or Streamlit (or something else)
## Additional Context
Add any other context about the problem here, such as:
- Does this happen consistently or intermittently?
- Were there any recent changes that might be related?
- Any workarounds you've discovered?
## Possible Solution
If you have suggestions on how to fix the issue or what might be causing it.

5
original_archon/.github/ISSUE_TEMPLATE/config.yml vendored
View File

@@ -1,5 +0,0 @@
blank_issues_enabled: false
contact_links:
- name: Archon Community
url: https://thinktank.ottomator.ai/c/archon/30
about: Please ask questions and start conversations about Archon here in the oTTomator Think Tank!

19
original_archon/.github/ISSUE_TEMPLATE/feature_request.md vendored
View File

@@ -1,19 +0,0 @@
---
name: Feature Request
about: Suggest an idea for Archon
title: '[FEATURE] '
labels: enhancement
assignees: ''
---
## Describe the feature you'd like and why
A clear and concise description of what you want to happen.
## User Impact
Who would benefit from this feature and how?
## Implementation Details (optional)
Any thoughts on how this might be implemented?
## Additional context
Add any other screenshots, mockups, or context about the feature request here.

58
original_archon/.github/dependabot.yml vendored
View File

@@ -1,58 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
# Check for updates to GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
# Check for updates to Python packages in root (actual iteration)
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
# Check for updates to Python packages in mcp
- package-ecosystem: "pip"
directory: "/mcp"
schedule:
interval: "weekly"
# Check for updates in Dockerfile
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
# Check for updates in MCP Dockerfile
- package-ecosystem: "docker"
directory: "/mcp"
schedule:
interval: "weekly"
# Aditional: Structure to maintain previous iterations
# Update Version 1: Single Agent
# - package-ecosystem: "pip"
# directory: "/iterations/v1-single-agent"
# schedule:
# interval: "monthly"
# Update Version 2: Agentic Workflow
# - package-ecosystem: "pip"
# directory: "/iterations/v2-agentic-workflow"
# schedule:
# interval: "monthly"
# Upate Version 3: MCP Support
# - package-ecosystem: "pip"
# directory: "/iterations/v3-mcp-support"
# schedule:
# interval: "monthly"

100
original_archon/.github/workflows/build.yml vendored
View File

@@ -1,100 +0,0 @@
name: Build Archon
on:
push:
branches:
[ main, master ]
pull_request:
branches:
[ main, master ]
permissions:
contents: read
jobs:
build-locally:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12', '3.13']
include:
- python-version: '3.10'
experimental: true
- python-version: '3.12'
experimental: true
- python-version: '3.13'
experimental: true
fail-fast: false
# Test on newer Python versions
continue-on-error: ${{ matrix.experimental || false }}
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@42375524e23c412d93fb67b49958b491fce71c38
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
- name: Verify code compilation
run: |
source venv/bin/activate
python -m compileall -f .
build-docker:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12', '3.13']
include:
- python-version: '3.10'
experimental: true
- python-version: '3.13'
experimental: true
fail-fast: false
# Test on newer Python versions
continue-on-error: ${{ matrix.experimental || false }}
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@42375524e23c412d93fb67b49958b491fce71c38
with:
python-version: ${{ matrix.python-version }}
- name: Modify run_docker.py for CI environment
run: |
cp run_docker.py run_docker_ci.py
# Modify the script to just build and verify containers without running them indefinitely
sed -i 's/"-d",/"-d", "--rm",/g' run_docker_ci.py
- name: Run Docker setup script
run: |
chmod +x run_docker_ci.py
python run_docker_ci.py
- name: Verify containers are built
run: |
docker images | grep archon
docker images | grep archon-mcp
- name: Test container running status
run: |
docker ps -a | grep archon-container
- name: Stop containers
run: |
docker stop archon-container || true
docker rm archon-container || true
docker stop archon-mcp || true
docker rm archon-mcp || true
docker ps -a | grep archon || echo "All containers successfully removed"

11
original_archon/.gitignore vendored
View File

@@ -1,11 +0,0 @@
# Folders
workbench
__pycache__
venv
.langgraph_api
# Files
.env
.env.temp
.env.test
env_vars.json

6
original_archon/.streamlit/config.toml
View File

@@ -1,6 +0,0 @@
[client]
showErrorDetails = "none"
[theme]
primaryColor = "#EB2D8C"
base="dark"

28
original_archon/Dockerfile
View File

@@ -1,28 +0,0 @@
FROM python:3.12-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements first for better caching
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application
COPY . .
# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH=/app
# Expose port for Streamlit
EXPOSE 8501
# Expose port for the Archon Service (started within Streamlit)
EXPOSE 8100
# Set the entrypoint to run Streamlit directly
CMD ["streamlit", "run", "streamlit_ui.py", "--server.port=8501", "--server.address=0.0.0.0"]

21
original_archon/LICENSE
View File

@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2025 oTTomator and Archon contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

325
original_archon/README.md
View File

@@ -1,325 +0,0 @@
# Archon - AI Agent Builder
<img src="public/Archon.png" alt="Archon Logo" />
<div align="center" style="margin-top: 20px;margin-bottom: 30px">
<h3>🚀 **CURRENT VERSION** 🚀</h3>
**[ V6 - Tool Library and MCP Integration ]**
*Prebuilt tools, examples, and MCP server integration*
</div>
> **🔄 IMPORTANT UPDATE (March 31st)**: Archon now includes a library of prebuilt tools, examples, and MCP server integrations. Archon can now incorporate these resources when building new agents, significantly enhancing capabilities and reducing hallucinations. Note that the examples/tool library for Archon is just starting out. Please feel free to contribute examples, MCP servers, and prebuilt tools!
Archon is the world's first **"Agenteer"**, an AI agent designed to autonomously build, refine, and optimize other AI agents.
It serves both as a practical tool for developers and as an educational framework demonstrating the evolution of agentic systems.
Archon will be developed in iterations, starting with just a simple Pydantic AI agent that can build other Pydantic AI agents,
all the way to a full agentic workflow using LangGraph that can build other AI agents with any framework.
Through its iterative development, Archon showcases the power of planning, feedback loops, and domain-specific knowledge in creating robust AI agents.
## Important Links
- The current version of Archon is V6 as mentioned above - see [V6 Documentation](iterations/v6-tool-library-integration/README.md) for details.
- I **just** created the [Archon community](https://thinktank.ottomator.ai/c/archon/30) forum over in the oTTomator Think Tank! Please post any questions you have there!
- [GitHub Kanban board](https://github.com/users/coleam00/projects/1) for feature implementation and bug squashing.
## Vision
Archon demonstrates three key principles in modern AI development:
1. **Agentic Reasoning**: Planning, iterative feedback, and self-evaluation overcome the limitations of purely reactive systems
2. **Domain Knowledge Integration**: Seamless embedding of frameworks like Pydantic AI and LangGraph within autonomous workflows
3. **Scalable Architecture**: Modular design supporting maintainability, cost optimization, and ethical AI practices
## Getting Started with V6 (current version)
Since V6 is the current version of Archon, all the code for V6 is in both the main directory and `archon/iterations/v6-tool-library-integration` directory.
Note that the examples/tool library for Archon is just starting out. Please feel free to contribute examples, MCP servers, and prebuilt tools!
### Prerequisites
- Docker (optional but preferred)
- Python 3.11+
- Supabase account (for vector database)
- OpenAI/Anthropic/OpenRouter API key or Ollama for local LLMs (note that only OpenAI supports streaming in the Streamlit UI currently)
### Installation
#### Option 1: Docker (Recommended)
1. Clone the repository:
```bash
git clone https://github.com/coleam00/archon.git
cd archon
```
2. Run the Docker setup script:
```bash
# This will build both containers and start Archon
python run_docker.py
```
3. Access the Streamlit UI at http://localhost:8501.
> **Note:** `run_docker.py` will automatically:
> - Build the MCP server container
> - Build the main Archon container
> - Run Archon with the appropriate port mappings
> - Use environment variables from `.env` file if it exists
#### Option 2: Local Python Installation
1. Clone the repository:
```bash
git clone https://github.com/coleam00/archon.git
cd archon
```
2. Install dependencies:
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```
3. Start the Streamlit UI:
```bash
streamlit run streamlit_ui.py
```
4. Access the Streamlit UI at http://localhost:8501.
### Setup Process
After installation, follow the guided setup process in the Intro section of the Streamlit UI:
- **Environment**: Configure your API keys and model settings - all stored in `workbench/env_vars.json`
- **Database**: Set up your Supabase vector database
- **Documentation**: Crawl and index the Pydantic AI documentation
- **Agent Service**: Start the agent service for generating agents
- **Chat**: Interact with Archon to create AI agents
- **MCP** (optional): Configure integration with AI IDEs
The Streamlit interface will guide you through each step with clear instructions and interactive elements.
There are a good amount of steps for the setup but it goes quick!
### Troubleshooting
If you encounter any errors when using Archon, please first check the logs in the "Agent Service" tab.
Logs specifically for MCP are also logged to `workbench/logs.txt` (file is automatically created) so please
check there. The goal is for you to have a clear error message before creating a bug here in the GitHub repo
### Updating Archon
#### Option 1: Docker
To get the latest updates for Archon when using Docker:
```bash
# Pull the latest changes from the repository (from within the archon directory)
git pull
# Rebuild and restart the containers with the latest changes
python run_docker.py
```
The `run_docker.py` script will automatically:
- Detect and remove any existing Archon containers (whether running or stopped)
- Rebuild the containers with the latest code
- Start fresh containers with the updated version
#### Option 2: Local Python Installation
To get the latest updates for Archon when using local Python installation:
```bash
# Pull the latest changes from the repository (from within the archon directory)
git pull
# Install any new dependencies
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
# Restart the Streamlit UI
# (If you're already running it, stop with Ctrl+C first)
streamlit run streamlit_ui.py
```
This ensures you're always running the most recent version of Archon with all the latest features and bug fixes.
## Project Evolution
### V1: Single-Agent Foundation
- Basic RAG-powered agent using Pydantic AI
- Supabase vector database for documentation storage
- Simple code generation without validation
- [Learn more about V1](iterations/v1-single-agent/README.md)
### V2: Agentic Workflow (LangGraph)
- Multi-agent system with planning and execution separation
- Reasoning LLM (O3-mini/R1) for architecture planning
- LangGraph for workflow orchestration
- Support for local LLMs via Ollama
- [Learn more about V2](iterations/v2-agentic-workflow/README.md)
### V3: MCP Support
- Integration with AI IDEs like Windsurf and Cursor
- Automated file creation and dependency management
- FastAPI service for agent generation
- Improved project structure and organization
- [Learn more about V3](iterations/v3-mcp-support/README.md)
### V4: Streamlit UI Overhaul
- Docker support
- Comprehensive Streamlit interface for managing all aspects of Archon
- Guided setup process with interactive tabs
- Environment variable management through the UI
- Database setup and documentation crawling simplified
- Agent service control and monitoring
- MCP configuration through the UI
- [Learn more about V4](iterations/v4-streamlit-ui-overhaul/README.md)
### V5: Multi-Agent Coding Workflow
- Specialized refiner agents for different autonomously improving the initially generated agent
- Prompt refiner agent for optimizing system prompts
- Tools refiner agent for specialized tool implementation
- Agent refiner for optimizing agent configuration and dependencies
- Cohesive initial agent structure before specialized refinement
- Improved workflow orchestration with LangGraph
- [Learn more about V5](iterations/v5-parallel-specialized-agents/README.md)
### V6: Current - Tool Library and MCP Integration
- Comprehensive library of prebuilt tools, examples, and agent templates
- Integration with MCP servers for massive amounts of prebuilt tools
- Advisor agent that recommends relevant tools and examples based on user requirements
- Automatic incorporation of prebuilt components into new agents
- Specialized tools refiner agent also validates and optimizes MCP server configurations
- Streamlined access to external services through MCP integration
- Reduced development time through component reuse
- [Learn more about V6](iterations/v6-tool-library-integration/README.md)
### Future Iterations
- V7: LangGraph Documentation - Allow Archon to build Pydantic AI AND LangGraph agents
- V8: Self-Feedback Loop - Automated validation and error correction
- V9: Self Agent Execution - Testing and iterating on agents in an isolated environment
- V10: Multi-Framework Support - Framework-agnostic agent generation
- V11: Autonomous Framework Learning - Self-updating framework adapters
- V12: Advanced RAG Techniques - Enhanced retrieval and incorporation of framework documentation
- V13: MCP Agent Marketplace - Integrating Archon agents as MCP servers and publishing to marketplaces
### Future Integrations
- LangSmith
- MCP marketplace
- Other frameworks besides Pydantic AI
- Other vector databases besides Supabase
- [Local AI package](https://github.com/coleam00/local-ai-packaged) for the agent environment
## Archon Agents Architecture
The below diagram from the LangGraph studio is a visual representation of the Archon agent graph.
<img src="public/ArchonGraph.png" alt="Archon Graph" />
The flow works like this:
1. You describe the initial AI agent you want to create
2. The reasoner LLM creates the high level scope for the agent
3. The primary coding agent uses the scope and documentation to create the initial agent
4. Control is passed back to you to either give feedback or ask Archon to 'refine' the agent autonomously
5. If refining autonomously, the specialized agents are invoked to improve the prompt, tools, and agent configuration
6. The primary coding agent is invoked again with either user or specialized agent feedback
7. The process goes back to step 4 until you say the agent is complete
8. Once the agent is complete, Archon spits out the full code again with instructions for running it
## File Architecture
### Core Files
- `streamlit_ui.py`: Comprehensive web interface for managing all aspects of Archon
- `graph_service.py`: FastAPI service that handles the agentic workflow
- `run_docker.py`: Script to build and run Archon Docker containers
- `Dockerfile`: Container definition for the main Archon application
### MCP Integration
- `mcp/`: Model Context Protocol server implementation
- `mcp_server.py`: MCP server script for AI IDE integration
- `Dockerfile`: Container definition for the MCP server
### Archon Package
- `archon/`: Core agent and workflow implementation
- `archon_graph.py`: LangGraph workflow definition and agent coordination
- `pydantic_ai_coder.py`: Main coding agent with RAG capabilities
- `refiner_agents/`: Specialized agents for refining different aspects of the created agent
- `prompt_refiner_agent.py`: Optimizes system prompts
- `tools_refiner_agent.py`: Specializes in tool implementation
- `agent_refiner_agent.py`: Refines agent configuration and dependencies
- `crawl_pydantic_ai_docs.py`: Documentation crawler and processor
### Utilities
- `utils/`: Utility functions and database setup
- `utils.py`: Shared utility functions
- `site_pages.sql`: Database setup commands
### Workbench
- `workbench/`: Created at runtime, files specific to your environment
- `env_vars.json`: Environment variables defined in the UI are stored here (included in .gitignore, file is created automatically)
- `logs.txt`: Low level logs for all Archon processes go here
- `scope.md`: The detailed scope document created by the reasoner model at the start of each Archon execution
## Deployment Options
- **Docker Containers**: Run Archon in isolated containers with all dependencies included
- Main container: Runs the Streamlit UI and graph service
- MCP container: Provides MCP server functionality for AI IDEs
- **Local Python**: Run directly on your system with a Python virtual environment
### Docker Architecture
The Docker implementation consists of two containers:
1. **Main Archon Container**:
- Runs the Streamlit UI on port 8501
- Hosts the Graph Service on port 8100
- Built from the root Dockerfile
- Handles all agent functionality and user interactions
2. **MCP Container**:
- Implements the Model Context Protocol for AI IDE integration
- Built from the mcp/Dockerfile
- Communicates with the main container's Graph Service
- Provides a standardized interface for AI IDEs like Windsurf, Cursor, Cline, and Roo Code
When running with Docker, the `run_docker.py` script automates building and starting both containers with the proper configuration.
## Database Setup
The Supabase database uses the following schema:
```sql
CREATE TABLE site_pages (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
url TEXT,
chunk_number INTEGER,
title TEXT,
summary TEXT,
content TEXT,
metadata JSONB,
embedding VECTOR(1536) -- Adjust dimensions as necessary (i.e. 768 for nomic-embed-text)
);
```
The Streamlit UI provides an interface to set up this database structure automatically.
## Contributing
We welcome contributions! Whether you're fixing bugs, adding features, or improving documentation, please feel free to submit a Pull Request.
## License
[MIT License](LICENSE)
---
For version-specific details:
- [V1 Documentation](iterations/v1-single-agent/README.md)
- [V2 Documentation](iterations/v2-agentic-workflow/README.md)
- [V3 Documentation](iterations/v3-mcp-support/README.md)
- [V4 Documentation](iterations/v4-streamlit-ui-overhaul/README.md)
- [V5 Documentation](iterations/v5-parallel-specialized-agents/README.md)
- [V6 Documentation](iterations/v6-tool-library-integration/README.md)

173
original_archon/agent-resources/examples/pydantic_github_agent.py
View File

@@ -1,173 +0,0 @@
from __future__ import annotations as _annotations
import asyncio
import os
from dataclasses import dataclass
from typing import Any, List, Dict
import tempfile
from pathlib import Path
from dotenv import load_dotenv
import shutil
import time
import re
import json
import httpx
import logfire
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.providers.openai import OpenAIProvider
from pydantic_ai.models.openai import OpenAIModel
from devtools import debug
load_dotenv()
llm = os.getenv('LLM_MODEL', 'deepseek/deepseek-chat')
model = OpenAIModel(
llm,
provider=OpenAIProvider(base_url="https://openrouter.ai/api/v1", api_key=os.getenv('OPEN_ROUTER_API_KEY'))
) if os.getenv('OPEN_ROUTER_API_KEY', None) else OpenAIModel(llm)
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class GitHubDeps:
client: httpx.AsyncClient
github_token: str | None = None
system_prompt = """
You are a coding expert with access to GitHub to help the user manage their repository and get information from it.
Your only job is to assist with this and you don't answer other questions besides describing what you are able to do.
Don't ask the user before taking an action, just do it. Always make sure you look at the repository with the provided tools before answering the user's question unless you have already.
When answering a question about the repo, always start your answer with the full repo URL in brackets and then give your answer on a newline. Like:
[Using https://github.com/[repo URL from the user]]
Your answer here...
"""
github_agent = Agent(
model,
system_prompt=system_prompt,
deps_type=GitHubDeps,
retries=2
)
@github_agent.tool
async def get_repo_info(ctx: RunContext[GitHubDeps], github_url: str) -> str:
"""Get repository information including size and description using GitHub API.
Args:
ctx: The context.
github_url: The GitHub repository URL.
Returns:
str: Repository information as a formatted string.
"""
match = re.search(r'github\.com[:/]([^/]+)/([^/]+?)(?:\.git)?$', github_url)
if not match:
return "Invalid GitHub URL format"
owner, repo = match.groups()
headers = {'Authorization': f'token {ctx.deps.github_token}'} if ctx.deps.github_token else {}
response = await ctx.deps.client.get(
f'https://api.github.com/repos/{owner}/{repo}',
headers=headers
)
if response.status_code != 200:
return f"Failed to get repository info: {response.text}"
data = response.json()
size_mb = data['size'] / 1024
return (
f"Repository: {data['full_name']}\n"
f"Description: {data['description']}\n"
f"Size: {size_mb:.1f}MB\n"
f"Stars: {data['stargazers_count']}\n"
f"Language: {data['language']}\n"
f"Created: {data['created_at']}\n"
f"Last Updated: {data['updated_at']}"
)
@github_agent.tool
async def get_repo_structure(ctx: RunContext[GitHubDeps], github_url: str) -> str:
"""Get the directory structure of a GitHub repository.
Args:
ctx: The context.
github_url: The GitHub repository URL.
Returns:
str: Directory structure as a formatted string.
"""
match = re.search(r'github\.com[:/]([^/]+)/([^/]+?)(?:\.git)?$', github_url)
if not match:
return "Invalid GitHub URL format"
owner, repo = match.groups()
headers = {'Authorization': f'token {ctx.deps.github_token}'} if ctx.deps.github_token else {}
response = await ctx.deps.client.get(
f'https://api.github.com/repos/{owner}/{repo}/git/trees/main?recursive=1',
headers=headers
)
if response.status_code != 200:
# Try with master branch if main fails
response = await ctx.deps.client.get(
f'https://api.github.com/repos/{owner}/{repo}/git/trees/master?recursive=1',
headers=headers
)
if response.status_code != 200:
return f"Failed to get repository structure: {response.text}"
data = response.json()
tree = data['tree']
# Build directory structure
structure = []
for item in tree:
if not any(excluded in item['path'] for excluded in ['.git/', 'node_modules/', '__pycache__/']):
structure.append(f"{'📁 ' if item['type'] == 'tree' else '📄 '}{item['path']}")
return "\n".join(structure)
@github_agent.tool
async def get_file_content(ctx: RunContext[GitHubDeps], github_url: str, file_path: str) -> str:
"""Get the content of a specific file from the GitHub repository.
Args:
ctx: The context.
github_url: The GitHub repository URL.
file_path: Path to the file within the repository.
Returns:
str: File content as a string.
"""
match = re.search(r'github\.com[:/]([^/]+)/([^/]+?)(?:\.git)?$', github_url)
if not match:
return "Invalid GitHub URL format"
owner, repo = match.groups()
headers = {'Authorization': f'token {ctx.deps.github_token}'} if ctx.deps.github_token else {}
response = await ctx.deps.client.get(
f'https://raw.githubusercontent.com/{owner}/{repo}/main/{file_path}',
headers=headers
)
if response.status_code != 200:
# Try with master branch if main fails
response = await ctx.deps.client.get(
f'https://raw.githubusercontent.com/{owner}/{repo}/master/{file_path}',
headers=headers
)
if response.status_code != 200:
return f"Failed to get file content: {response.text}"
return response.text

33
original_archon/agent-resources/examples/pydantic_mcp_agent.py
View File

@@ -1,33 +0,0 @@
from pydantic_ai.providers.openai import OpenAIProvider
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.mcp import MCPServerStdio
from pydantic_ai import Agent
from dotenv import load_dotenv
import asyncio
import os
load_dotenv()
def get_model():
llm = os.getenv('MODEL_CHOICE', 'gpt-4o-mini')
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-api-key-provided')
return OpenAIModel(llm, provider=OpenAIProvider(base_url=base_url, api_key=api_key))
server = MCPServerStdio(
'npx',
['-y', '@modelcontextprotocol/server-brave-search', 'stdio'],
env={"BRAVE_API_KEY": os.getenv("BRAVE_API_KEY")}
)
agent = Agent(get_model(), mcp_servers=[server])
async def main():
async with agent.run_mcp_servers():
result = await agent.run('What is new with Gemini 2.5 Pro?')
print(result.data)
user_input = input("Press enter to quit...")
if __name__ == '__main__':
asyncio.run(main())

110
original_archon/agent-resources/examples/pydantic_web_search_agent.py
View File

@@ -1,110 +0,0 @@
from __future__ import annotations as _annotations
import asyncio
import os
from dataclasses import dataclass
from datetime import datetime
from typing import Any
import logfire
from devtools import debug
from httpx import AsyncClient
from dotenv import load_dotenv
from openai import AsyncOpenAI
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai import Agent, ModelRetry, RunContext
load_dotenv()
llm = os.getenv('LLM_MODEL', 'gpt-4o')
client = AsyncOpenAI(
base_url = 'http://localhost:11434/v1',
api_key='ollama'
)
model = OpenAIModel(llm) if llm.lower().startswith("gpt") else OpenAIModel(llm, openai_client=client)
# 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class Deps:
client: AsyncClient
brave_api_key: str | None
web_search_agent = Agent(
model,
system_prompt=f'You are an expert at researching the web to answer user questions. The current date is: {datetime.now().strftime("%Y-%m-%d")}',
deps_type=Deps,
retries=2
)
@web_search_agent.tool
async def search_web(
ctx: RunContext[Deps], web_query: str
) -> str:
"""Search the web given a query defined to answer the user's question.
Args:
ctx: The context.
web_query: The query for the web search.
Returns:
str: The search results as a formatted string.
"""
if ctx.deps.brave_api_key is None:
return "This is a test web search result. Please provide a Brave API key to get real search results."
headers = {
'X-Subscription-Token': ctx.deps.brave_api_key,
'Accept': 'application/json',
}
with logfire.span('calling Brave search API', query=web_query) as span:
r = await ctx.deps.client.get(
'https://api.search.brave.com/res/v1/web/search',
params={
'q': web_query,
'count': 5,
'text_decorations': True,
'search_lang': 'en'
},
headers=headers
)
r.raise_for_status()
data = r.json()
span.set_attribute('response', data)
results = []
# Add web results in a nice formatted way
web_results = data.get('web', {}).get('results', [])
for item in web_results[:3]:
title = item.get('title', '')
description = item.get('description', '')
url = item.get('url', '')
if title and description:
results.append(f"Title: {title}\nSummary: {description}\nSource: {url}\n")
return "\n".join(results) if results else "No results found for the query."
async def main():
async with AsyncClient() as client:
brave_api_key = os.getenv('BRAVE_API_KEY', None)
deps = Deps(client=client, brave_api_key=brave_api_key)
result = await web_search_agent.run(
'Give me some articles talking about the new release of React 19.', deps=deps
)
debug(result)
print('Response:', result.data)
if __name__ == '__main__':
asyncio.run(main())

14
original_archon/agent-resources/mcps/airtable.json
View File

@@ -1,14 +0,0 @@
{
"mcpServers": {
"airtable": {
"command": "npx",
"args": [
"-y",
"airtable-mcp-server"
],
"env": {
"AIRTABLE_API_KEY": "pat123.abc123"
}
}
}
}

14
original_archon/agent-resources/mcps/brave_search.json
View File

@@ -1,14 +0,0 @@
{
"mcpServers": {
"brave-search": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-brave-search"
],
"env": {
"BRAVE_API_KEY": "YOUR_API_KEY_HERE"
}
}
}
}

14
original_archon/agent-resources/mcps/chroma.json
View File

@@ -1,14 +0,0 @@
{
"mcpServers": {
"chroma": {
"command": "uvx",
"args": [
"chroma-mcp",
"--client-type",
"persistent",
"--data-dir",
"/full/path/to/your/data/directory"
]
}
}
}

13
original_archon/agent-resources/mcps/file_system.json
View File

@@ -1,13 +0,0 @@
{
"mcpServers": {
"filesystem": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-filesystem",
"/Users/username/Desktop",
"/path/to/other/allowed/dir"
]
}
}
}

19
original_archon/agent-resources/mcps/firecrawl.json
View File

@@ -1,19 +0,0 @@
{
"mcpServers": {
"mcp-server-firecrawl": {
"command": "npx",
"args": ["-y", "firecrawl-mcp"],
"env": {
"FIRECRAWL_API_KEY": "YOUR_API_KEY_HERE",
"FIRECRAWL_RETRY_MAX_ATTEMPTS": "5",
"FIRECRAWL_RETRY_INITIAL_DELAY": "2000",
"FIRECRAWL_RETRY_MAX_DELAY": "30000",
"FIRECRAWL_RETRY_BACKOFF_FACTOR": "3",
"FIRECRAWL_CREDIT_WARNING_THRESHOLD": "2000",
"FIRECRAWL_CREDIT_CRITICAL_THRESHOLD": "500"
}
}
}
}

16
original_archon/agent-resources/mcps/git.json
View File

@@ -1,16 +0,0 @@
{
"mcpServers": {
"git": {
"command": "docker",
"args": [
"run",
"--rm",
"-i",
"--mount", "type=bind,src=/Users/username/Desktop,dst=/projects/Desktop",
"--mount", "type=bind,src=/path/to/other/allowed/dir,dst=/projects/other/allowed/dir,ro",
"--mount", "type=bind,src=/path/to/file.txt,dst=/projects/path/to/file.txt",
"mcp/git"
]
}
}
}

14
original_archon/agent-resources/mcps/github.json
View File

@@ -1,14 +0,0 @@
{
"mcpServers": {
"github": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-github"
],
"env": {
"GITHUB_PERSONAL_ACCESS_TOKEN": "<YOUR_TOKEN>"
}
}
}
}

8
original_archon/agent-resources/mcps/google_drive.json
View File

@@ -1,8 +0,0 @@
{
"mcpServers": {
"gdrive": {
"command": "docker",
"args": ["run", "-i", "--rm", "-v", "mcp-gdrive:/gdrive-server", "-e", "GDRIVE_CREDENTIALS_PATH=/gdrive-server/credentials.json", "mcp/gdrive"]
}
}
}

12
original_archon/agent-resources/mcps/qdrant.json
View File

@@ -1,12 +0,0 @@
{
"qdrant": {
"command": "uvx",
"args": ["mcp-server-qdrant"],
"env": {
"QDRANT_URL": "https://xyz-example.eu-central.aws.cloud.qdrant.io:6333",
"QDRANT_API_KEY": "your_api_key",
"COLLECTION_NAME": "your-collection-name",
"EMBEDDING_MODEL": "sentence-transformers/all-MiniLM-L6-v2"
}
}
}

12
original_archon/agent-resources/mcps/redis.json
View File

@@ -1,12 +0,0 @@
{
"mcpServers": {
"redis": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-redis",
"redis://localhost:6379"
]
}
}
}

15
original_archon/agent-resources/mcps/slack.json
View File

@@ -1,15 +0,0 @@
{
"mcpServers": {
"slack": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-slack"
],
"env": {
"SLACK_BOT_TOKEN": "xoxb-your-bot-token",
"SLACK_TEAM_ID": "T01234567"
}
}
}
}

17
original_archon/agent-resources/mcps/sqlite.json
View File

@@ -1,17 +0,0 @@
{
"mcpServers": {
"sqlite": {
"command": "docker",
"args": [
"run",
"--rm",
"-i",
"-v",
"mcp-test:/mcp",
"mcp/sqlite",
"--db-path",
"/mcp/test.db"
]
}
}
}

34
original_archon/agent-resources/tools/get_github_file.py
View File

@@ -1,34 +0,0 @@
@github_agent.tool
async def get_file_content(ctx: RunContext[GitHubDeps], github_url: str, file_path: str) -> str:
"""Get the content of a specific file from the GitHub repository.
Args:
ctx: The context.
github_url: The GitHub repository URL.
file_path: Path to the file within the repository.
Returns:
str: File content as a string.
"""
match = re.search(r'github\.com[:/]([^/]+)/([^/]+?)(?:\.git)?$', github_url)
if not match:
return "Invalid GitHub URL format"
owner, repo = match.groups()
headers = {'Authorization': f'token {ctx.deps.github_token}'} if ctx.deps.github_token else {}
response = await ctx.deps.client.get(
f'https://raw.githubusercontent.com/{owner}/{repo}/main/{file_path}',
headers=headers
)
if response.status_code != 200:
# Try with master branch if main fails
response = await ctx.deps.client.get(
f'https://raw.githubusercontent.com/{owner}/{repo}/master/{file_path}',
headers=headers
)
if response.status_code != 200:
return f"Failed to get file content: {response.text}"
return response.text

42
original_archon/agent-resources/tools/get_github_file_structure.py
View File

@@ -1,42 +0,0 @@
@github_agent.tool
async def get_repo_structure(ctx: RunContext[GitHubDeps], github_url: str) -> str:
"""Get the directory structure of a GitHub repository.
Args:
ctx: The context.
github_url: The GitHub repository URL.
Returns:
str: Directory structure as a formatted string.
"""
match = re.search(r'github\.com[:/]([^/]+)/([^/]+?)(?:\.git)?$', github_url)
if not match:
return "Invalid GitHub URL format"
owner, repo = match.groups()
headers = {'Authorization': f'token {ctx.deps.github_token}'} if ctx.deps.github_token else {}
response = await ctx.deps.client.get(
f'https://api.github.com/repos/{owner}/{repo}/git/trees/main?recursive=1',
headers=headers
)
if response.status_code != 200:
# Try with master branch if main fails
response = await ctx.deps.client.get(
f'https://api.github.com/repos/{owner}/{repo}/git/trees/master?recursive=1',
headers=headers
)
if response.status_code != 200:
return f"Failed to get repository structure: {response.text}"
data = response.json()
tree = data['tree']
# Build directory structure
structure = []
for item in tree:
if not any(excluded in item['path'] for excluded in ['.git/', 'node_modules/', '__pycache__/']):
structure.append(f"{'📁 ' if item['type'] == 'tree' else '📄 '}{item['path']}")
return "\n".join(structure)

38
original_archon/agent-resources/tools/get_github_repo_info.py
View File

@@ -1,38 +0,0 @@
@github_agent.tool
async def get_repo_info(ctx: RunContext[GitHubDeps], github_url: str) -> str:
"""Get repository information including size and description using GitHub API.
Args:
ctx: The context.
github_url: The GitHub repository URL.
Returns:
str: Repository information as a formatted string.
"""
match = re.search(r'github\.com[:/]([^/]+)/([^/]+?)(?:\.git)?$', github_url)
if not match:
return "Invalid GitHub URL format"
owner, repo = match.groups()
headers = {'Authorization': f'token {ctx.deps.github_token}'} if ctx.deps.github_token else {}
response = await ctx.deps.client.get(
f'https://api.github.com/repos/{owner}/{repo}',
headers=headers
)
if response.status_code != 200:
return f"Failed to get repository info: {response.text}"
data = response.json()
size_mb = data['size'] / 1024
return (
f"Repository: {data['full_name']}\n"
f"Description: {data['description']}\n"
f"Size: {size_mb:.1f}MB\n"
f"Stars: {data['stargazers_count']}\n"
f"Language: {data['language']}\n"
f"Created: {data['created_at']}\n"
f"Last Updated: {data['updated_at']}"
)

48
original_archon/agent-resources/tools/web_search.py
View File

@@ -1,48 +0,0 @@
@web_search_agent.tool
async def search_web(
ctx: RunContext[Deps], web_query: str
) -> str:
"""Search the web given a query defined to answer the user's question.
Args:
ctx: The context.
web_query: The query for the web search.
Returns:
str: The search results as a formatted string.
"""
if ctx.deps.brave_api_key is None:
return "This is a test web search result. Please provide a Brave API key to get real search results."
headers = {
'X-Subscription-Token': ctx.deps.brave_api_key,
'Accept': 'application/json',
}
with logfire.span('calling Brave search API', query=web_query) as span:
r = await ctx.deps.client.get(
'https://api.search.brave.com/res/v1/web/search',
params={
'q': web_query,
'count': 5,
'text_decorations': True,
'search_lang': 'en'
},
headers=headers
)
r.raise_for_status()
data = r.json()
span.set_attribute('response', data)
results = []
# Add web results in a nice formatted way
web_results = data.get('web', {}).get('results', [])
for item in web_results[:3]:
title = item.get('title', '')
description = item.get('description', '')
url = item.get('url', '')
if title and description:
results.append(f"Title: {title}\nSummary: {description}\nSource: {url}\n")
return "\n".join(results) if results else "No results found for the query."

0
original_archon/archon/__init__.py
View File

70
original_archon/archon/advisor_agent.py
View File

@@ -1,70 +0,0 @@
from __future__ import annotations as _annotations
from dataclasses import dataclass
from dotenv import load_dotenv
import logfire
import asyncio
import httpx
import os
import sys
import json
from typing import List
from pydantic import BaseModel
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.models.openai import OpenAIModel
from openai import AsyncOpenAI
from supabase import Client
# Add the parent directory to sys.path to allow importing from the parent directory
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from utils.utils import get_env_var
from archon.agent_prompts import advisor_prompt
from archon.agent_tools import get_file_content_tool
load_dotenv()
provider = get_env_var('LLM_PROVIDER') or 'OpenAI'
llm = get_env_var('PRIMARY_MODEL') or 'gpt-4o-mini'
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
model = AnthropicModel(llm, api_key=api_key) if provider == "Anthropic" else OpenAIModel(llm, base_url=base_url, api_key=api_key)
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class AdvisorDeps:
file_list: List[str]
advisor_agent = Agent(
model,
system_prompt=advisor_prompt,
deps_type=AdvisorDeps,
retries=2
)
@advisor_agent.system_prompt
def add_file_list(ctx: RunContext[str]) -> str:
joined_files = "\n".join(ctx.deps.file_list)
return f"""
Here is the list of all the files that you can pull the contents of with the
'get_file_content' tool if the example/tool/MCP server is relevant to the
agent the user is trying to build:
{joined_files}
"""
@advisor_agent.tool_plain
def get_file_content(file_path: str) -> str:
"""
Retrieves the content of a specific file. Use this to get the contents of an example, tool, config for an MCP server
Args:
file_path: The path to the file
Returns:
The raw contents of the file
"""
return get_file_content_tool(file_path)

334
original_archon/archon/agent_prompts.py
View File

@@ -1,334 +0,0 @@
advisor_prompt = """
You are an AI agent engineer specialized in using example code and prebuilt tools/MCP servers
and synthesizing these prebuilt components into a recommended starting point for the primary coding agent.
You will be given a prompt from the user for the AI agent they want to build, and also a list of examples,
prebuilt tools, and MCP servers you can use to aid in creating the agent so the least amount of code possible
has to be recreated.
Use the file name to determine if the example/tool/MCP server is relevant to the agent the user is requesting.
Examples will be in the examples/ folder. These are examples of AI agents to use as a starting point if applicable.
Prebuilt tools will be in the tools/ folder. Use some or none of these depending on if any of the prebuilt tools
would be needed for the agent.
MCP servers will be in the mcps/ folder. These are all config files that show the necessary parameters to set up each
server. MCP servers are just pre-packaged tools that you can include in the agent.
Take a look at examples/pydantic_mpc_agent.py to see how to incorporate MCP servers into the agents.
For example, if the Brave Search MCP config is:
{
"mcpServers": {
"brave-search": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-brave-search"
],
"env": {
"BRAVE_API_KEY": "YOUR_API_KEY_HERE"
}
}
}
}
Then the way to connect that into the agent is:
server = MCPServerStdio(
'npx',
['-y', '@modelcontextprotocol/server-brave-search', 'stdio'],
env={"BRAVE_API_KEY": os.getenv("BRAVE_API_KEY")}
)
agent = Agent(get_model(), mcp_servers=[server])
So you can see how you would map the config parameters to the MCPServerStdio instantiation.
You are given a single tool to look at the contents of any file, so call this as many times as you need to look
at the different files given to you that you think are relevant for the AI agent being created.
IMPORTANT: Only look at a few examples/tools/servers. Keep your search concise.
Your primary job at the end of looking at examples/tools/MCP servers is to provide a recommendation for a starting
point of an AI agent that uses applicable resources you pulled. Only focus on the examples/tools/servers that
are actually relevant to the AI agent the user requested.
"""
prompt_refiner_prompt = """
You are an AI agent engineer specialized in refining prompts for the agents.
Your only job is to take the current prompt from the conversation, and refine it so the agent being created
has optimal instructions to carry out its role and tasks.
You want the prompt to:
1. Clearly describe the role of the agent
2. Provide concise and easy to understand goals
3. Help the agent understand when and how to use each tool provided
4. Give interactaction guidelines
5. Provide instructions for handling issues/errors
Output the new prompt and nothing else.
"""
tools_refiner_prompt = """
You are an AI agent engineer specialized in refining tools for the agents.
You have comprehensive access to the Pydantic AI documentation, including API references, usage guides, and implementation examples.
You also have access to a list of files mentioned below that give you examples, prebuilt tools, and MCP servers
you can reference when vaildating the tools and MCP servers given to the current agent.
Your only job is to take the current tools/MCP servers from the conversation, and refine them so the agent being created
has the optimal tooling to fulfill its role and tasks. Also make sure the tools are coded properly
and allow the agent to solve the problems they are meant to help with.
For each tool, ensure that it:
1. Has a clear docstring to help the agent understand when and how to use it
2. Has correct arguments
3. Uses the run context properly if applicable (not all tools need run context)
4. Is coded properly (uses API calls correctly for the services, returns the correct data, etc.)
5. Handles errors properly
For each MCP server:
1. Get the contents of the JSON config for the server
2. Make sure the name of the server and arguments match what is in the config
3. Make sure the correct environment variables are used
Only change what is necessary to refine the tools and MCP server definitions, don't go overboard
unless of course the tools are broken and need a lot of fixing.
Output the new code for the tools/MCP servers and nothing else.
"""
agent_refiner_prompt = """
You are an AI agent engineer specialized in refining agent definitions in code.
There are other agents handling refining the prompt and tools, so your job is to make sure the higher
level definition of the agent (depedencies, setting the LLM, etc.) is all correct.
You have comprehensive access to the Pydantic AI documentation, including API references, usage guides, and implementation examples.
Your only job is to take the current agent definition from the conversation, and refine it so the agent being created
has dependencies, the LLM, the prompt, etc. all configured correctly. Use the Pydantic AI documentation tools to
confirm that the agent is set up properly, and only change the current definition if it doesn't align with
the documentation.
Output the agent depedency and definition code if it needs to change and nothing else.
"""
primary_coder_prompt = """
[ROLE AND CONTEXT]
You are a specialized AI agent engineer focused on building robust Pydantic AI agents. You have comprehensive access to the Pydantic AI documentation, including API references, usage guides, and implementation examples.
[CORE RESPONSIBILITIES]
1. Agent Development
- Create new agents from user requirements
- Complete partial agent implementations
- Optimize and debug existing agents
- Guide users through agent specification if needed
2. Documentation Integration
- Systematically search documentation using RAG before any implementation
- Cross-reference multiple documentation pages for comprehensive understanding
- Validate all implementations against current best practices
- Notify users if documentation is insufficient for any requirement
[CODE STRUCTURE AND DELIVERABLES]
All new agents must include these files with complete, production-ready code:
1. agent.py
- Primary agent definition and configuration
- Core agent logic and behaviors
- No tool implementations allowed here
2. agent_tools.py
- All tool function implementations
- Tool configurations and setup
- External service integrations
3. agent_prompts.py
- System prompts
- Task-specific prompts
- Conversation templates
- Instruction sets
4. .env.example
- Required environment variables
- Clear setup instructions in a comment above the variable for how to do so
- API configuration templates
5. requirements.txt
- Core dependencies without versions
- User-specified packages included
[DOCUMENTATION WORKFLOW]
1. Initial Research
- Begin with RAG search for relevant documentation
- List all documentation pages using list_documentation_pages
- Retrieve specific page content using get_page_content
- Cross-reference the weather agent example for best practices
2. Implementation
- Provide complete, working code implementations
- Never leave placeholder functions
- Include all necessary error handling
- Implement proper logging and monitoring
3. Quality Assurance
- Verify all tool implementations are complete
- Ensure proper separation of concerns
- Validate environment variable handling
- Test critical path functionality
[INTERACTION GUIDELINES]
- Take immediate action without asking for permission
- Always verify documentation before implementation
- Provide honest feedback about documentation gaps
- Include specific enhancement suggestions
- Request user feedback on implementations
- Maintain code consistency across files
- After providing code, ask the user at the end if they want you to refine the agent autonomously,
otherwise they can give feedback for you to use. The can specifically say 'refine' for you to continue
working on the agent through self reflection.
[ERROR HANDLING]
- Implement robust error handling in all tools
- Provide clear error messages
- Include recovery mechanisms
- Log important state changes
[BEST PRACTICES]
- Follow Pydantic AI naming conventions
- Implement proper type hints
- Include comprehensive docstrings, the agent uses this to understand what tools are for.
- Maintain clean code structure
- Use consistent formatting
Here is a good example of a Pydantic AI agent:
```python
from __future__ import annotations as _annotations
import asyncio
import os
from dataclasses import dataclass
from typing import Any
import logfire
from devtools import debug
from httpx import AsyncClient
from pydantic_ai import Agent, ModelRetry, RunContext
# 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class Deps:
client: AsyncClient
weather_api_key: str | None
geo_api_key: str | None
weather_agent = Agent(
'openai:gpt-4o',
# 'Be concise, reply with one sentence.' is enough for some models (like openai) to use
# the below tools appropriately, but others like anthropic and gemini require a bit more direction.
system_prompt=(
'Be concise, reply with one sentence.'
'Use the `get_lat_lng` tool to get the latitude and longitude of the locations, '
'then use the `get_weather` tool to get the weather.'
),
deps_type=Deps,
retries=2,
)
@weather_agent.tool
async def get_lat_lng(
ctx: RunContext[Deps], location_description: str
) -> dict[str, float]:
\"\"\"Get the latitude and longitude of a location.
Args:
ctx: The context.
location_description: A description of a location.
\"\"\"
if ctx.deps.geo_api_key is None:
# if no API key is provided, return a dummy response (London)
return {'lat': 51.1, 'lng': -0.1}
params = {
'q': location_description,
'api_key': ctx.deps.geo_api_key,
}
with logfire.span('calling geocode API', params=params) as span:
r = await ctx.deps.client.get('https://geocode.maps.co/search', params=params)
r.raise_for_status()
data = r.json()
span.set_attribute('response', data)
if data:
return {'lat': data[0]['lat'], 'lng': data[0]['lon']}
else:
raise ModelRetry('Could not find the location')
@weather_agent.tool
async def get_weather(ctx: RunContext[Deps], lat: float, lng: float) -> dict[str, Any]:
\"\"\"Get the weather at a location.
Args:
ctx: The context.
lat: Latitude of the location.
lng: Longitude of the location.
\"\"\"
if ctx.deps.weather_api_key is None:
# if no API key is provided, return a dummy response
return {'temperature': '21 °C', 'description': 'Sunny'}
params = {
'apikey': ctx.deps.weather_api_key,
'location': f'{lat},{lng}',
'units': 'metric',
}
with logfire.span('calling weather API', params=params) as span:
r = await ctx.deps.client.get(
'https://api.tomorrow.io/v4/weather/realtime', params=params
)
r.raise_for_status()
data = r.json()
span.set_attribute('response', data)
values = data['data']['values']
# https://docs.tomorrow.io/reference/data-layers-weather-codes
code_lookup = {
...
}
return {
'temperature': f'{values["temperatureApparent"]:0.0f}°C',
'description': code_lookup.get(values['weatherCode'], 'Unknown'),
}
async def main():
async with AsyncClient() as client:
# create a free API key at https://www.tomorrow.io/weather-api/
weather_api_key = os.getenv('WEATHER_API_KEY')
# create a free API key at https://geocode.maps.co/
geo_api_key = os.getenv('GEO_API_KEY')
deps = Deps(
client=client, weather_api_key=weather_api_key, geo_api_key=geo_api_key
)
result = await weather_agent.run(
'What is the weather like in London and in Wiltshire?', deps=deps
)
debug(result)
print('Response:', result.data)
if __name__ == '__main__':
asyncio.run(main())
```
"""

141
original_archon/archon/agent_tools.py
View File

@@ -1,141 +0,0 @@
from typing import Dict, Any, List, Optional
from openai import AsyncOpenAI
from supabase import Client
import sys
import os
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from utils.utils import get_env_var
embedding_model = get_env_var('EMBEDDING_MODEL') or 'text-embedding-3-small'
async def get_embedding(text: str, embedding_client: AsyncOpenAI) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await embedding_client.embeddings.create(
model=embedding_model,
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
async def retrieve_relevant_documentation_tool(supabase: Client, embedding_client: AsyncOpenAI, user_query: str) -> str:
try:
# Get the embedding for the query
query_embedding = await get_embedding(user_query, embedding_client)
# Query Supabase for relevant documents
result = supabase.rpc(
'match_site_pages',
{
'query_embedding': query_embedding,
'match_count': 4,
'filter': {'source': 'pydantic_ai_docs'}
}
).execute()
if not result.data:
return "No relevant documentation found."
# Format the results
formatted_chunks = []
for doc in result.data:
chunk_text = f"""
# {doc['title']}
{doc['content']}
"""
formatted_chunks.append(chunk_text)
# Join all chunks with a separator
return "\n\n---\n\n".join(formatted_chunks)
except Exception as e:
print(f"Error retrieving documentation: {e}")
return f"Error retrieving documentation: {str(e)}"
async def list_documentation_pages_tool(supabase: Client) -> List[str]:
"""
Function to retrieve a list of all available Pydantic AI documentation pages.
This is called by the list_documentation_pages tool and also externally
to fetch documentation pages for the reasoner LLM.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
try:
# Query Supabase for unique URLs where source is pydantic_ai_docs
result = supabase.from_('site_pages') \
.select('url') \
.eq('metadata->>source', 'pydantic_ai_docs') \
.execute()
if not result.data:
return []
# Extract unique URLs
urls = sorted(set(doc['url'] for doc in result.data))
return urls
except Exception as e:
print(f"Error retrieving documentation pages: {e}")
return []
async def get_page_content_tool(supabase: Client, url: str) -> str:
"""
Retrieve the full content of a specific documentation page by combining all its chunks.
Args:
ctx: The context including the Supabase client
url: The URL of the page to retrieve
Returns:
str: The complete page content with all chunks combined in order
"""
try:
# Query Supabase for all chunks of this URL, ordered by chunk_number
result = supabase.from_('site_pages') \
.select('title, content, chunk_number') \
.eq('url', url) \
.eq('metadata->>source', 'pydantic_ai_docs') \
.order('chunk_number') \
.execute()
if not result.data:
return f"No content found for URL: {url}"
# Format the page with its title and all chunks
page_title = result.data[0]['title'].split(' - ')[0] # Get the main title
formatted_content = [f"# {page_title}\n"]
# Add each chunk's content
for chunk in result.data:
formatted_content.append(chunk['content'])
# Join everything together but limit the characters in case the page is massive (there are a coule big ones)
# This will be improved later so if the page is too big RAG will be performed on the page itself
return "\n\n".join(formatted_content)[:20000]
except Exception as e:
print(f"Error retrieving page content: {e}")
return f"Error retrieving page content: {str(e)}"
def get_file_content_tool(file_path: str) -> str:
"""
Retrieves the content of a specific file. Use this to get the contents of an example, tool, config for an MCP server
Args:
file_path: The path to the file
Returns:
The raw contents of the file
"""
try:
with open(file_path, "r") as file:
file_contents = file.read()
return file_contents
except Exception as e:
print(f"Error retrieving file contents: {e}")
return f"Error retrieving file contents: {str(e)}"

342
original_archon/archon/archon_graph.py
View File

@@ -1,342 +0,0 @@
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai import Agent, RunContext
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver
from typing import TypedDict, Annotated, List, Any
from langgraph.config import get_stream_writer
from langgraph.types import interrupt
from dotenv import load_dotenv
from openai import AsyncOpenAI
from supabase import Client
import logfire
import os
import sys
# Import the message classes from Pydantic AI
from pydantic_ai.messages import (
ModelMessage,
ModelMessagesTypeAdapter
)
# Add the parent directory to Python path
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from archon.pydantic_ai_coder import pydantic_ai_coder, PydanticAIDeps
from archon.advisor_agent import advisor_agent, AdvisorDeps
from archon.refiner_agents.prompt_refiner_agent import prompt_refiner_agent
from archon.refiner_agents.tools_refiner_agent import tools_refiner_agent, ToolsRefinerDeps
from archon.refiner_agents.agent_refiner_agent import agent_refiner_agent, AgentRefinerDeps
from archon.agent_tools import list_documentation_pages_tool
from utils.utils import get_env_var, get_clients
# Load environment variables
load_dotenv()
# Configure logfire to suppress warnings (optional)
logfire.configure(send_to_logfire='never')
provider = get_env_var('LLM_PROVIDER') or 'OpenAI'
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
is_anthropic = provider == "Anthropic"
is_openai = provider == "OpenAI"
reasoner_llm_model_name = get_env_var('REASONER_MODEL') or 'o3-mini'
reasoner_llm_model = AnthropicModel(reasoner_llm_model_name, api_key=api_key) if is_anthropic else OpenAIModel(reasoner_llm_model_name, base_url=base_url, api_key=api_key)
reasoner = Agent(
reasoner_llm_model,
system_prompt='You are an expert at coding AI agents with Pydantic AI and defining the scope for doing so.',
)
primary_llm_model_name = get_env_var('PRIMARY_MODEL') or 'gpt-4o-mini'
primary_llm_model = AnthropicModel(primary_llm_model_name, api_key=api_key) if is_anthropic else OpenAIModel(primary_llm_model_name, base_url=base_url, api_key=api_key)
router_agent = Agent(
primary_llm_model,
system_prompt='Your job is to route the user message either to the end of the conversation or to continue coding the AI agent.',
)
end_conversation_agent = Agent(
primary_llm_model,
system_prompt='Your job is to end a conversation for creating an AI agent by giving instructions for how to execute the agent and they saying a nice goodbye to the user.',
)
# Initialize clients
embedding_client, supabase = get_clients()
# Define state schema
class AgentState(TypedDict):
latest_user_message: str
messages: Annotated[List[bytes], lambda x, y: x + y]
scope: str
advisor_output: str
file_list: List[str]
refined_prompt: str
refined_tools: str
refined_agent: str
# Scope Definition Node with Reasoner LLM
async def define_scope_with_reasoner(state: AgentState):
# First, get the documentation pages so the reasoner can decide which ones are necessary
documentation_pages = await list_documentation_pages_tool(supabase)
documentation_pages_str = "\n".join(documentation_pages)
# Then, use the reasoner to define the scope
prompt = f"""
User AI Agent Request: {state['latest_user_message']}
Create detailed scope document for the AI agent including:
- Architecture diagram
- Core components
- External dependencies
- Testing strategy
Also based on these documentation pages available:
{documentation_pages_str}
Include a list of documentation pages that are relevant to creating this agent for the user in the scope document.
"""
result = await reasoner.run(prompt)
scope = result.data
# Get the directory one level up from the current file
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
scope_path = os.path.join(parent_dir, "workbench", "scope.md")
os.makedirs(os.path.join(parent_dir, "workbench"), exist_ok=True)
with open(scope_path, "w", encoding="utf-8") as f:
f.write(scope)
return {"scope": scope}
# Advisor agent - create a starting point based on examples and prebuilt tools/MCP servers
async def advisor_with_examples(state: AgentState):
# Get the directory one level up from the current file (archon_graph.py)
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
# The agent-resources folder is adjacent to the parent folder of archon_graph.py
agent_resources_dir = os.path.join(parent_dir, "agent-resources")
# Get a list of all files in the agent-resources directory and its subdirectories
file_list = []
for root, dirs, files in os.walk(agent_resources_dir):
for file in files:
# Get the full path to the file
file_path = os.path.join(root, file)
# Use the full path instead of relative path
file_list.append(file_path)
# Then, prompt the advisor with the list of files it can use for examples and tools
deps = AdvisorDeps(file_list=file_list)
result = await advisor_agent.run(state['latest_user_message'], deps=deps)
advisor_output = result.data
return {"file_list": file_list, "advisor_output": advisor_output}
# Coding Node with Feedback Handling
async def coder_agent(state: AgentState, writer):
# Prepare dependencies
deps = PydanticAIDeps(
supabase=supabase,
embedding_client=embedding_client,
reasoner_output=state['scope'],
advisor_output=state['advisor_output']
)
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
# The prompt either needs to be the user message (initial agent request or feedback)
# or the refined prompt/tools/agent if we are in that stage of the agent creation process
if 'refined_prompt' in state and state['refined_prompt']:
prompt = f"""
I need you to refine the agent you created.
Here is the refined prompt:\n
{state['refined_prompt']}\n\n
Here are the refined tools:\n
{state['refined_tools']}\n
And finally, here are the changes to the agent definition to make if any:\n
{state['refined_agent']}\n\n
Output any changes necessary to the agent code based on these refinements.
"""
else:
prompt = state['latest_user_message']
# Run the agent in a stream
if not is_openai:
writer = get_stream_writer()
result = await pydantic_ai_coder.run(prompt, deps=deps, message_history=message_history)
writer(result.data)
else:
async with pydantic_ai_coder.run_stream(
state['latest_user_message'],
deps=deps,
message_history=message_history
) as result:
# Stream partial text as it arrives
async for chunk in result.stream_text(delta=True):
writer(chunk)
# print(ModelMessagesTypeAdapter.validate_json(result.new_messages_json()))
# Add the new conversation history (including tool calls)
# Reset the refined properties in case they were just used to refine the agent
return {
"messages": [result.new_messages_json()],
"refined_prompt": "",
"refined_tools": "",
"refined_agent": ""
}
# Interrupt the graph to get the user's next message
def get_next_user_message(state: AgentState):
value = interrupt({})
# Set the user's latest message for the LLM to continue the conversation
return {
"latest_user_message": value
}
# Determine if the user is finished creating their AI agent or not
async def route_user_message(state: AgentState):
prompt = f"""
The user has sent a message:
{state['latest_user_message']}
If the user wants to end the conversation, respond with just the text "finish_conversation".
If the user wants to continue coding the AI agent and gave feedback, respond with just the text "coder_agent".
If the user asks specifically to "refine" the agent, respond with just the text "refine".
"""
result = await router_agent.run(prompt)
if result.data == "finish_conversation": return "finish_conversation"
if result.data == "refine": return ["refine_prompt", "refine_tools", "refine_agent"]
return "coder_agent"
# Refines the prompt for the AI agent
async def refine_prompt(state: AgentState):
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
prompt = "Based on the current conversation, refine the prompt for the agent."
# Run the agent to refine the prompt for the agent being created
result = await prompt_refiner_agent.run(prompt, message_history=message_history)
return {"refined_prompt": result.data}
# Refines the tools for the AI agent
async def refine_tools(state: AgentState):
# Prepare dependencies
deps = ToolsRefinerDeps(
supabase=supabase,
embedding_client=embedding_client,
file_list=state['file_list']
)
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
prompt = "Based on the current conversation, refine the tools for the agent."
# Run the agent to refine the tools for the agent being created
result = await tools_refiner_agent.run(prompt, deps=deps, message_history=message_history)
return {"refined_tools": result.data}
# Refines the defintion for the AI agent
async def refine_agent(state: AgentState):
# Prepare dependencies
deps = AgentRefinerDeps(
supabase=supabase,
embedding_client=embedding_client
)
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
prompt = "Based on the current conversation, refine the agent definition."
# Run the agent to refine the definition for the agent being created
result = await agent_refiner_agent.run(prompt, deps=deps, message_history=message_history)
return {"refined_agent": result.data}
# End of conversation agent to give instructions for executing the agent
async def finish_conversation(state: AgentState, writer):
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
# Run the agent in a stream
if not is_openai:
writer = get_stream_writer()
result = await end_conversation_agent.run(state['latest_user_message'], message_history= message_history)
writer(result.data)
else:
async with end_conversation_agent.run_stream(
state['latest_user_message'],
message_history= message_history
) as result:
# Stream partial text as it arrives
async for chunk in result.stream_text(delta=True):
writer(chunk)
return {"messages": [result.new_messages_json()]}
# Build workflow
builder = StateGraph(AgentState)
# Add nodes
builder.add_node("define_scope_with_reasoner", define_scope_with_reasoner)
builder.add_node("advisor_with_examples", advisor_with_examples)
builder.add_node("coder_agent", coder_agent)
builder.add_node("get_next_user_message", get_next_user_message)
builder.add_node("refine_prompt", refine_prompt)
builder.add_node("refine_tools", refine_tools)
builder.add_node("refine_agent", refine_agent)
builder.add_node("finish_conversation", finish_conversation)
# Set edges
builder.add_edge(START, "define_scope_with_reasoner")
builder.add_edge(START, "advisor_with_examples")
builder.add_edge("define_scope_with_reasoner", "coder_agent")
builder.add_edge("advisor_with_examples", "coder_agent")
builder.add_edge("coder_agent", "get_next_user_message")
builder.add_conditional_edges(
"get_next_user_message",
route_user_message,
["coder_agent", "finish_conversation", "refine_prompt", "refine_tools", "refine_agent"]
)
builder.add_edge("refine_prompt", "coder_agent")
builder.add_edge("refine_tools", "coder_agent")
builder.add_edge("refine_agent", "coder_agent")
builder.add_edge("finish_conversation", END)
# Configure persistence
memory = MemorySaver()
agentic_flow = builder.compile(checkpointer=memory)

513
original_archon/archon/crawl_pydantic_ai_docs.py
View File

@@ -1,513 +0,0 @@
import os
import sys
import asyncio
import threading
import subprocess
import requests
import json
import time
from typing import List, Dict, Any, Optional, Callable
from xml.etree import ElementTree
from dataclasses import dataclass
from datetime import datetime, timezone
from urllib.parse import urlparse
from dotenv import load_dotenv
from openai import AsyncOpenAI
import re
import html2text
# Add the parent directory to sys.path to allow importing from the parent directory
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from utils.utils import get_env_var, get_clients
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
load_dotenv()
# Initialize embedding and Supabase clients
embedding_client, supabase = get_clients()
# Define the embedding model for embedding the documentation for RAG
embedding_model = get_env_var('EMBEDDING_MODEL') or 'text-embedding-3-small'
# LLM client setup
llm_client = None
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-api-key-provided'
provider = get_env_var('LLM_PROVIDER') or 'OpenAI'
# Setup OpenAI client for LLM
if provider == "Ollama":
if api_key == "NOT_REQUIRED":
api_key = "ollama" # Use a dummy key for Ollama
llm_client = AsyncOpenAI(base_url=base_url, api_key=api_key)
else:
llm_client = AsyncOpenAI(base_url=base_url, api_key=api_key)
# Initialize HTML to Markdown converter
html_converter = html2text.HTML2Text()
html_converter.ignore_links = False
html_converter.ignore_images = False
html_converter.ignore_tables = False
html_converter.body_width = 0 # No wrapping
@dataclass
class ProcessedChunk:
url: str
chunk_number: int
title: str
summary: str
content: str
metadata: Dict[str, Any]
embedding: List[float]
class CrawlProgressTracker:
"""Class to track progress of the crawling process."""
def __init__(self,
progress_callback: Optional[Callable[[Dict[str, Any]], None]] = None):
"""Initialize the progress tracker.
Args:
progress_callback: Function to call with progress updates
"""
self.progress_callback = progress_callback
self.urls_found = 0
self.urls_processed = 0
self.urls_succeeded = 0
self.urls_failed = 0
self.chunks_stored = 0
self.logs = []
self.is_running = False
self.start_time = None
self.end_time = None
def log(self, message: str):
"""Add a log message and update progress."""
timestamp = datetime.now().strftime("%H:%M:%S")
log_entry = f"[{timestamp}] {message}"
self.logs.append(log_entry)
print(message) # Also print to console
# Call the progress callback if provided
if self.progress_callback:
self.progress_callback(self.get_status())
def start(self):
"""Mark the crawling process as started."""
self.is_running = True
self.start_time = datetime.now()
self.log("Crawling process started")
# Call the progress callback if provided
if self.progress_callback:
self.progress_callback(self.get_status())
def complete(self):
"""Mark the crawling process as completed."""
self.is_running = False
self.end_time = datetime.now()
duration = self.end_time - self.start_time if self.start_time else None
duration_str = str(duration).split('.')[0] if duration else "unknown"
self.log(f"Crawling process completed in {duration_str}")
# Call the progress callback if provided
if self.progress_callback:
self.progress_callback(self.get_status())
def get_status(self) -> Dict[str, Any]:
"""Get the current status of the crawling process."""
return {
"is_running": self.is_running,
"urls_found": self.urls_found,
"urls_processed": self.urls_processed,
"urls_succeeded": self.urls_succeeded,
"urls_failed": self.urls_failed,
"chunks_stored": self.chunks_stored,
"progress_percentage": (self.urls_processed / self.urls_found * 100) if self.urls_found > 0 else 0,
"logs": self.logs,
"start_time": self.start_time,
"end_time": self.end_time
}
@property
def is_completed(self) -> bool:
"""Return True if the crawling process is completed."""
return not self.is_running and self.end_time is not None
@property
def is_successful(self) -> bool:
"""Return True if the crawling process completed successfully."""
return self.is_completed and self.urls_failed == 0 and self.urls_succeeded > 0
def chunk_text(text: str, chunk_size: int = 5000) -> List[str]:
"""Split text into chunks, respecting code blocks and paragraphs."""
chunks = []
start = 0
text_length = len(text)
while start < text_length:
# Calculate end position
end = start + chunk_size
# If we're at the end of the text, just take what's left
if end >= text_length:
chunks.append(text[start:].strip())
break
# Try to find a code block boundary first (```)
chunk = text[start:end]
code_block = chunk.rfind('```')
if code_block != -1 and code_block > chunk_size * 0.3:
end = start + code_block
# If no code block, try to break at a paragraph
elif '\n\n' in chunk:
# Find the last paragraph break
last_break = chunk.rfind('\n\n')
if last_break > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_break
# If no paragraph break, try to break at a sentence
elif '. ' in chunk:
# Find the last sentence break
last_period = chunk.rfind('. ')
if last_period > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_period + 1
# Extract chunk and clean it up
chunk = text[start:end].strip()
if chunk:
chunks.append(chunk)
# Move start position for next chunk
start = max(start + 1, end)
return chunks
async def get_title_and_summary(chunk: str, url: str) -> Dict[str, str]:
"""Extract title and summary using GPT-4."""
system_prompt = """You are an AI that extracts titles and summaries from documentation chunks.
Return a JSON object with 'title' and 'summary' keys.
For the title: If this seems like the start of a document, extract its title. If it's a middle chunk, derive a descriptive title.
For the summary: Create a concise summary of the main points in this chunk.
Keep both title and summary concise but informative."""
try:
response = await llm_client.chat.completions.create(
model=get_env_var("PRIMARY_MODEL") or "gpt-4o-mini",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"URL: {url}\n\nContent:\n{chunk[:1000]}..."} # Send first 1000 chars for context
],
response_format={ "type": "json_object" }
)
return json.loads(response.choices[0].message.content)
except Exception as e:
print(f"Error getting title and summary: {e}")
return {"title": "Error processing title", "summary": "Error processing summary"}
async def get_embedding(text: str) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await embedding_client.embeddings.create(
model=embedding_model,
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
async def process_chunk(chunk: str, chunk_number: int, url: str) -> ProcessedChunk:
"""Process a single chunk of text."""
# Get title and summary
extracted = await get_title_and_summary(chunk, url)
# Get embedding
embedding = await get_embedding(chunk)
# Create metadata
metadata = {
"source": "pydantic_ai_docs",
"chunk_size": len(chunk),
"crawled_at": datetime.now(timezone.utc).isoformat(),
"url_path": urlparse(url).path
}
return ProcessedChunk(
url=url,
chunk_number=chunk_number,
title=extracted['title'],
summary=extracted['summary'],
content=chunk, # Store the original chunk content
metadata=metadata,
embedding=embedding
)
async def insert_chunk(chunk: ProcessedChunk):
"""Insert a processed chunk into Supabase."""
try:
data = {
"url": chunk.url,
"chunk_number": chunk.chunk_number,
"title": chunk.title,
"summary": chunk.summary,
"content": chunk.content,
"metadata": chunk.metadata,
"embedding": chunk.embedding
}
result = supabase.table("site_pages").insert(data).execute()
print(f"Inserted chunk {chunk.chunk_number} for {chunk.url}")
return result
except Exception as e:
print(f"Error inserting chunk: {e}")
return None
async def process_and_store_document(url: str, markdown: str, tracker: Optional[CrawlProgressTracker] = None):
"""Process a document and store its chunks in parallel."""
# Split into chunks
chunks = chunk_text(markdown)
if tracker:
tracker.log(f"Split document into {len(chunks)} chunks for {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Split document into {len(chunks)} chunks for {url}")
# Process chunks in parallel
tasks = [
process_chunk(chunk, i, url)
for i, chunk in enumerate(chunks)
]
processed_chunks = await asyncio.gather(*tasks)
if tracker:
tracker.log(f"Processed {len(processed_chunks)} chunks for {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Processed {len(processed_chunks)} chunks for {url}")
# Store chunks in parallel
insert_tasks = [
insert_chunk(chunk)
for chunk in processed_chunks
]
await asyncio.gather(*insert_tasks)
if tracker:
tracker.chunks_stored += len(processed_chunks)
tracker.log(f"Stored {len(processed_chunks)} chunks for {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Stored {len(processed_chunks)} chunks for {url}")
def fetch_url_content(url: str) -> str:
"""Fetch content from a URL using requests and convert to markdown."""
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
}
try:
response = requests.get(url, headers=headers, timeout=30)
response.raise_for_status()
# Convert HTML to Markdown
markdown = html_converter.handle(response.text)
# Clean up the markdown
markdown = re.sub(r'\n{3,}', '\n\n', markdown) # Remove excessive newlines
return markdown
except Exception as e:
raise Exception(f"Error fetching {url}: {str(e)}")
async def crawl_parallel_with_requests(urls: List[str], tracker: Optional[CrawlProgressTracker] = None, max_concurrent: int = 5):
"""Crawl multiple URLs in parallel with a concurrency limit using direct HTTP requests."""
# Create a semaphore to limit concurrency
semaphore = asyncio.Semaphore(max_concurrent)
async def process_url(url: str):
async with semaphore:
if tracker:
tracker.log(f"Crawling: {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Crawling: {url}")
try:
# Use a thread pool to run the blocking HTTP request
loop = asyncio.get_running_loop()
if tracker:
tracker.log(f"Fetching content from: {url}")
else:
print(f"Fetching content from: {url}")
markdown = await loop.run_in_executor(None, fetch_url_content, url)
if markdown:
if tracker:
tracker.urls_succeeded += 1
tracker.log(f"Successfully crawled: {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Successfully crawled: {url}")
await process_and_store_document(url, markdown, tracker)
else:
if tracker:
tracker.urls_failed += 1
tracker.log(f"Failed: {url} - No content retrieved")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Failed: {url} - No content retrieved")
except Exception as e:
if tracker:
tracker.urls_failed += 1
tracker.log(f"Error processing {url}: {str(e)}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Error processing {url}: {str(e)}")
finally:
if tracker:
tracker.urls_processed += 1
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
time.sleep(2)
# Process all URLs in parallel with limited concurrency
if tracker:
tracker.log(f"Processing {len(urls)} URLs with concurrency {max_concurrent}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Processing {len(urls)} URLs with concurrency {max_concurrent}")
await asyncio.gather(*[process_url(url) for url in urls])
def get_pydantic_ai_docs_urls() -> List[str]:
"""Get URLs from Pydantic AI docs sitemap."""
sitemap_url = "https://ai.pydantic.dev/sitemap.xml"
try:
response = requests.get(sitemap_url)
response.raise_for_status()
# Parse the XML
root = ElementTree.fromstring(response.content)
# Extract all URLs from the sitemap
namespace = {'ns': 'http://www.sitemaps.org/schemas/sitemap/0.9'}
urls = [loc.text for loc in root.findall('.//ns:loc', namespace)]
return urls
except Exception as e:
print(f"Error fetching sitemap: {e}")
return []
def clear_existing_records():
"""Clear all existing records with source='pydantic_ai_docs' from the site_pages table."""
try:
result = supabase.table("site_pages").delete().eq("metadata->>source", "pydantic_ai_docs").execute()
print("Cleared existing pydantic_ai_docs records from site_pages")
return result
except Exception as e:
print(f"Error clearing existing records: {e}")
return None
async def main_with_requests(tracker: Optional[CrawlProgressTracker] = None):
"""Main function using direct HTTP requests instead of browser automation."""
try:
# Start tracking if tracker is provided
if tracker:
tracker.start()
else:
print("Starting crawling process...")
# Clear existing records first
if tracker:
tracker.log("Clearing existing Pydantic AI docs records...")
else:
print("Clearing existing Pydantic AI docs records...")
clear_existing_records()
if tracker:
tracker.log("Existing records cleared")
else:
print("Existing records cleared")
# Get URLs from Pydantic AI docs
if tracker:
tracker.log("Fetching URLs from Pydantic AI sitemap...")
else:
print("Fetching URLs from Pydantic AI sitemap...")
urls = get_pydantic_ai_docs_urls()
if not urls:
if tracker:
tracker.log("No URLs found to crawl")
tracker.complete()
else:
print("No URLs found to crawl")
return
if tracker:
tracker.urls_found = len(urls)
tracker.log(f"Found {len(urls)} URLs to crawl")
else:
print(f"Found {len(urls)} URLs to crawl")
# Crawl the URLs using direct HTTP requests
await crawl_parallel_with_requests(urls, tracker)
# Mark as complete if tracker is provided
if tracker:
tracker.complete()
else:
print("Crawling process completed")
except Exception as e:
if tracker:
tracker.log(f"Error in crawling process: {str(e)}")
tracker.complete()
else:
print(f"Error in crawling process: {str(e)}")
def start_crawl_with_requests(progress_callback: Optional[Callable[[Dict[str, Any]], None]] = None) -> CrawlProgressTracker:
"""Start the crawling process using direct HTTP requests in a separate thread and return the tracker."""
tracker = CrawlProgressTracker(progress_callback)
def run_crawl():
try:
asyncio.run(main_with_requests(tracker))
except Exception as e:
print(f"Error in crawl thread: {e}")
tracker.log(f"Thread error: {str(e)}")
tracker.complete()
# Start the crawling process in a separate thread
thread = threading.Thread(target=run_crawl)
thread.daemon = True
thread.start()
return tracker
if __name__ == "__main__":
# Run the main function directly
print("Starting crawler...")
asyncio.run(main_with_requests())
print("Crawler finished.")

7
original_archon/archon/langgraph.json
View File

@@ -1,7 +0,0 @@
{
"dependencies": ["."],
"graphs": {
"agent": "./archon_graph.py:agentic_flow"
},
"env": "../.env"
}

102
original_archon/archon/pydantic_ai_coder.py
View File

@@ -1,102 +0,0 @@
from __future__ import annotations as _annotations
from dataclasses import dataclass
from dotenv import load_dotenv
import logfire
import asyncio
import httpx
import os
import sys
import json
from typing import List
from pydantic import BaseModel
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.models.openai import OpenAIModel
from openai import AsyncOpenAI
from supabase import Client
# Add the parent directory to sys.path to allow importing from the parent directory
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from utils.utils import get_env_var
from archon.agent_prompts import primary_coder_prompt
from archon.agent_tools import (
retrieve_relevant_documentation_tool,
list_documentation_pages_tool,
get_page_content_tool
)
load_dotenv()
provider = get_env_var('LLM_PROVIDER') or 'OpenAI'
llm = get_env_var('PRIMARY_MODEL') or 'gpt-4o-mini'
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
model = AnthropicModel(llm, api_key=api_key) if provider == "Anthropic" else OpenAIModel(llm, base_url=base_url, api_key=api_key)
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class PydanticAIDeps:
supabase: Client
embedding_client: AsyncOpenAI
reasoner_output: str
advisor_output: str
pydantic_ai_coder = Agent(
model,
system_prompt=primary_coder_prompt,
deps_type=PydanticAIDeps,
retries=2
)
@pydantic_ai_coder.system_prompt
def add_reasoner_output(ctx: RunContext[str]) -> str:
return f"""
Additional thoughts/instructions from the reasoner LLM.
This scope includes documentation pages for you to search as well:
{ctx.deps.reasoner_output}
Recommended starting point from the advisor agent:
{ctx.deps.advisor_output}
"""
@pydantic_ai_coder.tool
async def retrieve_relevant_documentation(ctx: RunContext[PydanticAIDeps], user_query: str) -> str:
"""
Retrieve relevant documentation chunks based on the query with RAG.
Args:
ctx: The context including the Supabase client and OpenAI client
user_query: The user's question or query
Returns:
A formatted string containing the top 4 most relevant documentation chunks
"""
return await retrieve_relevant_documentation_tool(ctx.deps.supabase, ctx.deps.embedding_client, user_query)
@pydantic_ai_coder.tool
async def list_documentation_pages(ctx: RunContext[PydanticAIDeps]) -> List[str]:
"""
Retrieve a list of all available Pydantic AI documentation pages.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
return await list_documentation_pages_tool(ctx.deps.supabase)
@pydantic_ai_coder.tool
async def get_page_content(ctx: RunContext[PydanticAIDeps], url: str) -> str:
"""
Retrieve the full content of a specific documentation page by combining all its chunks.
Args:
ctx: The context including the Supabase client
url: The URL of the page to retrieve
Returns:
str: The complete page content with all chunks combined in order
"""
return await get_page_content_tool(ctx.deps.supabase, url)

92
original_archon/archon/refiner_agents/agent_refiner_agent.py
View File

@@ -1,92 +0,0 @@
from __future__ import annotations as _annotations
from dataclasses import dataclass
from dotenv import load_dotenv
import logfire
import asyncio
import httpx
import os
import sys
import json
from typing import List
from pydantic import BaseModel
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.models.openai import OpenAIModel
from openai import AsyncOpenAI
from supabase import Client
# Add the parent directory to sys.path to allow importing from the parent directory
sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
from utils.utils import get_env_var
from archon.agent_prompts import agent_refiner_prompt
from archon.agent_tools import (
retrieve_relevant_documentation_tool,
list_documentation_pages_tool,
get_page_content_tool
)
load_dotenv()
provider = get_env_var('LLM_PROVIDER') or 'OpenAI'
llm = get_env_var('PRIMARY_MODEL') or 'gpt-4o-mini'
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
model = AnthropicModel(llm, api_key=api_key) if provider == "Anthropic" else OpenAIModel(llm, base_url=base_url, api_key=api_key)
embedding_model = get_env_var('EMBEDDING_MODEL') or 'text-embedding-3-small'
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class AgentRefinerDeps:
supabase: Client
embedding_client: AsyncOpenAI
agent_refiner_agent = Agent(
model,
system_prompt=agent_refiner_prompt,
deps_type=AgentRefinerDeps,
retries=2
)
@agent_refiner_agent.tool
async def retrieve_relevant_documentation(ctx: RunContext[AgentRefinerDeps], query: str) -> str:
"""
Retrieve relevant documentation chunks based on the query with RAG.
Make sure your searches always focus on implementing the agent itself.
Args:
ctx: The context including the Supabase client and OpenAI client
query: Your query to retrieve relevant documentation for implementing agents
Returns:
A formatted string containing the top 4 most relevant documentation chunks
"""
return await retrieve_relevant_documentation_tool(ctx.deps.supabase, ctx.deps.embedding_client, query)
@agent_refiner_agent.tool
async def list_documentation_pages(ctx: RunContext[AgentRefinerDeps]) -> List[str]:
"""
Retrieve a list of all available Pydantic AI documentation pages.
This will give you all pages available, but focus on the ones related to configuring agents and their dependencies.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
return await list_documentation_pages_tool(ctx.deps.supabase)
@agent_refiner_agent.tool
async def get_page_content(ctx: RunContext[AgentRefinerDeps], url: str) -> str:
"""
Retrieve the full content of a specific documentation page by combining all its chunks.
Only use this tool to get pages related to setting up agents with Pydantic AI.
Args:
ctx: The context including the Supabase client
url: The URL of the page to retrieve
Returns:
str: The complete page content with all chunks combined in order
"""
return await get_page_content_tool(ctx.deps.supabase, url)

31
original_archon/archon/refiner_agents/prompt_refiner_agent.py
View File

@@ -1,31 +0,0 @@
from __future__ import annotations as _annotations
import logfire
import os
import sys
from pydantic_ai import Agent
from dotenv import load_dotenv
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.models.openai import OpenAIModel
from supabase import Client
# Add the parent directory to sys.path to allow importing from the parent directory
sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
from utils.utils import get_env_var
from archon.agent_prompts import prompt_refiner_prompt
load_dotenv()
provider = get_env_var('LLM_PROVIDER') or 'OpenAI'
llm = get_env_var('PRIMARY_MODEL') or 'gpt-4o-mini'
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
model = AnthropicModel(llm, api_key=api_key) if provider == "Anthropic" else OpenAIModel(llm, base_url=base_url, api_key=api_key)
logfire.configure(send_to_logfire='if-token-present')
prompt_refiner_agent = Agent(
model,
system_prompt=prompt_refiner_prompt
)

120
original_archon/archon/refiner_agents/tools_refiner_agent.py
View File

@@ -1,120 +0,0 @@
from __future__ import annotations as _annotations
from dataclasses import dataclass
from dotenv import load_dotenv
import logfire
import asyncio
import httpx
import os
import sys
import json
from typing import List
from pydantic import BaseModel
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.models.openai import OpenAIModel
from openai import AsyncOpenAI
from supabase import Client
# Add the parent directory to sys.path to allow importing from the parent directory
sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
from utils.utils import get_env_var
from archon.agent_prompts import tools_refiner_prompt
from archon.agent_tools import (
retrieve_relevant_documentation_tool,
list_documentation_pages_tool,
get_page_content_tool,
get_file_content_tool
)
load_dotenv()
provider = get_env_var('LLM_PROVIDER') or 'OpenAI'
llm = get_env_var('PRIMARY_MODEL') or 'gpt-4o-mini'
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
model = AnthropicModel(llm, api_key=api_key) if provider == "Anthropic" else OpenAIModel(llm, base_url=base_url, api_key=api_key)
embedding_model = get_env_var('EMBEDDING_MODEL') or 'text-embedding-3-small'
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class ToolsRefinerDeps:
supabase: Client
embedding_client: AsyncOpenAI
file_list: List[str]
tools_refiner_agent = Agent(
model,
system_prompt=tools_refiner_prompt,
deps_type=ToolsRefinerDeps,
retries=2
)
@tools_refiner_agent.system_prompt
def add_file_list(ctx: RunContext[str]) -> str:
joined_files = "\n".join(ctx.deps.file_list)
return f"""
Here is the list of all the files that you can pull the contents of with the
'get_file_content' tool if the example/tool/MCP server is relevant to the
agent the user is trying to build:
{joined_files}
"""
@tools_refiner_agent.tool
async def retrieve_relevant_documentation(ctx: RunContext[ToolsRefinerDeps], query: str) -> str:
"""
Retrieve relevant documentation chunks based on the query with RAG.
Make sure your searches always focus on implementing tools.
Args:
ctx: The context including the Supabase client and OpenAI client
query: Your query to retrieve relevant documentation for implementing tools
Returns:
A formatted string containing the top 4 most relevant documentation chunks
"""
return await retrieve_relevant_documentation_tool(ctx.deps.supabase, ctx.deps.embedding_client, query)
@tools_refiner_agent.tool
async def list_documentation_pages(ctx: RunContext[ToolsRefinerDeps]) -> List[str]:
"""
Retrieve a list of all available Pydantic AI documentation pages.
This will give you all pages available, but focus on the ones related to tools.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
return await list_documentation_pages_tool(ctx.deps.supabase)
@tools_refiner_agent.tool
async def get_page_content(ctx: RunContext[ToolsRefinerDeps], url: str) -> str:
"""
Retrieve the full content of a specific documentation page by combining all its chunks.
Only use this tool to get pages related to using tools with Pydantic AI.
Args:
ctx: The context including the Supabase client
url: The URL of the page to retrieve
Returns:
str: The complete page content with all chunks combined in order
"""
return await get_page_content_tool(ctx.deps.supabase, url)
@tools_refiner_agent.tool_plain
def get_file_content(file_path: str) -> str:
"""
Retrieves the content of a specific file. Use this to get the contents of an example, tool, config for an MCP server
Args:
file_path: The path to the file
Returns:
The raw contents of the file
"""
return get_file_content_tool(file_path)

70
original_archon/graph_service.py
View File

@@ -1,70 +0,0 @@
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import Optional, Dict, Any
from archon.archon_graph import agentic_flow
from langgraph.types import Command
from utils.utils import write_to_log
app = FastAPI()
class InvokeRequest(BaseModel):
message: str
thread_id: str
is_first_message: bool = False
config: Optional[Dict[str, Any]] = None
@app.get("/health")
async def health_check():
"""Health check endpoint"""
return {"status": "ok"}
@app.post("/invoke")
async def invoke_agent(request: InvokeRequest):
"""Process a message through the agentic flow and return the complete response.
The agent streams the response but this API endpoint waits for the full output
before returning so it's a synchronous operation for MCP.
Another endpoint will be made later to fully stream the response from the API.
Args:
request: The InvokeRequest containing message and thread info
Returns:
dict: Contains the complete response from the agent
"""
try:
config = request.config or {
"configurable": {
"thread_id": request.thread_id
}
}
response = ""
if request.is_first_message:
write_to_log(f"Processing first message for thread {request.thread_id}")
async for msg in agentic_flow.astream(
{"latest_user_message": request.message},
config,
stream_mode="custom"
):
response += str(msg)
else:
write_to_log(f"Processing continuation for thread {request.thread_id}")
async for msg in agentic_flow.astream(
Command(resume=request.message),
config,
stream_mode="custom"
):
response += str(msg)
write_to_log(f"Final response for thread {request.thread_id}: {response}")
return {"response": response}
except Exception as e:
print(f"Exception invoking Archon for thread {request.thread_id}: {str(e)}")
write_to_log(f"Error processing message for thread {request.thread_id}: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8100)

19
original_archon/iterations/v1-single-agent/.env.example
View File

@@ -1,19 +0,0 @@
# Get your Open AI API Key by following these instructions -
# https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# You only need this environment variable set if you are using GPT (and not Ollama)
OPENAI_API_KEY=
# For the Supabase version (sample_supabase_agent.py), set your Supabase URL and Service Key.
# Get your SUPABASE_URL from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
SUPABASE_URL=
# Get your SUPABASE_SERVICE_KEY from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
# On this page it is called the service_role secret.
SUPABASE_SERVICE_KEY=
# The LLM you want to use from OpenAI. See the list of models here:
# https://platform.openai.com/docs/models
# Example: gpt-4o-mini
LLM_MODEL=

122
original_archon/iterations/v1-single-agent/README.md
View File

@@ -1,122 +0,0 @@
# Archon V1 - Basic Pydantic AI Agent to Build other Pydantic AI Agents
This is the first iteration of the Archon project - no use of LangGraph and built with a single AI agent to keep things very simple and introductory.
An intelligent documentation crawler and RAG (Retrieval-Augmented Generation) agent built using Pydantic AI and Supabase that is capable of building other Pydantic AI agents. The agent crawls the Pydantic AI documentation, stores content in a vector database, and provides Pydantic AI agent code by retrieving and analyzing relevant documentation chunks.
## Features
- Pydantic AI documentation crawling and chunking
- Vector database storage with Supabase
- Semantic search using OpenAI embeddings
- RAG-based question answering
- Support for code block preservation
- Streamlit UI for interactive querying
## Prerequisites
- Python 3.11+
- Supabase account and database
- OpenAI API key
- Streamlit (for web interface)
## Installation
1. Clone the repository:
```bash
git clone https://github.com/coleam00/archon.git
cd archon/iterations/v1-single-agent
```
2. Install dependencies (recommended to use a Python virtual environment):
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```
3. Set up environment variables:
- Rename `.env.example` to `.env`
- Edit `.env` with your API keys and preferences:
```env
OPENAI_API_KEY=your_openai_api_key
SUPABASE_URL=your_supabase_url
SUPABASE_SERVICE_KEY=your_supabase_service_key
LLM_MODEL=gpt-4o-mini # or your preferred OpenAI model
```
## Usage
### Database Setup
Execute the SQL commands in `site_pages.sql` to:
1. Create the necessary tables
2. Enable vector similarity search
3. Set up Row Level Security policies
In Supabase, do this by going to the "SQL Editor" tab and pasting in the SQL into the editor there. Then click "Run".
### Crawl Documentation
To crawl and store documentation in the vector database:
```bash
python crawl_pydantic_ai_docs.py
```
This will:
1. Fetch URLs from the documentation sitemap
2. Crawl each page and split into chunks
3. Generate embeddings and store in Supabase
### Streamlit Web Interface
For an interactive web interface to query the documentation:
```bash
streamlit run streamlit_ui.py
```
The interface will be available at `http://localhost:8501`
## Configuration
### Database Schema
The Supabase database uses the following schema:
```sql
CREATE TABLE site_pages (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
url TEXT,
chunk_number INTEGER,
title TEXT,
summary TEXT,
content TEXT,
metadata JSONB,
embedding VECTOR(1536)
);
```
### Chunking Configuration
You can configure chunking parameters in `crawl_pydantic_ai_docs.py`:
```python
chunk_size = 5000 # Characters per chunk
```
The chunker intelligently preserves:
- Code blocks
- Paragraph boundaries
- Sentence boundaries
## Project Structure
- `crawl_pydantic_ai_docs.py`: Documentation crawler and processor
- `pydantic_ai_expert.py`: RAG agent implementation
- `streamlit_ui.py`: Web interface
- `site_pages.sql`: Database setup commands
- `requirements.txt`: Project dependencies
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.

245
original_archon/iterations/v1-single-agent/crawl_pydantic_ai_docs.py
View File

@@ -1,245 +0,0 @@
import os
import sys
import json
import asyncio
import requests
from xml.etree import ElementTree
from typing import List, Dict, Any
from dataclasses import dataclass
from datetime import datetime, timezone
from urllib.parse import urlparse
from dotenv import load_dotenv
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
from openai import AsyncOpenAI
from supabase import create_client, Client
load_dotenv()
# Initialize OpenAI and Supabase clients
openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
supabase: Client = create_client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_SERVICE_KEY")
)
@dataclass
class ProcessedChunk:
url: str
chunk_number: int
title: str
summary: str
content: str
metadata: Dict[str, Any]
embedding: List[float]
def chunk_text(text: str, chunk_size: int = 5000) -> List[str]:
"""Split text into chunks, respecting code blocks and paragraphs."""
chunks = []
start = 0
text_length = len(text)
while start < text_length:
# Calculate end position
end = start + chunk_size
# If we're at the end of the text, just take what's left
if end >= text_length:
chunks.append(text[start:].strip())
break
# Try to find a code block boundary first (```)
chunk = text[start:end]
code_block = chunk.rfind('```')
if code_block != -1 and code_block > chunk_size * 0.3:
end = start + code_block
# If no code block, try to break at a paragraph
elif '\n\n' in chunk:
# Find the last paragraph break
last_break = chunk.rfind('\n\n')
if last_break > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_break
# If no paragraph break, try to break at a sentence
elif '. ' in chunk:
# Find the last sentence break
last_period = chunk.rfind('. ')
if last_period > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_period + 1
# Extract chunk and clean it up
chunk = text[start:end].strip()
if chunk:
chunks.append(chunk)
# Move start position for next chunk
start = max(start + 1, end)
return chunks
async def get_title_and_summary(chunk: str, url: str) -> Dict[str, str]:
"""Extract title and summary using GPT-4."""
system_prompt = """You are an AI that extracts titles and summaries from documentation chunks.
Return a JSON object with 'title' and 'summary' keys.
For the title: If this seems like the start of a document, extract its title. If it's a middle chunk, derive a descriptive title.
For the summary: Create a concise summary of the main points in this chunk.
Keep both title and summary concise but informative."""
try:
response = await openai_client.chat.completions.create(
model=os.getenv("LLM_MODEL", "gpt-4o-mini"),
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"URL: {url}\n\nContent:\n{chunk[:1000]}..."} # Send first 1000 chars for context
],
response_format={ "type": "json_object" }
)
return json.loads(response.choices[0].message.content)
except Exception as e:
print(f"Error getting title and summary: {e}")
return {"title": "Error processing title", "summary": "Error processing summary"}
async def get_embedding(text: str) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await openai_client.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
async def process_chunk(chunk: str, chunk_number: int, url: str) -> ProcessedChunk:
"""Process a single chunk of text."""
# Get title and summary
extracted = await get_title_and_summary(chunk, url)
# Get embedding
embedding = await get_embedding(chunk)
# Create metadata
metadata = {
"source": "pydantic_ai_docs",
"chunk_size": len(chunk),
"crawled_at": datetime.now(timezone.utc).isoformat(),
"url_path": urlparse(url).path
}
return ProcessedChunk(
url=url,
chunk_number=chunk_number,
title=extracted['title'],
summary=extracted['summary'],
content=chunk, # Store the original chunk content
metadata=metadata,
embedding=embedding
)
async def insert_chunk(chunk: ProcessedChunk):
"""Insert a processed chunk into Supabase."""
try:
data = {
"url": chunk.url,
"chunk_number": chunk.chunk_number,
"title": chunk.title,
"summary": chunk.summary,
"content": chunk.content,
"metadata": chunk.metadata,
"embedding": chunk.embedding
}
result = supabase.table("site_pages").insert(data).execute()
print(f"Inserted chunk {chunk.chunk_number} for {chunk.url}")
return result
except Exception as e:
print(f"Error inserting chunk: {e}")
return None
async def process_and_store_document(url: str, markdown: str):
"""Process a document and store its chunks in parallel."""
# Split into chunks
chunks = chunk_text(markdown)
# Process chunks in parallel
tasks = [
process_chunk(chunk, i, url)
for i, chunk in enumerate(chunks)
]
processed_chunks = await asyncio.gather(*tasks)
# Store chunks in parallel
insert_tasks = [
insert_chunk(chunk)
for chunk in processed_chunks
]
await asyncio.gather(*insert_tasks)
async def crawl_parallel(urls: List[str], max_concurrent: int = 5):
"""Crawl multiple URLs in parallel with a concurrency limit."""
browser_config = BrowserConfig(
headless=True,
verbose=False,
extra_args=["--disable-gpu", "--disable-dev-shm-usage", "--no-sandbox"],
)
crawl_config = CrawlerRunConfig(cache_mode=CacheMode.BYPASS)
# Create the crawler instance
crawler = AsyncWebCrawler(config=browser_config)
await crawler.start()
try:
# Create a semaphore to limit concurrency
semaphore = asyncio.Semaphore(max_concurrent)
async def process_url(url: str):
async with semaphore:
result = await crawler.arun(
url=url,
config=crawl_config,
session_id="session1"
)
if result.success:
print(f"Successfully crawled: {url}")
await process_and_store_document(url, result.markdown_v2.raw_markdown)
else:
print(f"Failed: {url} - Error: {result.error_message}")
# Process all URLs in parallel with limited concurrency
await asyncio.gather(*[process_url(url) for url in urls])
finally:
await crawler.close()
def get_pydantic_ai_docs_urls() -> List[str]:
"""Get URLs from Pydantic AI docs sitemap."""
sitemap_url = "https://ai.pydantic.dev/sitemap.xml"
try:
response = requests.get(sitemap_url)
response.raise_for_status()
# Parse the XML
root = ElementTree.fromstring(response.content)
# Extract all URLs from the sitemap
namespace = {'ns': 'http://www.sitemaps.org/schemas/sitemap/0.9'}
urls = [loc.text for loc in root.findall('.//ns:loc', namespace)]
return urls
except Exception as e:
print(f"Error fetching sitemap: {e}")
return []
async def main():
# Get URLs from Pydantic AI docs
urls = get_pydantic_ai_docs_urls()
if not urls:
print("No URLs found to crawl")
return
print(f"Found {len(urls)} URLs to crawl")
await crawl_parallel(urls)
if __name__ == "__main__":
asyncio.run(main())

193
original_archon/iterations/v1-single-agent/pydantic_ai_coder.py
View File

@@ -1,193 +0,0 @@
from __future__ import annotations as _annotations
from dataclasses import dataclass
from dotenv import load_dotenv
import logfire
import asyncio
import httpx
import os
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.models.openai import OpenAIModel
from openai import AsyncOpenAI
from supabase import Client
from typing import List
load_dotenv()
llm = os.getenv('LLM_MODEL', 'gpt-4o-mini')
model = OpenAIModel(llm)
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class PydanticAIDeps:
supabase: Client
openai_client: AsyncOpenAI
system_prompt = """
~~ CONTEXT: ~~
You are an expert at Pydantic AI - a Python AI agent framework that you have access to all the documentation to,
including examples, an API reference, and other resources to help you build Pydantic AI agents.
~~ GOAL: ~~
Your only job is to help the user create an AI agent with Pydantic AI.
The user will describe the AI agent they want to build, or if they don't, guide them towards doing so.
You will take their requirements, and then search through the Pydantic AI documentation with the tools provided
to find all the necessary information to create the AI agent with correct code.
It's important for you to search through multiple Pydantic AI documentation pages to get all the information you need.
Almost never stick to just one page - use RAG and the other documentation tools multiple times when you are creating
an AI agent from scratch for the user.
~~ STRUCTURE: ~~
When you build an AI agent from scratch, split the agent into this files and give the code for each:
- `agent.py`: The main agent file, which is where the Pydantic AI agent is defined.
- `agent_tools.py`: A tools file for the agent, which is where all the tool functions are defined. Use this for more complex agents.
- `agent_prompts.py`: A prompts file for the agent, which includes all system prompts and other prompts used by the agent. Use this when there are many prompts or large ones.
- `.env.example`: An example `.env` file - specify each variable that the user will need to fill in and a quick comment above each one for how to do so.
- `requirements.txt`: Don't include any versions, just the top level package names needed for the agent.
~~ INSTRUCTIONS: ~~
- Don't ask the user before taking an action, just do it. Always make sure you look at the documentation with the provided tools before writing any code.
- When you first look at the documentation, always start with RAG.
Then also always check the list of available documentation pages and retrieve the content of page(s) if it'll help.
- Always let the user know when you didn't find the answer in the documentation or the right URL - be honest.
- Helpful tip: when starting a new AI agent build, it's a good idea to look at the 'weather agent' in the docs as an example.
- When starting a new AI agent build, always produce the full code for the AI agent - never tell the user to finish a tool/function.
- When refining an existing AI agent build in a conversation, just share the code changes necessary.
"""
pydantic_ai_coder = Agent(
model,
system_prompt=system_prompt,
deps_type=PydanticAIDeps,
retries=2
)
async def get_embedding(text: str, openai_client: AsyncOpenAI) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await openai_client.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
@pydantic_ai_coder.tool
async def retrieve_relevant_documentation(ctx: RunContext[PydanticAIDeps], user_query: str) -> str:
"""
Retrieve relevant documentation chunks based on the query with RAG.
Args:
ctx: The context including the Supabase client and OpenAI client
user_query: The user's question or query
Returns:
A formatted string containing the top 5 most relevant documentation chunks
"""
try:
# Get the embedding for the query
query_embedding = await get_embedding(user_query, ctx.deps.openai_client)
# Query Supabase for relevant documents
result = ctx.deps.supabase.rpc(
'match_site_pages',
{
'query_embedding': query_embedding,
'match_count': 5,
'filter': {'source': 'pydantic_ai_docs'}
}
).execute()
if not result.data:
return "No relevant documentation found."
# Format the results
formatted_chunks = []
for doc in result.data:
chunk_text = f"""
# {doc['title']}
{doc['content']}
"""
formatted_chunks.append(chunk_text)
# Join all chunks with a separator
return "\n\n---\n\n".join(formatted_chunks)
except Exception as e:
print(f"Error retrieving documentation: {e}")
return f"Error retrieving documentation: {str(e)}"
@pydantic_ai_coder.tool
async def list_documentation_pages(ctx: RunContext[PydanticAIDeps]) -> List[str]:
"""
Retrieve a list of all available Pydantic AI documentation pages.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
try:
# Query Supabase for unique URLs where source is pydantic_ai_docs
result = ctx.deps.supabase.from_('site_pages') \
.select('url') \
.eq('metadata->>source', 'pydantic_ai_docs') \
.execute()
if not result.data:
return []
# Extract unique URLs
urls = sorted(set(doc['url'] for doc in result.data))
return urls
except Exception as e:
print(f"Error retrieving documentation pages: {e}")
return []
@pydantic_ai_coder.tool
async def get_page_content(ctx: RunContext[PydanticAIDeps], url: str) -> str:
"""
Retrieve the full content of a specific documentation page by combining all its chunks.
Args:
ctx: The context including the Supabase client
url: The URL of the page to retrieve
Returns:
str: The complete page content with all chunks combined in order
"""
try:
# Query Supabase for all chunks of this URL, ordered by chunk_number
result = ctx.deps.supabase.from_('site_pages') \
.select('title, content, chunk_number') \
.eq('url', url) \
.eq('metadata->>source', 'pydantic_ai_docs') \
.order('chunk_number') \
.execute()
if not result.data:
return f"No content found for URL: {url}"
# Format the page with its title and all chunks
page_title = result.data[0]['title'].split(' - ')[0] # Get the main title
formatted_content = [f"# {page_title}\n"]
# Add each chunk's content
for chunk in result.data:
formatted_content.append(chunk['content'])
# Join everything together
return "\n\n".join(formatted_content)
except Exception as e:
print(f"Error retrieving page content: {e}")
return f"Error retrieving page content: {str(e)}"

BIN
original_archon/iterations/v1-single-agent/requirements.txt
View File

Binary file not shown.

72
original_archon/iterations/v1-single-agent/site_pages.sql
View File

@@ -1,72 +0,0 @@
-- Enable the pgvector extension
create extension if not exists vector;
-- Create the documentation chunks table
create table site_pages (
id bigserial primary key,
url varchar not null,
chunk_number integer not null,
title varchar not null,
summary varchar not null,
content text not null, -- Added content column
metadata jsonb not null default '{}'::jsonb, -- Added metadata column
embedding vector(1536), -- OpenAI embeddings are 1536 dimensions
created_at timestamp with time zone default timezone('utc'::text, now()) not null,
-- Add a unique constraint to prevent duplicate chunks for the same URL
unique(url, chunk_number)
);
-- Create an index for better vector similarity search performance
create index on site_pages using ivfflat (embedding vector_cosine_ops);
-- Create an index on metadata for faster filtering
create index idx_site_pages_metadata on site_pages using gin (metadata);
-- Create a function to search for documentation chunks
create function match_site_pages (
query_embedding vector(1536),
match_count int default 10,
filter jsonb DEFAULT '{}'::jsonb
) returns table (
id bigint,
url varchar,
chunk_number integer,
title varchar,
summary varchar,
content text,
metadata jsonb,
similarity float
)
language plpgsql
as $$
#variable_conflict use_column
begin
return query
select
id,
url,
chunk_number,
title,
summary,
content,
metadata,
1 - (site_pages.embedding <=> query_embedding) as similarity
from site_pages
where metadata @> filter
order by site_pages.embedding <=> query_embedding
limit match_count;
end;
$$;
-- Everything above will work for any PostgreSQL database. The below commands are for Supabase security
-- Enable RLS on the table
alter table site_pages enable row level security;
-- Create a policy that allows anyone to read
create policy "Allow public read access"
on site_pages
for select
to public
using (true);

143
original_archon/iterations/v1-single-agent/streamlit_ui.py
View File

@@ -1,143 +0,0 @@
from __future__ import annotations
from typing import Literal, TypedDict
import asyncio
import os
import streamlit as st
import json
import logfire
from supabase import Client
from openai import AsyncOpenAI
# Import all the message part classes
from pydantic_ai.messages import (
ModelMessage,
ModelRequest,
ModelResponse,
SystemPromptPart,
UserPromptPart,
TextPart,
ToolCallPart,
ToolReturnPart,
RetryPromptPart,
ModelMessagesTypeAdapter
)
from pydantic_ai_coder import pydantic_ai_coder, PydanticAIDeps
# Load environment variables
from dotenv import load_dotenv
load_dotenv()
openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
supabase: Client = Client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_SERVICE_KEY")
)
# Configure logfire to suppress warnings (optional)
logfire.configure(send_to_logfire='never')
class ChatMessage(TypedDict):
"""Format of messages sent to the browser/API."""
role: Literal['user', 'model']
timestamp: str
content: str
def display_message_part(part):
"""
Display a single part of a message in the Streamlit UI.
Customize how you display system prompts, user prompts,
tool calls, tool returns, etc.
"""
# system-prompt
if part.part_kind == 'system-prompt':
with st.chat_message("system"):
st.markdown(f"**System**: {part.content}")
# user-prompt
elif part.part_kind == 'user-prompt':
with st.chat_message("user"):
st.markdown(part.content)
# text
elif part.part_kind == 'text':
with st.chat_message("assistant"):
st.markdown(part.content)
async def run_agent_with_streaming(user_input: str):
"""
Run the agent with streaming text for the user_input prompt,
while maintaining the entire conversation in `st.session_state.messages`.
"""
# Prepare dependencies
deps = PydanticAIDeps(
supabase=supabase,
openai_client=openai_client
)
# Run the agent in a stream
async with pydantic_ai_coder.run_stream(
user_input,
deps=deps,
message_history= st.session_state.messages[:-1], # pass entire conversation so far
) as result:
# We'll gather partial text to show incrementally
partial_text = ""
message_placeholder = st.empty()
# Render partial text as it arrives
async for chunk in result.stream_text(delta=True):
partial_text += chunk
message_placeholder.markdown(partial_text)
# Now that the stream is finished, we have a final result.
# Add new messages from this run, excluding user-prompt messages
filtered_messages = [msg for msg in result.new_messages()
if not (hasattr(msg, 'parts') and
any(part.part_kind == 'user-prompt' for part in msg.parts))]
st.session_state.messages.extend(filtered_messages)
# Add the final response to the messages
st.session_state.messages.append(
ModelResponse(parts=[TextPart(content=partial_text)])
)
async def main():
st.title("Archon - Agent Builder")
st.write("Describe to me an AI agent you want to build and I'll code it for you with Pydantic AI.")
# Initialize chat history in session state if not present
if "messages" not in st.session_state:
st.session_state.messages = []
# Display all messages from the conversation so far
# Each message is either a ModelRequest or ModelResponse.
# We iterate over their parts to decide how to display them.
for msg in st.session_state.messages:
if isinstance(msg, ModelRequest) or isinstance(msg, ModelResponse):
for part in msg.parts:
display_message_part(part)
# Chat input for the user
user_input = st.chat_input("What do you want to build today?")
if user_input:
# We append a new request to the conversation explicitly
st.session_state.messages.append(
ModelRequest(parts=[UserPromptPart(content=user_input)])
)
# Display user prompt in the UI
with st.chat_message("user"):
st.markdown(user_input)
# Display the assistant's partial response while streaming
with st.chat_message("assistant"):
# Actually run the agent now, streaming the text
await run_agent_with_streaming(user_input)
if __name__ == "__main__":
asyncio.run(main())

38
original_archon/iterations/v2-agentic-workflow/.env.example
View File

@@ -1,38 +0,0 @@
# Base URL for the OpenAI instance (default is https://api.openai.com/v1)
# OpenAI: https://api.openai.com/v1
# Ollama (example): http://localhost:11434/v1
# OpenRouter: https://openrouter.ai/api/v1
BASE_URL=
# Get your Open AI API Key by following these instructions -
# https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# Even if using OpenRouter/Ollama, you still need to set this for the embedding model.
# Future versions of Archon will be more flexible with this.
OPENAI_API_KEY=
# For OpenAI: https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# For OpenRouter: https://openrouter.ai/keys
LLM_API_KEY=
# For the Supabase version (sample_supabase_agent.py), set your Supabase URL and Service Key.
# Get your SUPABASE_URL from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
SUPABASE_URL=
# Get your SUPABASE_SERVICE_KEY from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
# On this page it is called the service_role secret.
SUPABASE_SERVICE_KEY=
# The LLM you want to use for the reasoner (o3-mini, R1, QwQ, etc.).
# Example: o3-mini
# Example: deepseek-r1:7b-8k
REASONER_MODEL=
# The LLM you want to use for the primary agent/coder.
# Example: qwen2.5:14b-instruct-8k
PRIMARY_MODEL=
# Embedding model you want to use (nomic-embed-text:latest, text-embedding-3-small)
# Example: nomic-embed-text:latest
EMBEDDING_MODEL=

1
original_archon/iterations/v2-agentic-workflow/.gitignore vendored
View File

@@ -1 +0,0 @@
.env

132
original_archon/iterations/v2-agentic-workflow/README.md
View File

@@ -1,132 +0,0 @@
# Archon V2 - Agentic Workflow for Building Pydantic AI Agents
This is the second iteration of the Archon project, building upon V1 by introducing LangGraph for a full agentic workflow. The system starts with a reasoning LLM (like O3-mini or R1) that analyzes user requirements and documentation to create a detailed scope, which then guides specialized coding and routing agents in generating high-quality Pydantic AI agents.
An intelligent documentation crawler and RAG (Retrieval-Augmented Generation) system built using Pydantic AI, LangGraph, and Supabase that is capable of building other Pydantic AI agents. The system crawls the Pydantic AI documentation, stores content in a vector database, and provides Pydantic AI agent code by retrieving and analyzing relevant documentation chunks.
This version also supports local LLMs with Ollama for the main agent and reasoning LLM.
Note that we are still relying on OpenAI for embeddings no matter what, but future versions of Archon will change that.
## Features
- Multi-agent workflow using LangGraph
- Specialized agents for reasoning, routing, and coding
- Pydantic AI documentation crawling and chunking
- Vector database storage with Supabase
- Semantic search using OpenAI embeddings
- RAG-based question answering
- Support for code block preservation
- Streamlit UI for interactive querying
## Prerequisites
- Python 3.11+
- Supabase account and database
- OpenAI/OpenRouter API key or Ollama for local LLMs
- Streamlit (for web interface)
## Installation
1. Clone the repository:
```bash
git clone https://github.com/coleam00/archon.git
cd archon/iterations/v2-agentic-workflow
```
2. Install dependencies (recommended to use a Python virtual environment):
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```
3. Set up environment variables:
- Rename `.env.example` to `.env`
- Edit `.env` with your API keys and preferences:
```env
BASE_URL=https://api.openai.com/v1 for OpenAI, https://api.openrouter.ai/v1 for OpenRouter, or your Ollama URL
LLM_API_KEY=your_openai_or_openrouter_api_key
OPENAI_API_KEY=your_openai_api_key
SUPABASE_URL=your_supabase_url
SUPABASE_SERVICE_KEY=your_supabase_service_key
PRIMARY_MODEL=gpt-4o-mini # or your preferred OpenAI model for main agent
REASONER_MODEL=o3-mini # or your preferred OpenAI model for reasoning
```
## Usage
### Database Setup
Execute the SQL commands in `site_pages.sql` to:
1. Create the necessary tables
2. Enable vector similarity search
3. Set up Row Level Security policies
In Supabase, do this by going to the "SQL Editor" tab and pasting in the SQL into the editor there. Then click "Run".
### Crawl Documentation
To crawl and store documentation in the vector database:
```bash
python crawl_pydantic_ai_docs.py
```
This will:
1. Fetch URLs from the documentation sitemap
2. Crawl each page and split into chunks
3. Generate embeddings and store in Supabase
### Chunking Configuration
You can configure chunking parameters in `crawl_pydantic_ai_docs.py`:
```python
chunk_size = 5000 # Characters per chunk
```
The chunker intelligently preserves:
- Code blocks
- Paragraph boundaries
- Sentence boundaries
### Streamlit Web Interface
For an interactive web interface to query the documentation and create agents:
```bash
streamlit run streamlit_ui.py
```
The interface will be available at `http://localhost:8501`
## Configuration
### Database Schema
The Supabase database uses the following schema:
```sql
CREATE TABLE site_pages (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
url TEXT,
chunk_number INTEGER,
title TEXT,
summary TEXT,
content TEXT,
metadata JSONB,
embedding VECTOR(1536)
);
```
## Project Structure
- `archon_graph.py`: LangGraph workflow definition and agent coordination
- `pydantic_ai_coder.py`: Main coding agent with RAG capabilities
- `crawl_pydantic_ai_docs.py`: Documentation crawler and processor
- `streamlit_ui.py`: Web interface with streaming support
- `site_pages.sql`: Database setup commands
- `requirements.txt`: Project dependencies
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.

207
original_archon/iterations/v2-agentic-workflow/archon_graph.py
View File

@@ -1,207 +0,0 @@
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai import Agent, RunContext
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver
from typing import TypedDict, Annotated, List, Any
from langgraph.config import get_stream_writer
from langgraph.types import interrupt
from dotenv import load_dotenv
from openai import AsyncOpenAI
from supabase import Client
import logfire
import os
# Import the message classes from Pydantic AI
from pydantic_ai.messages import (
ModelMessage,
ModelMessagesTypeAdapter
)
from pydantic_ai_coder import pydantic_ai_coder, PydanticAIDeps, list_documentation_pages_helper
# Load environment variables
load_dotenv()
# Configure logfire to suppress warnings (optional)
logfire.configure(send_to_logfire='never')
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-llm-api-key-provided')
is_ollama = "localhost" in base_url.lower()
reasoner_llm_model = os.getenv('REASONER_MODEL', 'o3-mini')
reasoner = Agent(
OpenAIModel(reasoner_llm_model, base_url=base_url, api_key=api_key),
system_prompt='You are an expert at coding AI agents with Pydantic AI and defining the scope for doing so.',
)
primary_llm_model = os.getenv('PRIMARY_MODEL', 'gpt-4o-mini')
router_agent = Agent(
OpenAIModel(primary_llm_model, base_url=base_url, api_key=api_key),
system_prompt='Your job is to route the user message either to the end of the conversation or to continue coding the AI agent.',
)
end_conversation_agent = Agent(
OpenAIModel(primary_llm_model, base_url=base_url, api_key=api_key),
system_prompt='Your job is to end a conversation for creating an AI agent by giving instructions for how to execute the agent and they saying a nice goodbye to the user.',
)
openai_client=None
if is_ollama:
openai_client = AsyncOpenAI(base_url=base_url,api_key=api_key)
else:
openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
supabase: Client = Client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_SERVICE_KEY")
)
# Define state schema
class AgentState(TypedDict):
latest_user_message: str
messages: Annotated[List[bytes], lambda x, y: x + y]
scope: str
# Scope Definition Node with Reasoner LLM
async def define_scope_with_reasoner(state: AgentState):
# First, get the documentation pages so the reasoner can decide which ones are necessary
documentation_pages = await list_documentation_pages_helper(supabase)
documentation_pages_str = "\n".join(documentation_pages)
# Then, use the reasoner to define the scope
prompt = f"""
User AI Agent Request: {state['latest_user_message']}
Create detailed scope document for the AI agent including:
- Architecture diagram
- Core components
- External dependencies
- Testing strategy
Also based on these documentation pages available:
{documentation_pages_str}
Include a list of documentation pages that are relevant to creating this agent for the user in the scope document.
"""
result = await reasoner.run(prompt)
scope = result.data
# Save the scope to a file
scope_path = os.path.join("workbench", "scope.md")
os.makedirs("workbench", exist_ok=True)
with open(scope_path, "w", encoding="utf-8") as f:
f.write(scope)
return {"scope": scope}
# Coding Node with Feedback Handling
async def coder_agent(state: AgentState, writer):
# Prepare dependencies
deps = PydanticAIDeps(
supabase=supabase,
openai_client=openai_client,
reasoner_output=state['scope']
)
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
# Run the agent in a stream
if is_ollama:
writer = get_stream_writer()
result = await pydantic_ai_coder.run(state['latest_user_message'], deps=deps, message_history= message_history)
writer(result.data)
else:
async with pydantic_ai_coder.run_stream(
state['latest_user_message'],
deps=deps,
message_history= message_history
) as result:
# Stream partial text as it arrives
async for chunk in result.stream_text(delta=True):
writer(chunk)
# print(ModelMessagesTypeAdapter.validate_json(result.new_messages_json()))
return {"messages": [result.new_messages_json()]}
# Interrupt the graph to get the user's next message
def get_next_user_message(state: AgentState):
value = interrupt({})
# Set the user's latest message for the LLM to continue the conversation
return {
"latest_user_message": value
}
# Determine if the user is finished creating their AI agent or not
async def route_user_message(state: AgentState):
prompt = f"""
The user has sent a message:
{state['latest_user_message']}
If the user wants to end the conversation, respond with just the text "finish_conversation".
If the user wants to continue coding the AI agent, respond with just the text "coder_agent".
"""
result = await router_agent.run(prompt)
next_action = result.data
if next_action == "finish_conversation":
return "finish_conversation"
else:
return "coder_agent"
# End of conversation agent to give instructions for executing the agent
async def finish_conversation(state: AgentState, writer):
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
# Run the agent in a stream
if is_ollama:
writer = get_stream_writer()
result = await end_conversation_agent.run(state['latest_user_message'], message_history= message_history)
writer(result.data)
else:
async with end_conversation_agent.run_stream(
state['latest_user_message'],
message_history= message_history
) as result:
# Stream partial text as it arrives
async for chunk in result.stream_text(delta=True):
writer(chunk)
return {"messages": [result.new_messages_json()]}
# Build workflow
builder = StateGraph(AgentState)
# Add nodes
builder.add_node("define_scope_with_reasoner", define_scope_with_reasoner)
builder.add_node("coder_agent", coder_agent)
builder.add_node("get_next_user_message", get_next_user_message)
builder.add_node("finish_conversation", finish_conversation)
# Set edges
builder.add_edge(START, "define_scope_with_reasoner")
builder.add_edge("define_scope_with_reasoner", "coder_agent")
builder.add_edge("coder_agent", "get_next_user_message")
builder.add_conditional_edges(
"get_next_user_message",
route_user_message,
{"coder_agent": "coder_agent", "finish_conversation": "finish_conversation"}
)
builder.add_edge("finish_conversation", END)
# Configure persistence
memory = MemorySaver()
agentic_flow = builder.compile(checkpointer=memory)

258
original_archon/iterations/v2-agentic-workflow/crawl_pydantic_ai_docs.py
View File

@@ -1,258 +0,0 @@
import os
import sys
import json
import asyncio
import requests
from xml.etree import ElementTree
from typing import List, Dict, Any
from dataclasses import dataclass
from datetime import datetime, timezone
from urllib.parse import urlparse
from dotenv import load_dotenv
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
from openai import AsyncOpenAI
from supabase import create_client, Client
load_dotenv()
# Initialize OpenAI and Supabase clients
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-llm-api-key-provided')
is_ollama = "localhost" in base_url.lower()
embedding_model = os.getenv('EMBEDDING_MODEL', 'text-embedding-3-small')
openai_client=None
if is_ollama:
openai_client = AsyncOpenAI(base_url=base_url,api_key=api_key)
else:
openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
supabase: Client = create_client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_SERVICE_KEY")
)
@dataclass
class ProcessedChunk:
url: str
chunk_number: int
title: str
summary: str
content: str
metadata: Dict[str, Any]
embedding: List[float]
def chunk_text(text: str, chunk_size: int = 5000) -> List[str]:
"""Split text into chunks, respecting code blocks and paragraphs."""
chunks = []
start = 0
text_length = len(text)
while start < text_length:
# Calculate end position
end = start + chunk_size
# If we're at the end of the text, just take what's left
if end >= text_length:
chunks.append(text[start:].strip())
break
# Try to find a code block boundary first (```)
chunk = text[start:end]
code_block = chunk.rfind('```')
if code_block != -1 and code_block > chunk_size * 0.3:
end = start + code_block
# If no code block, try to break at a paragraph
elif '\n\n' in chunk:
# Find the last paragraph break
last_break = chunk.rfind('\n\n')
if last_break > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_break
# If no paragraph break, try to break at a sentence
elif '. ' in chunk:
# Find the last sentence break
last_period = chunk.rfind('. ')
if last_period > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_period + 1
# Extract chunk and clean it up
chunk = text[start:end].strip()
if chunk:
chunks.append(chunk)
# Move start position for next chunk
start = max(start + 1, end)
return chunks
async def get_title_and_summary(chunk: str, url: str) -> Dict[str, str]:
"""Extract title and summary using GPT-4."""
system_prompt = """You are an AI that extracts titles and summaries from documentation chunks.
Return a JSON object with 'title' and 'summary' keys.
For the title: If this seems like the start of a document, extract its title. If it's a middle chunk, derive a descriptive title.
For the summary: Create a concise summary of the main points in this chunk.
Keep both title and summary concise but informative."""
try:
response = await openai_client.chat.completions.create(
model=os.getenv("PRIMARY_MODEL", "gpt-4o-mini"),
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"URL: {url}\n\nContent:\n{chunk[:1000]}..."} # Send first 1000 chars for context
],
response_format={ "type": "json_object" }
)
return json.loads(response.choices[0].message.content)
except Exception as e:
print(f"Error getting title and summary: {e}")
return {"title": "Error processing title", "summary": "Error processing summary"}
async def get_embedding(text: str) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await openai_client.embeddings.create(
model= embedding_model,
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
async def process_chunk(chunk: str, chunk_number: int, url: str) -> ProcessedChunk:
"""Process a single chunk of text."""
# Get title and summary
extracted = await get_title_and_summary(chunk, url)
# Get embedding
embedding = await get_embedding(chunk)
# Create metadata
metadata = {
"source": "pydantic_ai_docs",
"chunk_size": len(chunk),
"crawled_at": datetime.now(timezone.utc).isoformat(),
"url_path": urlparse(url).path
}
return ProcessedChunk(
url=url,
chunk_number=chunk_number,
title=extracted['title'],
summary=extracted['summary'],
content=chunk, # Store the original chunk content
metadata=metadata,
embedding=embedding
)
async def insert_chunk(chunk: ProcessedChunk):
"""Insert a processed chunk into Supabase."""
try:
data = {
"url": chunk.url,
"chunk_number": chunk.chunk_number,
"title": chunk.title,
"summary": chunk.summary,
"content": chunk.content,
"metadata": chunk.metadata,
"embedding": chunk.embedding
}
result = supabase.table("site_pages").insert(data).execute()
print(f"Inserted chunk {chunk.chunk_number} for {chunk.url}")
return result
except Exception as e:
print(f"Error inserting chunk: {e}")
return None
async def process_and_store_document(url: str, markdown: str):
"""Process a document and store its chunks in parallel."""
# Split into chunks
chunks = chunk_text(markdown)
# Process chunks in parallel
tasks = [
process_chunk(chunk, i, url)
for i, chunk in enumerate(chunks)
]
processed_chunks = await asyncio.gather(*tasks)
# Store chunks in parallel
insert_tasks = [
insert_chunk(chunk)
for chunk in processed_chunks
]
await asyncio.gather(*insert_tasks)
async def crawl_parallel(urls: List[str], max_concurrent: int = 5):
"""Crawl multiple URLs in parallel with a concurrency limit."""
browser_config = BrowserConfig(
headless=True,
verbose=False,
extra_args=["--disable-gpu", "--disable-dev-shm-usage", "--no-sandbox"],
)
crawl_config = CrawlerRunConfig(cache_mode=CacheMode.BYPASS)
# Create the crawler instance
crawler = AsyncWebCrawler(config=browser_config)
await crawler.start()
try:
# Create a semaphore to limit concurrency
semaphore = asyncio.Semaphore(max_concurrent)
async def process_url(url: str):
async with semaphore:
result = await crawler.arun(
url=url,
config=crawl_config,
session_id="session1"
)
if result.success:
print(f"Successfully crawled: {url}")
await process_and_store_document(url, result.markdown_v2.raw_markdown)
else:
print(f"Failed: {url} - Error: {result.error_message}")
# Process all URLs in parallel with limited concurrency
await asyncio.gather(*[process_url(url) for url in urls])
finally:
await crawler.close()
def get_pydantic_ai_docs_urls() -> List[str]:
"""Get URLs from Pydantic AI docs sitemap."""
sitemap_url = "https://ai.pydantic.dev/sitemap.xml"
try:
response = requests.get(sitemap_url)
response.raise_for_status()
# Parse the XML
root = ElementTree.fromstring(response.content)
# Extract all URLs from the sitemap
namespace = {'ns': 'http://www.sitemaps.org/schemas/sitemap/0.9'}
urls = [loc.text for loc in root.findall('.//ns:loc', namespace)]
return urls
except Exception as e:
print(f"Error fetching sitemap: {e}")
return []
async def main():
# Get URLs from Pydantic AI docs
urls = get_pydantic_ai_docs_urls()
if not urls:
print("No URLs found to crawl")
return
print(f"Found {len(urls)} URLs to crawl")
await crawl_parallel(urls)
if __name__ == "__main__":
asyncio.run(main())

7
original_archon/iterations/v2-agentic-workflow/langgraph.json
View File

@@ -1,7 +0,0 @@
{
"dependencies": ["."],
"graphs": {
"agent": "./archon_graph.py:agentic_flow"
},
"env": ".env"
}

72
original_archon/iterations/v2-agentic-workflow/ollama_site_pages.sql
View File

@@ -1,72 +0,0 @@
-- Enable the pgvector extension
create extension if not exists vector;
-- Create the documentation chunks table
create table site_pages (
id bigserial primary key,
url varchar not null,
chunk_number integer not null,
title varchar not null,
summary varchar not null,
content text not null, -- Added content column
metadata jsonb not null default '{}'::jsonb, -- Added metadata column
embedding vector(768), -- Ollama nomic-embed-text embeddings are 768 dimensions
created_at timestamp with time zone default timezone('utc'::text, now()) not null,
-- Add a unique constraint to prevent duplicate chunks for the same URL
unique(url, chunk_number)
);
-- Create an index for better vector similarity search performance
create index on site_pages using ivfflat (embedding vector_cosine_ops);
-- Create an index on metadata for faster filtering
create index idx_site_pages_metadata on site_pages using gin (metadata);
-- Create a function to search for documentation chunks
create function match_site_pages (
query_embedding vector(768),
match_count int default 10,
filter jsonb DEFAULT '{}'::jsonb
) returns table (
id bigint,
url varchar,
chunk_number integer,
title varchar,
summary varchar,
content text,
metadata jsonb,
similarity float
)
language plpgsql
as $$
#variable_conflict use_column
begin
return query
select
id,
url,
chunk_number,
title,
summary,
content,
metadata,
1 - (site_pages.embedding <=> query_embedding) as similarity
from site_pages
where metadata @> filter
order by site_pages.embedding <=> query_embedding
limit match_count;
end;
$$;
-- Everything above will work for any PostgreSQL database. The below commands are for Supabase security
-- Enable RLS on the table
alter table site_pages enable row level security;
-- Create a policy that allows anyone to read
create policy "Allow public read access"
on site_pages
for select
to public
using (true);

270
original_archon/iterations/v2-agentic-workflow/pydantic_ai_coder.py
View File

@@ -1,270 +0,0 @@
from __future__ import annotations as _annotations
from dataclasses import dataclass
from dotenv import load_dotenv
import logfire
import asyncio
import httpx
import os
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.models.openai import OpenAIModel
from openai import AsyncOpenAI
from supabase import Client
from typing import List
load_dotenv()
llm = os.getenv('PRIMARY_MODEL', 'gpt-4o-mini')
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-llm-api-key-provided')
model = OpenAIModel(llm, base_url=base_url, api_key=api_key)
embedding_model = os.getenv('EMBEDDING_MODEL', 'text-embedding-3-small')
logfire.configure(send_to_logfire='if-token-present')
is_ollama = "localhost" in base_url.lower()
@dataclass
class PydanticAIDeps:
supabase: Client
openai_client: AsyncOpenAI
reasoner_output: str
system_prompt = """
[ROLE AND CONTEXT]
You are a specialized AI agent engineer focused on building robust Pydantic AI agents. You have comprehensive access to the Pydantic AI documentation, including API references, usage guides, and implementation examples.
[CORE RESPONSIBILITIES]
1. Agent Development
- Create new agents from user requirements
- Complete partial agent implementations
- Optimize and debug existing agents
- Guide users through agent specification if needed
2. Documentation Integration
- Systematically search documentation using RAG before any implementation
- Cross-reference multiple documentation pages for comprehensive understanding
- Validate all implementations against current best practices
- Notify users if documentation is insufficient for any requirement
[CODE STRUCTURE AND DELIVERABLES]
All new agents must include these files with complete, production-ready code:
1. agent.py
- Primary agent definition and configuration
- Core agent logic and behaviors
- No tool implementations allowed here
2. agent_tools.py
- All tool function implementations
- Tool configurations and setup
- External service integrations
3. agent_prompts.py
- System prompts
- Task-specific prompts
- Conversation templates
- Instruction sets
4. .env.example
- Required environment variables
- Clear setup instructions in a comment above the variable for how to do so
- API configuration templates
5. requirements.txt
- Core dependencies without versions
- User-specified packages included
[DOCUMENTATION WORKFLOW]
1. Initial Research
- Begin with RAG search for relevant documentation
- List all documentation pages using list_documentation_pages
- Retrieve specific page content using get_page_content
- Cross-reference the weather agent example for best practices
2. Implementation
- Provide complete, working code implementations
- Never leave placeholder functions
- Include all necessary error handling
- Implement proper logging and monitoring
3. Quality Assurance
- Verify all tool implementations are complete
- Ensure proper separation of concerns
- Validate environment variable handling
- Test critical path functionality
[INTERACTION GUIDELINES]
- Take immediate action without asking for permission
- Always verify documentation before implementation
- Provide honest feedback about documentation gaps
- Include specific enhancement suggestions
- Request user feedback on implementations
- Maintain code consistency across files
[ERROR HANDLING]
- Implement robust error handling in all tools
- Provide clear error messages
- Include recovery mechanisms
- Log important state changes
[BEST PRACTICES]
- Follow Pydantic AI naming conventions
- Implement proper type hints
- Include comprehensive docstrings, the agent uses this to understand what tools are for.
- Maintain clean code structure
- Use consistent formatting
"""
pydantic_ai_coder = Agent(
model,
system_prompt=system_prompt,
deps_type=PydanticAIDeps,
retries=2
)
@pydantic_ai_coder.system_prompt
def add_reasoner_output(ctx: RunContext[str]) -> str:
return f"""
\n\nAdditional thoughts/instructions from the reasoner LLM.
This scope includes documentation pages for you to search as well:
{ctx.deps.reasoner_output}
"""
# Add this in to get some crazy tool calling:
# You must get ALL documentation pages listed in the scope.
async def get_embedding(text: str, openai_client: AsyncOpenAI) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await openai_client.embeddings.create(
model= embedding_model,
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
@pydantic_ai_coder.tool
async def retrieve_relevant_documentation(ctx: RunContext[PydanticAIDeps], user_query: str) -> str:
"""
Retrieve relevant documentation chunks based on the query with RAG.
Args:
ctx: The context including the Supabase client and OpenAI client
user_query: The user's question or query
Returns:
A formatted string containing the top 5 most relevant documentation chunks
"""
try:
# Get the embedding for the query
query_embedding = await get_embedding(user_query, ctx.deps.openai_client)
# Query Supabase for relevant documents
result = ctx.deps.supabase.rpc(
'match_site_pages',
{
'query_embedding': query_embedding,
'match_count': 5,
'filter': {'source': 'pydantic_ai_docs'}
}
).execute()
if not result.data:
return "No relevant documentation found."
# Format the results
formatted_chunks = []
for doc in result.data:
chunk_text = f"""
# {doc['title']}
{doc['content']}
"""
formatted_chunks.append(chunk_text)
# Join all chunks with a separator
return "\n\n---\n\n".join(formatted_chunks)
except Exception as e:
print(f"Error retrieving documentation: {e}")
return f"Error retrieving documentation: {str(e)}"
async def list_documentation_pages_helper(supabase: Client) -> List[str]:
"""
Function to retrieve a list of all available Pydantic AI documentation pages.
This is called by the list_documentation_pages tool and also externally
to fetch documentation pages for the reasoner LLM.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
try:
# Query Supabase for unique URLs where source is pydantic_ai_docs
result = supabase.from_('site_pages') \
.select('url') \
.eq('metadata->>source', 'pydantic_ai_docs') \
.execute()
if not result.data:
return []
# Extract unique URLs
urls = sorted(set(doc['url'] for doc in result.data))
return urls
except Exception as e:
print(f"Error retrieving documentation pages: {e}")
return []
@pydantic_ai_coder.tool
async def list_documentation_pages(ctx: RunContext[PydanticAIDeps]) -> List[str]:
"""
Retrieve a list of all available Pydantic AI documentation pages.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
return await list_documentation_pages_helper(ctx.deps.supabase)
@pydantic_ai_coder.tool
async def get_page_content(ctx: RunContext[PydanticAIDeps], url: str) -> str:
"""
Retrieve the full content of a specific documentation page by combining all its chunks.
Args:
ctx: The context including the Supabase client
url: The URL of the page to retrieve
Returns:
str: The complete page content with all chunks combined in order
"""
try:
# Query Supabase for all chunks of this URL, ordered by chunk_number
result = ctx.deps.supabase.from_('site_pages') \
.select('title, content, chunk_number') \
.eq('url', url) \
.eq('metadata->>source', 'pydantic_ai_docs') \
.order('chunk_number') \
.execute()
if not result.data:
return f"No content found for URL: {url}"
# Format the page with its title and all chunks
page_title = result.data[0]['title'].split(' - ')[0] # Get the main title
formatted_content = [f"# {page_title}\n"]
# Add each chunk's content
for chunk in result.data:
formatted_content.append(chunk['content'])
# Join everything together
return "\n\n".join(formatted_content)
except Exception as e:
print(f"Error retrieving page content: {e}")
return f"Error retrieving page content: {str(e)}"

BIN
original_archon/iterations/v2-agentic-workflow/requirements.txt
View File

Binary file not shown.

72
original_archon/iterations/v2-agentic-workflow/site_pages.sql
View File

@@ -1,72 +0,0 @@
-- Enable the pgvector extension
create extension if not exists vector;
-- Create the documentation chunks table
create table site_pages (
id bigserial primary key,
url varchar not null,
chunk_number integer not null,
title varchar not null,
summary varchar not null,
content text not null, -- Added content column
metadata jsonb not null default '{}'::jsonb, -- Added metadata column
embedding vector(1536), -- OpenAI embeddings are 1536 dimensions
created_at timestamp with time zone default timezone('utc'::text, now()) not null,
-- Add a unique constraint to prevent duplicate chunks for the same URL
unique(url, chunk_number)
);
-- Create an index for better vector similarity search performance
create index on site_pages using ivfflat (embedding vector_cosine_ops);
-- Create an index on metadata for faster filtering
create index idx_site_pages_metadata on site_pages using gin (metadata);
-- Create a function to search for documentation chunks
create function match_site_pages (
query_embedding vector(1536),
match_count int default 10,
filter jsonb DEFAULT '{}'::jsonb
) returns table (
id bigint,
url varchar,
chunk_number integer,
title varchar,
summary varchar,
content text,
metadata jsonb,
similarity float
)
language plpgsql
as $$
#variable_conflict use_column
begin
return query
select
id,
url,
chunk_number,
title,
summary,
content,
metadata,
1 - (site_pages.embedding <=> query_embedding) as similarity
from site_pages
where metadata @> filter
order by site_pages.embedding <=> query_embedding
limit match_count;
end;
$$;
-- Everything above will work for any PostgreSQL database. The below commands are for Supabase security
-- Enable RLS on the table
alter table site_pages enable row level security;
-- Create a policy that allows anyone to read
create policy "Allow public read access"
on site_pages
for select
to public
using (true);

124
original_archon/iterations/v2-agentic-workflow/streamlit_ui.py
View File

@@ -1,124 +0,0 @@
from __future__ import annotations
from typing import Literal, TypedDict
from langgraph.types import Command
from openai import AsyncOpenAI
from supabase import Client
import streamlit as st
import logfire
import asyncio
import json
import uuid
import os
# Import all the message part classes
from pydantic_ai.messages import (
ModelMessage,
ModelRequest,
ModelResponse,
SystemPromptPart,
UserPromptPart,
TextPart,
ToolCallPart,
ToolReturnPart,
RetryPromptPart,
ModelMessagesTypeAdapter
)
from archon_graph import agentic_flow
# Load environment variables
from dotenv import load_dotenv
load_dotenv()
openai_client=None
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-llm-api-key-provided')
is_ollama = "localhost" in base_url.lower()
if is_ollama:
openai_client = AsyncOpenAI(base_url=base_url,api_key=api_key)
else:
openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
supabase: Client = Client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_SERVICE_KEY")
)
# Configure logfire to suppress warnings (optional)
logfire.configure(send_to_logfire='never')
@st.cache_resource
def get_thread_id():
return str(uuid.uuid4())
thread_id = get_thread_id()
async def run_agent_with_streaming(user_input: str):
"""
Run the agent with streaming text for the user_input prompt,
while maintaining the entire conversation in `st.session_state.messages`.
"""
config = {
"configurable": {
"thread_id": thread_id
}
}
# First message from user
if len(st.session_state.messages) == 1:
async for msg in agentic_flow.astream(
{"latest_user_message": user_input}, config, stream_mode="custom"
):
yield msg
# Continue the conversation
else:
async for msg in agentic_flow.astream(
Command(resume=user_input), config, stream_mode="custom"
):
yield msg
async def main():
st.title("Archon - Agent Builder")
st.write("Describe to me an AI agent you want to build and I'll code it for you with Pydantic AI.")
st.write("Example: Build me an AI agent that can search the web with the Brave API.")
# Initialize chat history in session state if not present
if "messages" not in st.session_state:
st.session_state.messages = []
# Display chat messages from history on app rerun
for message in st.session_state.messages:
message_type = message["type"]
if message_type in ["human", "ai", "system"]:
with st.chat_message(message_type):
st.markdown(message["content"])
# Chat input for the user
user_input = st.chat_input("What do you want to build today?")
if user_input:
# We append a new request to the conversation explicitly
st.session_state.messages.append({"type": "human", "content": user_input})
# Display user prompt in the UI
with st.chat_message("user"):
st.markdown(user_input)
# Display assistant response in chat message container
response_content = ""
with st.chat_message("assistant"):
message_placeholder = st.empty() # Placeholder for updating the message
# Run the async generator to fetch responses
async for chunk in run_agent_with_streaming(user_input):
response_content += chunk
# Update the placeholder with the current response content
message_placeholder.markdown(response_content)
st.session_state.messages.append({"type": "ai", "content": response_content})
if __name__ == "__main__":
asyncio.run(main())

41
original_archon/iterations/v3-mcp-support/.env.example
View File

@@ -1,41 +0,0 @@
# Base URL for the OpenAI instance (default is https://api.openai.com/v1)
# OpenAI: https://api.openai.com/v1
# Ollama (example): http://localhost:11434/v1
# OpenRouter: https://openrouter.ai/api/v1
BASE_URL=
# For OpenAI: https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# For OpenRouter: https://openrouter.ai/keys
# For Ollama, no need to set this unless you specifically configured an API key
LLM_API_KEY=
# Get your Open AI API Key by following these instructions -
# https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# Even if using OpenRouter, you still need to set this for the embedding model.
# No need to set this if using Ollama.
OPENAI_API_KEY=
# For the Supabase version (sample_supabase_agent.py), set your Supabase URL and Service Key.
# Get your SUPABASE_URL from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
SUPABASE_URL=
# Get your SUPABASE_SERVICE_KEY from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
# On this page it is called the service_role secret.
SUPABASE_SERVICE_KEY=
# The LLM you want to use for the reasoner (o3-mini, R1, QwQ, etc.).
# Example: o3-mini
# Example: deepseek-r1:7b-8k
REASONER_MODEL=
# The LLM you want to use for the primary agent/coder.
# Example: gpt-4o-mini
# Example: qwen2.5:14b-instruct-8k
PRIMARY_MODEL=
# Embedding model you want to use
# Example for Ollama: nomic-embed-text
# Example for OpenAI: text-embedding-3-small
EMBEDDING_MODEL=

198
original_archon/iterations/v3-mcp-support/README.md
View File

@@ -1,198 +0,0 @@
# Archon V3 - AI Agent Generator with MCP Support
This is the third iteration of the Archon project, building upon V2 by adding MCP server support for seamless integration with AI IDEs like Windsurf and Cursor. The system starts with a reasoning LLM that analyzes user requirements and documentation to create a detailed scope, which then guides specialized coding and routing agents in generating high-quality Pydantic AI agents.
What makes V3 special is its ability to run as an MCP server, allowing AI IDEs to directly leverage Archon's agent generation capabilities. When you ask your AI IDE to create a new agent, Archon can not only generate the code but the IDE can automatically write it to the appropriate files, manage dependencies, and help you test the agent - creating a powerful synergy between agent generation and development environment.
The core remains an intelligent documentation crawler and RAG (Retrieval-Augmented Generation) system built using Pydantic AI, LangGraph, and Supabase. The system crawls the Pydantic AI documentation, stores content in a vector database, and provides Pydantic AI agent code by retrieving and analyzing relevant documentation chunks.
This version supports both local LLMs with Ollama and cloud-based LLMs through OpenAI/OpenRouter.
## Features
- MCP server support for AI IDE integration
- Multi-agent workflow using LangGraph
- Specialized agents for reasoning, routing, and coding
- Pydantic AI documentation crawling and chunking
- Vector database storage with Supabase
- Semantic search using OpenAI embeddings
- RAG-based question answering
- Support for code block preservation
- Streamlit UI for interactive querying
## Prerequisites
- Python 3.11+
- Supabase account and database
- OpenAI/OpenRouter API key or Ollama for local LLMs
- Streamlit (for web interface)
- Windsurf, Cursor, or another MCP-compatible AI IDE (optional)
## Installation
There are two ways to install Archon V3:
### Option 1: Standard Installation (for using the Streamlit UI)
1. Clone the repository:
```bash
git clone https://github.com/coleam00/archon.git
cd archon/iterations/v3-mcp-support
```
2. Install dependencies:
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```
### Option 2: MCP Server Setup (for AI IDE integration)
1. Clone the repository as above
2. Run the MCP setup script:
```bash
python setup_mcp.py
```
For running the crawler and graph service later, activate the virtual environment too:
```bash
source venv/bin/activate # On Windows: venv\Scripts\activate
```
This will:
- Create a virtual environment if it doesn't exist
- Install dependencies from requirements.txt
- Generate an MCP configuration file
3. Configure your AI IDE:
- **In Windsurf**:
- Click on the hammer icon above the chat input
- Click on "Configure"
- Paste the JSON that `setup_mcp.py` gave you as the MCP config
- Click "Refresh" next to "Configure"
- **In Cursor**:
- Go to Cursor Settings > Features > MCP
- Click on "+ Add New MCP Server"
- Name: Archon
- Type: command (equivalent to stdio)
- Command: Paste the command that `setup_mcp.py` gave for Cursor
NOTE that this MCP server will only be functional once you complete the steps below!
Be sure to restart your MCP server after finishing all steps.
## Environment Setup
1. Set up environment variables:
- Rename `.env.example` to `.env`
- Edit `.env` with your API keys and preferences:
```env
BASE_URL=https://api.openai.com/v1 for OpenAI, https://api.openrouter.ai/v1 for OpenRouter, or your Ollama URL
LLM_API_KEY=your_openai_or_openrouter_api_key
OPENAI_API_KEY=your_openai_api_key
SUPABASE_URL=your_supabase_url
SUPABASE_SERVICE_KEY=your_supabase_service_key
PRIMARY_MODEL=your_main_coding_llm
REASONER_MODEL=your_reasoning_llm
EMBEDDING_MODEL=your_embedding_model
```
## Usage
### Database Setup
Execute the SQL commands in `utils/site_pages.sql` to:
1. Create the necessary tables
2. Enable vector similarity search
3. Set up Row Level Security policies
In Supabase, do this by going to the "SQL Editor" tab and pasting in the SQL into the editor there. Then click "Run".
If using Ollama with the nomic-embed-text embedding model or another with 786 dimensions, either update site_pages.sql so that the dimensions are 768 instead of 1536 or use `utils/ollama_site_pages.sql`
### Crawl Documentation
To crawl and store documentation in the vector database:
```bash
python archon/crawl_pydantic_ai_docs.py
```
This will:
1. Fetch URLs from the documentation sitemap
2. Crawl each page and split into chunks
3. Generate embeddings and store in Supabase
### Using with AI IDEs (MCP Support)
1. After crawling the documentation, start the graph service:
```bash
python graph_service.py
```
Archon runs as a separate API endpoint for MCP instead of directly in the MCP server because that way Archon can be updated separately without having to restart the MCP server, and the communication protocols for MCP seemed to interfere with LLM calls when done directly within the MCP server.
2. Restart the MCP server in your AI IDE
3. You can now ask your AI IDE to create agents with Archon
4. Be sure to specify when you want to use Archon - not necessary but it helps a lot
### Using the Streamlit UI
For an interactive web interface:
```bash
streamlit run streamlit_ui.py
```
The interface will be available at `http://localhost:8501`
## Configuration
### Database Schema
The Supabase database uses the following schema:
```sql
CREATE TABLE site_pages (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
url TEXT,
chunk_number INTEGER,
title TEXT,
summary TEXT,
content TEXT,
metadata JSONB,
embedding VECTOR(1536) -- Adjust dimensions as necessary (i.e. 768 for nomic-embed-text)
);
```
## Project Structure
### Core Files
- `mcp_server.py`: MCP server script for AI IDE integration
- `graph_service.py`: FastAPI service that handles the agentic workflow
- `setup_mcp.py`: MCP setup script
- `streamlit_ui.py`: Web interface with streaming support
- `requirements.txt`: Project dependencies
- `.env.example`: Example environment variables
### Archon Package
- `archon/`: Core agent and workflow implementation
- `archon_graph.py`: LangGraph workflow definition and agent coordination
- `pydantic_ai_coder.py`: Main coding agent with RAG capabilities
- `crawl_pydantic_ai_docs.py`: Documentation crawler and processor
### Utilities
- `utils/`: Utility functions and database setup
- `utils.py`: Shared utility functions
- `site_pages.sql`: Database setup commands
- `site_pages_ollama.sql`: Database setup commands with vector dimensions updated for nomic-embed-text
### Runtime
- `workbench/`: Runtime files and logs
- `venv/`: Python virtual environment (created by setup)
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.

0
original_archon/iterations/v3-mcp-support/archon/__init__.py
View File

212
original_archon/iterations/v3-mcp-support/archon/archon_graph.py
View File

@@ -1,212 +0,0 @@
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai import Agent, RunContext
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver
from typing import TypedDict, Annotated, List, Any
from langgraph.config import get_stream_writer
from langgraph.types import interrupt
from dotenv import load_dotenv
from openai import AsyncOpenAI
from supabase import Client
import logfire
import os
import sys
# Import the message classes from Pydantic AI
from pydantic_ai.messages import (
ModelMessage,
ModelMessagesTypeAdapter
)
# Add the parent directory to Python path
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from archon.pydantic_ai_coder import pydantic_ai_coder, PydanticAIDeps, list_documentation_pages_helper
# Load environment variables
load_dotenv()
# Configure logfire to suppress warnings (optional)
logfire.configure(send_to_logfire='never')
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-llm-api-key-provided')
is_ollama = "localhost" in base_url.lower()
reasoner_llm_model = os.getenv('REASONER_MODEL', 'o3-mini')
reasoner = Agent(
OpenAIModel(reasoner_llm_model, base_url=base_url, api_key=api_key),
system_prompt='You are an expert at coding AI agents with Pydantic AI and defining the scope for doing so.',
)
primary_llm_model = os.getenv('PRIMARY_MODEL', 'gpt-4o-mini')
router_agent = Agent(
OpenAIModel(primary_llm_model, base_url=base_url, api_key=api_key),
system_prompt='Your job is to route the user message either to the end of the conversation or to continue coding the AI agent.',
)
end_conversation_agent = Agent(
OpenAIModel(primary_llm_model, base_url=base_url, api_key=api_key),
system_prompt='Your job is to end a conversation for creating an AI agent by giving instructions for how to execute the agent and they saying a nice goodbye to the user.',
)
openai_client=None
if is_ollama:
openai_client = AsyncOpenAI(base_url=base_url,api_key=api_key)
else:
openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
supabase: Client = Client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_SERVICE_KEY")
)
# Define state schema
class AgentState(TypedDict):
latest_user_message: str
messages: Annotated[List[bytes], lambda x, y: x + y]
scope: str
# Scope Definition Node with Reasoner LLM
async def define_scope_with_reasoner(state: AgentState):
# First, get the documentation pages so the reasoner can decide which ones are necessary
documentation_pages = await list_documentation_pages_helper(supabase)
documentation_pages_str = "\n".join(documentation_pages)
# Then, use the reasoner to define the scope
prompt = f"""
User AI Agent Request: {state['latest_user_message']}
Create detailed scope document for the AI agent including:
- Architecture diagram
- Core components
- External dependencies
- Testing strategy
Also based on these documentation pages available:
{documentation_pages_str}
Include a list of documentation pages that are relevant to creating this agent for the user in the scope document.
"""
result = await reasoner.run(prompt)
scope = result.data
# Get the directory one level up from the current file
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
scope_path = os.path.join(parent_dir, "workbench", "scope.md")
os.makedirs(os.path.join(parent_dir, "workbench"), exist_ok=True)
with open(scope_path, "w", encoding="utf-8") as f:
f.write(scope)
return {"scope": scope}
# Coding Node with Feedback Handling
async def coder_agent(state: AgentState, writer):
# Prepare dependencies
deps = PydanticAIDeps(
supabase=supabase,
openai_client=openai_client,
reasoner_output=state['scope']
)
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
# Run the agent in a stream
if is_ollama:
writer = get_stream_writer()
result = await pydantic_ai_coder.run(state['latest_user_message'], deps=deps, message_history= message_history)
writer(result.data)
else:
async with pydantic_ai_coder.run_stream(
state['latest_user_message'],
deps=deps,
message_history= message_history
) as result:
# Stream partial text as it arrives
async for chunk in result.stream_text(delta=True):
writer(chunk)
# print(ModelMessagesTypeAdapter.validate_json(result.new_messages_json()))
return {"messages": [result.new_messages_json()]}
# Interrupt the graph to get the user's next message
def get_next_user_message(state: AgentState):
value = interrupt({})
# Set the user's latest message for the LLM to continue the conversation
return {
"latest_user_message": value
}
# Determine if the user is finished creating their AI agent or not
async def route_user_message(state: AgentState):
prompt = f"""
The user has sent a message:
{state['latest_user_message']}
If the user wants to end the conversation, respond with just the text "finish_conversation".
If the user wants to continue coding the AI agent, respond with just the text "coder_agent".
"""
result = await router_agent.run(prompt)
next_action = result.data
if next_action == "finish_conversation":
return "finish_conversation"
else:
return "coder_agent"
# End of conversation agent to give instructions for executing the agent
async def finish_conversation(state: AgentState, writer):
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
# Run the agent in a stream
if is_ollama:
writer = get_stream_writer()
result = await end_conversation_agent.run(state['latest_user_message'], message_history= message_history)
writer(result.data)
else:
async with end_conversation_agent.run_stream(
state['latest_user_message'],
message_history= message_history
) as result:
# Stream partial text as it arrives
async for chunk in result.stream_text(delta=True):
writer(chunk)
return {"messages": [result.new_messages_json()]}
# Build workflow
builder = StateGraph(AgentState)
# Add nodes
builder.add_node("define_scope_with_reasoner", define_scope_with_reasoner)
builder.add_node("coder_agent", coder_agent)
builder.add_node("get_next_user_message", get_next_user_message)
builder.add_node("finish_conversation", finish_conversation)
# Set edges
builder.add_edge(START, "define_scope_with_reasoner")
builder.add_edge("define_scope_with_reasoner", "coder_agent")
builder.add_edge("coder_agent", "get_next_user_message")
builder.add_conditional_edges(
"get_next_user_message",
route_user_message,
{"coder_agent": "coder_agent", "finish_conversation": "finish_conversation"}
)
builder.add_edge("finish_conversation", END)
# Configure persistence
memory = MemorySaver()
agentic_flow = builder.compile(checkpointer=memory)

271
original_archon/iterations/v3-mcp-support/archon/crawl_pydantic_ai_docs.py
View File

@@ -1,271 +0,0 @@
import os
import sys
import json
import asyncio
import requests
from xml.etree import ElementTree
from typing import List, Dict, Any
from dataclasses import dataclass
from datetime import datetime, timezone
from urllib.parse import urlparse
from dotenv import load_dotenv
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
from openai import AsyncOpenAI
from supabase import create_client, Client
load_dotenv()
# Initialize OpenAI and Supabase clients
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-llm-api-key-provided')
is_ollama = "localhost" in base_url.lower()
embedding_model = os.getenv('EMBEDDING_MODEL', 'text-embedding-3-small')
openai_client=None
if is_ollama:
openai_client = AsyncOpenAI(base_url=base_url,api_key=api_key)
else:
openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
supabase: Client = create_client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_SERVICE_KEY")
)
@dataclass
class ProcessedChunk:
url: str
chunk_number: int
title: str
summary: str
content: str
metadata: Dict[str, Any]
embedding: List[float]
def chunk_text(text: str, chunk_size: int = 5000) -> List[str]:
"""Split text into chunks, respecting code blocks and paragraphs."""
chunks = []
start = 0
text_length = len(text)
while start < text_length:
# Calculate end position
end = start + chunk_size
# If we're at the end of the text, just take what's left
if end >= text_length:
chunks.append(text[start:].strip())
break
# Try to find a code block boundary first (```)
chunk = text[start:end]
code_block = chunk.rfind('```')
if code_block != -1 and code_block > chunk_size * 0.3:
end = start + code_block
# If no code block, try to break at a paragraph
elif '\n\n' in chunk:
# Find the last paragraph break
last_break = chunk.rfind('\n\n')
if last_break > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_break
# If no paragraph break, try to break at a sentence
elif '. ' in chunk:
# Find the last sentence break
last_period = chunk.rfind('. ')
if last_period > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_period + 1
# Extract chunk and clean it up
chunk = text[start:end].strip()
if chunk:
chunks.append(chunk)
# Move start position for next chunk
start = max(start + 1, end)
return chunks
async def get_title_and_summary(chunk: str, url: str) -> Dict[str, str]:
"""Extract title and summary using GPT-4."""
system_prompt = """You are an AI that extracts titles and summaries from documentation chunks.
Return a JSON object with 'title' and 'summary' keys.
For the title: If this seems like the start of a document, extract its title. If it's a middle chunk, derive a descriptive title.
For the summary: Create a concise summary of the main points in this chunk.
Keep both title and summary concise but informative."""
try:
response = await openai_client.chat.completions.create(
model=os.getenv("PRIMARY_MODEL", "gpt-4o-mini"),
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"URL: {url}\n\nContent:\n{chunk[:1000]}..."} # Send first 1000 chars for context
],
response_format={ "type": "json_object" }
)
return json.loads(response.choices[0].message.content)
except Exception as e:
print(f"Error getting title and summary: {e}")
return {"title": "Error processing title", "summary": "Error processing summary"}
async def get_embedding(text: str) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await openai_client.embeddings.create(
model= embedding_model,
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
async def process_chunk(chunk: str, chunk_number: int, url: str) -> ProcessedChunk:
"""Process a single chunk of text."""
# Get title and summary
extracted = await get_title_and_summary(chunk, url)
# Get embedding
embedding = await get_embedding(chunk)
# Create metadata
metadata = {
"source": "pydantic_ai_docs",
"chunk_size": len(chunk),
"crawled_at": datetime.now(timezone.utc).isoformat(),
"url_path": urlparse(url).path
}
return ProcessedChunk(
url=url,
chunk_number=chunk_number,
title=extracted['title'],
summary=extracted['summary'],
content=chunk, # Store the original chunk content
metadata=metadata,
embedding=embedding
)
async def insert_chunk(chunk: ProcessedChunk):
"""Insert a processed chunk into Supabase."""
try:
data = {
"url": chunk.url,
"chunk_number": chunk.chunk_number,
"title": chunk.title,
"summary": chunk.summary,
"content": chunk.content,
"metadata": chunk.metadata,
"embedding": chunk.embedding
}
result = supabase.table("site_pages").insert(data).execute()
print(f"Inserted chunk {chunk.chunk_number} for {chunk.url}")
return result
except Exception as e:
print(f"Error inserting chunk: {e}")
return None
async def process_and_store_document(url: str, markdown: str):
"""Process a document and store its chunks in parallel."""
# Split into chunks
chunks = chunk_text(markdown)
# Process chunks in parallel
tasks = [
process_chunk(chunk, i, url)
for i, chunk in enumerate(chunks)
]
processed_chunks = await asyncio.gather(*tasks)
# Store chunks in parallel
insert_tasks = [
insert_chunk(chunk)
for chunk in processed_chunks
]
await asyncio.gather(*insert_tasks)
async def crawl_parallel(urls: List[str], max_concurrent: int = 5):
"""Crawl multiple URLs in parallel with a concurrency limit."""
browser_config = BrowserConfig(
headless=True,
verbose=False,
extra_args=["--disable-gpu", "--disable-dev-shm-usage", "--no-sandbox"],
)
crawl_config = CrawlerRunConfig(cache_mode=CacheMode.BYPASS)
# Create the crawler instance
crawler = AsyncWebCrawler(config=browser_config)
await crawler.start()
try:
# Create a semaphore to limit concurrency
semaphore = asyncio.Semaphore(max_concurrent)
async def process_url(url: str):
async with semaphore:
result = await crawler.arun(
url=url,
config=crawl_config,
session_id="session1"
)
if result.success:
print(f"Successfully crawled: {url}")
await process_and_store_document(url, result.markdown_v2.raw_markdown)
else:
print(f"Failed: {url} - Error: {result.error_message}")
# Process all URLs in parallel with limited concurrency
await asyncio.gather(*[process_url(url) for url in urls])
finally:
await crawler.close()
def get_pydantic_ai_docs_urls() -> List[str]:
"""Get URLs from Pydantic AI docs sitemap."""
sitemap_url = "https://ai.pydantic.dev/sitemap.xml"
try:
response = requests.get(sitemap_url)
response.raise_for_status()
# Parse the XML
root = ElementTree.fromstring(response.content)
# Extract all URLs from the sitemap
namespace = {'ns': 'http://www.sitemaps.org/schemas/sitemap/0.9'}
urls = [loc.text for loc in root.findall('.//ns:loc', namespace)]
return urls
except Exception as e:
print(f"Error fetching sitemap: {e}")
return []
async def clear_existing_records():
"""Clear all existing records with source='pydantic_ai_docs' from the site_pages table."""
try:
result = supabase.table("site_pages").delete().eq("metadata->>source", "pydantic_ai_docs").execute()
print("Cleared existing pydantic_ai_docs records from site_pages")
return result
except Exception as e:
print(f"Error clearing existing records: {e}")
return None
async def main():
# Clear existing records first
await clear_existing_records()
# Get URLs from Pydantic AI docs
urls = get_pydantic_ai_docs_urls()
if not urls:
print("No URLs found to crawl")
return
print(f"Found {len(urls)} URLs to crawl")
await crawl_parallel(urls)
if __name__ == "__main__":
asyncio.run(main())

7
original_archon/iterations/v3-mcp-support/archon/langgraph.json
View File

@@ -1,7 +0,0 @@
{
"dependencies": ["."],
"graphs": {
"agent": "./archon_graph.py:agentic_flow"
},
"env": "../.env"
}

222
original_archon/iterations/v3-mcp-support/archon/pydantic_ai_coder.py
View File

@@ -1,222 +0,0 @@
from __future__ import annotations as _annotations
from dataclasses import dataclass
from dotenv import load_dotenv
import logfire
import asyncio
import httpx
import os
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.models.openai import OpenAIModel
from openai import AsyncOpenAI
from supabase import Client
from typing import List
load_dotenv()
llm = os.getenv('PRIMARY_MODEL', 'gpt-4o-mini')
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-llm-api-key-provided')
model = OpenAIModel(llm, base_url=base_url, api_key=api_key)
embedding_model = os.getenv('EMBEDDING_MODEL', 'text-embedding-3-small')
logfire.configure(send_to_logfire='if-token-present')
is_ollama = "localhost" in base_url.lower()
@dataclass
class PydanticAIDeps:
supabase: Client
openai_client: AsyncOpenAI
reasoner_output: str
system_prompt = """
~~ CONTEXT: ~~
You are an expert at Pydantic AI - a Python AI agent framework that you have access to all the documentation to,
including examples, an API reference, and other resources to help you build Pydantic AI agents.
~~ GOAL: ~~
Your only job is to help the user create an AI agent with Pydantic AI.
The user will describe the AI agent they want to build, or if they don't, guide them towards doing so.
You will take their requirements, and then search through the Pydantic AI documentation with the tools provided
to find all the necessary information to create the AI agent with correct code.
It's important for you to search through multiple Pydantic AI documentation pages to get all the information you need.
Almost never stick to just one page - use RAG and the other documentation tools multiple times when you are creating
an AI agent from scratch for the user.
~~ STRUCTURE: ~~
When you build an AI agent from scratch, split the agent into this files and give the code for each:
- `agent.py`: The main agent file, which is where the Pydantic AI agent is defined.
- `agent_tools.py`: A tools file for the agent, which is where all the tool functions are defined. Use this for more complex agents.
- `agent_prompts.py`: A prompts file for the agent, which includes all system prompts and other prompts used by the agent. Use this when there are many prompts or large ones.
- `.env.example`: An example `.env` file - specify each variable that the user will need to fill in and a quick comment above each one for how to do so.
- `requirements.txt`: Don't include any versions, just the top level package names needed for the agent.
~~ INSTRUCTIONS: ~~
- Don't ask the user before taking an action, just do it. Always make sure you look at the documentation with the provided tools before writing any code.
- When you first look at the documentation, always start with RAG.
Then also always check the list of available documentation pages and retrieve the content of page(s) if it'll help.
- Always let the user know when you didn't find the answer in the documentation or the right URL - be honest.
- Helpful tip: when starting a new AI agent build, it's a good idea to look at the 'weather agent' in the docs as an example.
- When starting a new AI agent build, always produce the full code for the AI agent - never tell the user to finish a tool/function.
- When refining an existing AI agent build in a conversation, just share the code changes necessary.
- Each time you respond to the user, ask them to let you know either if they need changes or the code looks good.
"""
pydantic_ai_coder = Agent(
model,
system_prompt=system_prompt,
deps_type=PydanticAIDeps,
retries=2
)
@pydantic_ai_coder.system_prompt
def add_reasoner_output(ctx: RunContext[str]) -> str:
return f"""
\n\nAdditional thoughts/instructions from the reasoner LLM.
This scope includes documentation pages for you to search as well:
{ctx.deps.reasoner_output}
"""
# Add this in to get some crazy tool calling:
# You must get ALL documentation pages listed in the scope.
async def get_embedding(text: str, openai_client: AsyncOpenAI) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await openai_client.embeddings.create(
model=embedding_model,
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
@pydantic_ai_coder.tool
async def retrieve_relevant_documentation(ctx: RunContext[PydanticAIDeps], user_query: str) -> str:
"""
Retrieve relevant documentation chunks based on the query with RAG.
Args:
ctx: The context including the Supabase client and OpenAI client
user_query: The user's question or query
Returns:
A formatted string containing the top 5 most relevant documentation chunks
"""
try:
# Get the embedding for the query
query_embedding = await get_embedding(user_query, ctx.deps.openai_client)
# Query Supabase for relevant documents
result = ctx.deps.supabase.rpc(
'match_site_pages',
{
'query_embedding': query_embedding,
'match_count': 5,
'filter': {'source': 'pydantic_ai_docs'}
}
).execute()
if not result.data:
return "No relevant documentation found."
# Format the results
formatted_chunks = []
for doc in result.data:
chunk_text = f"""
# {doc['title']}
{doc['content']}
"""
formatted_chunks.append(chunk_text)
# Join all chunks with a separator
return "\n\n---\n\n".join(formatted_chunks)
except Exception as e:
print(f"Error retrieving documentation: {e}")
return f"Error retrieving documentation: {str(e)}"
async def list_documentation_pages_helper(supabase: Client) -> List[str]:
"""
Function to retrieve a list of all available Pydantic AI documentation pages.
This is called by the list_documentation_pages tool and also externally
to fetch documentation pages for the reasoner LLM.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
try:
# Query Supabase for unique URLs where source is pydantic_ai_docs
result = supabase.from_('site_pages') \
.select('url') \
.eq('metadata->>source', 'pydantic_ai_docs') \
.execute()
if not result.data:
return []
# Extract unique URLs
urls = sorted(set(doc['url'] for doc in result.data))
return urls
except Exception as e:
print(f"Error retrieving documentation pages: {e}")
return []
@pydantic_ai_coder.tool
async def list_documentation_pages(ctx: RunContext[PydanticAIDeps]) -> List[str]:
"""
Retrieve a list of all available Pydantic AI documentation pages.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
return await list_documentation_pages_helper(ctx.deps.supabase)
@pydantic_ai_coder.tool
async def get_page_content(ctx: RunContext[PydanticAIDeps], url: str) -> str:
"""
Retrieve the full content of a specific documentation page by combining all its chunks.
Args:
ctx: The context including the Supabase client
url: The URL of the page to retrieve
Returns:
str: The complete page content with all chunks combined in order
"""
try:
# Query Supabase for all chunks of this URL, ordered by chunk_number
result = ctx.deps.supabase.from_('site_pages') \
.select('title, content, chunk_number') \
.eq('url', url) \
.eq('metadata->>source', 'pydantic_ai_docs') \
.order('chunk_number') \
.execute()
if not result.data:
return f"No content found for URL: {url}"
# Format the page with its title and all chunks
page_title = result.data[0]['title'].split(' - ')[0] # Get the main title
formatted_content = [f"# {page_title}\n"]
# Add each chunk's content
for chunk in result.data:
formatted_content.append(chunk['content'])
# Join everything together
return "\n\n".join(formatted_content)
except Exception as e:
print(f"Error retrieving page content: {e}")
return f"Error retrieving page content: {str(e)}"

69
original_archon/iterations/v3-mcp-support/graph_service.py
View File

@@ -1,69 +0,0 @@
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import Optional, Dict, Any
from archon.archon_graph import agentic_flow
from langgraph.types import Command
from utils.utils import write_to_log
app = FastAPI()
class InvokeRequest(BaseModel):
message: str
thread_id: str
is_first_message: bool = False
config: Optional[Dict[str, Any]] = None
@app.get("/health")
async def health_check():
"""Health check endpoint"""
return {"status": "ok"}
@app.post("/invoke")
async def invoke_agent(request: InvokeRequest):
"""Process a message through the agentic flow and return the complete response.
The agent streams the response but this API endpoint waits for the full output
before returning so it's a synchronous operation for MCP.
Another endpoint will be made later to fully stream the response from the API.
Args:
request: The InvokeRequest containing message and thread info
Returns:
dict: Contains the complete response from the agent
"""
try:
config = request.config or {
"configurable": {
"thread_id": request.thread_id
}
}
response = ""
if request.is_first_message:
write_to_log(f"Processing first message for thread {request.thread_id}")
async for msg in agentic_flow.astream(
{"latest_user_message": request.message},
config,
stream_mode="custom"
):
response += str(msg)
else:
write_to_log(f"Processing continuation for thread {request.thread_id}")
async for msg in agentic_flow.astream(
Command(resume=request.message),
config,
stream_mode="custom"
):
response += str(msg)
write_to_log(f"Final response for thread {request.thread_id}: {response}")
return {"response": response}
except Exception as e:
write_to_log(f"Error processing message for thread {request.thread_id}: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="127.0.0.1", port=8100)

10
original_archon/iterations/v3-mcp-support/mcp-config.json
View File

@@ -1,10 +0,0 @@
{
"mcpServers": {
"archon": {
"command": "[path to Archon]\\archon\\venv\\Scripts\\python.exe",
"args": [
"[path to Archon]\\archon\\mcp_server.py"
]
}
}
}

95
original_archon/iterations/v3-mcp-support/mcp_server.py
View File

@@ -1,95 +0,0 @@
import os
import sys
import asyncio
import threading
from mcp.server.fastmcp import FastMCP
import requests
from typing import Dict, List
import uuid
from utils.utils import write_to_log
from graph_service import app
import uvicorn
# Initialize FastMCP server
mcp = FastMCP("archon")
# Store active threads
active_threads: Dict[str, List[str]] = {}
# FastAPI service URL
GRAPH_SERVICE_URL = "http://127.0.0.1:8100"
@mcp.tool()
async def create_thread() -> str:
"""Create a new conversation thread for Archon.
Always call this tool before invoking Archon for the first time in a conversation.
(if you don't already have a thread ID)
Returns:
str: A unique thread ID for the conversation
"""
thread_id = str(uuid.uuid4())
active_threads[thread_id] = []
write_to_log(f"Created new thread: {thread_id}")
return thread_id
def _make_request(thread_id: str, user_input: str, config: dict) -> str:
"""Make synchronous request to graph service"""
response = requests.post(
f"{GRAPH_SERVICE_URL}/invoke",
json={
"message": user_input,
"thread_id": thread_id,
"is_first_message": not active_threads[thread_id],
"config": config
}
)
response.raise_for_status()
return response.json()
@mcp.tool()
async def run_agent(thread_id: str, user_input: str) -> str:
"""Run the Archon agent with user input.
Only use this tool after you have called create_thread in this conversation to get a unique thread ID.
If you already created a thread ID in this conversation, do not create another one. Reuse the same ID.
After you receive the code from Archon, always implement it into the codebase unless asked not to.
Args:
thread_id: The conversation thread ID
user_input: The user's message to process
Returns:
str: The agent's response which generally includes the code for the agent
"""
if thread_id not in active_threads:
write_to_log(f"Error: Thread not found - {thread_id}")
raise ValueError("Thread not found")
write_to_log(f"Processing message for thread {thread_id}: {user_input}")
config = {
"configurable": {
"thread_id": thread_id
}
}
try:
result = await asyncio.to_thread(_make_request, thread_id, user_input, config)
active_threads[thread_id].append(user_input)
return result['response']
except Exception as e:
raise
if __name__ == "__main__":
write_to_log("Starting MCP server")
# Run MCP server
mcp.run(transport='stdio')

176
original_archon/iterations/v3-mcp-support/requirements.txt
View File

@@ -1,176 +0,0 @@
aiofiles==24.1.0
aiohappyeyeballs==2.4.4
aiohttp==3.11.11
aiosignal==1.3.2
aiosqlite==0.20.0
altair==5.5.0
annotated-types==0.7.0
anthropic==0.42.0
anyio==4.8.0
attrs==24.3.0
beautifulsoup4==4.12.3
blinker==1.9.0
cachetools==5.5.0
certifi==2024.12.14
cffi==1.17.1
charset-normalizer==3.4.1
click==8.1.8
cohere==5.13.12
colorama==0.4.6
Crawl4AI==0.4.247
cryptography==43.0.3
Deprecated==1.2.15
deprecation==2.1.0
distro==1.9.0
dnspython==2.7.0
email_validator==2.2.0
eval_type_backport==0.2.2
executing==2.1.0
fake-http-header==0.3.5
fastapi==0.115.8
fastapi-cli==0.0.7
fastavro==1.10.0
filelock==3.16.1
frozenlist==1.5.0
fsspec==2024.12.0
gitdb==4.0.12
GitPython==3.1.44
google-auth==2.37.0
googleapis-common-protos==1.66.0
gotrue==2.11.1
greenlet==3.1.1
griffe==1.5.4
groq==0.15.0
h11==0.14.0
h2==4.1.0
hpack==4.0.0
httpcore==1.0.7
httptools==0.6.4
httpx==0.27.2
httpx-sse==0.4.0
huggingface-hub==0.27.1
hyperframe==6.0.1
idna==3.10
importlib_metadata==8.5.0
iniconfig==2.0.0
itsdangerous==2.2.0
Jinja2==3.1.5
jiter==0.8.2
joblib==1.4.2
jsonpatch==1.33
jsonpath-python==1.0.6
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
jsonschema_rs==0.25.1
langchain-core==0.3.33
langgraph==0.2.69
langgraph-api==0.0.22
langgraph-checkpoint==2.0.10
langgraph-cli==0.1.71
langgraph-sdk==0.1.51
langsmith==0.3.6
litellm==1.57.8
logfire==3.1.0
logfire-api==3.1.0
lxml==5.3.0
markdown-it-py==3.0.0
MarkupSafe==3.0.2
mcp==1.2.1
mdurl==0.1.2
mistralai==1.2.6
mockito==1.5.3
msgpack==1.1.0
multidict==6.1.0
mypy-extensions==1.0.0
narwhals==1.21.1
nltk==3.9.1
numpy==2.2.1
openai==1.59.6
opentelemetry-api==1.29.0
opentelemetry-exporter-otlp-proto-common==1.29.0
opentelemetry-exporter-otlp-proto-http==1.29.0
opentelemetry-instrumentation==0.50b0
opentelemetry-proto==1.29.0
opentelemetry-sdk==1.29.0
opentelemetry-semantic-conventions==0.50b0
orjson==3.10.15
packaging==24.2
pandas==2.2.3
pillow==10.4.0
playwright==1.49.1
pluggy==1.5.0
postgrest==0.19.1
propcache==0.2.1
protobuf==5.29.3
psutil==6.1.1
pyarrow==18.1.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycparser==2.22
pydantic==2.10.5
pydantic-ai==0.0.22
pydantic-ai-slim==0.0.22
pydantic-extra-types==2.10.2
pydantic-graph==0.0.22
pydantic-settings==2.7.1
pydantic_core==2.27.2
pydeck==0.9.1
pyee==12.0.0
Pygments==2.19.1
PyJWT==2.10.1
pyOpenSSL==24.3.0
pytest==8.3.4
pytest-mockito==0.0.4
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-multipart==0.0.20
pytz==2024.2
PyYAML==6.0.2
rank-bm25==0.2.2
realtime==2.1.0
referencing==0.35.1
regex==2024.11.6
requests==2.32.3
requests-toolbelt==1.0.0
rich==13.9.4
rich-toolkit==0.13.2
rpds-py==0.22.3
rsa==4.9
shellingham==1.5.4
six==1.17.0
smmap==5.0.2
sniffio==1.3.1
snowballstemmer==2.2.0
soupsieve==2.6
sse-starlette==2.1.3
starlette==0.45.3
storage3==0.11.0
streamlit==1.41.1
StrEnum==0.4.15
structlog==24.4.0
supabase==2.11.0
supafunc==0.9.0
tenacity==9.0.0
tf-playwright-stealth==1.1.0
tiktoken==0.8.0
tokenizers==0.21.0
toml==0.10.2
tornado==6.4.2
tqdm==4.67.1
typer==0.15.1
types-requests==2.32.0.20241016
typing-inspect==0.9.0
typing_extensions==4.12.2
tzdata==2024.2
ujson==5.10.0
urllib3==2.3.0
uvicorn==0.34.0
watchdog==6.0.0
watchfiles==1.0.4
websockets==13.1
wrapt==1.17.1
xxhash==3.5.0
yarl==1.18.3
zipp==3.21.0
zstandard==0.23.0

60
original_archon/iterations/v3-mcp-support/setup_mcp.py
View File

@@ -1,60 +0,0 @@
import os
import json
import subprocess
import sys
def setup_venv():
# Get the absolute path to the current directory
base_path = os.path.abspath(os.path.dirname(__file__))
venv_path = os.path.join(base_path, 'venv')
venv_created = False
# Create virtual environment if it doesn't exist
if not os.path.exists(venv_path):
print("Creating virtual environment...")
subprocess.run([sys.executable, '-m', 'venv', venv_path], check=True)
print("Virtual environment created successfully!")
venv_created = True
else:
print("Virtual environment already exists.")
# Install requirements if we just created the venv
if venv_created:
print("\nInstalling requirements...")
# Use the venv's pip to install requirements
pip_path = os.path.join(venv_path, 'Scripts', 'pip.exe')
requirements_path = os.path.join(base_path, 'requirements.txt')
subprocess.run([pip_path, 'install', '-r', requirements_path], check=True)
print("Requirements installed successfully!")
def generate_mcp_config():
# Get the absolute path to the current directory
base_path = os.path.abspath(os.path.dirname(__file__))
# Construct the paths
python_path = os.path.join(base_path, 'venv', 'Scripts', 'python.exe')
server_script_path = os.path.join(base_path, 'mcp_server.py')
# Create the config dictionary
config = {
"mcpServers": {
"archon": {
"command": python_path,
"args": [server_script_path]
}
}
}
# Write the config to a file
config_path = os.path.join(base_path, 'mcp-config.json')
with open(config_path, 'w') as f:
json.dump(config, f, indent=2)
print(f"\nMCP configuration has been written to: {config_path}")
print(f"\nMCP configuration for Cursor:\n\n{python_path} {server_script_path}")
print("\nMCP configuration for Windsurf/Claude Desktop:")
print(json.dumps(config, indent=2))
if __name__ == '__main__':
setup_venv()
generate_mcp_config()

127
original_archon/iterations/v3-mcp-support/streamlit_ui.py
View File

@@ -1,127 +0,0 @@
from __future__ import annotations
from typing import Literal, TypedDict
from langgraph.types import Command
from openai import AsyncOpenAI
from supabase import Client
import streamlit as st
import logfire
import asyncio
import json
import uuid
import os
import sys
# Import all the message part classes
from pydantic_ai.messages import (
ModelMessage,
ModelRequest,
ModelResponse,
SystemPromptPart,
UserPromptPart,
TextPart,
ToolCallPart,
ToolReturnPart,
RetryPromptPart,
ModelMessagesTypeAdapter
)
# Add the current directory to Python path
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
from archon.archon_graph import agentic_flow
# Load environment variables
from dotenv import load_dotenv
load_dotenv()
openai_client=None
base_url = os.getenv('BASE_URL', 'https://api.openai.com/v1')
api_key = os.getenv('LLM_API_KEY', 'no-llm-api-key-provided')
is_ollama = "localhost" in base_url.lower()
if is_ollama:
openai_client = AsyncOpenAI(base_url=base_url,api_key=api_key)
else:
openai_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
supabase: Client = Client(
os.getenv("SUPABASE_URL"),
os.getenv("SUPABASE_SERVICE_KEY")
)
# Configure logfire to suppress warnings (optional)
logfire.configure(send_to_logfire='never')
@st.cache_resource
def get_thread_id():
return str(uuid.uuid4())
thread_id = get_thread_id()
async def run_agent_with_streaming(user_input: str):
"""
Run the agent with streaming text for the user_input prompt,
while maintaining the entire conversation in `st.session_state.messages`.
"""
config = {
"configurable": {
"thread_id": thread_id
}
}
# First message from user
if len(st.session_state.messages) == 1:
async for msg in agentic_flow.astream(
{"latest_user_message": user_input}, config, stream_mode="custom"
):
yield msg
# Continue the conversation
else:
async for msg in agentic_flow.astream(
Command(resume=user_input), config, stream_mode="custom"
):
yield msg
async def main():
st.title("Archon - Agent Builder")
st.write("Describe to me an AI agent you want to build and I'll code it for you with Pydantic AI.")
st.write("Example: Build me an AI agent that can search the web with the Brave API.")
# Initialize chat history in session state if not present
if "messages" not in st.session_state:
st.session_state.messages = []
# Display chat messages from history on app rerun
for message in st.session_state.messages:
message_type = message["type"]
if message_type in ["human", "ai", "system"]:
with st.chat_message(message_type):
st.markdown(message["content"])
# Chat input for the user
user_input = st.chat_input("What do you want to build today?")
if user_input:
# We append a new request to the conversation explicitly
st.session_state.messages.append({"type": "human", "content": user_input})
# Display user prompt in the UI
with st.chat_message("user"):
st.markdown(user_input)
# Display assistant response in chat message container
response_content = ""
with st.chat_message("assistant"):
message_placeholder = st.empty() # Placeholder for updating the message
# Run the async generator to fetch responses
async for chunk in run_agent_with_streaming(user_input):
response_content += chunk
# Update the placeholder with the current response content
message_placeholder.markdown(response_content)
st.session_state.messages.append({"type": "ai", "content": response_content})
if __name__ == "__main__":
asyncio.run(main())

72
original_archon/iterations/v3-mcp-support/utils/ollama_site_pages.sql
View File

@@ -1,72 +0,0 @@
-- Enable the pgvector extension
create extension if not exists vector;
-- Create the documentation chunks table
create table site_pages (
id bigserial primary key,
url varchar not null,
chunk_number integer not null,
title varchar not null,
summary varchar not null,
content text not null, -- Added content column
metadata jsonb not null default '{}'::jsonb, -- Added metadata column
embedding vector(768), -- Ollama nomic-embed-text embeddings are 768 dimensions
created_at timestamp with time zone default timezone('utc'::text, now()) not null,
-- Add a unique constraint to prevent duplicate chunks for the same URL
unique(url, chunk_number)
);
-- Create an index for better vector similarity search performance
create index on site_pages using ivfflat (embedding vector_cosine_ops);
-- Create an index on metadata for faster filtering
create index idx_site_pages_metadata on site_pages using gin (metadata);
-- Create a function to search for documentation chunks
create function match_site_pages (
query_embedding vector(768),
match_count int default 10,
filter jsonb DEFAULT '{}'::jsonb
) returns table (
id bigint,
url varchar,
chunk_number integer,
title varchar,
summary varchar,
content text,
metadata jsonb,
similarity float
)
language plpgsql
as $$
#variable_conflict use_column
begin
return query
select
id,
url,
chunk_number,
title,
summary,
content,
metadata,
1 - (site_pages.embedding <=> query_embedding) as similarity
from site_pages
where metadata @> filter
order by site_pages.embedding <=> query_embedding
limit match_count;
end;
$$;
-- Everything above will work for any PostgreSQL database. The below commands are for Supabase security
-- Enable RLS on the table
alter table site_pages enable row level security;
-- Create a policy that allows anyone to read
create policy "Allow public read access"
on site_pages
for select
to public
using (true);

72
original_archon/iterations/v3-mcp-support/utils/site_pages.sql
View File

@@ -1,72 +0,0 @@
-- Enable the pgvector extension
create extension if not exists vector;
-- Create the documentation chunks table
create table site_pages (
id bigserial primary key,
url varchar not null,
chunk_number integer not null,
title varchar not null,
summary varchar not null,
content text not null, -- Added content column
metadata jsonb not null default '{}'::jsonb, -- Added metadata column
embedding vector(1536), -- OpenAI embeddings are 1536 dimensions
created_at timestamp with time zone default timezone('utc'::text, now()) not null,
-- Add a unique constraint to prevent duplicate chunks for the same URL
unique(url, chunk_number)
);
-- Create an index for better vector similarity search performance
create index on site_pages using ivfflat (embedding vector_cosine_ops);
-- Create an index on metadata for faster filtering
create index idx_site_pages_metadata on site_pages using gin (metadata);
-- Create a function to search for documentation chunks
create function match_site_pages (
query_embedding vector(1536),
match_count int default 10,
filter jsonb DEFAULT '{}'::jsonb
) returns table (
id bigint,
url varchar,
chunk_number integer,
title varchar,
summary varchar,
content text,
metadata jsonb,
similarity float
)
language plpgsql
as $$
#variable_conflict use_column
begin
return query
select
id,
url,
chunk_number,
title,
summary,
content,
metadata,
1 - (site_pages.embedding <=> query_embedding) as similarity
from site_pages
where metadata @> filter
order by site_pages.embedding <=> query_embedding
limit match_count;
end;
$$;
-- Everything above will work for any PostgreSQL database. The below commands are for Supabase security
-- Enable RLS on the table
alter table site_pages enable row level security;
-- Create a policy that allows anyone to read
create policy "Allow public read access"
on site_pages
for select
to public
using (true);

42
original_archon/iterations/v3-mcp-support/utils/utils.py
View File

@@ -1,42 +0,0 @@
import os
from datetime import datetime
from functools import wraps
import inspect
def write_to_log(message: str):
"""Write a message to the logs.txt file in the workbench directory.
Args:
message: The message to log
"""
# Get the directory one level up from the current file
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
workbench_dir = os.path.join(parent_dir, "workbench")
log_path = os.path.join(workbench_dir, "logs.txt")
os.makedirs(workbench_dir, exist_ok=True)
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
log_entry = f"[{timestamp}] {message}\n"
with open(log_path, "a", encoding="utf-8") as f:
f.write(log_entry)
def log_node_execution(func):
"""Decorator to log the start and end of graph node execution.
Args:
func: The async function to wrap
"""
@wraps(func)
async def wrapper(*args, **kwargs):
func_name = func.__name__
write_to_log(f"Starting node: {func_name}")
try:
result = await func(*args, **kwargs)
write_to_log(f"Completed node: {func_name}")
return result
except Exception as e:
write_to_log(f"Error in node {func_name}: {str(e)}")
raise
return wrapper

38
original_archon/iterations/v4-streamlit-ui-overhaul/.dockerignore
View File

@@ -1,38 +0,0 @@
# Ignore specified folders
iterations/
venv/
.langgraph_api/
.github/
__pycache__/
.env
# Git related
.git/
.gitignore
.gitattributes
# Python cache
*.pyc
*.pyo
*.pyd
.Python
*.so
.pytest_cache/
# Environment files
.env.local
.env.development.local
.env.test.local
.env.production.local
# Logs
*.log
# IDE specific files
.idea/
.vscode/
*.swp
*.swo
# Keep the example env file for reference
!.env.example

43
original_archon/iterations/v4-streamlit-ui-overhaul/.env.example
View File

@@ -1,43 +0,0 @@
# Base URL for the OpenAI instance (default is https://api.openai.com/v1)
# OpenAI: https://api.openai.com/v1
# Ollama (example): http://localhost:11434/v1
# OpenRouter: https://openrouter.ai/api/v1
# Anthropic: https://api.anthropic.com/v1
BASE_URL=
# For OpenAI: https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# For Anthropic: https://console.anthropic.com/account/keys
# For OpenRouter: https://openrouter.ai/keys
# For Ollama, no need to set this unless you specifically configured an API key
LLM_API_KEY=
# Get your Open AI API Key by following these instructions -
# https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
# Even if using Anthropic or OpenRouter, you still need to set this for the embedding model.
# No need to set this if using Ollama.
OPENAI_API_KEY=
# For the Supabase version (sample_supabase_agent.py), set your Supabase URL and Service Key.
# Get your SUPABASE_URL from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
SUPABASE_URL=
# Get your SUPABASE_SERVICE_KEY from the API section of your Supabase project settings -
# https://supabase.com/dashboard/project/<your project ID>/settings/api
# On this page it is called the service_role secret.
SUPABASE_SERVICE_KEY=
# The LLM you want to use for the reasoner (o3-mini, R1, QwQ, etc.).
# Example: o3-mini
# Example: deepseek-r1:7b-8k
REASONER_MODEL=
# The LLM you want to use for the primary agent/coder.
# Example: gpt-4o-mini
# Example: qwen2.5:14b-instruct-8k
PRIMARY_MODEL=
# Embedding model you want to use
# Example for Ollama: nomic-embed-text
# Example for OpenAI: text-embedding-3-small
EMBEDDING_MODEL=

11
original_archon/iterations/v4-streamlit-ui-overhaul/.gitignore vendored
View File

@@ -1,11 +0,0 @@
# Folders
workbench
__pycache__
venv
.langgraph_api
# Files
.env
.env.temp
.env.test
env_vars.json

6
original_archon/iterations/v4-streamlit-ui-overhaul/.streamlit/config.toml
View File

@@ -1,6 +0,0 @@
[client]
showErrorDetails = "none"
[theme]
primaryColor = "#FF69B4"
base="dark"

28
original_archon/iterations/v4-streamlit-ui-overhaul/Dockerfile
View File

@@ -1,28 +0,0 @@
FROM python:3.12-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements first for better caching
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application
COPY . .
# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH=/app
# Expose port for Streamlit
EXPOSE 8501
# Expose port for the Archon Service (started within Streamlit)
EXPOSE 8100
# Set the entrypoint to run Streamlit directly
CMD ["streamlit", "run", "streamlit_ui.py", "--server.port=8501", "--server.address=0.0.0.0"]

186
original_archon/iterations/v4-streamlit-ui-overhaul/README.md
View File

@@ -1,186 +0,0 @@
# Archon V4 - Streamlit UI Overhaul (and Docker Support)
This is the fourth iteration of the Archon project, building upon V3 by adding a comprehensive Streamlit UI for managing all aspects of Archon and Docker support. The system retains the core LangGraph workflow and MCP support from V3, but now provides a unified interface for environment configuration, database setup, documentation crawling, agent service management, and MCP integration.
What makes V4 special is its guided setup process that walks users through each step of configuring and running Archon. The Streamlit UI eliminates the need for manual configuration of environment variables, database setup, and service management, making Archon much more accessible to users without extensive technical knowledge.
The core remains an intelligent documentation crawler and RAG (Retrieval-Augmented Generation) system built using Pydantic AI, LangGraph, and Supabase. The system crawls the Pydantic AI documentation, stores content in a vector database, and provides Pydantic AI agent code by retrieving and analyzing relevant documentation chunks.
This version continues to support both local LLMs with Ollama and cloud-based LLMs through OpenAI/OpenRouter.
## Key Features
- **Comprehensive Streamlit UI**: Unified interface for all Archon functionality
- **Docker Support**: Containerized deployment with automated build and run scripts
- **Guided Setup Process**: Step-by-step instructions for configuration
- **Environment Variable Management**: Configure all settings through the UI
- **Database Setup**: Automated creation of Supabase tables and indexes
- **Documentation Crawler**: Fetch and process documentation for RAG
- **Agent Service Management**: Start/stop the agent service from the UI
- **MCP Integration**: Configure and manage MCP for AI IDE integration
- **Multiple LLM Support**: OpenAI, OpenRouter, and local Ollama models
- **Multi-agent workflow using LangGraph**: Manage multiple agents simultaneously
## Prerequisites
- Docker (optional but preferred)
- Python 3.11+
- Supabase account (for vector database)
- OpenAI/OpenRouter/Anthropic API key or Ollama for local LLMs
## Installation
### Option 1: Docker (Recommended)
1. Clone the repository:
```bash
git clone https://github.com/coleam00/archon.git
cd archon/iterations/v4-streamlit-ui-overhaul
```
2. Run the Docker setup script:
```bash
# This will build both containers and start Archon
python run_docker.py
```
3. Access the Streamlit UI at http://localhost:8501.
> **Note:** `run_docker.py` will automatically:
> - Build the MCP server container
> - Build the main Archon container
> - Run Archon with the appropriate port mappings
> - Use environment variables from `.env` file if it exists
### Option 2: Local Python Installation
1. Clone the repository:
```bash
git clone https://github.com/coleam00/archon.git
cd archon/iterations/v4-streamlit-ui-overhaul
```
2. Install dependencies:
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```
3. Start the Streamlit UI:
```bash
streamlit run streamlit_ui.py
```
4. Access the Streamlit UI at http://localhost:8501.
### Streamlit UI Tabs
The Streamlit interface will guide you through each step with clear instructions and interactive elements.
There are a good amount of steps for the setup but it goes quick!
The Streamlit UI provides the following tabs:
1. **Intro**: Overview and guided setup process
2. **Environment**: Configure API keys and model settings
3. **Database**: Set up your Supabase vector database
4. **Documentation**: Crawl and index the Pydantic AI documentation
5. **Agent Service**: Start and monitor the agent service
6. **Chat**: Interact with Archon to create AI agents
7. **MCP**: Configure integration with AI IDEs
### Environment Configuration
The Environment tab allows you to set and manage all environment variables through the UI:
- Base URL for API endpoints
- API keys for LLM providers
- Supabase connection details
- Model selections for different agent roles
- Embedding model configuration
All settings are saved to an `env_vars.json` file, which is automatically loaded when Archon starts.
### Database Setup
The Database tab simplifies the process of setting up your Supabase database:
- Select embedding dimensions based on your model
- View SQL commands for table creation
- Get instructions for executing SQL in Supabase
- Clear existing data if needed
### Documentation Management
The Documentation tab provides an interface for crawling and managing documentation:
- Start and monitor the crawling process with progress tracking
- View logs of the crawling process
- Clear existing documentation
- View database statistics
### Agent Service Control
The Agent Service tab allows you to manage the agent service:
- Start, restart, and stop the service
- Monitor service output in real-time
- Clear output logs
- Auto-refresh for continuous monitoring
### MCP Configuration
The MCP tab simplifies the process of configuring MCP for AI IDEs:
- Select your IDE (Windsurf, Cursor, Cline, or Roo Code)
- Generate configuration commands or JSON
- Copy configuration to clipboard
- Get step-by-step instructions for your specific IDE
## Architecture
### Core Files
- `streamlit_ui.py`: Comprehensive web interface for managing all aspects of Archon
- `graph_service.py`: FastAPI service that handles the agentic workflow
- `run_docker.py`: Script to build and run Archon Docker containers
- `Dockerfile`: Container definition for the main Archon application
### MCP Integration
- `mcp/`: Model Context Protocol server implementation
- `mcp_server.py`: MCP server script for AI IDE integration
- `Dockerfile`: Container definition for the MCP server
### Archon Package
- `archon/`: Core agent and workflow implementation
- `archon_graph.py`: LangGraph workflow definition and agent coordination
- `pydantic_ai_coder.py`: Main coding agent with RAG capabilities
- `crawl_pydantic_ai_docs.py`: Documentation crawler and processor
### Utilities
- `utils/`: Utility functions and database setup
- `utils.py`: Shared utility functions
- `site_pages.sql`: Database setup commands
- `env_vars.json`: Environment variables defined in the UI are stored here (included in .gitignore, file is created automatically)
## Deployment Options
- **Docker Containers**: Run Archon in isolated containers with all dependencies included
- Main container: Runs the Streamlit UI and graph service
- MCP container: Provides MCP server functionality for AI IDEs
- **Local Python**: Run directly on your system with a Python virtual environment
### Docker Architecture
The Docker implementation consists of two containers:
1. **Main Archon Container**:
- Runs the Streamlit UI on port 8501
- Hosts the Graph Service on port 8100
- Built from the root Dockerfile
- Handles all agent functionality and user interactions
2. **MCP Container**:
- Implements the Model Context Protocol for AI IDE integration
- Built from the mcp/Dockerfile
- Communicates with the main container's Graph Service
- Provides a standardized interface for AI IDEs like Windsurf, Cursor, Cline and Roo Code
When running with Docker, the `run_docker.py` script automates building and starting both containers with the proper configuration.
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.

0
original_archon/iterations/v4-streamlit-ui-overhaul/archon/__init__.py
View File

227
original_archon/iterations/v4-streamlit-ui-overhaul/archon/archon_graph.py
View File

@@ -1,227 +0,0 @@
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai import Agent, RunContext
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver
from typing import TypedDict, Annotated, List, Any
from langgraph.config import get_stream_writer
from langgraph.types import interrupt
from dotenv import load_dotenv
from openai import AsyncOpenAI
from supabase import Client
import logfire
import os
import sys
# Import the message classes from Pydantic AI
from pydantic_ai.messages import (
ModelMessage,
ModelMessagesTypeAdapter
)
# Add the parent directory to Python path
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from archon.pydantic_ai_coder import pydantic_ai_coder, PydanticAIDeps, list_documentation_pages_helper
from utils.utils import get_env_var
# Load environment variables
load_dotenv()
# Configure logfire to suppress warnings (optional)
logfire.configure(send_to_logfire='never')
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
is_ollama = "localhost" in base_url.lower()
is_anthropic = "anthropic" in base_url.lower()
is_openai = "openai" in base_url.lower()
reasoner_llm_model_name = get_env_var('REASONER_MODEL') or 'o3-mini'
reasoner_llm_model = AnthropicModel(reasoner_llm_model_name, api_key=api_key) if is_anthropic else OpenAIModel(reasoner_llm_model_name, base_url=base_url, api_key=api_key)
reasoner = Agent(
reasoner_llm_model,
system_prompt='You are an expert at coding AI agents with Pydantic AI and defining the scope for doing so.',
)
primary_llm_model_name = get_env_var('PRIMARY_MODEL') or 'gpt-4o-mini'
primary_llm_model = AnthropicModel(primary_llm_model_name, api_key=api_key) if is_anthropic else OpenAIModel(primary_llm_model_name, base_url=base_url, api_key=api_key)
router_agent = Agent(
primary_llm_model,
system_prompt='Your job is to route the user message either to the end of the conversation or to continue coding the AI agent.',
)
end_conversation_agent = Agent(
primary_llm_model,
system_prompt='Your job is to end a conversation for creating an AI agent by giving instructions for how to execute the agent and they saying a nice goodbye to the user.',
)
openai_client=None
if is_ollama:
openai_client = AsyncOpenAI(base_url=base_url,api_key=api_key)
elif get_env_var("OPENAI_API_KEY"):
openai_client = AsyncOpenAI(api_key=get_env_var("OPENAI_API_KEY"))
else:
openai_client = None
if get_env_var("SUPABASE_URL"):
supabase: Client = Client(
get_env_var("SUPABASE_URL"),
get_env_var("SUPABASE_SERVICE_KEY")
)
else:
supabase = None
# Define state schema
class AgentState(TypedDict):
latest_user_message: str
messages: Annotated[List[bytes], lambda x, y: x + y]
scope: str
# Scope Definition Node with Reasoner LLM
async def define_scope_with_reasoner(state: AgentState):
# First, get the documentation pages so the reasoner can decide which ones are necessary
documentation_pages = await list_documentation_pages_helper(supabase)
documentation_pages_str = "\n".join(documentation_pages)
# Then, use the reasoner to define the scope
prompt = f"""
User AI Agent Request: {state['latest_user_message']}
Create detailed scope document for the AI agent including:
- Architecture diagram
- Core components
- External dependencies
- Testing strategy
Also based on these documentation pages available:
{documentation_pages_str}
Include a list of documentation pages that are relevant to creating this agent for the user in the scope document.
"""
result = await reasoner.run(prompt)
scope = result.data
# Get the directory one level up from the current file
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
scope_path = os.path.join(parent_dir, "workbench", "scope.md")
os.makedirs(os.path.join(parent_dir, "workbench"), exist_ok=True)
with open(scope_path, "w", encoding="utf-8") as f:
f.write(scope)
return {"scope": scope}
# Coding Node with Feedback Handling
async def coder_agent(state: AgentState, writer):
# Prepare dependencies
deps = PydanticAIDeps(
supabase=supabase,
openai_client=openai_client,
reasoner_output=state['scope']
)
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
# Run the agent in a stream
if not is_openai:
writer = get_stream_writer()
result = await pydantic_ai_coder.run(state['latest_user_message'], deps=deps, message_history= message_history)
writer(result.data)
else:
async with pydantic_ai_coder.run_stream(
state['latest_user_message'],
deps=deps,
message_history= message_history
) as result:
# Stream partial text as it arrives
async for chunk in result.stream_text(delta=True):
writer(chunk)
# print(ModelMessagesTypeAdapter.validate_json(result.new_messages_json()))
return {"messages": [result.new_messages_json()]}
# Interrupt the graph to get the user's next message
def get_next_user_message(state: AgentState):
value = interrupt({})
# Set the user's latest message for the LLM to continue the conversation
return {
"latest_user_message": value
}
# Determine if the user is finished creating their AI agent or not
async def route_user_message(state: AgentState):
prompt = f"""
The user has sent a message:
{state['latest_user_message']}
If the user wants to end the conversation, respond with just the text "finish_conversation".
If the user wants to continue coding the AI agent, respond with just the text "coder_agent".
"""
result = await router_agent.run(prompt)
next_action = result.data
if next_action == "finish_conversation":
return "finish_conversation"
else:
return "coder_agent"
# End of conversation agent to give instructions for executing the agent
async def finish_conversation(state: AgentState, writer):
# Get the message history into the format for Pydantic AI
message_history: list[ModelMessage] = []
for message_row in state['messages']:
message_history.extend(ModelMessagesTypeAdapter.validate_json(message_row))
# Run the agent in a stream
if not is_openai:
writer = get_stream_writer()
result = await end_conversation_agent.run(state['latest_user_message'], message_history= message_history)
writer(result.data)
else:
async with end_conversation_agent.run_stream(
state['latest_user_message'],
message_history= message_history
) as result:
# Stream partial text as it arrives
async for chunk in result.stream_text(delta=True):
writer(chunk)
return {"messages": [result.new_messages_json()]}
# Build workflow
builder = StateGraph(AgentState)
# Add nodes
builder.add_node("define_scope_with_reasoner", define_scope_with_reasoner)
builder.add_node("coder_agent", coder_agent)
builder.add_node("get_next_user_message", get_next_user_message)
builder.add_node("finish_conversation", finish_conversation)
# Set edges
builder.add_edge(START, "define_scope_with_reasoner")
builder.add_edge("define_scope_with_reasoner", "coder_agent")
builder.add_edge("coder_agent", "get_next_user_message")
builder.add_conditional_edges(
"get_next_user_message",
route_user_message,
{"coder_agent": "coder_agent", "finish_conversation": "finish_conversation"}
)
builder.add_edge("finish_conversation", END)
# Configure persistence
memory = MemorySaver()
agentic_flow = builder.compile(checkpointer=memory)

511
original_archon/iterations/v4-streamlit-ui-overhaul/archon/crawl_pydantic_ai_docs.py
View File

@@ -1,511 +0,0 @@
import os
import sys
import asyncio
import threading
import subprocess
import requests
import json
from typing import List, Dict, Any, Optional, Callable
from xml.etree import ElementTree
from dataclasses import dataclass
from datetime import datetime, timezone
from urllib.parse import urlparse
from dotenv import load_dotenv
import re
import html2text
# Add the parent directory to sys.path to allow importing from the parent directory
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from utils.utils import get_env_var
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
from openai import AsyncOpenAI
from supabase import create_client, Client
load_dotenv()
# Initialize OpenAI and Supabase clients
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
is_ollama = "localhost" in base_url.lower()
embedding_model = get_env_var('EMBEDDING_MODEL') or 'text-embedding-3-small'
openai_client=None
if is_ollama:
openai_client = AsyncOpenAI(base_url=base_url,api_key=api_key)
else:
openai_client = AsyncOpenAI(api_key=get_env_var("OPENAI_API_KEY"))
supabase: Client = create_client(
get_env_var("SUPABASE_URL"),
get_env_var("SUPABASE_SERVICE_KEY")
)
# Initialize HTML to Markdown converter
html_converter = html2text.HTML2Text()
html_converter.ignore_links = False
html_converter.ignore_images = False
html_converter.ignore_tables = False
html_converter.body_width = 0 # No wrapping
@dataclass
class ProcessedChunk:
url: str
chunk_number: int
title: str
summary: str
content: str
metadata: Dict[str, Any]
embedding: List[float]
class CrawlProgressTracker:
"""Class to track progress of the crawling process."""
def __init__(self,
progress_callback: Optional[Callable[[Dict[str, Any]], None]] = None):
"""Initialize the progress tracker.
Args:
progress_callback: Function to call with progress updates
"""
self.progress_callback = progress_callback
self.urls_found = 0
self.urls_processed = 0
self.urls_succeeded = 0
self.urls_failed = 0
self.chunks_stored = 0
self.logs = []
self.is_running = False
self.start_time = None
self.end_time = None
def log(self, message: str):
"""Add a log message and update progress."""
timestamp = datetime.now().strftime("%H:%M:%S")
log_entry = f"[{timestamp}] {message}"
self.logs.append(log_entry)
print(message) # Also print to console
# Call the progress callback if provided
if self.progress_callback:
self.progress_callback(self.get_status())
def start(self):
"""Mark the crawling process as started."""
self.is_running = True
self.start_time = datetime.now()
self.log("Crawling process started")
# Call the progress callback if provided
if self.progress_callback:
self.progress_callback(self.get_status())
def complete(self):
"""Mark the crawling process as completed."""
self.is_running = False
self.end_time = datetime.now()
duration = self.end_time - self.start_time if self.start_time else None
duration_str = str(duration).split('.')[0] if duration else "unknown"
self.log(f"Crawling process completed in {duration_str}")
# Call the progress callback if provided
if self.progress_callback:
self.progress_callback(self.get_status())
def get_status(self) -> Dict[str, Any]:
"""Get the current status of the crawling process."""
return {
"is_running": self.is_running,
"urls_found": self.urls_found,
"urls_processed": self.urls_processed,
"urls_succeeded": self.urls_succeeded,
"urls_failed": self.urls_failed,
"chunks_stored": self.chunks_stored,
"progress_percentage": (self.urls_processed / self.urls_found * 100) if self.urls_found > 0 else 0,
"logs": self.logs,
"start_time": self.start_time,
"end_time": self.end_time
}
@property
def is_completed(self) -> bool:
"""Return True if the crawling process is completed."""
return not self.is_running and self.end_time is not None
@property
def is_successful(self) -> bool:
"""Return True if the crawling process completed successfully."""
return self.is_completed and self.urls_failed == 0 and self.urls_succeeded > 0
def chunk_text(text: str, chunk_size: int = 5000) -> List[str]:
"""Split text into chunks, respecting code blocks and paragraphs."""
chunks = []
start = 0
text_length = len(text)
while start < text_length:
# Calculate end position
end = start + chunk_size
# If we're at the end of the text, just take what's left
if end >= text_length:
chunks.append(text[start:].strip())
break
# Try to find a code block boundary first (```)
chunk = text[start:end]
code_block = chunk.rfind('```')
if code_block != -1 and code_block > chunk_size * 0.3:
end = start + code_block
# If no code block, try to break at a paragraph
elif '\n\n' in chunk:
# Find the last paragraph break
last_break = chunk.rfind('\n\n')
if last_break > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_break
# If no paragraph break, try to break at a sentence
elif '. ' in chunk:
# Find the last sentence break
last_period = chunk.rfind('. ')
if last_period > chunk_size * 0.3: # Only break if we're past 30% of chunk_size
end = start + last_period + 1
# Extract chunk and clean it up
chunk = text[start:end].strip()
if chunk:
chunks.append(chunk)
# Move start position for next chunk
start = max(start + 1, end)
return chunks
async def get_title_and_summary(chunk: str, url: str) -> Dict[str, str]:
"""Extract title and summary using GPT-4."""
system_prompt = """You are an AI that extracts titles and summaries from documentation chunks.
Return a JSON object with 'title' and 'summary' keys.
For the title: If this seems like the start of a document, extract its title. If it's a middle chunk, derive a descriptive title.
For the summary: Create a concise summary of the main points in this chunk.
Keep both title and summary concise but informative."""
try:
response = await openai_client.chat.completions.create(
model=get_env_var("PRIMARY_MODEL") or "gpt-4o-mini",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": f"URL: {url}\n\nContent:\n{chunk[:1000]}..."} # Send first 1000 chars for context
],
response_format={ "type": "json_object" }
)
return json.loads(response.choices[0].message.content)
except Exception as e:
print(f"Error getting title and summary: {e}")
return {"title": "Error processing title", "summary": "Error processing summary"}
async def get_embedding(text: str) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await openai_client.embeddings.create(
model= embedding_model,
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
async def process_chunk(chunk: str, chunk_number: int, url: str) -> ProcessedChunk:
"""Process a single chunk of text."""
# Get title and summary
extracted = await get_title_and_summary(chunk, url)
# Get embedding
embedding = await get_embedding(chunk)
# Create metadata
metadata = {
"source": "pydantic_ai_docs",
"chunk_size": len(chunk),
"crawled_at": datetime.now(timezone.utc).isoformat(),
"url_path": urlparse(url).path
}
return ProcessedChunk(
url=url,
chunk_number=chunk_number,
title=extracted['title'],
summary=extracted['summary'],
content=chunk, # Store the original chunk content
metadata=metadata,
embedding=embedding
)
async def insert_chunk(chunk: ProcessedChunk):
"""Insert a processed chunk into Supabase."""
try:
data = {
"url": chunk.url,
"chunk_number": chunk.chunk_number,
"title": chunk.title,
"summary": chunk.summary,
"content": chunk.content,
"metadata": chunk.metadata,
"embedding": chunk.embedding
}
result = supabase.table("site_pages").insert(data).execute()
print(f"Inserted chunk {chunk.chunk_number} for {chunk.url}")
return result
except Exception as e:
print(f"Error inserting chunk: {e}")
return None
async def process_and_store_document(url: str, markdown: str, tracker: Optional[CrawlProgressTracker] = None):
"""Process a document and store its chunks in parallel."""
# Split into chunks
chunks = chunk_text(markdown)
if tracker:
tracker.log(f"Split document into {len(chunks)} chunks for {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Split document into {len(chunks)} chunks for {url}")
# Process chunks in parallel
tasks = [
process_chunk(chunk, i, url)
for i, chunk in enumerate(chunks)
]
processed_chunks = await asyncio.gather(*tasks)
if tracker:
tracker.log(f"Processed {len(processed_chunks)} chunks for {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Processed {len(processed_chunks)} chunks for {url}")
# Store chunks in parallel
insert_tasks = [
insert_chunk(chunk)
for chunk in processed_chunks
]
await asyncio.gather(*insert_tasks)
if tracker:
tracker.chunks_stored += len(processed_chunks)
tracker.log(f"Stored {len(processed_chunks)} chunks for {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Stored {len(processed_chunks)} chunks for {url}")
def fetch_url_content(url: str) -> str:
"""Fetch content from a URL using requests and convert to markdown."""
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
}
try:
response = requests.get(url, headers=headers, timeout=30)
response.raise_for_status()
# Convert HTML to Markdown
markdown = html_converter.handle(response.text)
# Clean up the markdown
markdown = re.sub(r'\n{3,}', '\n\n', markdown) # Remove excessive newlines
return markdown
except Exception as e:
raise Exception(f"Error fetching {url}: {str(e)}")
async def crawl_parallel_with_requests(urls: List[str], tracker: Optional[CrawlProgressTracker] = None, max_concurrent: int = 5):
"""Crawl multiple URLs in parallel with a concurrency limit using direct HTTP requests."""
# Create a semaphore to limit concurrency
semaphore = asyncio.Semaphore(max_concurrent)
async def process_url(url: str):
async with semaphore:
if tracker:
tracker.log(f"Crawling: {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Crawling: {url}")
try:
# Use a thread pool to run the blocking HTTP request
loop = asyncio.get_running_loop()
if tracker:
tracker.log(f"Fetching content from: {url}")
else:
print(f"Fetching content from: {url}")
markdown = await loop.run_in_executor(None, fetch_url_content, url)
if markdown:
if tracker:
tracker.urls_succeeded += 1
tracker.log(f"Successfully crawled: {url}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Successfully crawled: {url}")
await process_and_store_document(url, markdown, tracker)
else:
if tracker:
tracker.urls_failed += 1
tracker.log(f"Failed: {url} - No content retrieved")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Failed: {url} - No content retrieved")
except Exception as e:
if tracker:
tracker.urls_failed += 1
tracker.log(f"Error processing {url}: {str(e)}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Error processing {url}: {str(e)}")
finally:
if tracker:
tracker.urls_processed += 1
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
# Process all URLs in parallel with limited concurrency
if tracker:
tracker.log(f"Processing {len(urls)} URLs with concurrency {max_concurrent}")
# Ensure UI gets updated
if tracker.progress_callback:
tracker.progress_callback(tracker.get_status())
else:
print(f"Processing {len(urls)} URLs with concurrency {max_concurrent}")
await asyncio.gather(*[process_url(url) for url in urls])
def get_pydantic_ai_docs_urls() -> List[str]:
"""Get URLs from Pydantic AI docs sitemap."""
sitemap_url = "https://ai.pydantic.dev/sitemap.xml"
try:
response = requests.get(sitemap_url)
response.raise_for_status()
# Parse the XML
root = ElementTree.fromstring(response.content)
# Extract all URLs from the sitemap
namespace = {'ns': 'http://www.sitemaps.org/schemas/sitemap/0.9'}
urls = [loc.text for loc in root.findall('.//ns:loc', namespace)]
return urls
except Exception as e:
print(f"Error fetching sitemap: {e}")
return []
async def clear_existing_records():
"""Clear all existing records with source='pydantic_ai_docs' from the site_pages table."""
try:
result = supabase.table("site_pages").delete().eq("metadata->>source", "pydantic_ai_docs").execute()
print("Cleared existing pydantic_ai_docs records from site_pages")
return result
except Exception as e:
print(f"Error clearing existing records: {e}")
return None
async def main_with_requests(tracker: Optional[CrawlProgressTracker] = None):
"""Main function using direct HTTP requests instead of browser automation."""
try:
# Start tracking if tracker is provided
if tracker:
tracker.start()
else:
print("Starting crawling process...")
# Clear existing records first
if tracker:
tracker.log("Clearing existing Pydantic AI docs records...")
else:
print("Clearing existing Pydantic AI docs records...")
await clear_existing_records()
if tracker:
tracker.log("Existing records cleared")
else:
print("Existing records cleared")
# Get URLs from Pydantic AI docs
if tracker:
tracker.log("Fetching URLs from Pydantic AI sitemap...")
else:
print("Fetching URLs from Pydantic AI sitemap...")
urls = get_pydantic_ai_docs_urls()
if not urls:
if tracker:
tracker.log("No URLs found to crawl")
tracker.complete()
else:
print("No URLs found to crawl")
return
if tracker:
tracker.urls_found = len(urls)
tracker.log(f"Found {len(urls)} URLs to crawl")
else:
print(f"Found {len(urls)} URLs to crawl")
# Crawl the URLs using direct HTTP requests
await crawl_parallel_with_requests(urls, tracker)
# Mark as complete if tracker is provided
if tracker:
tracker.complete()
else:
print("Crawling process completed")
except Exception as e:
if tracker:
tracker.log(f"Error in crawling process: {str(e)}")
tracker.complete()
else:
print(f"Error in crawling process: {str(e)}")
def start_crawl_with_requests(progress_callback: Optional[Callable[[Dict[str, Any]], None]] = None) -> CrawlProgressTracker:
"""Start the crawling process using direct HTTP requests in a separate thread and return the tracker."""
tracker = CrawlProgressTracker(progress_callback)
def run_crawl():
try:
asyncio.run(main_with_requests(tracker))
except Exception as e:
print(f"Error in crawl thread: {e}")
tracker.log(f"Thread error: {str(e)}")
tracker.complete()
# Start the crawling process in a separate thread
thread = threading.Thread(target=run_crawl)
thread.daemon = True
thread.start()
return tracker
if __name__ == "__main__":
# Run the main function directly
print("Starting crawler...")
asyncio.run(main_with_requests())
print("Crawler finished.")

7
original_archon/iterations/v4-streamlit-ui-overhaul/archon/langgraph.json
View File

@@ -1,7 +0,0 @@
{
"dependencies": ["."],
"graphs": {
"agent": "./archon_graph.py:agentic_flow"
},
"env": "../.env"
}

408
original_archon/iterations/v4-streamlit-ui-overhaul/archon/pydantic_ai_coder.py
View File

@@ -1,408 +0,0 @@
from __future__ import annotations as _annotations
from dataclasses import dataclass
from dotenv import load_dotenv
import logfire
import asyncio
import httpx
import os
import sys
import json
from typing import Dict, Any, List, Optional
from pydantic import BaseModel
from pydantic_ai import Agent, ModelRetry, RunContext
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.models.openai import OpenAIModel
from openai import AsyncOpenAI
from supabase import Client
# Add the parent directory to sys.path to allow importing from the parent directory
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from utils.utils import get_env_var
load_dotenv()
llm = get_env_var('PRIMARY_MODEL') or 'gpt-4o-mini'
base_url = get_env_var('BASE_URL') or 'https://api.openai.com/v1'
api_key = get_env_var('LLM_API_KEY') or 'no-llm-api-key-provided'
is_ollama = "localhost" in base_url.lower()
is_anthropic = "anthropic" in base_url.lower()
model = AnthropicModel(llm, api_key=api_key) if is_anthropic else OpenAIModel(llm, base_url=base_url, api_key=api_key)
embedding_model = get_env_var('EMBEDDING_MODEL') or 'text-embedding-3-small'
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class PydanticAIDeps:
supabase: Client
openai_client: AsyncOpenAI
reasoner_output: str
system_prompt = """
[ROLE AND CONTEXT]
You are a specialized AI agent engineer focused on building robust Pydantic AI agents. You have comprehensive access to the Pydantic AI documentation, including API references, usage guides, and implementation examples.
[CORE RESPONSIBILITIES]
1. Agent Development
- Create new agents from user requirements
- Complete partial agent implementations
- Optimize and debug existing agents
- Guide users through agent specification if needed
2. Documentation Integration
- Systematically search documentation using RAG before any implementation
- Cross-reference multiple documentation pages for comprehensive understanding
- Validate all implementations against current best practices
- Notify users if documentation is insufficient for any requirement
[CODE STRUCTURE AND DELIVERABLES]
All new agents must include these files with complete, production-ready code:
1. agent.py
- Primary agent definition and configuration
- Core agent logic and behaviors
- No tool implementations allowed here
2. agent_tools.py
- All tool function implementations
- Tool configurations and setup
- External service integrations
3. agent_prompts.py
- System prompts
- Task-specific prompts
- Conversation templates
- Instruction sets
4. .env.example
- Required environment variables
- Clear setup instructions in a comment above the variable for how to do so
- API configuration templates
5. requirements.txt
- Core dependencies without versions
- User-specified packages included
[DOCUMENTATION WORKFLOW]
1. Initial Research
- Begin with RAG search for relevant documentation
- List all documentation pages using list_documentation_pages
- Retrieve specific page content using get_page_content
- Cross-reference the weather agent example for best practices
2. Implementation
- Provide complete, working code implementations
- Never leave placeholder functions
- Include all necessary error handling
- Implement proper logging and monitoring
3. Quality Assurance
- Verify all tool implementations are complete
- Ensure proper separation of concerns
- Validate environment variable handling
- Test critical path functionality
[INTERACTION GUIDELINES]
- Take immediate action without asking for permission
- Always verify documentation before implementation
- Provide honest feedback about documentation gaps
- Include specific enhancement suggestions
- Request user feedback on implementations
- Maintain code consistency across files
[ERROR HANDLING]
- Implement robust error handling in all tools
- Provide clear error messages
- Include recovery mechanisms
- Log important state changes
[BEST PRACTICES]
- Follow Pydantic AI naming conventions
- Implement proper type hints
- Include comprehensive docstrings, the agent uses this to understand what tools are for.
- Maintain clean code structure
- Use consistent formatting
Here is a good example of a Pydantic AI agent:
```python
from __future__ import annotations as _annotations
import asyncio
import os
from dataclasses import dataclass
from typing import Any
import logfire
from devtools import debug
from httpx import AsyncClient
from pydantic_ai import Agent, ModelRetry, RunContext
# 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured
logfire.configure(send_to_logfire='if-token-present')
@dataclass
class Deps:
client: AsyncClient
weather_api_key: str | None
geo_api_key: str | None
weather_agent = Agent(
'openai:gpt-4o',
# 'Be concise, reply with one sentence.' is enough for some models (like openai) to use
# the below tools appropriately, but others like anthropic and gemini require a bit more direction.
system_prompt=(
'Be concise, reply with one sentence.'
'Use the `get_lat_lng` tool to get the latitude and longitude of the locations, '
'then use the `get_weather` tool to get the weather.'
),
deps_type=Deps,
retries=2,
)
@weather_agent.tool
async def get_lat_lng(
ctx: RunContext[Deps], location_description: str
) -> dict[str, float]:
\"\"\"Get the latitude and longitude of a location.
Args:
ctx: The context.
location_description: A description of a location.
\"\"\"
if ctx.deps.geo_api_key is None:
# if no API key is provided, return a dummy response (London)
return {'lat': 51.1, 'lng': -0.1}
params = {
'q': location_description,
'api_key': ctx.deps.geo_api_key,
}
with logfire.span('calling geocode API', params=params) as span:
r = await ctx.deps.client.get('https://geocode.maps.co/search', params=params)
r.raise_for_status()
data = r.json()
span.set_attribute('response', data)
if data:
return {'lat': data[0]['lat'], 'lng': data[0]['lon']}
else:
raise ModelRetry('Could not find the location')
@weather_agent.tool
async def get_weather(ctx: RunContext[Deps], lat: float, lng: float) -> dict[str, Any]:
\"\"\"Get the weather at a location.
Args:
ctx: The context.
lat: Latitude of the location.
lng: Longitude of the location.
\"\"\"
if ctx.deps.weather_api_key is None:
# if no API key is provided, return a dummy response
return {'temperature': '21 °C', 'description': 'Sunny'}
params = {
'apikey': ctx.deps.weather_api_key,
'location': f'{lat},{lng}',
'units': 'metric',
}
with logfire.span('calling weather API', params=params) as span:
r = await ctx.deps.client.get(
'https://api.tomorrow.io/v4/weather/realtime', params=params
)
r.raise_for_status()
data = r.json()
span.set_attribute('response', data)
values = data['data']['values']
# https://docs.tomorrow.io/reference/data-layers-weather-codes
code_lookup = {
...
}
return {
'temperature': f'{values["temperatureApparent"]:0.0f}°C',
'description': code_lookup.get(values['weatherCode'], 'Unknown'),
}
async def main():
async with AsyncClient() as client:
# create a free API key at https://www.tomorrow.io/weather-api/
weather_api_key = os.getenv('WEATHER_API_KEY')
# create a free API key at https://geocode.maps.co/
geo_api_key = os.getenv('GEO_API_KEY')
deps = Deps(
client=client, weather_api_key=weather_api_key, geo_api_key=geo_api_key
)
result = await weather_agent.run(
'What is the weather like in London and in Wiltshire?', deps=deps
)
debug(result)
print('Response:', result.data)
if __name__ == '__main__':
asyncio.run(main())
```
"""
pydantic_ai_coder = Agent(
model,
system_prompt=system_prompt,
deps_type=PydanticAIDeps,
retries=2
)
@pydantic_ai_coder.system_prompt
def add_reasoner_output(ctx: RunContext[str]) -> str:
return f"""
\n\nAdditional thoughts/instructions from the reasoner LLM.
This scope includes documentation pages for you to search as well:
{ctx.deps.reasoner_output}
"""
# Add this in to get some crazy tool calling:
# You must get ALL documentation pages listed in the scope.
async def get_embedding(text: str, openai_client: AsyncOpenAI) -> List[float]:
"""Get embedding vector from OpenAI."""
try:
response = await openai_client.embeddings.create(
model=embedding_model,
input=text
)
return response.data[0].embedding
except Exception as e:
print(f"Error getting embedding: {e}")
return [0] * 1536 # Return zero vector on error
@pydantic_ai_coder.tool
async def retrieve_relevant_documentation(ctx: RunContext[PydanticAIDeps], user_query: str) -> str:
"""
Retrieve relevant documentation chunks based on the query with RAG.
Args:
ctx: The context including the Supabase client and OpenAI client
user_query: The user's question or query
Returns:
A formatted string containing the top 4 most relevant documentation chunks
"""
try:
# Get the embedding for the query
query_embedding = await get_embedding(user_query, ctx.deps.openai_client)
# Query Supabase for relevant documents
result = ctx.deps.supabase.rpc(
'match_site_pages',
{
'query_embedding': query_embedding,
'match_count': 4,
'filter': {'source': 'pydantic_ai_docs'}
}
).execute()
if not result.data:
return "No relevant documentation found."
# Format the results
formatted_chunks = []
for doc in result.data:
chunk_text = f"""
# {doc['title']}
{doc['content']}
"""
formatted_chunks.append(chunk_text)
# Join all chunks with a separator
return "\n\n---\n\n".join(formatted_chunks)
except Exception as e:
print(f"Error retrieving documentation: {e}")
return f"Error retrieving documentation: {str(e)}"
async def list_documentation_pages_helper(supabase: Client) -> List[str]:
"""
Function to retrieve a list of all available Pydantic AI documentation pages.
This is called by the list_documentation_pages tool and also externally
to fetch documentation pages for the reasoner LLM.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
try:
# Query Supabase for unique URLs where source is pydantic_ai_docs
result = supabase.from_('site_pages') \
.select('url') \
.eq('metadata->>source', 'pydantic_ai_docs') \
.execute()
if not result.data:
return []
# Extract unique URLs
urls = sorted(set(doc['url'] for doc in result.data))
return urls
except Exception as e:
print(f"Error retrieving documentation pages: {e}")
return []
@pydantic_ai_coder.tool
async def list_documentation_pages(ctx: RunContext[PydanticAIDeps]) -> List[str]:
"""
Retrieve a list of all available Pydantic AI documentation pages.
Returns:
List[str]: List of unique URLs for all documentation pages
"""
return await list_documentation_pages_helper(ctx.deps.supabase)
@pydantic_ai_coder.tool
async def get_page_content(ctx: RunContext[PydanticAIDeps], url: str) -> str:
"""
Retrieve the full content of a specific documentation page by combining all its chunks.
Args:
ctx: The context including the Supabase client
url: The URL of the page to retrieve
Returns:
str: The complete page content with all chunks combined in order
"""
try:
# Query Supabase for all chunks of this URL, ordered by chunk_number
result = ctx.deps.supabase.from_('site_pages') \
.select('title, content, chunk_number') \
.eq('url', url) \
.eq('metadata->>source', 'pydantic_ai_docs') \
.order('chunk_number') \
.execute()
if not result.data:
return f"No content found for URL: {url}"
# Format the page with its title and all chunks
page_title = result.data[0]['title'].split(' - ')[0] # Get the main title
formatted_content = [f"# {page_title}\n"]
# Add each chunk's content
for chunk in result.data:
formatted_content.append(chunk['content'])
# Join everything together but limit the characters in case the page is massive (there are a coule big ones)
# This will be improved later so if the page is too big RAG will be performed on the page itself
return "\n\n".join(formatted_content)[:20000]
except Exception as e:
print(f"Error retrieving page content: {e}")
return f"Error retrieving page content: {str(e)}"

831
original_archon/iterations/v4-streamlit-ui-overhaul/future_enhancements.py
View File

@@ -1,831 +0,0 @@
import streamlit as st
def future_enhancements_tab():
# Display the future enhancements and integrations interface
st.write("## Future Enhancements")
st.write("Explore what's coming next for Archon - from specialized multi-agent workflows to autonomous framework learning.")
# Future Iterations section
st.write("### Future Iterations")
# V5: Multi-Agent Coding Workflow
with st.expander("V5: Multi-Agent Coding Workflow"):
st.write("Specialized agents for different parts of the agent creation process")
# Create a visual representation of multi-agent workflow
st.write("#### Multi-Agent Coding Architecture")
# Describe the parallel architecture
st.markdown("""
The V5 architecture introduces specialized parallel agents that work simultaneously on different aspects of agent creation:
1. **Reasoner Agent**: Analyzes requirements and plans the overall agent architecture
2. **Parallel Coding Agents**:
- **Prompt Engineering Agent**: Designs optimal prompts for the agent
- **Tool Definition Agent**: Creates tool specifications and interfaces
- **Dependencies Agent**: Identifies required libraries and dependencies
- **Model Selection Agent**: Determines the best model configuration
3. **Final Coding Agent**: Integrates all components into a cohesive agent
4. **Human-in-the-Loop**: Iterative refinement with the final coding agent
""")
# Display parallel agents
st.write("#### Parallel Coding Agents")
col1, col2, col3, col4 = st.columns(4)
with col1:
st.info("**Prompt Engineering Agent**\n\nDesigns optimal prompts for different agent scenarios")
with col2:
st.success("**Tool Definition Agent**\n\nCreates tool specifications and interfaces")
with col3:
st.warning("**Dependencies Agent**\n\nIdentifies required libraries and dependencies")
with col4:
st.error("**Model Selection Agent**\n\nDetermines the best model configuration")
# Updated flow chart visualization with better colors for ovals
st.graphviz_chart('''
digraph {
rankdir=LR;
node [shape=box, style=filled, color=lightblue];
User [label="User Request", shape=ellipse, style=filled, color=purple, fontcolor=black];
Reasoner [label="Reasoner\nAgent"];
subgraph cluster_parallel {
label = "Parallel Coding Agents";
color = lightgrey;
style = filled;
Prompt [label="Prompt\nEngineering\nAgent", color=lightskyblue];
Tools [label="Tool\nDefinition\nAgent", color=green];
Dependencies [label="Dependencies\nAgent", color=yellow];
Model [label="Model\nSelection\nAgent", color=pink];
}
Final [label="Final\nCoding\nAgent"];
Human [label="Human-in-the-Loop\nIteration", shape=ellipse, style=filled, color=orange, fontcolor=black];
User -> Reasoner;
Reasoner -> Prompt;
Reasoner -> Tools;
Reasoner -> Dependencies;
Reasoner -> Model;
Prompt -> Final;
Tools -> Final;
Dependencies -> Final;
Model -> Final;
Final -> Human;
Human -> Final [label="Feedback Loop", color=red, constraint=false];
}
''')
st.write("#### Benefits of Parallel Agent Architecture")
st.markdown("""
- **Specialization**: Each agent focuses on its area of expertise
- **Efficiency**: Parallel processing reduces overall development time
- **Quality**: Specialized agents produce higher quality components
- **Flexibility**: Easy to add new specialized agents as needed
- **Scalability**: Architecture can handle complex agent requirements
""")
# V6: Tool Library and Example Integration
with st.expander("V6: Tool Library and Example Integration"):
st.write("Pre-built external tool and agent examples incorporation")
st.write("""
With pre-built tools, the agent can pull full functions from the tool library so it doesn't have to
create them from scratch. On top of that, pre-built agents will give Archon a starting point
so it doesn't have to build the agent structure from scratch either.
""")
st.write("#### Example Integration Configuration")
# Add tabs for different aspects of V6
tool_tab, example_tab = st.tabs(["Tool Library", "Example Agents"])
with tool_tab:
st.write("##### Example Tool Library Config (could be a RAG implementation too, still deciding)")
sample_config = """
{
"tool_library": {
"web_tools": {
"web_search": {
"type": "search_engine",
"api_key_env": "SEARCH_API_KEY",
"description": "Search the web for information"
},
"web_browser": {
"type": "browser",
"description": "Navigate web pages and extract content"
}
},
"data_tools": {
"database_query": {
"type": "sql_executor",
"description": "Execute SQL queries against databases"
},
"data_analysis": {
"type": "pandas_processor",
"description": "Analyze data using pandas"
}
},
"ai_service_tools": {
"image_generation": {
"type": "text_to_image",
"api_key_env": "IMAGE_GEN_API_KEY",
"description": "Generate images from text descriptions"
},
"text_to_speech": {
"type": "tts_converter",
"api_key_env": "TTS_API_KEY",
"description": "Convert text to spoken audio"
}
}
}
}
"""
st.code(sample_config, language="json")
st.write("##### Pydantic AI Tool Definition Example")
pydantic_tool_example = """
from pydantic_ai import Agent, RunContext, Tool
from typing import Union, List, Dict, Any
import requests
@agent.tool
async def weather_tool(ctx: RunContext[Dict[str, Any]], location: str) -> str:
\"\"\"Get current weather information for a location.
Args:
location: The city and state/country (e.g., 'San Francisco, CA')
Returns:
A string with current weather conditions and temperature
\"\"\"
api_key = ctx.deps.get("WEATHER_API_KEY")
if not api_key:
return "Error: Weather API key not configured"
try:
url = f"https://api.weatherapi.com/v1/current.json?key={api_key}&q={location}"
response = requests.get(url)
data = response.json()
if "error" in data:
return f"Error: {data['error']['message']}"
current = data["current"]
location_name = f"{data['location']['name']}, {data['location']['country']}"
condition = current["condition"]["text"]
temp_c = current["temp_c"]
temp_f = current["temp_f"]
humidity = current["humidity"]
return f"Weather in {location_name}: {condition}, {temp_c}°C ({temp_f}°F), {humidity}% humidity"
except Exception as e:
return f"Error retrieving weather data: {str(e)}"
"""
st.code(pydantic_tool_example, language="python")
st.write("##### Tool Usage in Agent")
tool_usage_example = """
async def use_weather_tool(location: str) -> str:
\"\"\"Search for weather information\"\"\"
tool = agent.get_tool("get_weather")
result = await tool.execute({"location": location})
return result.content
"""
st.code(tool_usage_example, language="python")
with example_tab:
st.write("##### Example Agents")
st.markdown("""
V6 will include pre-built example agents that serve as templates and learning resources. These examples will be baked directly into agent prompts to improve results and consistency.
**Benefits of Example Agents:**
- Provide concrete implementation patterns for common agent types
- Demonstrate best practices for tool usage and error handling
- Serve as starting points that can be customized for specific needs
- Improve consistency in agent behavior and output format
- Reduce the learning curve for new users
""")
st.write("##### Example Agent Types")
example_agents = {
"Research Assistant": {
"description": "Performs comprehensive research on topics using web search and content analysis",
"tools": ["web_search", "web_browser", "summarization"],
"example_prompt": "Research the latest advancements in quantum computing and provide a summary"
},
"Data Analyst": {
"description": "Analyzes datasets, generates visualizations, and provides insights",
"tools": ["database_query", "data_analysis", "chart_generation"],
"example_prompt": "Analyze this sales dataset and identify key trends over the past quarter"
},
"Content Creator": {
"description": "Generates various types of content including text, images, and code",
"tools": ["text_generation", "image_generation", "code_generation"],
"example_prompt": "Create a blog post about sustainable living with accompanying images"
},
"Conversational Assistant": {
"description": "Engages in helpful, informative conversations with natural dialogue",
"tools": ["knowledge_base", "memory_management", "personalization"],
"example_prompt": "I'd like to learn more about machine learning. Where should I start?"
}
}
# Create a table of example agents
example_data = {
"Agent Type": list(example_agents.keys()),
"Description": [example_agents[a]["description"] for a in example_agents],
"Core Tools": [", ".join(example_agents[a]["tools"]) for a in example_agents]
}
st.dataframe(example_data, use_container_width=True)
st.write("##### Example Agent Implementation")
st.code("""
# Example Weather Agent based on Pydantic AI documentation
from pydantic_ai import Agent, RunContext
from typing import Dict, Any
from dataclasses import dataclass
from httpx import AsyncClient
@dataclass
class WeatherDeps:
client: AsyncClient
weather_api_key: str | None
geo_api_key: str | None
# Create the agent with appropriate system prompt
weather_agent = Agent(
'openai:gpt-4o',
system_prompt=(
'Be concise, reply with one sentence. '
'Use the `get_lat_lng` tool to get the latitude and longitude of locations, '
'then use the `get_weather` tool to get the weather.'
),
deps_type=WeatherDeps,
)
@weather_agent.tool
async def get_lat_lng(ctx: RunContext[WeatherDeps], location_description: str) -> Dict[str, float]:
\"\"\"Get the latitude and longitude of a location.
Args:
location_description: A description of a location (e.g., 'London, UK')
Returns:
Dictionary with lat and lng keys
\"\"\"
if ctx.deps.geo_api_key is None:
# Return dummy data if no API key
return {'lat': 51.1, 'lng': -0.1}
# Call geocoding API
params = {'q': location_description, 'api_key': ctx.deps.geo_api_key}
r = await ctx.deps.client.get('https://geocode.maps.co/search', params=params)
r.raise_for_status()
data = r.json()
if data:
return {'lat': float(data[0]['lat']), 'lng': float(data[0]['lon'])}
else:
return {'error': 'Location not found'}
@weather_agent.tool
async def get_weather(ctx: RunContext[WeatherDeps], lat: float, lng: float) -> Dict[str, Any]:
\"\"\"Get the weather at a location.
Args:
lat: Latitude of the location
lng: Longitude of the location
Returns:
Dictionary with temperature and description
\"\"\"
if ctx.deps.weather_api_key is None:
# Return dummy data if no API key
return {'temperature': '21°C', 'description': 'Sunny'}
# Call weather API
params = {
'apikey': ctx.deps.weather_api_key,
'location': f'{lat},{lng}',
'units': 'metric',
}
r = await ctx.deps.client.get(
'https://api.tomorrow.io/v4/weather/realtime',
params=params
)
r.raise_for_status()
data = r.json()
values = data['data']['values']
weather_codes = {
1000: 'Clear, Sunny',
1100: 'Mostly Clear',
1101: 'Partly Cloudy',
4001: 'Rain',
5000: 'Snow',
8000: 'Thunderstorm',
}
return {
'temperature': f'{values["temperatureApparent"]:0.0f}°C',
'description': weather_codes.get(values['weatherCode'], 'Unknown'),
}
# Example usage
async def get_weather_report(location: str) -> str:
\"\"\"Get weather report for a location.\"\"\"
async with AsyncClient() as client:
deps = WeatherDeps(
client=client,
weather_api_key="YOUR_API_KEY", # Replace with actual key
geo_api_key="YOUR_API_KEY", # Replace with actual key
)
result = await weather_agent.run(
f"What is the weather like in {location}?",
deps=deps
)
return result.data
""", language="python")
st.info("""
**In-Context Learning with Examples**
These example agents will be used in the system prompt for Archon, providing concrete examples that help the LLM understand the expected structure and quality of agent code. This approach leverages in-context learning to significantly improve code generation quality and consistency.
""")
# V7: LangGraph Documentation
with st.expander("V7: LangGraph Documentation"):
st.write("Integrating LangGraph for complex agent workflows")
st.markdown("""
### Pydantic AI vs LangGraph with Pydantic AI
V7 will integrate LangGraph to enable complex agent workflows while maintaining compatibility with Pydantic AI agents.
This allows for creating sophisticated multi-agent systems with well-defined state management and workflow control.
""")
col1, col2 = st.columns(2)
with col1:
st.markdown("#### Pydantic AI Agent")
st.markdown("Simple, standalone agent with tools")
pydantic_agent_code = """
# Simple Pydantic AI Weather Agent
from pydantic_ai import Agent, RunContext
from typing import Dict, Any
from dataclasses import dataclass
from httpx import AsyncClient
@dataclass
class WeatherDeps:
client: AsyncClient
weather_api_key: str | None
# Create the agent
weather_agent = Agent(
'openai:gpt-4o',
system_prompt="You provide weather information.",
deps_type=WeatherDeps,
)
@weather_agent.tool
async def get_weather(
ctx: RunContext[WeatherDeps],
location: str
) -> Dict[str, Any]:
\"\"\"Get weather for a location.\"\"\"
# Implementation details...
return {"temperature": "21°C", "description": "Sunny"}
# Usage
async def main():
async with AsyncClient() as client:
deps = WeatherDeps(
client=client,
weather_api_key="API_KEY"
)
result = await weather_agent.run(
"What's the weather in London?",
deps=deps
)
print(result.data)
"""
st.code(pydantic_agent_code, language="python")
with col2:
st.markdown("#### LangGraph with Pydantic AI Agent")
st.markdown("Complex workflow using Pydantic AI agents in a graph")
langgraph_code = """
# LangGraph with Pydantic AI Agents
from pydantic_ai import Agent, RunContext
from typing import TypedDict, Literal
from dataclasses import dataclass
from httpx import AsyncClient
from langgraph.graph import StateGraph, START, END
# Define state for LangGraph
class GraphState(TypedDict):
query: str
weather_result: str
verified: bool
response: str
# Create a verifier agent
verifier_agent = Agent(
'openai:gpt-4o',
system_prompt=(
"You verify weather information for accuracy and completeness. "
"Check if the weather report includes temperature, conditions, "
"and is properly formatted."
)
)
# Define nodes for the graph
async def get_weather_info(state: GraphState) -> GraphState:
\"\"\"Use the weather agent to get weather information.\"\"\"
# Simply use the weather agent directly
async with AsyncClient() as client:
deps = WeatherDeps(
client=client,
weather_api_key="API_KEY"
)
result = await weather_agent.run(
state["query"],
deps=deps
)
return {"weather_result": result.data}
async def verify_information(state: GraphState) -> GraphState:
\"\"\"Use the verifier agent to check the weather information.\"\"\"
result = await verifier_agent.run(
f"Verify this weather information: {state['weather_result']}"
)
# Simple verification logic
verified = "accurate" in result.data.lower()
return {"verified": verified}
async def route(state: GraphState) -> Literal["regenerate", "finalize"]:
"\"\"Decide whether to regenerate or finalize based on verification.\"\"\"
if state["verified"]:
return "finalize"
else:
return "regenerate"
async def regenerate_response(state: GraphState) -> GraphState:
\"\"\"Regenerate a better response if verification failed.\"\"\"
result = await verifier_agent.run(
result = await weather_agent.run(
f"Please provide more detailed weather information for: {state['query']}"
)
return {"weather_result": result.data, "verified": True}
async def finalize_response(state: GraphState) -> GraphState:
\"\"\"Format the final response.\"\"\"
return {"response": f"Verified Weather Report: {state['weather_result']}"}
# Build the graph
workflow = StateGraph(GraphState)
# Add nodes
workflow.add_node("get_weather", get_weather_info)
workflow.add_node("verify", verify_information)
workflow.add_node("regenerate", regenerate_response)
workflow.add_node("finalize", finalize_response)
# Add edges
workflow.add_edge(START, "get_weather")
workflow.add_edge("get_weather", "verify")
# Add conditional edges based on verification
workflow.add_conditional_edges(
"verify",
route,
{
"regenerate": "regenerate",
"finalize": "finalize"
}
)
workflow.add_edge("regenerate", "finalize")
workflow.add_edge("finalize", END)
# Compile the graph
app = workflow.compile()
# Usage
async def main():
result = await app.ainvoke({
"query": "What's the weather in London?",
"verified": False
})
print(result["response"])
"""
st.code(langgraph_code, language="python")
st.markdown("""
### Key Benefits of Integration
1. **Workflow Management**: LangGraph provides a structured way to define complex agent workflows with clear state transitions.
2. **Reusability**: Pydantic AI agents can be reused within LangGraph nodes, maintaining their tool capabilities.
3. **Visualization**: LangGraph offers built-in visualization of agent workflows, making it easier to understand and debug complex systems.
4. **State Management**: The typed state in LangGraph ensures type safety and clear data flow between nodes.
5. **Parallel Execution**: LangGraph supports parallel execution of nodes, enabling more efficient processing.
6. **Human-in-the-Loop**: Both frameworks support human intervention points, which can be combined for powerful interactive systems.
""")
st.image("https://blog.langchain.dev/content/images/2024/01/simple_multi_agent_diagram--1-.png",
caption="Example LangGraph Multi-Agent Workflow", width=600)
# V8: Self-Feedback Loop
with st.expander("V8: Self-Feedback Loop"):
st.write("Automated validation and error correction")
# Create a visual feedback loop
st.graphviz_chart('''
digraph {
rankdir=TB;
node [shape=box, style=filled, color=lightblue];
Agent [label="Agent Generation"];
Test [label="Automated Testing"];
Validate [label="Validation"];
Error [label="Error Detection"];
Fix [label="Self-Correction"];
Agent -> Test;
Test -> Validate;
Validate -> Error [label="Issues Found"];
Error -> Fix;
Fix -> Agent [label="Regenerate"];
Validate -> Agent [label="Success", color=green];
}
''')
st.write("#### Validation Process")
st.info("""
1. Generate agent code
2. Run automated tests
3. Analyze test results
4. Identify errors or improvement areas
5. Apply self-correction algorithms
6. Regenerate improved code
7. Repeat until validation passes
""")
# V9: Self Agent Execution
with st.expander("V9: Self Agent Execution"):
st.write("Testing and iterating on agents in an isolated environment")
st.write("#### Agent Execution Process")
execution_process = [
{"phase": "Sandbox Creation", "description": "Set up isolated environment using Local AI package"},
{"phase": "Agent Deployment", "description": "Load the generated agent into the testing environment"},
{"phase": "Test Execution", "description": "Run the agent against predefined scenarios and user queries"},
{"phase": "Performance Monitoring", "description": "Track response quality, latency, and resource usage"},
{"phase": "Error Detection", "description": "Identify runtime errors and logical inconsistencies"},
{"phase": "Iterative Improvement", "description": "Refine agent based on execution results"}
]
for i, phase in enumerate(execution_process):
st.write(f"**{i+1}. {phase['phase']}:** {phase['description']}")
st.write("#### Local AI Package Integration")
st.markdown("""
The [Local AI package](https://github.com/coleam00/local-ai-packaged) provides a containerized environment for:
- Running LLMs locally for agent testing
- Simulating API calls and external dependencies
- Monitoring agent behavior in a controlled setting
- Collecting performance metrics for optimization
""")
st.info("This enables Archon to test and refine agents in a controlled environment before deployment, significantly improving reliability and performance through empirical iteration.")
# V10: Multi-Framework Support
with st.expander("V10: Multi-Framework Support"):
st.write("Framework-agnostic agent generation")
frameworks = {
"Pydantic AI": {"status": "Supported", "description": "Native support for function-based agents"},
"LangGraph": {"status": "Coming in V7", "description": "Declarative multi-agent orchestration"},
"LangChain": {"status": "Planned", "description": "Popular agent framework with extensive tools"},
"Agno (Phidata)": {"status": "Planned", "description": "Multi-agent workflow framework"},
"CrewAI": {"status": "Planned", "description": "Role-based collaborative agents"},
"LlamaIndex": {"status": "Planned", "description": "RAG-focused agent framework"}
}
# Create a frameworks comparison table
df_data = {
"Framework": list(frameworks.keys()),
"Status": [frameworks[f]["status"] for f in frameworks],
"Description": [frameworks[f]["description"] for f in frameworks]
}
st.dataframe(df_data, use_container_width=True)
# V11: Autonomous Framework Learning
with st.expander("V11: Autonomous Framework Learning"):
st.write("Self-learning from mistakes and continuous improvement")
st.write("#### Self-Improvement Process")
improvement_process = [
{"phase": "Error Detection", "description": "Identifies patterns in failed agent generations and runtime errors"},
{"phase": "Root Cause Analysis", "description": "Analyzes error patterns to determine underlying issues in prompts or examples"},
{"phase": "Prompt Refinement", "description": "Automatically updates system prompts to address identified weaknesses"},
{"phase": "Example Augmentation", "description": "Adds new examples to the prompt library based on successful generations"},
{"phase": "Tool Enhancement", "description": "Creates or modifies tools to handle edge cases and common failure modes"},
{"phase": "Validation", "description": "Tests improvements against historical failure cases to ensure progress"}
]
for i, phase in enumerate(improvement_process):
st.write(f"**{i+1}. {phase['phase']}:** {phase['description']}")
st.info("This enables Archon to stay updated with the latest AI frameworks without manual intervention.")
# V12: Advanced RAG Techniques
with st.expander("V12: Advanced RAG Techniques"):
st.write("Enhanced retrieval and incorporation of framework documentation")
st.write("#### Advanced RAG Components")
col1, col2 = st.columns(2)
with col1:
st.markdown("#### Document Processing")
st.markdown("""
- **Hierarchical Chunking**: Multi-level chunking strategy that preserves document structure
- **Semantic Headers**: Extraction of meaningful section headers for better context
- **Code-Text Separation**: Specialized embedding models for code vs. natural language
- **Metadata Enrichment**: Automatic tagging with framework version, function types, etc.
""")
st.markdown("#### Query Processing")
st.markdown("""
- **Query Decomposition**: Breaking complex queries into sub-queries
- **Framework Detection**: Identifying which framework the query relates to
- **Intent Classification**: Determining if query is about usage, concepts, or troubleshooting
- **Query Expansion**: Adding relevant framework-specific terminology
""")
with col2:
st.markdown("#### Retrieval Enhancements")
st.markdown("""
- **Hybrid Search**: Combining dense and sparse retrievers for optimal results
- **Re-ranking**: Post-retrieval scoring based on relevance to the specific task
- **Cross-Framework Retrieval**: Finding analogous patterns across different frameworks
- **Code Example Prioritization**: Boosting practical examples in search results
""")
st.markdown("#### Knowledge Integration")
st.markdown("""
- **Context Stitching**: Intelligently combining information from multiple chunks
- **Framework Translation**: Converting patterns between frameworks (e.g., LangChain to LangGraph)
- **Version Awareness**: Handling differences between framework versions
- **Adaptive Retrieval**: Learning from successful and unsuccessful retrievals
""")
st.info("This enables Archon to more effectively retrieve and incorporate framework documentation, leading to more accurate and contextually appropriate agent generation.")
# V13: MCP Agent Marketplace
with st.expander("V13: MCP Agent Marketplace"):
st.write("Integrating Archon agents as MCP servers and publishing to marketplaces")
st.write("#### MCP Integration Process")
mcp_integration_process = [
{"phase": "Protocol Implementation", "description": "Implement the Model Context Protocol to enable IDE integration"},
{"phase": "Agent Conversion", "description": "Transform Archon-generated agents into MCP-compatible servers"},
{"phase": "Specialized Agent Creation", "description": "Build purpose-specific agents for code review, refactoring, and testing"},
{"phase": "Marketplace Publishing", "description": "Package and publish agents to MCP marketplaces for distribution"},
{"phase": "IDE Integration", "description": "Enable seamless operation within Windsurf, Cursor, and other MCP-enabled IDEs"}
]
for i, phase in enumerate(mcp_integration_process):
st.write(f"**{i+1}. {phase['phase']}:** {phase['description']}")
st.info("This enables Archon to create specialized agents that operate directly within IDEs through the MCP protocol, while also making them available through marketplace distribution channels.")
# Future Integrations section
st.write("### Future Integrations")
# LangSmith
with st.expander("LangSmith"):
st.write("Integration with LangChain's tracing and monitoring platform")
st.image("https://docs.smith.langchain.com/assets/images/trace-9510284b5b15ba55fc1cca6af2404657.png", width=600)
st.write("#### LangSmith Benefits")
st.markdown("""
- **Tracing**: Monitor agent execution steps and decisions
- **Debugging**: Identify issues in complex agent workflows
- **Analytics**: Track performance and cost metrics
- **Evaluation**: Assess agent quality with automated testing
- **Feedback Collection**: Gather human feedback to improve agents
""")
# MCP Marketplace
with st.expander("MCP Marketplace"):
st.write("Integration with AI IDE marketplaces")
st.write("#### MCP Marketplace Integration")
st.markdown("""
- Publish Archon itself as a premium agent in MCP marketplaces
- Create specialized Archon variants for different development needs
- Enable one-click installation directly from within IDEs
- Integrate seamlessly with existing development workflows
""")
st.warning("The Model Context Protocol (MCP) is an emerging standard for AI assistant integration with IDEs like Windsurf, Cursor, and Cline.")
# Other Frameworks
with st.expander("Other Frameworks besides Pydantic AI"):
st.write("Support for additional agent frameworks")
st.write("#### Framework Adapter Architecture")
st.graphviz_chart('''
digraph {
rankdir=TB;
node [shape=box, style=filled, color=lightblue];
Archon [label="Archon Core"];
Adapter [label="Framework Adapter Layer"];
Pydantic [label="Pydantic AI", color=lightskyblue];
LangGraph [label="LangGraph", color=lightskyblue];
LangChain [label="LangChain", color=lightskyblue];
Agno [label="Agno", color=lightskyblue];
CrewAI [label="CrewAI", color=lightskyblue];
LlamaIndex [label="LlamaIndex", color=lightskyblue];
Archon -> Adapter;
Adapter -> Pydantic;
Adapter -> LangGraph;
Adapter -> LangChain;
Adapter -> Agno;
Adapter -> CrewAI;
Adapter -> LlamaIndex;
}
''')
# Vector Databases
with st.expander("Other Vector Databases besides Supabase"):
st.write("Support for additional vector databases")
vector_dbs = {
"Supabase": {"status": "Supported", "features": ["pgvector integration", "SQL API", "Real-time subscriptions"]},
"Pinecone": {"status": "Planned", "features": ["High scalability", "Low latency", "Serverless"]},
"Qdrant": {"status": "Planned", "features": ["Filtering", "Self-hosted option", "REST API"]},
"Milvus": {"status": "Planned", "features": ["Horizontal scaling", "Cloud-native", "Hybrid search"]},
"Chroma": {"status": "Planned", "features": ["Local-first", "Lightweight", "Simple API"]},
"Weaviate": {"status": "Planned", "features": ["GraphQL", "Multi-modal", "RESTful API"]}
}
# Create vector DB comparison table
df_data = {
"Vector Database": list(vector_dbs.keys()),
"Status": [vector_dbs[db]["status"] for db in vector_dbs],
"Key Features": [", ".join(vector_dbs[db]["features"]) for db in vector_dbs]
}
st.dataframe(df_data, use_container_width=True)
# Local AI Package
with st.expander("Local AI Package Integration"):
st.write("Integration with [Local AI Package](https://github.com/coleam00/local-ai-packaged)")
st.markdown("""
The Local AI Package enables running models entirely locally, providing:
- **Complete Privacy**: No data leaves your machine
- **Cost Savings**: Eliminate API usage fees
- **Offline Operation**: Work without internet connectivity
- **Custom Fine-tuning**: Adapt models to specific domains
- **Lower Latency**: Reduce response times for better UX
""")
st.info("This integration will allow Archon to operate fully offline with local models for both agent creation and execution.")

69
original_archon/iterations/v4-streamlit-ui-overhaul/graph_service.py
View File

@@ -1,69 +0,0 @@
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import Optional, Dict, Any
from archon.archon_graph import agentic_flow
from langgraph.types import Command
from utils.utils import write_to_log
app = FastAPI()
class InvokeRequest(BaseModel):
message: str
thread_id: str
is_first_message: bool = False
config: Optional[Dict[str, Any]] = None
@app.get("/health")
async def health_check():
"""Health check endpoint"""
return {"status": "ok"}
@app.post("/invoke")
async def invoke_agent(request: InvokeRequest):
"""Process a message through the agentic flow and return the complete response.
The agent streams the response but this API endpoint waits for the full output
before returning so it's a synchronous operation for MCP.
Another endpoint will be made later to fully stream the response from the API.
Args:
request: The InvokeRequest containing message and thread info
Returns:
dict: Contains the complete response from the agent
"""
try:
config = request.config or {
"configurable": {
"thread_id": request.thread_id
}
}
response = ""
if request.is_first_message:
write_to_log(f"Processing first message for thread {request.thread_id}")
async for msg in agentic_flow.astream(
{"latest_user_message": request.message},
config,
stream_mode="custom"
):
response += str(msg)
else:
write_to_log(f"Processing continuation for thread {request.thread_id}")
async for msg in agentic_flow.astream(
Command(resume=request.message),
config,
stream_mode="custom"
):
response += str(msg)
write_to_log(f"Final response for thread {request.thread_id}: {response}")
return {"response": response}
except Exception as e:
write_to_log(f"Error processing message for thread {request.thread_id}: {str(e)}")
raise HTTPException(status_code=500, detail=str(e))
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8100)

38
original_archon/iterations/v4-streamlit-ui-overhaul/mcp/.dockerignore
View File

@@ -1,38 +0,0 @@
# Ignore specified folders
iterations/
venv/
.langgraph_api/
.github/
__pycache__/
.env
# Git related
.git/
.gitignore
.gitattributes
# Python cache
*.pyc
*.pyo
*.pyd
.Python
*.so
.pytest_cache/
# Environment files
.env.local
.env.development.local
.env.test.local
.env.production.local
# Logs
*.log
# IDE specific files
.idea/
.vscode/
*.swp
*.swo
# Keep the example env file for reference
!.env.example

19
original_archon/iterations/v4-streamlit-ui-overhaul/mcp/Dockerfile
View File

@@ -1,19 +0,0 @@
FROM python:3.12-slim
WORKDIR /app
# Copy requirements file and install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy the MCP server files
COPY . .
# Expose port for MCP server
EXPOSE 8100
# Set environment variables
ENV PYTHONUNBUFFERED=1
# Command to run the MCP server
CMD ["python", "mcp_server.py"]

112
original_archon/iterations/v4-streamlit-ui-overhaul/mcp/mcp_server.py
View File

@@ -1,112 +0,0 @@
from mcp.server.fastmcp import FastMCP
from datetime import datetime
from dotenv import load_dotenv
from typing import Dict, List
import threading
import requests
import asyncio
import uuid
import sys
import os
# Load environment variables from .env file
load_dotenv()
# Initialize FastMCP server
mcp = FastMCP("archon")
# Store active threads
active_threads: Dict[str, List[str]] = {}
# FastAPI service URL
GRAPH_SERVICE_URL = os.getenv("GRAPH_SERVICE_URL", "http://localhost:8100")
def write_to_log(message: str):
"""Write a message to the logs.txt file in the workbench directory.
Args:
message: The message to log
"""
# Get the directory one level up from the current file
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
workbench_dir = os.path.join(parent_dir, "workbench")
log_path = os.path.join(workbench_dir, "logs.txt")
os.makedirs(workbench_dir, exist_ok=True)
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
log_entry = f"[{timestamp}] {message}\n"
with open(log_path, "a", encoding="utf-8") as f:
f.write(log_entry)
@mcp.tool()
async def create_thread() -> str:
"""Create a new conversation thread for Archon.
Always call this tool before invoking Archon for the first time in a conversation.
(if you don't already have a thread ID)
Returns:
str: A unique thread ID for the conversation
"""
thread_id = str(uuid.uuid4())
active_threads[thread_id] = []
write_to_log(f"Created new thread: {thread_id}")
return thread_id
def _make_request(thread_id: str, user_input: str, config: dict) -> str:
"""Make synchronous request to graph service"""
response = requests.post(
f"{GRAPH_SERVICE_URL}/invoke",
json={
"message": user_input,
"thread_id": thread_id,
"is_first_message": not active_threads[thread_id],
"config": config
}
)
response.raise_for_status()
return response.json()
@mcp.tool()
async def run_agent(thread_id: str, user_input: str) -> str:
"""Run the Archon agent with user input.
Only use this tool after you have called create_thread in this conversation to get a unique thread ID.
If you already created a thread ID in this conversation, do not create another one. Reuse the same ID.
After you receive the code from Archon, always implement it into the codebase unless asked not to.
Args:
thread_id: The conversation thread ID
user_input: The user's message to process
Returns:
str: The agent's response which generally includes the code for the agent
"""
if thread_id not in active_threads:
write_to_log(f"Error: Thread not found - {thread_id}")
raise ValueError("Thread not found")
write_to_log(f"Processing message for thread {thread_id}: {user_input}")
config = {
"configurable": {
"thread_id": thread_id
}
}
try:
result = await asyncio.to_thread(_make_request, thread_id, user_input, config)
active_threads[thread_id].append(user_input)
return result['response']
except Exception as e:
raise
if __name__ == "__main__":
write_to_log("Starting MCP server")
# Run MCP server
mcp.run(transport='stdio')

3
original_archon/iterations/v4-streamlit-ui-overhaul/mcp/requirements.txt
View File

@@ -1,3 +0,0 @@
mcp==1.2.1
python-dotenv==1.0.1
requests==2.32.3

95
original_archon/iterations/v4-streamlit-ui-overhaul/mcp_server.py
View File

@@ -1,95 +0,0 @@
import os
import sys
import asyncio
import threading
from mcp.server.fastmcp import FastMCP
import requests
from typing import Dict, List
import uuid
from utils.utils import write_to_log
from graph_service import app
import uvicorn
# Initialize FastMCP server
mcp = FastMCP("archon")
# Store active threads
active_threads: Dict[str, List[str]] = {}
# FastAPI service URL
GRAPH_SERVICE_URL = "http://127.0.0.1:8100"
@mcp.tool()
async def create_thread() -> str:
"""Create a new conversation thread for Archon.
Always call this tool before invoking Archon for the first time in a conversation.
(if you don't already have a thread ID)
Returns:
str: A unique thread ID for the conversation
"""
thread_id = str(uuid.uuid4())
active_threads[thread_id] = []
write_to_log(f"Created new thread: {thread_id}")
return thread_id
def _make_request(thread_id: str, user_input: str, config: dict) -> str:
"""Make synchronous request to graph service"""
response = requests.post(
f"{GRAPH_SERVICE_URL}/invoke",
json={
"message": user_input,
"thread_id": thread_id,
"is_first_message": not active_threads[thread_id],
"config": config
}
)
response.raise_for_status()
return response.json()
@mcp.tool()
async def run_agent(thread_id: str, user_input: str) -> str:
"""Run the Archon agent with user input.
Only use this tool after you have called create_thread in this conversation to get a unique thread ID.
If you already created a thread ID in this conversation, do not create another one. Reuse the same ID.
After you receive the code from Archon, always implement it into the codebase unless asked not to.
Args:
thread_id: The conversation thread ID
user_input: The user's message to process
Returns:
str: The agent's response which generally includes the code for the agent
"""
if thread_id not in active_threads:
write_to_log(f"Error: Thread not found - {thread_id}")
raise ValueError("Thread not found")
write_to_log(f"Processing message for thread {thread_id}: {user_input}")
config = {
"configurable": {
"thread_id": thread_id
}
}
try:
result = await asyncio.to_thread(_make_request, thread_id, user_input, config)
active_threads[thread_id].append(user_input)
return result['response']
except Exception as e:
raise
if __name__ == "__main__":
write_to_log("Starting MCP server")
# Run MCP server
mcp.run(transport='stdio')

BIN
original_archon/iterations/v4-streamlit-ui-overhaul/public/Archon.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 576 KiB

BIN
original_archon/iterations/v4-streamlit-ui-overhaul/public/ArchonLightGrey.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 327 KiB

176
original_archon/iterations/v4-streamlit-ui-overhaul/requirements.txt
View File

@@ -1,176 +0,0 @@
aiofiles==24.1.0
aiohappyeyeballs==2.4.4
aiohttp==3.11.11
aiosignal==1.3.2
aiosqlite==0.20.0
altair==5.5.0
annotated-types==0.7.0
anthropic==0.42.0
anyio==4.8.0
attrs==24.3.0
beautifulsoup4==4.12.3
blinker==1.9.0
cachetools==5.5.0
certifi==2024.12.14
cffi==1.17.1
charset-normalizer==3.4.1
click==8.1.8
cohere==5.13.12
colorama==0.4.6
Crawl4AI==0.4.247
cryptography==43.0.3
Deprecated==1.2.15
deprecation==2.1.0
distro==1.9.0
dnspython==2.7.0
email_validator==2.2.0
eval_type_backport==0.2.2
executing==2.1.0
fake-http-header==0.3.5
fastapi==0.115.8
fastapi-cli==0.0.7
fastavro==1.10.0
filelock==3.16.1
frozenlist==1.5.0
fsspec==2024.12.0
gitdb==4.0.12
GitPython==3.1.44
google-auth==2.37.0
googleapis-common-protos==1.66.0
gotrue==2.11.1
greenlet==3.1.1
griffe==1.5.4
groq==0.15.0
h11==0.14.0
h2==4.1.0
hpack==4.0.0
html2text==2024.2.26
httpcore==1.0.7
httptools==0.6.4
httpx==0.27.2
httpx-sse==0.4.0
huggingface-hub==0.27.1
hyperframe==6.0.1
idna==3.10
importlib_metadata==8.5.0
iniconfig==2.0.0
itsdangerous==2.2.0
Jinja2==3.1.5
jiter==0.8.2
joblib==1.4.2
jsonpatch==1.33
jsonpath-python==1.0.6
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
jsonschema_rs==0.25.1
langchain-core==0.3.33
langgraph==0.2.69
langgraph-checkpoint==2.0.10
langgraph-cli==0.1.71
langgraph-sdk==0.1.51
langsmith==0.3.6
litellm==1.57.8
logfire==3.1.0
logfire-api==3.1.0
lxml==5.3.0
markdown-it-py==3.0.0
MarkupSafe==3.0.2
mcp==1.2.1
mdurl==0.1.2
mistralai==1.2.6
mockito==1.5.3
msgpack==1.1.0
multidict==6.1.0
mypy-extensions==1.0.0
narwhals==1.21.1
nltk==3.9.1
numpy==2.2.1
openai==1.59.6
opentelemetry-api==1.29.0
opentelemetry-exporter-otlp-proto-common==1.29.0
opentelemetry-exporter-otlp-proto-http==1.29.0
opentelemetry-instrumentation==0.50b0
opentelemetry-proto==1.29.0
opentelemetry-sdk==1.29.0
opentelemetry-semantic-conventions==0.50b0
orjson==3.10.15
packaging==24.2
pandas==2.2.3
pillow==10.4.0
playwright==1.49.1
pluggy==1.5.0
postgrest==0.19.1
propcache==0.2.1
protobuf==5.29.3
psutil==6.1.1
pyarrow==18.1.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycparser==2.22
pydantic==2.10.5
pydantic-ai==0.0.22
pydantic-ai-slim==0.0.22
pydantic-extra-types==2.10.2
pydantic-graph==0.0.22
pydantic-settings==2.7.1
pydantic_core==2.27.2
pydeck==0.9.1
pyee==12.0.0
Pygments==2.19.1
PyJWT==2.10.1
pyOpenSSL==24.3.0
pytest==8.3.4
pytest-mockito==0.0.4
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-multipart==0.0.20
pytz==2024.2
PyYAML==6.0.2
rank-bm25==0.2.2
realtime==2.1.0
referencing==0.35.1
regex==2024.11.6
requests==2.32.3
requests-toolbelt==1.0.0
rich==13.9.4
rich-toolkit==0.13.2
rpds-py==0.22.3
rsa==4.9
shellingham==1.5.4
six==1.17.0
smmap==5.0.2
sniffio==1.3.1
snowballstemmer==2.2.0
soupsieve==2.6
sse-starlette==2.1.3
starlette==0.45.3
storage3==0.11.0
streamlit==1.41.1
StrEnum==0.4.15
structlog==24.4.0
supabase==2.11.0
supafunc==0.9.0
tenacity==9.0.0
tf-playwright-stealth==1.1.0
tiktoken==0.8.0
tokenizers==0.21.0
toml==0.10.2
tornado==6.4.2
tqdm==4.67.1
typer==0.15.1
types-requests==2.32.0.20241016
typing-inspect==0.9.0
typing_extensions==4.12.2
tzdata==2024.2
ujson==5.10.0
urllib3==2.3.0
uvicorn==0.34.0
watchdog==6.0.0
watchfiles==1.0.4
websockets==13.1
wrapt==1.17.1
xxhash==3.5.0
yarl==1.18.3
zipp==3.21.0
zstandard==0.23.0

126
original_archon/iterations/v4-streamlit-ui-overhaul/run_docker.py
View File

@@ -1,126 +0,0 @@
#!/usr/bin/env python
"""
Simple script to build and run Archon Docker containers.
"""
import os
import subprocess
import platform
import time
from pathlib import Path
def run_command(command, cwd=None):
"""Run a command and print output in real-time."""
print(f"Running: {' '.join(command)}")
process = subprocess.Popen(
command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=False,
cwd=cwd
)
for line in process.stdout:
try:
decoded_line = line.decode('utf-8', errors='replace')
print(decoded_line.strip())
except Exception as e:
print(f"Error processing output: {e}")
process.wait()
return process.returncode
def check_docker():
"""Check if Docker is installed and running."""
try:
subprocess.run(
["docker", "--version"],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
return True
except (subprocess.SubprocessError, FileNotFoundError):
print("Error: Docker is not installed or not in PATH")
return False
def main():
"""Main function to build and run Archon containers."""
# Check if Docker is available
if not check_docker():
return 1
# Get the base directory
base_dir = Path(__file__).parent.absolute()
# Check for .env file
env_file = base_dir / ".env"
env_args = []
if env_file.exists():
print(f"Using environment file: {env_file}")
env_args = ["--env-file", str(env_file)]
else:
print("No .env file found. Continuing without environment variables.")
# Build the MCP container
print("\n=== Building Archon MCP container ===")
mcp_dir = base_dir / "mcp"
if run_command(["docker", "build", "-t", "archon-mcp:latest", "."], cwd=mcp_dir) != 0:
print("Error building MCP container")
return 1
# Build the main Archon container
print("\n=== Building main Archon container ===")
if run_command(["docker", "build", "-t", "archon:latest", "."], cwd=base_dir) != 0:
print("Error building main Archon container")
return 1
# Check if the container is already running
try:
result = subprocess.run(
["docker", "ps", "-q", "--filter", "name=archon-container"],
check=True,
capture_output=True,
text=True
)
if result.stdout.strip():
print("\n=== Stopping existing Archon container ===")
run_command(["docker", "stop", "archon-container"])
run_command(["docker", "rm", "archon-container"])
except subprocess.SubprocessError:
pass
# Run the Archon container
print("\n=== Starting Archon container ===")
cmd = [
"docker", "run", "-d",
"--name", "archon-container",
"-p", "8501:8501",
"-p", "8100:8100",
"--add-host", "host.docker.internal:host-gateway"
]
# Add environment variables if .env exists
if env_args:
cmd.extend(env_args)
# Add image name
cmd.append("archon:latest")
if run_command(cmd) != 0:
print("Error starting Archon container")
return 1
# Wait a moment for the container to start
time.sleep(2)
# Print success message
print("\n=== Archon is now running! ===")
print("-> Access the Streamlit UI at: http://localhost:8501")
print("-> MCP container is ready to use - see the MCP tab in the UI.")
print("\nTo stop Archon, run: docker stop archon-container && docker rm archon-container")
return 0
if __name__ == "__main__":
exit(main())

Some files were not shown because too many files have changed in this diff Show More