Improve development environment with Docker Compose profiles (#435)

* Add improved development environment with backend in Docker and frontend locally

- Created dev.bat script to run backend services in Docker and frontend locally
- Added docker-compose.backend.yml for backend-only Docker setup
- Updated package.json to run frontend on port 3737
- Fixed api.ts to use default port 8181 instead of throwing error
- Script automatically stops production containers to avoid port conflicts
- Provides instant HMR for frontend development

* Refactor development environment setup: replace dev.bat with Makefile for cross-platform support and enhanced commands

* Enhance development environment: add environment variable checks and update test commands for frontend and backend

* Improve development environment with Docker Compose profiles

This commit enhances the development workflow by replacing the separate
docker-compose.backend.yml file with Docker Compose profiles, fixing
critical service discovery issues, and adding comprehensive developer
tooling through an improved Makefile system.

Key improvements:
- Replace docker-compose.backend.yml with cleaner profile approach
- Fix service discovery by maintaining consistent container names
- Fix port mappings (3737:3737 instead of 3737:5173)
- Add make doctor for environment validation
- Fix port configuration and frontend HMR
- Improve error handling with .SHELLFLAGS in Makefile
- Add comprehensive port configuration via environment variables
- Simplify make dev-local to only run essential services
- Add logging directory creation for local development
- Document profile strategy in docker-compose.yml

These changes provide three flexible development modes:
- Hybrid mode (default): Backend in Docker, frontend local with HMR
- Docker mode: Everything in Docker for production-like testing
- Local mode: API server and UI run locally

Co-authored-by: Zak Stam <zaksnet@users.noreply.github.com>

* Fix make stop command to properly handle Docker Compose profiles

The stop command now explicitly specifies all profiles to ensure
all containers are stopped regardless of how they were started.

* Fix README to document correct make commands

- Changed 'make lint' to 'make lint-frontend' and 'make lint-backend'
- Removed non-existent 'make logs-server' command
- Added 'make watch-mcp' and 'make watch-agents' commands
- All documented make commands now match what's available in Makefile

* fix: Address critical issues from code review #435

- Create robust environment validation script (check-env.js) that properly parses .env files
- Fix Docker healthcheck port mismatch (5173 -> 3737)
- Remove hard-coded port flags from package.json to allow environment configuration
- Fix Docker detection logic using /.dockerenv instead of HOSTNAME
- Normalize container names to lowercase (archon-server, archon-mcp, etc.)
- Improve stop-local command with port-based fallback for process killing
- Fix API configuration fallback chain to include VITE_PORT
- Fix Makefile shell variable expansion using runtime evaluation
- Update .PHONY targets with comprehensive list
- Add --profile flags to Docker Compose commands in README
- Add VITE_ARCHON_SERVER_PORT to docker-compose.yml
- Add Node.js 18+ to prerequisites
- Use dynamic ports in Makefile help messages
- Add lint alias combining frontend and backend linting
- Update .env.example documentation
- Scope .gitignore logs entry to /logs/

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix container name resolution for MCP server

- Add dynamic container name resolution with three-tier strategy
- Support environment variables for custom container names
- Add service discovery labels to docker-compose services
- Update BackendStartupError with correct container name references

* Fix frontend test failures in API configuration tests

- Update environment variable names to use VITE_ prefix that matches production code
- Fix MCP client service tests to use singleton instance export
- Update default behavior tests to expect fallback to port 8181
- All 77 frontend tests now pass

* Fix make stop-local to avoid Docker daemon interference

Replace aggressive kill -9 with targeted process termination:
- Filter processes by command name (node/vite/python/uvicorn) before killing
- Use graceful SIGTERM instead of SIGKILL
- Add process verification to avoid killing Docker-related processes
- Improve logging with descriptive step messages

* refactor: Simplify development workflow based on comprehensive review

- Reduced Makefile from 344 lines (43 targets) to 83 lines (8 essential targets)
- Removed unnecessary environment variables (*_CONTAINER_NAME variables)
- Fixed Windows compatibility by removing Unix-specific commands
- Added security fixes to check-env.js (path validation)
- Simplified MCP container discovery to use fixed container names
- Fixed 'make stop' to properly handle Docker Compose profiles
- Updated documentation to reflect simplified workflow
- Restored original .env.example with comprehensive Supabase key documentation

This addresses all critical issues from code review:
- Cross-platform compatibility ✅
- Security vulnerabilities fixed ✅
- 81% reduction in complexity ✅
- Maintains all essential functionality ✅

All tests pass: Frontend (77/77), Backend (267/267)

* feat: Add granular test and lint commands to Makefile

- Split test command into test-fe and test-be for targeted testing
- Split lint command into lint-fe and lint-be for targeted linting
- Keep original test and lint commands that run both
- Update help text with new commands for better developer experience

* feat: Improve Docker Compose detection and prefer modern syntax

- Prefer 'docker compose' (plugin) over 'docker-compose' (standalone)
- Add better error handling in Makefile with proper exit on failures
- Add Node.js check before running environment scripts
- Pass environment variables correctly to frontend in hybrid mode
- Update all documentation to use modern 'docker compose' syntax
- Auto-detect which Docker Compose version is available

* docs: Update CONTRIBUTING.md to reflect simplified development workflow

- Add Node.js 18+ as prerequisite for hybrid development
- Mark Make as optional throughout the documentation
- Update all docker-compose commands to modern 'docker compose' syntax
- Add Make command alternatives for testing (make test, test-fe, test-be)
- Document make dev for hybrid development mode
- Remove linting requirements until codebase errors are resolved

* fix: Rename frontend service to archon-frontend for consistency

Aligns frontend service naming with other services (archon-server, archon-mcp, archon-agents) for better consistency in Docker image naming patterns.

---------

Co-authored-by: Zak Stam <zakscomputers@hotmail.com>
Co-authored-by: Zak Stam <zaksnet@users.noreply.github.com>

python/src/server/api_routes/mcp_api.py

@@ -49,7 +49,7 @@ class MCPServerManager:
     """Manages the MCP Docker container lifecycle."""
 
     def __init__(self):
-        self.container_name = "Archon-MCP"  # Container name from docker-compose.yml
+        self.container_name = None  # Will be resolved dynamically
         self.docker_client = None
         self.container = None
         self.status: str = "stopped"
@@ -62,16 +62,29 @@ class MCPServerManager:
         self._min_operation_interval = 2.0  # Minimum 2 seconds between operations
         self._initialize_docker_client()
 
+    def _resolve_container(self):
+        """Simple container resolution - just use fixed name."""
+        if not self.docker_client:
+            return None
+        
+        try:
+            # Simple: Just look for the fixed container name
+            container = self.docker_client.containers.get("archon-mcp")
+            self.container_name = "archon-mcp"
+            mcp_logger.info("Found MCP container")
+            return container
+        except NotFound:
+            mcp_logger.warning("MCP container not found - is it running?")
+            self.container_name = "archon-mcp"
+            return None
+
     def _initialize_docker_client(self):
         """Initialize Docker client and get container reference."""
         try:
             self.docker_client = docker.from_env()
-            try:
-                self.container = self.docker_client.containers.get(self.container_name)
-                mcp_logger.info(f"Found Docker container: {self.container_name}")
-            except NotFound:
-                mcp_logger.warning(f"Docker container {self.container_name} not found")
-                self.container = None
+            self.container = self._resolve_container()
+            if not self.container:
+                mcp_logger.warning("MCP container not found during initialization")
         except Exception as e:
             mcp_logger.error(f"Failed to initialize Docker client: {str(e)}")
             self.docker_client = None
@@ -85,10 +98,17 @@ class MCPServerManager:
             if self.container:
                 self.container.reload()  # Refresh container info
             else:
-                self.container = self.docker_client.containers.get(self.container_name)
+                # Try to resolve container again if we don't have it
+                self.container = self._resolve_container()
+                if not self.container:
+                    return "not_found"
 
             return self.container.status
         except NotFound:
+            # Try to resolve again in case container was recreated
+            self.container = self._resolve_container()
+            if self.container:
+                return self.container.status
             return "not_found"
         except Exception as e:
             mcp_logger.error(f"Error getting container status: {str(e)}")