Mai/.planning/codebase/STRUCTURE.md

Codebase Structure

Analysis Date: 2026-01-26

Directory Layout

mai/
├── src/
│   ├── __main__.py              # CLI entry point
│   ├── mai.py                   # Core Mai class, orchestration
│   ├── models/
│   │   ├── __init__.py
│   │   ├── adapter.py           # Base model adapter interface
│   │   ├── ollama_adapter.py    # Ollama/LMStudio implementation
│   │   ├── model_manager.py     # Model selection and switching logic
│   │   └── resource_monitor.py  # CPU, RAM, GPU tracking
│   ├── memory/
│   │   ├── __init__.py
│   │   ├── store.py             # SQLite conversation store
│   │   ├── vector_search.py     # Semantic similarity search
│   │   ├── compression.py       # History compression and summarization
│   │   └── pattern_extractor.py # Learning and pattern recognition
│   ├── conversation/
│   │   ├── __init__.py
│   │   ├── engine.py            # Main conversation orchestration
│   │   ├── context_manager.py   # Token budget and window management
│   │   ├── turn_handler.py      # Single turn processing
│   │   └── reasoning.py         # Reasoning transparency and clarification
│   ├── personality/
│   │   ├── __init__.py
│   │   ├── core_rules.py        # Unshakeable core values enforcement
│   │   ├── learned_behaviors.py # Personality adaptation from interactions
│   │   ├── guardrails.py        # Safety constraints and refusal logic
│   │   └── config_loader.py     # YAML personality configuration
│   ├── safety/
│   │   ├── __init__.py
│   │   ├── executor.py          # Docker sandbox execution wrapper
│   │   ├── risk_analyzer.py     # Risk classification (LOW/MEDIUM/HIGH/BLOCKED)
│   │   ├── ast_validator.py     # Syntax and import validation
│   │   └── audit_log.py         # Immutable execution history
│   ├── selfmod/
│   │   ├── __init__.py
│   │   ├── analyzer.py          # Code analysis and improvement detection
│   │   ├── generator.py         # Improvement code generation
│   │   ├── scheduler.py         # Periodic and on-demand analysis trigger
│   │   ├── reviewer.py          # Second-agent review coordination
│   │   └── git_manager.py       # Git commit integration
│   ├── interfaces/
│   │   ├── __init__.py
│   │   ├── cli.py               # CLI chat interface
│   │   ├── discord_bot.py       # Discord bot implementation
│   │   ├── message_handler.py   # Shared message processing
│   │   ├── approval_handler.py  # Change approval workflow
│   │   └── offline_queue.py     # Message queueing during disconnection
│   └── utils/
│       ├── __init__.py
│       ├── config.py            # Configuration loading
│       ├── logging.py           # Structured logging setup
│       ├── validators.py        # Input validation helpers
│       └── helpers.py           # Shared utility functions
├── config/
│   ├── personality.yaml         # Core personality configuration
│   ├── models.yaml              # Model definitions and resource limits
│   ├── safety_rules.yaml        # Risk assessment rules
│   └── logging.yaml             # Logging configuration
├── tests/
│   ├── unit/
│   │   ├── test_models.py
│   │   ├── test_memory.py
│   │   ├── test_conversation.py
│   │   ├── test_personality.py
│   │   ├── test_safety.py
│   │   └── test_selfmod.py
│   ├── integration/
│   │   ├── test_conversation_flow.py
│   │   ├── test_selfmod_workflow.py
│   │   └── test_interfaces.py
│   └── fixtures/
│       ├── mock_models.py
│       ├── test_data.py
│       └── sample_conversations.json
├── scripts/
│   ├── setup_ollama.py          # Initial model downloading
│   ├── init_db.py               # Database schema initialization
│   └── verify_environment.py    # Pre-flight checks
├── docker/
│   └── Dockerfile               # Sandbox execution environment
├── .env.example                 # Environment variables template
├── pyproject.toml               # Project metadata and dependencies
├── requirements.txt             # Python dependencies
├── pytest.ini                   # Test configuration
├── Makefile                     # Development commands
└── README.md                    # Project overview

Directory Purposes

src/:

• Purpose: All application code
• Contains: Python modules organized by architectural layer
• Key files: mai.py (core), __main__.py (CLI entry)

src/models/:

• Purpose: Model inference abstraction
• Contains: Adapter interfaces, Ollama client, resource monitoring
• Key files: model_manager.py (selection logic), resource_monitor.py (constraints)

src/memory/:

• Purpose: Persistent storage and retrieval
• Contains: SQLite operations, vector search, compression
• Key files: store.py (main interface), vector_search.py (semantic search)

src/conversation/:

• Purpose: Multi-turn conversation orchestration
• Contains: Turn handling, context windowing, reasoning transparency
• Key files: engine.py (main coordinator), context_manager.py (token budget)

src/personality/:

• Purpose: Values enforcement and personality adaptation
• Contains: Core rules, learned behaviors, guardrails
• Key files: core_rules.py (unshakeable values), learned_behaviors.py (adaptation)

src/safety/:

• Purpose: Code execution sandboxing and risk assessment
• Contains: Docker wrapper, AST validation, risk classification, audit logging
• Key files: executor.py (sandbox wrapper), risk_analyzer.py (classification)

src/selfmod/:

• Purpose: Autonomous code improvement and review
• Contains: Code analysis, improvement generation, approval workflow
• Key files: analyzer.py (detection), reviewer.py (second-agent coordination)

src/interfaces/:

• Purpose: External communication adapters
• Contains: CLI handler, Discord bot, approval system
• Key files: cli.py (terminal UI), discord_bot.py (Discord integration)

src/utils/:

• Purpose: Shared utilities and helpers
• Contains: Configuration loading, logging, validation
• Key files: config.py (env/file loading), logging.py (structured logs)

config/:

• Purpose: Non-code configuration files
• Contains: YAML personality, models, safety rules definitions
• Key files: personality.yaml (core values), models.yaml (resource profiles)

tests/:

• Purpose: Test suites organized by type
• Contains: Unit tests (layer isolation), integration tests (flows), fixtures (test data)
• Key files: Each test file mirrors src/ structure

scripts/:

• Purpose: One-off setup and maintenance scripts
• Contains: Database initialization, environment verification
• Key files: setup_ollama.py (first-time model setup)

docker/:

• Purpose: Container configuration for sandboxed execution
• Contains: Dockerfile for isolation environment
• Key files: Dockerfile (build recipe)

Key File Locations

Entry Points:

• src/__main__.py: CLI entry, python -m mai launches here
• src/interfaces/discord_bot.py: Discord bot main loop
• src/mai.py: Core Mai class, system initialization

Configuration:

• config/personality.yaml: Core values, interaction patterns, refusal rules
• config/models.yaml: Available models, resource requirements, context windows
• .env.example: Required environment variables template

Core Logic:

• src/mai.py: Main orchestration
• src/conversation/engine.py: Conversation turn processing
• src/selfmod/analyzer.py: Improvement opportunity detection
• src/safety/executor.py: Safe code execution

Testing:

• tests/unit/: Layer-isolated tests (no dependencies between layers)
• tests/integration/: End-to-end flow tests
• tests/fixtures/: Mock objects and test data

Naming Conventions

Files:

• Module files: snake_case.py (e.g., model_manager.py)
• Entry points: __main__.py for packages, standalone scripts at package root
• Config files: snake_case.yaml (e.g., personality.yaml)
• Test files: test_*.py (e.g., test_conversation.py)

Directories:

• Feature areas: snake_case (e.g., src/selfmod/)
• No abbreviations except selfmod (self-modification) which is project standard
• Each layer is a top-level directory under src/

Functions/Classes:

• Classes: PascalCase (e.g., ModelManager, ConversationEngine)
• Functions: snake_case (e.g., generate_response(), validate_code())
• Constants: UPPER_SNAKE_CASE (e.g., MAX_CONTEXT_TOKENS)
• Private methods/functions: prefix with _ (e.g., _internal_method())

Types:

• Use type hints throughout: def process(msg: str) -> str:
• Complex types in src/utils/types.py or local to module

Where to Add New Code

New Feature (e.g., new communication interface like Slack):

• Primary code: src/interfaces/slack_adapter.py (new adapter following discord_bot.py pattern)
• Tests: tests/unit/test_slack_adapter.py and tests/integration/test_slack_interface.py
• Configuration: Add to src/interfaces/__init__.py imports and config/interfaces.yaml if needed
• Entry hook: Modify src/mai.py to initialize new adapter

New Component/Module (e.g., advanced memory with graph databases):

• Implementation: src/memory/graph_store.py (new module in appropriate layer)
• Interface: Follow existing patterns (e.g., inherit from src/memory/store.py base)
• Tests: Corresponding test in tests/unit/test_memory.py or new file if complex
• Integration: Modify src/mai.py initialization to use new component with feature flag

Utilities (e.g., new helper function):

• Shared helpers: src/utils/helpers.py (functions) or new file like src/utils/math_utils.py if substantial
• Internal helpers: Keep in the module where used (don't over-extract)
• Tests: Add to tests/unit/test_utils.py

Configuration:

• Static rules: Add to appropriate YAML in config/
• Dynamic config: Load in src/utils/config.py
• Env-driven: Add to .env.example with documentation

Special Directories

tests/fixtures/:

• Purpose: Reusable test data and mock objects
• Generated: No, hand-created
• Committed: Yes, part of repository

config/:

• Purpose: Non-code configuration
• Generated: No, hand-maintained
• Committed: Yes, except secrets (use .env)

.env (not committed):

• Purpose: Local environment overrides and secrets
• Generated: No, copied from .env.example and filled locally
• Committed: No (in .gitignore)

docker/:

• Purpose: Sandbox environment for safe execution
• Generated: No, hand-maintained
• Committed: Yes

---

Structure analysis: 2026-01-26