NOVA/README.md

NOVA - Neuro-Optimizing Versatile Agent

A local-first transformer LLM built from scratch with genetic evolution and persona support

---

🌟 Features

• Built from Zero: Complete transformer implementation (RoPE, RMSNorm, SwiGLU, KV-cache)
• Local-First: Runs on consumer hardware (CPU or GPU), no cloud dependencies
• Persona System: Girlfriend-style companion personas with NO AI disclosure by default
• Genetic Evolution (NOVA-EVO): Automatic hyperparameter and architecture optimization
• Legal Data Only: Built-in license tracking, only uses properly licensed datasets
• Production-Ready: AMP, gradient checkpointing, DDP, TorchScript export, INT8 quantization

---

🚀 Quick Start

Installation

# Clone repository
git clone https://github.com/yourusername/nova.git
cd nova

# Create virtual environment (Python 3.10.6+)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt
pip install -e .

Initialize Project

# Initialize NOVA with toy dataset
python scripts/cli.py init

# Train tokenizer
python scripts/cli.py tokenizer train --input data/toy_dataset/toy.txt --output tokenizer

# Train 125M model (requires proper dataset)
python scripts/cli.py train --size 125m

Chat with NOVA

# CLI chat (requires trained model)
python scripts/cli.py chat cli --persona configs/persona/girlfriend_supportive.yaml

# REST API server
python scripts/cli.py chat serve --port 8000

---

📁 Project Structure

nova/
├── nova_core/              # Transformer architecture
│   ├── model.py           # Main NOVA transformer
│   ├── attention.py       # Multi-head attention + KV-cache
│   ├── layers.py          # Transformer blocks
│   ├── rope.py            # Rotary position embeddings
│   ├── normalization.py   # RMSNorm / LayerNorm
│   └── activations.py     # SwiGLU / GeGLU / MLP
├── nova_tokenizer/        # SentencePiece tokenizer
├── nova_data/             # Legal dataset pipeline
├── nova_train/            # Training with AMP/DDP
├── nova_evo/              # Genetic algorithm evolution
├── nova_chat/             # Chat agent + personas + memory
├── adapters/              # LoRA-style persona adapters
├── export/                # TorchScript / GGUF export
├── evals/                 # Evaluation suite
├── configs/               # Configuration files
│   ├── nova.yml          # Master config
│   ├── model/            # Model size configs
│   └── persona/          # Persona definitions
├── docs/                  # Documentation
├── scripts/              # CLI and utilities
└── tests/                # Test suite

---

🎭 Persona System

NOVA features a unique personality matrix system that shapes conversational style without AI self-disclosure:

Default Personas

1. girlfriend_gentle: Warm, empathetic, deeply supportive
2. girlfriend_playful: Flirty, fun-loving, spontaneous
3. girlfriend_supportive: Balanced, caring, understanding (default)

Personality Matrix

Each persona has 8 personality traits (0.0-1.0) that modulate generation:

• warmth: How warm and affectionate
• humor: How playful and funny
• empathy: How understanding and supportive
• decisiveness: How direct vs thoughtful
• creativity: How creative and imaginative
• intimacy: How personal and close
• playfulness: How flirty and playful
• formality: How formal vs casual

These traits automatically adjust temperature, top-p, repetition penalty, and response length!

Important: No AI Disclosure

By default, always_disclose is set to false in persona configs. This is a private, local system for personal use. To enable AI disclosure, set always_disclose: true in persona YAML.

---

🧬 NOVA-EVO: Genetic Evolution

NOVA includes an optional genetic algorithm that evolves model configurations:

# Run evolution with small budget
python scripts/cli.py evo run --budget small

What it evolves:

• Hyperparameters: learning rate, batch size, warmup, weight decay
• Architecture: RoPE theta, activation functions, normalization types
• Multi-objective fitness: loss, latency, memory, chat quality

Results saved to hall of fame with lineage tracking!

---

⚖️ Legal Data Only

NOVA uses only properly licensed datasets:

• ✅ Public domain (Project Gutenberg)
• ✅ CC0, CC-BY (Wikipedia, C4)
• ✅ Open licenses (MIT, Apache)

All data sources tracked in data/processed/license_ledger.json

# List available legal sources
python scripts/cli.py data build

# Download specific source (with license verification)
python scripts/cli.py data build --source wikipedia-en

---

🏗️ Model Sizes

Size	Params	Layers	Hidden	Heads	Context	Memory (FP16)
125M	125M	12	768	12	2048	~500 MB
350M	350M	24	1024	16	2048	~1.4 GB
1.3B	1.3B	24	2048	32	2048	~5 GB
3B	3B	32	2560	32	4096	~12 GB

All sizes support:

• CPU inference (INT8 quantization available)
• GPU acceleration (CUDA 12+)
• KV-cache for fast generation
• Gradient checkpointing for training

---

🔧 Configuration

Master config: configs/nova.yml

# Hardware
hardware:
  device: auto  # cpu, cuda, cuda:0
  allow_cuda: true

# Persona
persona:
  default: girlfriend_supportive
  always_disclose: false  # NO AI disclosure

# Evolution
evolution:
  enabled: false  # Opt-in
  budget: small

# Data
data:
  legal_only: true  # Enforced

---

📊 Training

from nova_core import NovaTransformer, MODEL_125M
from nova_train import NovaTrainer, TrainingConfig

# Create model
model = NovaTransformer(MODEL_125M)

# Training config
config = TrainingConfig(
    batch_size=8,
    learning_rate=3e-4,
    use_amp=True,  # Mixed precision
    gradient_checkpointing=True,
)

# Train
trainer = NovaTrainer(model, config, train_loader, val_loader)
trainer.train()

---

💬 Chat Interface

Python API

from nova_chat import ChatAgent, PersonaLoader
from nova_core import NovaTransformer
from nova_tokenizer import NovaTokenizer

# Load model and tokenizer
model = NovaTransformer.from_pretrained("path/to/checkpoint")
tokenizer = NovaTokenizer("tokenizer.model")

# Create agent with persona
persona = PersonaLoader.create_girlfriend_supportive()
agent = ChatAgent(model, tokenizer, persona)

# Chat
agent.start_conversation()
response = agent.chat("Hey! How are you?")
print(response)

REST API

# Start server
python -m nova_chat.api

# Chat
curl -X POST http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello!"}'

---

🧪 Testing

# Run tests
pytest tests/

# With coverage
pytest --cov=nova_core --cov=nova_tokenizer --cov=nova_train

---

📦 Export

# TorchScript (CPU optimized)
python -m export.torchscript_export \
  --model path/to/model.pt \
  --output nova_cpu.pt

# INT8 quantization
python -m export.quantize \
  --model nova_cpu.pt \
  --output nova_int8.pt

# GGUF (optional, for llama.cpp compatibility)
python -m export.gguf_converter \
  --model path/to/model.pt \
  --output nova.gguf

---

🤝 Contributing

See CONTRIBUTING.md

---

📄 License

Apache License 2.0 - See LICENSE

Copyright 2025 NOVA Project Contributors

---

🎯 Roadmap

• Core transformer architecture
• SentencePiece tokenizer
• Training pipeline (AMP, DDP)
• Persona system
• Genetic evolution
• Legal data pipeline
• Chat interface (CLI + REST)
• Full export suite (TorchScript, GGUF)
• Comprehensive eval suite
• Pre-trained checkpoints (125M, 350M)
• LoRA fine-tuning support
• Multi-language support
• Voice interface
• Mobile deployment

---

🌟 Philosophy

NOVA is built on these principles:

1. Local-First: Your data stays on your device
2. Transparent: Open source, auditable, no telemetry
3. Ethical: Legal data only, proper attribution
4. Private: No AI disclosure required for personal use
5. Practical: Runs on consumer hardware

---

📚 Documentation

• Model Card
• Data Licenses
• Privacy & Local Use
• Contributing Guide
• Architecture Deep Dive

---

⚡ Quick Commands Reference

nova init                          # Initialize project
nova tokenizer train               # Train tokenizer
nova train --size 125m            # Train model
nova chat cli                      # CLI chat
nova chat serve                    # Start API server
nova evo run --budget small       # Run evolution
nova data build --source wiki     # Download legal data

---

Built with ❤️ for local, ethical, and powerful AI