Desktop Waifu 🎀

An AI-powered desktop companion with VRM model rendering and Discord integration.

Features

Current (MVP)

• ✅ Transparent desktop widget
• ✅ Draggable VRM character
• ✅ Always-on-top window
• ✅ Basic state management
• ⏳ VRM model rendering (in progress)
• ⏳ Sound effects on interaction
• ⏳ Text chat interface
• ⏳ Local LLM integration
• ⏳ Expression changes based on emotion
• ⏳ Discord bot integration

Planned

• Voice input/output (STT/TTS)
• Advanced animations
• System integration (notifications, app control)
• Memory persistence
• Proactive messaging
• Cross-platform support

Setup

Prerequisites

• Python 3.10+
• VRM model file (.vrm)
• Local LLM (Ollama, llama.cpp, etc.) - optional for testing

Installation

1. Clone/Download the project

2. Install dependencies:

pip install -r requirements.txt

3. Configure environment:

cp .env.example .env
# Edit .env with your settings

4. Add your VRM model:

   • Place your .vrm file in the models/ folder
   • Update VRM_MODEL_PATH in .env

5. Add sound effects (optional):

   • Place .wav files in assets/sounds/
   • Examples: squeak.wav, click.wav

Discord Setup (Optional)

See DISCORD_SETUP.md for detailed instructions.

Quick setup:

1. Create bot at https://discord.com/developers/applications
2. Enable "Message Content Intent" in Bot settings
3. Copy bot token to DISCORD_BOT_TOKEN in .env
4. Invite bot to your server using OAuth2 URL generator
5. Bot will automatically start with the desktop app!

LLM Setup (Optional)

Using Ollama:

# Install Ollama from https://ollama.ai
ollama pull llama2
# Update .env: LLM_API_URL=http://localhost:11434

Using llama.cpp:

pip install llama-cpp-python
# Configure model path in code

Running

python main.py

Usage

Desktop Interactions

• Click and drag: Move the waifu around your screen
• Double-click: Open chat interface (coming soon)
• Right-click: Context menu (coming soon)

Discord Commands

• !hello: Greet the waifu
• !status: Check current mood/status
• @mention: Talk to the waifu in any channel
• DM: Send direct messages

Project Structure

Waifu/
├── main.py                 # Entry point
├── requirements.txt        # Dependencies
├── .env                    # Configuration (create from .env.example)
├── PROJECT_PLAN.md         # Development plan
├── RESEARCH_FINDINGS.md    # Technical research
├── models/                 # VRM model files
│   └── .gitkeep
├── assets/
│   └── sounds/             # Sound effects
│       └── .gitkeep
└── src/
    ├── core/
    │   └── state_manager.py    # State synchronization
    ├── ui/
    │   ├── waifu_window.py     # Main window
    │   └── vrm_widget.py       # VRM renderer
    ├── discord_bot/
    │   └── bot.py              # Discord integration
    ├── llm/                    # LLM integration (TODO)
    └── audio/                  # Audio system (TODO)

Development Status

Phase 1: MVP (In Progress)

• Project structure
• State management system
• Transparent window framework
• Drag functionality
• Basic OpenGL setup
• VRM model loading
• VRM rendering with MToon shader
• Sound effects
• Chat interface
• LLM integration
• Discord bot
• Expression control

Phase 2: Enhancement (Planned)

• Voice I/O
• Advanced animations
• System integration
• Memory/database
• .exe packaging

Technical Details

Architecture

• UI: PyQt6 with transparent windows
• Rendering: OpenGL via QOpenGLWidget
• VRM Parsing: pygltflib
• State Management: Custom event-driven system
• Discord: discord.py
• LLM: Flexible (Ollama, llama.cpp, custom)

VRM Rendering

• VRM models are glTF 2.0 with extensions
• Custom MToon shader implementation needed
• Blend shapes for facial expressions
• Bone animations for gestures

See RESEARCH_FINDINGS.md for detailed technical research.

Contributing

This is a personal project, but suggestions and feedback are welcome!

License

TBD

Acknowledgments

• VRM Consortium for the VRM specification
• PyQt project for the excellent GUI framework
• discord.py for Discord integration