Dani
a657979bfd
- Project structure with modular architecture - State management system (emotion states, conversation history) - Transparent, draggable PyQt6 window - OpenGL rendering widget with placeholder cube - Discord bot framework (commands, event handlers) - Complete documentation (README, project plan, research findings) - Environment configuration template - Dependencies defined in requirements.txt MVP features working: - Transparent window appears at bottom-right - Window can be dragged around - Placeholder 3D cube renders and rotates - Emotion state changes on interaction - Event-driven state management Next steps: VRM model loading and rendering
183 lines
4.3 KiB
Markdown
183 lines
4.3 KiB
Markdown
# Desktop Waifu 🎀
|
|
|
|
An AI-powered desktop companion with VRM model rendering and Discord integration.
|
|
|
|
## Features
|
|
|
|
### Current (MVP)
|
|
- ✅ Transparent desktop widget
|
|
- ✅ Draggable VRM character
|
|
- ✅ Always-on-top window
|
|
- ✅ Basic state management
|
|
- ⏳ VRM model rendering (in progress)
|
|
- ⏳ Sound effects on interaction
|
|
- ⏳ Text chat interface
|
|
- ⏳ Local LLM integration
|
|
- ⏳ Expression changes based on emotion
|
|
- ⏳ Discord bot integration
|
|
|
|
### Planned
|
|
- Voice input/output (STT/TTS)
|
|
- Advanced animations
|
|
- System integration (notifications, app control)
|
|
- Memory persistence
|
|
- Proactive messaging
|
|
- Cross-platform support
|
|
|
|
## Setup
|
|
|
|
### Prerequisites
|
|
- Python 3.10+
|
|
- VRM model file (`.vrm`)
|
|
- Local LLM (Ollama, llama.cpp, etc.) - optional for testing
|
|
|
|
### Installation
|
|
|
|
1. **Clone/Download the project**
|
|
|
|
2. **Install dependencies:**
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
3. **Configure environment:**
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env with your settings
|
|
```
|
|
|
|
4. **Add your VRM model:**
|
|
- Place your `.vrm` file in the `models/` folder
|
|
- Update `VRM_MODEL_PATH` in `.env`
|
|
|
|
5. **Add sound effects (optional):**
|
|
- Place `.wav` files in `assets/sounds/`
|
|
- Examples: `squeak.wav`, `click.wav`
|
|
|
|
### Discord Setup (Optional)
|
|
|
|
1. Create a Discord bot at https://discord.com/developers/applications
|
|
2. Enable these intents:
|
|
- Message Content Intent
|
|
- Server Members Intent
|
|
3. Copy bot token to `DISCORD_BOT_TOKEN` in `.env`
|
|
4. Invite bot to your server with permissions:
|
|
- Send Messages
|
|
- Read Message History
|
|
- Use Slash Commands
|
|
|
|
### LLM Setup (Optional)
|
|
|
|
#### Using Ollama:
|
|
```bash
|
|
# Install Ollama from https://ollama.ai
|
|
ollama pull llama2
|
|
# Update .env: LLM_API_URL=http://localhost:11434
|
|
```
|
|
|
|
#### Using llama.cpp:
|
|
```bash
|
|
pip install llama-cpp-python
|
|
# Configure model path in code
|
|
```
|
|
|
|
## Running
|
|
|
|
```bash
|
|
python main.py
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Desktop Interactions
|
|
- **Click and drag**: Move the waifu around your screen
|
|
- **Double-click**: Open chat interface (coming soon)
|
|
- **Right-click**: Context menu (coming soon)
|
|
|
|
### Discord Commands
|
|
- `!hello`: Greet the waifu
|
|
- `!status`: Check current mood/status
|
|
- `@mention`: Talk to the waifu in any channel
|
|
- **DM**: Send direct messages
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
Waifu/
|
|
├── main.py # Entry point
|
|
├── requirements.txt # Dependencies
|
|
├── .env # Configuration (create from .env.example)
|
|
├── PROJECT_PLAN.md # Development plan
|
|
├── RESEARCH_FINDINGS.md # Technical research
|
|
├── models/ # VRM model files
|
|
│ └── .gitkeep
|
|
├── assets/
|
|
│ └── sounds/ # Sound effects
|
|
│ └── .gitkeep
|
|
└── src/
|
|
├── core/
|
|
│ └── state_manager.py # State synchronization
|
|
├── ui/
|
|
│ ├── waifu_window.py # Main window
|
|
│ └── vrm_widget.py # VRM renderer
|
|
├── discord_bot/
|
|
│ └── bot.py # Discord integration
|
|
├── llm/ # LLM integration (TODO)
|
|
└── audio/ # Audio system (TODO)
|
|
```
|
|
|
|
## Development Status
|
|
|
|
### Phase 1: MVP (In Progress)
|
|
- [x] Project structure
|
|
- [x] State management system
|
|
- [x] Transparent window framework
|
|
- [x] Drag functionality
|
|
- [x] Basic OpenGL setup
|
|
- [ ] VRM model loading
|
|
- [ ] VRM rendering with MToon shader
|
|
- [ ] Sound effects
|
|
- [ ] Chat interface
|
|
- [ ] LLM integration
|
|
- [ ] Discord bot
|
|
- [ ] Expression control
|
|
|
|
### Phase 2: Enhancement (Planned)
|
|
- [ ] Voice I/O
|
|
- [ ] Advanced animations
|
|
- [ ] System integration
|
|
- [ ] Memory/database
|
|
- [ ] .exe packaging
|
|
|
|
## Technical Details
|
|
|
|
### Architecture
|
|
- **UI**: PyQt6 with transparent windows
|
|
- **Rendering**: OpenGL via QOpenGLWidget
|
|
- **VRM Parsing**: pygltflib
|
|
- **State Management**: Custom event-driven system
|
|
- **Discord**: discord.py
|
|
- **LLM**: Flexible (Ollama, llama.cpp, custom)
|
|
|
|
### VRM Rendering
|
|
- VRM models are glTF 2.0 with extensions
|
|
- Custom MToon shader implementation needed
|
|
- Blend shapes for facial expressions
|
|
- Bone animations for gestures
|
|
|
|
See `RESEARCH_FINDINGS.md` for detailed technical research.
|
|
|
|
## Contributing
|
|
|
|
This is a personal project, but suggestions and feedback are welcome!
|
|
|
|
## License
|
|
|
|
TBD
|
|
|
|
## Acknowledgments
|
|
|
|
- VRM Consortium for the VRM specification
|
|
- PyQt project for the excellent GUI framework
|
|
- discord.py for Discord integration
|