Initial commit: Desktop Waifu MVP foundation

- Project structure with modular architecture
- State management system (emotion states, conversation history)
- Transparent, draggable PyQt6 window
- OpenGL rendering widget with placeholder cube
- Discord bot framework (commands, event handlers)
- Complete documentation (README, project plan, research findings)
- Environment configuration template
- Dependencies defined in requirements.txt

MVP features working:
- Transparent window appears at bottom-right
- Window can be dragged around
- Placeholder 3D cube renders and rotates
- Emotion state changes on interaction
- Event-driven state management

Next steps: VRM model loading and rendering
This commit is contained in:
2025-09-30 18:42:54 -04:00
commit a657979bfd
20 changed files with 1178 additions and 0 deletions

105
RESEARCH_FINDINGS.md Normal file
View File

@@ -0,0 +1,105 @@
# VRM Rendering Research Findings
## Date: 2025-09-30
## Problem
Need to render VRM 3D model in a transparent desktop window using Python.
## VRM Format
- VRM is based on glTF 2.0 with VRM-specific extensions
- Uses MToon shader for toon/cel-shaded rendering
- Contains blend shapes for facial expressions
- Contains bone data for animations
## Python VRM/glTF Libraries
### pygltflib (RECOMMENDED)
- **Pros**: Can parse VRM files, read extension data, well-maintained
- **Cons**: Parsing only, doesn't handle rendering
- **Use**: Load VRM data, extract meshes/materials/animations
### Alternatives
- `gltflib`: Fork of pygltflib with divergent features
- `pyrender`: Can render glTF but **doesn't support MToon shader properly** (produces holes)
## Rendering Solutions
### ✅ CHOSEN: PyQt6 + QOpenGLWidget
**Best fit for desktop widget**
**Pros:**
- Native transparent window support (`WA_TranslucentBackground`)
- Always-on-top windows (`WindowStaysOnTopHint`)
- Frameless windows (`FramelessWindowHint`)
- Direct OpenGL rendering via QOpenGLWidget
- Excellent for desktop widgets/mascots
- Can implement custom MToon shader in GLSL
**Implementation:**
```python
class VRMWidget(QOpenGLWidget):
def initializeGL(self):
# Setup OpenGL state, load shaders
def resizeGL(self, w, h):
# Handle window resize
def paintGL(self):
# Render VRM model
```
**Window Setup:**
```python
window.setAttribute(Qt.WA_TranslucentBackground)
window.setWindowFlags(Qt.FramelessWindowHint | Qt.WindowStaysOnTopHint)
```
### Alternative: Panda3D
**Not chosen due to:**
- Limited transparent window support
- Requires embedding in another GUI framework anyway
- More complexity for our use case
### Alternative: Blender Python API
**Not chosen due to:**
- Too heavyweight for a desktop widget
- Not designed for real-time desktop applications
## MToon Shader Implementation
Since Python libraries don't support MToon, we need to:
1. Parse VRM using pygltflib to get material properties
2. Implement custom GLSL MToon shader in OpenGL
3. Apply shader during rendering
MToon shader resources:
- Reference: https://github.com/Santarh/MToon
- Can port GLSL version to PyQt's OpenGL context
## Recommended Tech Stack
### Core
- **PyQt6**: Window management, transparency, always-on-top
- **QOpenGLWidget**: OpenGL rendering context
- **pygltflib**: VRM/glTF parsing
- **PyOpenGL**: OpenGL bindings for Python
### Audio
- **pygame**: Sound effects (squeaks, etc.)
- **pyttsx3** or **Coqui TTS**: Text-to-speech
- **whisper.cpp Python bindings**: Speech-to-text
### LLM
- **llama-cpp-python**: Local LLM inference
- **ollama-python**: Alternative if using Ollama
### Discord
- **discord.py**: Discord bot
### Packaging
- **PyInstaller** or **Nuitka**: .exe compilation
## Next Steps
1. Install PyQt6, PyOpenGL, pygltflib
2. Create basic transparent window
3. Load VRM using pygltflib
4. Implement basic OpenGL renderer
5. Port MToon shader to GLSL
6. Add interactivity (drag, click)