Files

Dani a657979bfd Initial commit: Desktop Waifu MVP foundation

- Project structure with modular architecture
- State management system (emotion states, conversation history)
- Transparent, draggable PyQt6 window
- OpenGL rendering widget with placeholder cube
- Discord bot framework (commands, event handlers)
- Complete documentation (README, project plan, research findings)
- Environment configuration template
- Dependencies defined in requirements.txt

MVP features working:
- Transparent window appears at bottom-right
- Window can be dragged around
- Placeholder 3D cube renders and rotates
- Emotion state changes on interaction
- Event-driven state management

Next steps: VRM model loading and rendering

2025-09-30 18:42:54 -04:00

2.9 KiB

Raw Blame History

VRM Rendering Research Findings

Date: 2025-09-30

Problem

Need to render VRM 3D model in a transparent desktop window using Python.

VRM Format

VRM is based on glTF 2.0 with VRM-specific extensions
Uses MToon shader for toon/cel-shaded rendering
Contains blend shapes for facial expressions
Contains bone data for animations

Python VRM/glTF Libraries

pygltflib (RECOMMENDED)

Pros: Can parse VRM files, read extension data, well-maintained
Cons: Parsing only, doesn't handle rendering
Use: Load VRM data, extract meshes/materials/animations

Alternatives

gltflib: Fork of pygltflib with divergent features
pyrender: Can render glTF but doesn't support MToon shader properly (produces holes)

Rendering Solutions

✅ CHOSEN: PyQt6 + QOpenGLWidget

Best fit for desktop widget

Pros:

Native transparent window support (WA_TranslucentBackground)
Always-on-top windows (WindowStaysOnTopHint)
Frameless windows (FramelessWindowHint)
Direct OpenGL rendering via QOpenGLWidget
Excellent for desktop widgets/mascots
Can implement custom MToon shader in GLSL

Implementation:

class VRMWidget(QOpenGLWidget):
    def initializeGL(self):
        # Setup OpenGL state, load shaders
    def resizeGL(self, w, h):
        # Handle window resize
    def paintGL(self):
        # Render VRM model

Window Setup:

window.setAttribute(Qt.WA_TranslucentBackground)
window.setWindowFlags(Qt.FramelessWindowHint | Qt.WindowStaysOnTopHint)

Alternative: Panda3D

Not chosen due to:

Limited transparent window support
Requires embedding in another GUI framework anyway
More complexity for our use case

Alternative: Blender Python API

Not chosen due to:

Too heavyweight for a desktop widget
Not designed for real-time desktop applications

MToon Shader Implementation

Since Python libraries don't support MToon, we need to:

Parse VRM using pygltflib to get material properties
Implement custom GLSL MToon shader in OpenGL
Apply shader during rendering

MToon shader resources:

Reference: https://github.com/Santarh/MToon
Can port GLSL version to PyQt's OpenGL context

Recommended Tech Stack

Core

PyQt6: Window management, transparency, always-on-top
QOpenGLWidget: OpenGL rendering context
pygltflib: VRM/glTF parsing
PyOpenGL: OpenGL bindings for Python

Audio

pygame: Sound effects (squeaks, etc.)
pyttsx3 or Coqui TTS: Text-to-speech
whisper.cpp Python bindings: Speech-to-text

LLM

llama-cpp-python: Local LLM inference
ollama-python: Alternative if using Ollama

Discord

discord.py: Discord bot

Packaging

PyInstaller or Nuitka: .exe compilation

Next Steps

Install PyQt6, PyOpenGL, pygltflib
Create basic transparent window
Load VRM using pygltflib
Implement basic OpenGL renderer
Port MToon shader to GLSL
Add interactivity (drag, click)

2.9 KiB Raw Blame History