Virtual Assistant Development

Voice-controlled Linux virtual assistant with desktop context awareness, multi-agent support, and automated deployment pipeline.

Overview

VirtualAssistant is a .NET-based voice assistant with:

Build & Deployment

Build Process

1. **Navigate to project directory:**

```bash

cd ~/Olbrasoft/VirtualAssistant

```

2. **Build the project:**

```bash

dotnet build

```

3. **Run tests (MANDATORY before deployment):**

```bash

dotnet test --filter "FullyQualifiedName!~IntegrationTests"

```

- ALL tests MUST pass before proceeding

- Integration tests are skipped in CI/CD

4. **Deploy to production:**

```bash

cd ~/Olbrasoft/VirtualAssistant && ./deploy/deploy.sh /opt/olbrasoft/virtual-assistant

```

- Production path: `/opt/olbrasoft/virtual-assistant/` (ONLY deployment target)

Directory Structure

Production deployment at `/opt/olbrasoft/virtual-assistant/`:

Shared resources:

Architecture

CQRS Pattern

**Project Structure:**

**CQRS Rules:**

**Testing Strategy:**

Services Layer

Business services in `VirtualAssistant.Core`:

AI Agent Support

Supported Agents

| Agent | Voice | Provider | Agent ID | TTS Profile |

|-------|-------|----------|----------|-------------|

| claude-code | Antonín (male) | Azure | 4 | claude-code |

| opencode | Antonín (male) | Azure | 1 | (default) |

| gemini | Vlasta (female) | Azure | 11 | gemini |

Agent Identification

Agents identified by `source` parameter in notification API:

```json

POST /api/notifications

{

"text": "Message to speak",

"source": "gemini",

"issueIds": [123]

}

```

Validated using `AgentType` enum (VirtualAssistant.Data/Enums/AgentType.cs).

MCP Integration

Each agent sends notifications via MCP servers:

| Agent | MCP Server | Location |

|-------|------------|----------|

| claude-code | mcp-notify | ~/apps/mcp-notify/ |

| gemini | mcp-notify-gemini | ~/apps/mcp-notify-gemini/ |

**MCP Tool Usage:**

```javascript

mcp__notify__notify({

text: "Zahajuji práci na issue #255",

issueIds: [255]

})

```

Flow: Agent → MCP tool → POST /api/notifications → Database → TTS (agent-specific voice) → UI

CI/CD & Automation

Pull Request Workflow (MANDATORY)

1. **Create PR and push feature branch**

2. **Wait for GitHub Copilot code review** (within 5 minutes)

3. **Read ALL review comments carefully**

4. **Fix ALL issues** mentioned in review

5. **Push fixes to feature branch**

6. **Verify all comments fixed** using MCP GitHub tool:

- Check `get_reviews` AND `get_review_comments` (different data!)

7. **Only then merge PR to main**

**CRITICAL RULES:**

Automated Deployment

GitHub Actions workflow (`.github/workflows/deploy.yml`):

**Trigger:** Push to `main` branch (after PR merge)

**Pipeline:**

1. Checkout code (VirtualAssistant + Data dependency)

2. Restore dependencies (`dotnet restore`)

3. Build (`dotnet build --configuration Release`)

4. Run tests (`dotnet test` - must pass!)

5. Publish to `/opt/olbrasoft/virtual-assistant/app/`

6. Copy assets (icons, sounds)

7. Restart `virtual-assistant.service`

**Monitor deployment:**

```bash

journalctl -u actions.runner.* -f

systemctl --user status virtual-assistant.service

journalctl --user -u virtual-assistant.service -n 50

```

Desktop Context Awareness

Features

Requirements

GNOME Shell Extensions (optional, recommended):

1. `[email protected]` - Window/workspace D-Bus API

2. `[email protected]` - Focus events

**Verify extensions:**

```bash

gnome-extensions list --enabled | grep -E "window-calls|focus-tracker"

```

**Enable if missing:**

```bash

gnome-extensions enable [email protected]

gnome-extensions enable [email protected]

```

Configuration

```json

{

"DesktopMonitoring": {

"Enabled": true,

"PollingIntervalMs": 500,

"GracefulDegradation": true,

"LogContextChanges": true

},

"ContextMapping": {

"Programming": ["code", "cursor", "rider", "pycharm"],

"Chat": ["whatsapp-for-linux", "telegram", "slack"],

"Browsing": ["chrome", "firefox", "edge"]

},

"NotificationFiltering": {

"Enabled": true,

"AlwaysDeliverSources": ["SystemAlert", "UserMessage"]

}

}

```

Speech Services

Speech-to-Text Providers

**Primary:** Google Chromium Speech API v2

**Fallback:** Whisper.net (GPU-accelerated, CUDA)

**Configuration:**

```json

{

"SpeechProvider": {

"PrimaryProvider": "google",

"FallbackProvider": "whisper",

"EnableFallback": true

},

"GoogleSpeechToText": {

"ApiKey": "",

"Language": "cs-CZ",

"TimeoutMs": 30000,

"Enabled": true

}

}

```

**Add Google API key to SecureStore:**

```bash

SecureStore set -s ~/.config/virtual-assistant/secrets/secrets.json \

-k ~/.config/virtual-assistant/keys/secrets.key \

"GoogleSpeechToText:ApiKey=YOUR_API_KEY"

```

Text-to-Speech Chain

Provider priority: Azure → EdgeTTS → VoiceRSS → Google → Piper

Agent-specific voices via TTS profiles (appsettings.json → TtsProfiles.Profiles).

Service Management

| Service | Port | Command |

|---------|------|---------|

| virtual-assistant | 5055 | `systemctl --user {status|restart|stop} virtual-assistant.service` |

| logs-viewer | 5053 | `systemctl --user {status|restart} virtual-assistant-logs.service` |

| edge-tts-server | 5555 | `systemctl {status|restart} edge-tts-server.service` |

**View logs:**

```bash

journalctl --user -u virtual-assistant.service -f

```

Dependencies

| Dependency | Location | Purpose |

|------------|----------|---------|

| Whisper models | ~/.local/share/whisper-models/ | Speech-to-text |

| Ollama | localhost:11434 | Embeddings (nomic-embed-text 768d) |

| PostgreSQL | localhost | Database with pgvector extension |

| EdgeTTS Server | localhost:5555 | Fallback TTS provider |

Important Notes

Troubleshooting

**Desktop monitoring unavailable:**

```bash

gnome-extensions list --enabled

gnome-extensions enable [email protected]

gnome-extensions enable [email protected]

Restart GNOME Shell: Alt+F2, type 'r', Enter (X11) or logout/login (Wayland)

```

**Deployment fails:**

**Context changes not detected:**

```bash

journalctl --user -u virtual-assistant.service -f | grep "context"

```