I created My Personal AI Assistant — Now It's a Privacy-First Telegram Bot (Gemma 4 sitting behind)

This is a submission for the GitHub Finish-Up-A-Thon Challenge

What I Built

Digital Secretary is a privacy-first personal AI assistant that runs on your Windows PC and talks to you through Telegram. No cloud subscription. No Docker. No data leaving your machine unless you choose it.

You send a message → your PC processes it → you get a smart, memory-aware reply. Voice in, voice out. Files, web search, reminders, OCR — all from Telegram.

GitHub: github.com/sree1959git/Digital-Secretary

Demo

Youtube: (https://youtu.be/23W33BQDmWQ)

Setup Wizard

The interactive wizard takes less than 3 minutes:

  ____  _       _ _        _   ____                    _
 |  _ \(_) __ _(_) |_ __ _| | / ___|  ___  ___ _ __ ___| |_ __ _ _ __ _   _
 | | | | |/ _` | | __/ _` | | \___ \ / _ \/ __| '__/ _ \ __/ _` | '__| | | |
 | |_| | | (_| | | || (_| | |  ___) |  __/ (__| | |  __/ || (_| | |  | |_| |
 |____/|_|\__, |_|\__\__,_|_| |____/ \___|\___| _|  \___|  \__\__,_|_|   \__, |
          |___/                                                              |___/

  Your personal AI secretary. On Telegram. On Windows.

  Step 1: Telegram Bot Token
  Step 2: OpenRouter API Key
  Step 3: Default AI Model
  Step 4: Name Your AI Secretary
  Step 5: Personality

Bot in Action

You: Remind me to call the dentist tomorrow at 10am
Digital Secretary: Done! I've set a reminder for tomorrow at 10:00 AM: "Call the dentist". 
                   I'll message you here on Telegram when it's time.

You: [sends photo of whiteboard]
Digital Secretary: I can see a whiteboard with what appears to be a sprint plan. 
                   The items listed are: [extracted text from OCR + vision analysis]

You: [voice message] "What did we discuss last week about the project?"
Digital Secretary: [voice reply] Based on our previous conversations, we talked about...

Architecture at a Glance

Telegram (Your Phone)
        │
        ▼
Digital Secretary (Windows PC)
        │
        ├── 🧠 AI: OpenRouter / Ollama / llama.cpp
        ├── 💾 Memory: SQLite FTS5 (persistent, searchable)
        ├── 🛠️ Tools: files, shell, web, OCR, MCP servers
        ├── 📅 Scheduler: APScheduler-backed reminders
        ├── 🎙️ STT: faster-whisper (local Whisper)
        └── 🔊 TTS: edge-tts (Microsoft neural voices)

The Comeback Story

Where It Started (Before)

Couple of weeks ago, I built the first version of this project. I wanted a personal AI assistant that:

1. Didn't require a monthly subscription
2. Ran on my Windows PC
3. Used Telegram (which I already had on my phone)

I got a working prototype in 3 days. The project sat there — functional but rough:

• Hard-coded paths (C:/Users/<username>/...) buried in the config
• The bot answered questions but forgot everything between restarts
• No voice support at all
• A single Python file that grew to 1,000 lines
• Zero documentation — I couldn't even remember how to set it up myself

I documented all this brutally honestly in CODE_REVIEW_REPORT.md:

"The config file has hard-coded personal paths... The memory only survives per-session... The OcrImageTool is registered in the Telegram channel but not in the main tool registry... The autostart task name is inconsistent between wizard.py and autostart.py..."

What I Changed (After)

When I picked this up again for this challenge, I treated it like arriving at a client's codebase for the first time. Here's what changed:

Architecture improvements:

• Split the monolith into a proper package structure: agent/, channels/, memory/, providers/, security/, tools/
• Added MemoryStore with SQLite FTS5 — the bot now remembers you across sessions
• Introduced ContextBuilder that assembles system prompts from identity files (SOUL.md, AGENTS.md, USER.md)
• Created a proper ToolRegistry with OpenAI-compatible function definitions

New features added:

• 🎙️ Voice I/O — Whisper (local STT) + Edge-TTS neural voices (3-tier voice system)
• 📅 Scheduler — "Remind me at 3pm daily to stand up and stretch"
• 🖼️ Vision + OCR — send a photo, get text extracted and image described
• 🔌 MCP Bridge — plug in any Model Context Protocol server
• 🔐 Hardware-bound encryption — secrets stored with Fernet keyed to Windows UUID

Quality of life:

• Interactive setup wizard (no more manual JSON editing)
• Auto-start via Windows Task Scheduler
• Proper graceful shutdown on Ctrl+C / SIGTERM
• Split responses at paragraph boundaries (no more mid-word Telegram truncation)
• /model command to switch AI models mid-conversation

The rebrand:

• Renamed from Antidote to Digital Secretary — the new name better reflects what it actually does: it's your personal secretary, not just "an AI"
• New CLI: digital_secretary and digital_secretary setup

My Experience Antigravity Extensions/Models (Claude)

Claude was genuinely invaluable during this revival. Here are the specific moments where it made the biggest difference:

1. Voice Pipeline Design

The 3-tier voice response system was the hardest design challenge:

• Tier 1: "Read it out" → TTS playback of last response, no LLM call
• Tier 2: "Explain this" → LLM re-explains in natural speech
• Tier 3: Default voice query → LLM summarizes its response for speech

I described the problem in English, and AI agent helped sketch the state machine and wrote the initial _is_readout_request() / _is_explain_request() pattern matching. What would have taken me an hour of thinking through edge cases was drafted in minutes.

2. Hardware-Bound Encryption

# Copilot suggested this elegant approach for Windows-specific key derivation
def _derive_key(uuid_bytes: bytes) -> bytes:
    return hashlib.pbkdf2_hmac(
        "sha256",
        uuid_bytes,
        salt=b"digital_secretary_v1",
        iterations=480_000,
    )

It caught that I should use PBKDF2 with high iterations rather than raw HMAC, and suggested the specific iteration count (480K) based on OWASP recommendations for 2024.

3. SQLite FTS5 Memory Search

I hadn't used FTS5 before. AI agent wrote the schema migration, the MATCH queries, and explained why rank needed to be negated for proper relevance ordering. I learned something new in the process.

4. Windows Signal Handling Bug

# This pattern only works on Unix:
loop.add_signal_handler(signal.SIGTERM, callback)

# Copilot caught this and suggested the Windows-compatible approach:
signal.signal(signal.SIGTERM, _signal_handler)

It spotted the platform incompatibility in a code review pass — something I'd have discovered only when a user reported it.

Use Cases

Digital Secretary is designed for people who want a personal AI that:

Technical Architecture Highlights

Memory That Persists

# Search semantic memory with FTS5
results = await memory.search("dentist appointment", limit=5)
# Returns: [Memory(content="User mentioned dentist on Tuesday...", category="fact")]

Every conversation is summarized and stored. The next time you ask a related question, the relevant memory is injected into the system prompt automatically.

Tool Use in Action

When you ask "What's the weather like tomorrow?", the LLM:

1. Decides to call web_search("weather tomorrow [your city]")
2. Gets results back
3. Calls fetch_webpage(url) for the detailed forecast
4. Synthesizes a natural language response

All in one Telegram exchange.

Multi-Provider AI

# Switch between providers transparently
provider = get_provider()  # reads config, returns right provider

# All providers implement the same interface
response = await provider.chat(messages=messages, tools=tool_defs)

Works with OpenRouter (100+ models), Ollama (local), and llama.cpp (ultra-local, GPU or CPU).

Getting Started

git clone https://github.com/sree1959git/Digital-Secretary.git
cd Digital-Secretary
.\start.ps1

The launcher handles everything — Python version check, venv creation, dependency install, and wizard launch.

Full setup guide: TELEGRAM_BOT_SETUP.md

What I have tested?

Uploaded an image file (like workflow diagram, medicine pic, handwritten text) and asked BOT to decipher & explain. It did this job clean as humans would do, instead of reading out entire output generated (The output generated is complete extraction of image file while what BOT tells me through voice is a gist of output). This happens when the user send a voice message along with uploaded document.
Then I asked it to readout the entire output, it did this job clean.
Most of the BOTs start reading out the inference (output) which could be very long, which is not desired by users. I have overcome this irritation of users.
BOT can do many things that openclaw/hermes agent would do by extending its skills, MCP configurations, connectors and inclusion of more opensource models. Gemma 4 is clearly the winner in this providing the foundation for localization.

What's Next

The project roadmap is honest about what's still to do:

• Multi-agent routing — separate /finance, /health, /work sub-agents with scoped memories
• Approval workflow — Telegram inline buttons to approve file writes and shell commands
• Durable conversation history — persist every turn, not just summaries
• Web UI — a dashboard to review memories, manage reminders, switch models

This challenge was exactly the kick I needed to stop leaving this on the shelf.

If you try it, let me know what you think in the comments. 👇

Tags: #devchallenge #githubchallenge #python #telegram #ai #windows #opensource