🧠 Your AI Writing Assistant Just Got Personal
Hey Product Hunt! 👋
I just shipped something that makes SourcePilot different from every other AI writing tool out there.
It learns which AI models work best for you. Automatically.
No settings to tweak. No preferences to configure. Just use it, and it gets better.
The Problem I Solved
Here's the thing about AI writing assistants: they're all using different AI models (GPT-4, Claude, Gemini, Mistral, etc.), and each model has its own personality.
Some are great at creative writing. Others excel at technical docs. Some are fast but shallow. Others are slow but thoughtful.
But here's what no one talks about: The "best" AI model is different for everyone.
What works for a novelist doesn't work for a technical writer. What works for blog posts doesn't work for emails.
And you know what's annoying?
Having to manually test and switch between models to find what works for you.
The Solution: LLM Arbitrage
I built a system that figures out your preferences automatically.
It's called LLM Arbitrage, and here's how it works in plain English:
Every time you interact with AI suggestions, you give feedback:
✅ Accept → "This is great, use this model more"
❌ Deny → "Not quite right, try something else"
🔄 Retry → "Close, but let me see another option"
The system learns from your choices:
User writes → AI suggests → User accepts/denies → System learns
After 10 interactions: "Hmm, user likes Model A more than Model B"
After 50 interactions: "Got it! User prefers Model A for emails,
Model C for creative writing"
After 100 interactions: "I know exactly what you like" 🎯
Next time, it automatically picks the best model for you:
Before LLM Arbitrage:
┌─────────────────────────────────────────┐
│ User request │
│ ↓ │
│ Random model selection │
│ ↓ │
│ Hit or miss suggestion │
└─────────────────────────────────────────┘
After LLM Arbitrage:
┌─────────────────────────────────────────┐
│ User request │
│ ↓ │
│ Analyze user's past preferences │
│ ↓ │
│ Pick best model for THIS user │
│ ↓ │
│ Better suggestion (higher accept rate) │
└─────────────────────────────────────────┘
How It Works (The Nerdy Bit)
Okay, for those who want the technical details... 🤓
SourcePilot use an algorithm called Thompson Sampling (the same one Netflix uses for recommendations).
Here's the flow:
USER FEEDBACK LOOP
Write ──→ AI Suggests ──→ Accept/Deny/Retry
↑ │
│ ↓
└────────── System Learns ←────┘
│
↓
┌────────────────┐
│ Model Stats DB │
│ │
│ Model A: 85% │ ← Success rates
│ Model B: 72% │
│ Model C: 91% │ ← Use this one!
└────────────────┘
│
↓
Thompson Sampling Algorithm
│
↓
Balances exploration
vs exploitation
│
┌──────────┴──────────┐
↓ ↓
Try new models Use proven models
(10% of time) (90% of time)
What's Thompson Sampling?
Think of it like this:
🎰 The Casino Analogy:
You walk into a casino with 5 slot machines.
Some pay out more than others, but you don't know which.
Strategy 1: Pick one machine and stick with it
❌ Problem: Might be the worst machine
Strategy 2: Try each machine equally forever
❌ Problem: Waste time on bad machines
Strategy 3: Thompson Sampling ✅
- Try all machines a few times
- Figure out which ones pay best
- Play the best ones more often
- But occasionally try the others (in case they improve)
That's exactly what I do with AI models!
Why This Matters to You
1. No More Guesswork
You don't need to know which AI model is "best" - the system figures it out for you.
2. Gets Better Over Time
The more you use SourcePilot, the better it gets at serving you. It's like having a personal assistant who learns your style.
3. Automatic Optimization
Works silently in the background. No settings to configure. No choices to make. Just write.
4. Privacy First 🔒
SourcePilot track which models you like (stats only). SourcePIlot never store your actual writing content.
Here's exactly what it tracks:
✅ Tracked:
- Model ID (e.g., "gpt-4", "claude-3")
- Your feedback (accept/deny/retry)
- Response time (how fast)
- Endpoint type (suggestion, completion, etc.)
❌ NOT Tracked:
- Your writing content
- Document text
- Personal information
- What you're writing about
5. Works With All AI Models
Cloud AI: OpenAI, Anthropic, Google, Mistral, Cohere
Local AI: Ollama, LM Studio (100% offline, 100% free)
Real-World Example
Let's say you're Sarah, a content marketer:
Week 1: Sarah writes blog posts and emails
SourcePilot tries different models randomly
Sarah accepts some, denies others
System starts noticing patterns
Week 2: Sarah has given 20+ pieces of feedback
System sees: Sarah loves GPT-4 for blog intros
But prefers Claude for professional emails
Mistral is fastest for quick edits
Week 3: Sarah just writes
SourcePilot automatically uses GPT-4 for blog posts
Switches to Claude when she's writing emails
Uses Mistral for quick grammar checks
Sarah's accept rate: 91% (vs 65% in Week 1)
Sarah doesn't configure anything. It just works.
How It Feels to Use
Day 1:
You: [writes something]
AI: [suggests completion - hit or miss]
You: [accept or deny]
Day 7:
You: [writes something]
AI: [suggests completion - getting better]
You: "Hmm, this AI seems to understand me more"
Day 30:
You: [writes something]
AI: [suggests completion - eerily good]
You: "HOW DID IT KNOW?!" 🤯
It's like magic, but it's just math. Really good math.
Technical Stats (For the Data Nerds)
┌─────────────────────────────────────────────┐
│ LLM Arbitrage Performance Metrics │
├─────────────────────────────────────────────┤
│ │
│ Learning Phase: │
│ • 0-10 interactions: Random exploration │
│ • 10-50 interactions: Smart selection │
│ • 50+ interactions: High confidence │
│ │
│ Exploration vs Exploitation: │
│ • 90% → Use best models (exploitation) │
│ • 10% → Try new models (exploration) │
│ │
│ Data Tracked Per Model: │
│ • Total requests │
│ • Accept rate (success metric) │
│ • Deny rate │
│ • Retry rate │
│ • Avg response time │
│ • Last used timestamp │
│ │
│ Privacy: │
│ • Content stored: ZERO │
│ • Metadata only: YES │
│ • User control: Full transparency │
│ │
└─────────────────────────────────────────────┘
What Else Is New in SourcePilot?
While we're here, I've shipped a bunch of other goodies this week:
📊 Word Counter with AI Tracking
See exactly how many words you've written vs AI-generated:
┌─────────────────────────────┐
│ 📝 Writing Stats │
├─────────────────────────────┤
│ Total Words: 1,247 │
│ Your Words: 1,089 │
│ AI Generated: 158 │
│ AI Ratio: 12.7% │
└─────────────────────────────┘
Know your stats. Own your content.
🎯 Smart Local AI Detection
Auto-detects Ollama and LM Studio
Works 100% offline
Completely free for local models
No API keys needed
💳 In-App Billing
Manage subscription without leaving the app
Upgrade/downgrade anytime
Apply promo codes
Cancel with one click
📈 Anonymous Analytics
We track MAU/DAU to improve the product
Zero personally identifiable information
Just usage patterns
Who Is This For?
Perfect for:
✍️ Writers who want AI that adapts to their style
📝 Content creators who write in different tones
💼 Professionals who need reliability
🔒 Privacy-conscious users who want control
🚀 Anyone who wants AI that gets better over time
Not for:
People who want to manually control everything
Users who only write one type of content (less benefit from adaptation)
Anyone uncomfortable with metadata tracking (even anonymous
The Vision
I believe AI writing assistants should be:
Personal - Adapt to each user's style and preferences
Private - Never store your actual writing
Powerful - Use the best AI models available
Flexible - Work offline or online, your choice
Honest - Show you what's yours vs AI-generated
LLM Arbitrage is step one in making AI writing truly personal.
Try It Today
Download SourcePilot → sourcepilot.app
Available for:
🍎 macOS (Apple Silicon + Intel)
🪟 Windows (64-bit)
🐧 Linux (Debian)
The system starts learning from your first interaction.
What's Next?
I'm working on:
🎨 Style Profiles - Auto-detect "blog post mode" vs "email mode"
🌍 Multilingual Support - Learn preferences per language
👥 Team Learning - Share preferences across team (opt-in)
📊 Advanced Analytics - See which models work best for what
🎯 Context Awareness Improvements - Branching text and more.
Join Us
We're building AI tools that respect users and get smarter over time.
Try SourcePilot today: sourcepilot.co/download
Read the deep dive: How Thompson Sampling Works
One More Thing...
If you use SourcePilot and love it, I'd appreciate:
🐦 A tweet about your experience
💬 Feedback (I actually read it all)
Every bit helps us build a better product for you.
Thanks for reading! Now go write something awesome. ✍️
P.S. - The irony of using AI to write about AI is not lost on us. This post was ~40% human, ~60% AI-assisted (with lots of accepts and denies). That's how we roll. 😄
Made with ❤️ by humans + AI
Replies
Querri
SourcePilot
@amy_ingram Thanks Amy! :)