🔬 R&D Model Strategy
Date: 2026-08-03 Status: Proposed (awaiting Pat’s approval)
Requirements
The R&D agent needs to:
- Analyze websites — fetch URLs, read content, extract structure
- Process YouTube — transcribe videos, summarize content
- TTS / Audio — generate or analyze speech
- Web scraping — pull data from pages programmatically
- Image analysis — screenshots, diagrams, visual content
- Write findings — to vault and sandbox experiments
Model Recommendation
Primary: openrouter/google/gemini-2.5-flash
Why:
- Natively multimodal (text, vision, audio, video, URL)
- Extremely cheap on OpenRouter (~$0.075/M input tokens)
- 1M token context window (can handle entire websites)
- Built-in URL fetching via OpenCode’s webfetch tool
- Proven working in our sandbox tests
Limitations through OpenCode:
- Can’t directly analyze images (OpenCode’s adapter strips vision)
- YouTube needs external tooling (yt-dlp for transcripts)
For Vision/Image Analysis: openrouter/openai/gpt-5.4-mini
When Gemini can’t handle it through OpenCode:
- GPT-5.4-mini can read images via file paths
- Affordable multimodal option
- Good for screenshot analysis, diagram reading
For Deep R&D: openrouter/qwen/qwen3-vl-30b-a3b-instruct
- Open-source vision-language model
- Can analyze images and code
- Good for technical document analysis
- Very cheap on OpenRouter
How R&D Tasks Actually Work
The R&D model won’t do everything in one shot. It orchestrates:
| Task | Tool | Model |
|---|---|---|
| Fetch website | curl / OpenCode webfetch | Gemini 2.5 Flash |
| YouTube transcript | yt-dlp → text | Gemini 2.5 Flash |
| Screenshot analysis | playwright → image → GPT-5.4-mini | GPT-5.4-mini |
| Web scraping | python + requests/bs4 | Gemini 2.5 Flash (code gen) |
| TTS generation | Hermes text_to_speech tool | N/A (uses configured TTS) |
| Document analysis | Read file → model | Gemini 2.5 Flash or Qwen VL |
| Write findings to vault | J.A.R.V.I.S. writes | N/A |
Updated Model Strategy Table
| Use Case | Model | Provider | Cost |
|---|---|---|---|
| Primary (Hermes daily) | glm-5.2 | z.ai | Lowest |
| Paperclip agents | deepseek-v4-flash-0731 | OpenRouter | $0.003-0.01/task |
| R&D / multimodal | gemini-2.5-flash | OpenRouter | ~$0.075/M |
| R&D vision | gpt-5.4-mini | OpenRouter | Moderate |
| Heavy reasoning (CEO review) | Claude Sonnet 5 | OpenRouter | Moderate |
| Heavy coding | GPT-5.6 Terra | OpenRouter | Moderate |