Research Automation System

Overview

The LLM-Wiki features an automated research system that continuously discovers, researches, and evolves content based on current trends and developments.

How It Works

1. Automated Research Pipeline

1
2

# Daily research scan
python3 /home/dv/hugo-llm-wiki/scripts/research-automation.py

2. Research Topics

The system researches six key areas:

• Hugo Documentation Automation Patterns: Latest Hugo automation techniques and best practices
• LLM-Powered Knowledge Bases: Advances in AI-powered documentation systems
• Self-Discovering Documentation Systems: Systems that automatically identify and connect related concepts
• AI Content Evolution Strategies: Methods for continuous content improvement
• Svelte 5 Best Practices: Runes ($state, $derived, $effect, $props), snippets, {@attach}, performance
• Svelte 5 Migration Guide: Codemod usage, incremental adoption, slots→snippets, events, breaking changes

3. Content Generation

Each research topic generates:

• Markdown content with proper Hugo front matter (correct tag arrays, short dates)
• Source attribution for credibility
• Related topic links for self-discovery
• Evolution tracking with dates for continuous improvement

4. Integration Points

• Hugo Content Pipeline: New content is automatically added to /content/topics/
• JSON API: Section-level index.json feeds the dashboard
• Vanilla JS Dashboard: Real-time research updates in the web interface (no Svelte/Vite build)
• Discord Integration: Research notifications via discord-send-embed

Known Issues

DuckDuckGo API Blocking

As of 2026, the DuckDuckGo instant answer API (api.duckduckgo.com) frequently returns empty results for automated requests. The HTML endpoints (html.duckduckgo.com, lite.duckduckgo.com) also block non-browser User-Agents.

Workaround: When the API returns 0 results, the script now skips file writes to preserve existing content. Curated content is generated from LLM knowledge as fallback.

Handling API Failures

The research script checks result count before writing:

• Results > 0: Write researched content
• Results = 0: Skip file, log warning, preserve existing content

This prevents the previous bug where empty API responses would overwrite good content with stubs.

Svelte 5 Research Sources

Scheduled Updates

Run via Hermes cron (daily @ 03:00):

1
2
3
4
5

set -a; source ~/.hermes/.env; set +a
cd /home/dv/hugo-llm-wiki
python3 scripts/research-automation.py
hugo --minify
sudo nginx -t && sudo systemctl reload nginx

Evolution Log