{ "generated_at": "2026-06-10T14:06:23.580695+00:00", "policy": "Danske app-beskrivelser genereres kun én gang pr. app_ref_id. Automatisk workflow må ikke løbende omskrive eksisterende beskrivelser; regenerering kræver manuel markering/godkendelse.", "method_tests": [ { "method_id": "openrouter:openai/gpt-4o-mini-search-preview", "provider_api": "openrouter", "model_or_endpoint": "openai/gpt-4o-mini-search-preview", "status": "PASS", "latency_s": 6.21, "cost_usd": 0.028, "input_tokens": 28, "output_tokens": 108, "sources_count": 1, "has_web_search": true, "answer": "Hermes Agent is an open-source autonomous AI agent developed by Nous Research, released in February 2026. It operates on your server, retains learned information, and becomes more capable over time. Hermes Agent is compatible with Linux, macOS, and WSL2, and can be installed with a single curl command. ([hermes-agent.org](https://hermes-agent.org/about/?utm_source=openai))\n\nThe official website for Hermes Agent is [https://hermes-agent.org/](https://hermes-agent.org/). ", "notes": "", "artifact_path": "/opt/data/llm-web-search-tests/openrouter/20260610T140011Z/01_gpt-4o-mini-search-preview_normalized.json", "raw_path": "/opt/data/llm-web-search-tests/openrouter/20260610T140011Z/01_gpt-4o-mini-search-preview_raw.json", "quality_note": "Hurtig, men smoke-testen fandt en ikke-canonical Hermes URL (hermes-agent.org) ift. vores app_directory official_url. Skal verificeres med domæne/redirect før den kaldes forkert.", "recommendation": "Brug kun til hurtige brede opslag med eftervalidering mod canonical SQL/URL-verifikation." }, { "method_id": "openrouter:openrouter/fusion", "provider_api": "openrouter", "model_or_endpoint": "openrouter/fusion", "status": "PASS", "latency_s": 63.59, "cost_usd": 0.359, "input_tokens": 9362, "output_tokens": 1129, "sources_count": 3, "has_web_search": true, "answer": "## Hermes Agent (Nous Research)\n\n**Official website:** https://hermes-agent.nousresearch.com/\n**GitHub repository:** https://github.com/NousResearch/hermes-agent\n\n### What it is\n\n**Hermes Agent** is an open-source (MIT-licensed) autonomous AI agent built by **Nous Research**, the AI lab known for its Hermes language models. Unlike a coding copilot tied to an IDE or a simple chatbot wrapper around a single API, it is designed as a persistent agent that runs on your own server (from a cheap VPS to a GPU cluster) and is meant to become more capable the longer it runs.\n\n### Purpose and key features\n\n- **Self-improving learning loop** — It builds reusable \"skills\" from experience, persists long-term memory, can search its own past conversations, and develops an evolving model of the user across sessions.\n- **Lives where you do** — Accessible across multiple platforms (Telegram, Discord, Slack, WhatsApp, Signal, Email, CLI), all sharing a single memory.\n- **Automation and delegation** — Natural-language scheduling (cron-style) for unattended tasks, plus the ability to spawn isolated subagents that work in parallel.\n- **Sandboxed execution** — Multiple isolated execution backends (e.g., local, Docker, SSH, Singularity, and serverless options like Modal/Daytona) with container hardening.\n- **Rich tooling** — Web search, browser automation, vision, image generation, and text-to-speech.\n- **Model-agnostic** — Works with many providers (Nous Portal, OpenRouter, OpenAI, NVIDIA NIM, and others) with no vendor lock-in.\n- **Research-ready** — Supports batch trajectory generation and reinforcement-learning training via Nous's Atropos framework.\n\n### Sources\n- Official site: https://hermes-agent.nousresearch.com/\n- Documentation: https://hermes-agent.nousresearch.com/docs/\n- GitHub: https://github.com/NousResearch/hermes-agent\n\n**A note on accuracy:** The capability descriptions above are drawn from Nous Research's own website and repository, so they reflect self-reported features rather than independent reviews. A few specifics are inconsistent even within Nous's own materials — for example, the tool count is given as \"40+\" in the GitHub README but \"60+\" on the site, and the exact number of sandbox backends varies (five vs. six) depending on which page you read. For the most current and precise figures, check the live site and repo directly.", "notes": "", "artifact_path": "/opt/data/llm-web-search-tests/openrouter/20260610T140011Z/02_openrouter-fusion_normalized.json", "raw_path": "/opt/data/llm-web-search-tests/openrouter/20260610T140011Z/02_openrouter-fusion_raw.json", "quality_note": "Ramte korrekt officiel URL og gav bedst syntese, men dyr og langsom.", "recommendation": "Brug til vigtig research, ikke batch for alle apps." }, { "method_id": "nanogpt-chat:linkup-research-low", "provider_api": "nanogpt-chat", "model_or_endpoint": "linkup-research-low", "status": "PASS", "latency_s": 42.18, "cost_usd": 0.2625, "input_tokens": 17, "output_tokens": 117, "sources_count": 2, "has_web_search": true, "answer": "", "notes": "Slow (deep research). Returns structured reasoning with answer, interpretation, evidence, confidence. Provides sources array with snippets. Most expensive option.", "quality_note": "Slow (deep research). Returns structured reasoning with answer, interpretation, evidence, confidence. Provides sources array with snippets. Most expensive option.", "recommendation": "Kan bruges selektivt", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/raw/linkup-research-low.txt" }, { "method_id": "nanogpt-chat:fastgpt", "provider_api": "nanogpt-chat", "model_or_endpoint": "fastgpt", "status": "FAIL", "latency_s": 0.19, "cost_usd": null, "input_tokens": null, "output_tokens": null, "sources_count": null, "has_web_search": false, "answer": "", "notes": "Model endpoint consistently returning 502. May be deprecated or temporarily down.", "quality_note": "Model endpoint consistently returning 502. May be deprecated or temporarily down.", "recommendation": "Undgå pt.", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/raw/fastgpt.txt" }, { "method_id": "nanogpt-chat:brave", "provider_api": "nanogpt-chat", "model_or_endpoint": "brave", "status": "PASS", "latency_s": 2.94, "cost_usd": 0.05692, "input_tokens": 10389, "output_tokens": 195, "sources_count": 3, "has_web_search": true, "answer": "", "notes": "Fast. Returns well-formatted answer with inline sources. Higher input token count suggests search context is injected.", "quality_note": "Fast. Returns well-formatted answer with inline sources. Higher input token count suggests search context is injected.", "recommendation": "Kan bruges selektivt", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/raw/brave.txt" }, { "method_id": "nanogpt-chat:exa-answer", "provider_api": "nanogpt-chat", "model_or_endpoint": "exa-answer", "status": "PASS", "latency_s": 2.9, "cost_usd": 0.005, "input_tokens": 27, "output_tokens": 120, "sources_count": 9, "has_web_search": true, "answer": "", "notes": "Fast and cheapest. Returns answer with structured citations array (title, url, image, favicon). Best value.", "quality_note": "Fast and cheapest. Returns answer with structured citations array (title, url, image, favicon). Best value.", "recommendation": "God kandidat", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/raw/exa-answer.txt" }, { "method_id": "nanogpt-web:linkup_standard_searchResults", "provider_api": "nanogpt-web", "model_or_endpoint": "linkup_standard_searchResults", "status": "PASS", "latency_s": 1.62, "cost_usd": 0.006, "input_tokens": null, "output_tokens": null, "sources_count": 15, "has_web_search": true, "answer": "", "notes": "Raw search results with full content snippets. Best for programmatic processing.", "quality_note": "Raw search results with full content snippets. Best for programmatic processing.", "recommendation": "Rå kildeindsamling", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" }, { "method_id": "nanogpt-web:linkup_standard_sourcedAnswer", "provider_api": "nanogpt-web", "model_or_endpoint": "linkup_standard_sourcedAnswer", "status": "PASS", "latency_s": 3.09, "cost_usd": 0.006, "input_tokens": null, "output_tokens": null, "sources_count": 15, "has_web_search": true, "answer": "", "notes": "LLM-synthesized answer + full source list. Best of both worlds.", "quality_note": "LLM-synthesized answer + full source list. Best of both worlds.", "recommendation": "Primær billig løsning", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" }, { "method_id": "nanogpt-web:linkup_standard_structured", "provider_api": "nanogpt-web", "model_or_endpoint": "linkup_standard_structured", "status": "FAIL", "latency_s": null, "cost_usd": null, "input_tokens": null, "output_tokens": null, "sources_count": null, "has_web_search": false, "answer": "", "notes": "Timed out with schema. May need simpler schema or retry.", "quality_note": "Timed out with schema. May need simpler schema or retry.", "recommendation": "Ikke anbefalet pt.", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" }, { "method_id": "nanogpt-web:linkup_deep_searchResults", "provider_api": "nanogpt-web", "model_or_endpoint": "linkup_deep_searchResults", "status": "PASS", "latency_s": null, "cost_usd": null, "input_tokens": null, "output_tokens": null, "sources_count": null, "has_web_search": true, "answer": "", "notes": "Works. Returns more comprehensive results than standard.", "quality_note": "Works. Returns more comprehensive results than standard.", "recommendation": "Rå kildeindsamling", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" }, { "method_id": "nanogpt-web:brave_standard_searchResults", "provider_api": "nanogpt-web", "model_or_endpoint": "brave_standard_searchResults", "status": "PASS", "latency_s": 1.12, "cost_usd": 0.005, "input_tokens": null, "output_tokens": null, "sources_count": 10, "has_web_search": true, "answer": "", "notes": "Fastest. Returns results with hostname field included.", "quality_note": "Fastest. Returns results with hostname field included.", "recommendation": "Rå kildeindsamling", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" }, { "method_id": "nanogpt-web:brave_standard_sourcedAnswer", "provider_api": "nanogpt-web", "model_or_endpoint": "brave_standard_sourcedAnswer", "status": "FAIL", "latency_s": null, "cost_usd": null, "input_tokens": null, "output_tokens": null, "sources_count": null, "has_web_search": false, "answer": "", "notes": "Brave provider only supports searchResults.", "quality_note": "Brave provider only supports searchResults.", "recommendation": "Ikke anbefalet pt.", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" }, { "method_id": "nanogpt-web:exa_standard_searchResults", "provider_api": "nanogpt-web", "model_or_endpoint": "exa_standard_searchResults", "status": "PASS", "latency_s": 1.3, "cost_usd": 0.005, "input_tokens": null, "output_tokens": null, "sources_count": 5, "has_web_search": true, "answer": "", "notes": "Fewer results than linkup/brave but includes metadata (publishedDate, imageUrl). Has internal exa cost breakdown.", "quality_note": "Fewer results than linkup/brave but includes metadata (publishedDate, imageUrl). Has internal exa cost breakdown.", "recommendation": "Rå kildeindsamling", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" }, { "method_id": "nanogpt-web:exa_standard_sourcedAnswer", "provider_api": "nanogpt-web", "model_or_endpoint": "exa_standard_sourcedAnswer", "status": "FAIL", "latency_s": null, "cost_usd": null, "input_tokens": null, "output_tokens": null, "sources_count": null, "has_web_search": false, "answer": "", "notes": "Exa provider only supports searchResults.", "quality_note": "Exa provider only supports searchResults.", "recommendation": "Ikke anbefalet pt.", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" }, { "method_id": "nanogpt-web:linkup_standard_answer_old", "provider_api": "nanogpt-web", "model_or_endpoint": "linkup_standard_answer_old", "status": "FAIL", "latency_s": null, "cost_usd": null, "input_tokens": null, "output_tokens": null, "sources_count": null, "has_web_search": false, "answer": "", "notes": "outputType 'answer' is not valid. Correct value is 'sourcedAnswer'.", "quality_note": "outputType 'answer' is not valid. Correct value is 'sourcedAnswer'.", "recommendation": "Ikke anbefalet pt.", "artifact_path": "/opt/data/llm-web-search-tests/nanogpt/20260610T135413Z/normalized/results.json", "raw_path": "" } ], "openrouter_catalog_search_hits": [ { "id": "openrouter/fusion", "name": "OpenRouter: Fusion", "matched_term": "web", "context_length": 128000 }, { "id": "x-ai/grok-4.20-multi-agent", "name": "xAI: Grok 4.20 Multi-Agent", "matched_term": "research", "context_length": 2000000 }, { "id": "relace/relace-search", "name": "Relace: Relace Search", "matched_term": "search", "context_length": 256000 }, { "id": "perplexity/sonar-pro-search", "name": "Perplexity: Sonar Pro Search", "matched_term": "search", "context_length": 200000 }, { "id": "openai/o3-deep-research", "name": "OpenAI: o3 Deep Research", "matched_term": "research", "context_length": 200000 }, { "id": "openai/o4-mini-deep-research", "name": "OpenAI: o4 Mini Deep Research", "matched_term": "research", "context_length": 200000 }, { "id": "nousresearch/hermes-4-70b", "name": "Nous: Hermes 4 70B", "matched_term": "Research", "context_length": 131072 }, { "id": "nousresearch/hermes-4-405b", "name": "Nous: Hermes 4 405B", "matched_term": "Research", "context_length": 131072 }, { "id": "bytedance/ui-tars-1.5-7b", "name": "ByteDance: UI-TARS 7B ", "matched_term": "web", "context_length": 128000 }, { "id": "openai/gpt-4o-mini-search-preview", "name": "OpenAI: GPT-4o-mini Search Preview", "matched_term": "search", "context_length": 128000 }, { "id": "openai/gpt-4o-search-preview", "name": "OpenAI: GPT-4o Search Preview", "matched_term": "search", "context_length": 128000 }, { "id": "perplexity/sonar-reasoning-pro", "name": "Perplexity: Sonar Reasoning Pro", "matched_term": "search", "context_length": 128000 }, { "id": "perplexity/sonar-pro", "name": "Perplexity: Sonar Pro", "matched_term": "search", "context_length": 200000 }, { "id": "perplexity/sonar-deep-research", "name": "Perplexity: Sonar Deep Research", "matched_term": "research", "context_length": 128000 }, { "id": "microsoft/phi-4", "name": "Microsoft: Phi 4", "matched_term": "Research", "context_length": 16384 }, { "id": "google/gemma-2-27b-it", "name": "Google: Gemma 2 27B", "matched_term": "research", "context_length": 8192 } ] }