Mike Swanson
1dd2f208a0
Mike's correction: web search (grok xsearch + gemini search) carries at least as much weight as live API probing - the searches gave the real leads this session (connector proxy, teleport setting path); blind endpoint-probing is "highly suspect" (mostly 404s). And the search bots MUST be properly fixed - both returned empty repeatedly on UniFi research despite the same-day partial grok fix. - docs/CT_THOUGHTS.md: Thought 2 (HIGH PRIORITY) - web-search reliability must-fix, with the observed failures + a proper-fix investigation plan (capture failing-query JSON; max-turns/streaming-json/ retry; cross-fallback grok<->gemini; 5/5 acceptance). - memory feedback_web_search_over_probing: lead with web search/docs; probe only to CONFIRM a hypothesis, never as primary discovery. Reading our own config is fine; guessing paths is not. - errorlog correction logged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1.8 KiB
1.8 KiB
name, description, metadata
| name | description | metadata | ||
|---|---|---|---|---|
| feedback_web_search_over_probing | For external API/capability discovery, LEAD with web search (grok/gemini) and bundled docs; use live endpoint-probing only to CONFIRM a search/doc-derived hypothesis - never as the primary discovery method. Mike's correction 2026-06-17. |
|
When figuring out an external system's API surface or capabilities, web search (grok xsearch / gemini search) and the vendor's own docs carry AT LEAST as much weight as live experimentation - usually more.
Why (Mike, 2026-06-17): blind endpoint-probing ("does /stat/openvpn exist? does
/cmd/vpnmgr?") is guessing - it mostly 404s and is "highly suspect" as a source of truth.
The genuinely valuable leads this session came from the searches: grok surfaced the UniFi
cloud connector proxy (/v1/connector/consoles/.../proxy/...); gemini surfaced the
Teleport /rest/setting/teleport path. Probing only confirmed those after the search
pointed the way.
How to apply:
- Discovery order: web search + bundled docs FIRST -> form a specific hypothesis -> then ONE targeted live call to CONFIRM it. Not: spray candidate URLs and infer from status codes.
- Reading a system's OWN config (e.g. our gateway's
networkconf) is fine - that's reading real data, not guessing endpoints. The "suspect" part is guessing unknown PATHS. - Do not present probe results as "authoritative" over web-search findings; weight them at least equally and reconcile.
- Corollary: the web-search bots being flaky is a real liability (see CT_THOUGHTS "Thought 2 - web-search reliability MUST FIX"); when they fail, say so plainly rather than silently falling back to guessing and calling it authoritative.
- Complements feedback_interview_ai_read_docs.