feat(agy): add keyless image-analyze + search modes

image-analyze: independent second-model vision over OAuth (pins the
gemini-3.1-pro-preview vision model; the default flash-lite router
hallucinates image content) — reads an image via read_file and describes it.
search: Google-grounded live web results with citation URLs (google_web_search).
Both verified working on the keyless Google OAuth. Image GENERATION
(nano-banana) still needs an AI Studio key + extension and stays Grok's lane.
Includes a scoped best-effort output sanitizer for image-analyze (preview
model occasionally leaks reasoning tokens); text/verify/review/search
unchanged. migrate-identity.sh now upgrades the gemini capabilities array.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-05 09:03:21 -07:00
parent 2d409a4e7a
commit ac0106f254
3 changed files with 113 additions and 10 deletions

View File

@@ -18,7 +18,8 @@ global, v0.45.1) for a genuinely independent, different-vendor second model.
AGY is the sibling of [`grok`](../grok/SKILL.md): both are second-opinion /
review routers. Use whichever you want a second model from (or both, to triangulate).
Verified working on this machine (2026-06-05): text, verify, review (single
file / file set / git diff).
file / file set / git diff), image-analyze (vision input), search (live Google
web search). All KEYLESS — they work on Google OAuth, no API key.
**Auth:** Gemini uses **Google login (OAuth)****no API key**. Creds live at
`~/.gemini/oauth_creds.json`. If calls fail with an auth error, run `gemini`
@@ -37,6 +38,8 @@ bash "$CLAUDETOOLS_ROOT/.claude/skills/agy/scripts/ask-gemini.sh" <mode> ...
| `review` | `ask-gemini.sh review <file-path> ["<instructions>"]` | Gemini reads the file itself (its `read_file` tool, read-only `plan` mode) and reviews it. Accepts absolute or repo-relative paths, and paths with spaces. Works even on gitignored files. |
| `review-files` | `ask-gemini.sh review-files [-i "<instr>"] <f1> [f2 …]` | Review a **set** of files together (cross-file consistency, multi-file change). Paths absolute or repo-relative; spaces OK. No code passed as a shell arg. |
| `review-diff` | `ask-gemini.sh review-diff [-C <repo-dir>] [-i "<instr>"] <gitref> [-- <pathspec>]` | Review a **git diff** (`git diff <gitref>` from `<repo-dir>`; default repo root, use `-C` for a submodule e.g. `-C projects/msp-tools/guru-rmm`). Diff goes via the prompt file; Gemini can `read_file` changed files for full context. |
| `image-analyze` | `ask-gemini.sh image-analyze <image-path> ["<question>"]` | **Vision** — Gemini `read_file`s the image and describes/answers about it. Pins the **pro vision model** (the default flash-lite router hallucinates image content). Path absolute or repo-relative; spaces OK. KEYLESS (works on OAuth). |
| `search` | `ask-gemini.sh search "<query>"` (or `search --prompt-file <path>`) | **Live Google web search** (sibling of `grok xsearch`) — Gemini uses its `google_web_search` tool and returns the answer **with source URLs**. KEYLESS (works on OAuth). |
| `raw` | `ask-gemini.sh raw <gemini args...>` | Escape hatch — passes args straight to `gemini`. |
The script runs Gemini headless with `-o json`, extracts the answer from
@@ -50,6 +53,17 @@ never corrupts the parse.
- `verify` / `review*` pin a **strong** model — `gemini-3.1-pro-preview` (verified
available on this account 2026-06-05; the CLI's own pro tier).
- Override either with `GEMINI_MODEL=<id>` (e.g. `GEMINI_MODEL=gemini-2.5-pro`).
- `image-analyze` and `search` also pin the strong model (`GEMINI_MODEL` still honored).
### Multimodal: image INPUT works, image GENERATION does not
- **Image INPUT (vision) works on OAuth** — `image-analyze` reads an image with the
pinned **pro vision model** and describes it correctly. The default flash-lite
router HALLUCINATES image content, which is why the pro model is pinned.
- **Image GENERATION (nano-banana) does NOT work on OAuth** — it needs a Google AI
Studio `NANOBANANA_API_KEY` plus the `nanobanana` extension. **Deferred** for now.
Image/video **generation** stays [GROK](../grok/SKILL.md)'s lane (`grok image` /
`grok video`); AGY's multimodal support is read/analyze only.
## Machine availability (fleet)
@@ -60,7 +74,7 @@ not. Availability is gated by `identity.json` (per-machine, gitignored):
"gemini": { "installed": true,
"binary": "C:/Users/guru/AppData/Roaming/npm/gemini",
"auth": "oauth", "is_fleet_host": true,
"capabilities": ["text","verify","review"] }
"capabilities": ["text","verify","review","image-analyze","search"] }
```
- If `gemini.installed` is `false` (or the block is absent), `ask-gemini.sh` exits
@@ -96,8 +110,9 @@ run both and compare — disagreement between them is a strong signal to slow do
- Editing this repo's code → Claude's own agents own the codebase work. Gemini's
`review*` modes are read-only (`--approval-mode plan`) by design; do not give
Gemini write access to this repo.
- Image / video generation → that's GROK's lane (`grok image` / `grok video`),
not Gemini here.
- Image / video **generation** → that's GROK's lane (`grok image` / `grok video`),
not Gemini here (nano-banana needs an API key — deferred). Gemini CAN analyze an
image you give it (`image-analyze`, vision input on OAuth).
- **Never** delegate unsupervised destructive / production actions to Gemini.
Always review Gemini output before acting on it — like Grok, it can over-claim.