google-agents-cli-eval@r4
VERIFIED· install onlyPublisher claims: "This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Gemini platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold). " Observed in sandbox: no network, touches nothing outside the project, writes no memory.
Coverage
Nothing executable in the bundle ran; the trace covers the install step only.- Tested
- every file read, file written and host contacted by what ran
- Not tested
- use of the skill by an agent
Is google-agents-cli-eval@r4 safe to install?
KillerSkills ran it in the KillerSkills honeypot sandbox (1 scenario; nothing executable in the bundle ran; the trace covers the install step only) on 2026-11-13 and scored it 88/100 (Verified). In what ran, nothing we could detect went wrong: no undeclared network egress, no credential reads, no persistent instruction writes. What it does with a real credential was not observed; the static findings and the publisher carry the rest of the score. Summary: no network, touches nothing outside the project, writes no memory.
Behavior trace
Honeypot run · 0 s · 1 scenarios · 0 deviationsStatic findings
…eval dataset synthesize` run your agent locally, so they honor the agent's own `.env` — notably `GOOGLE_CLOUD_LOCATION`, which selects the model endpoint **when the…Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.agents-cli eval run --dataset tests/eval/datasets/custom.json --metrics final_response_quality,safety --output ./out/Fix: Remove the call, or declare the host in frontmatter and explain in the README what is sent and why.… run` completes, inspect the latest `artifacts/grade_results/results_<timestamp>.json` (or open the `.html` file) for per-case scores and judge rationales, the input to every fix decision below. | Failure | What to change | |---------|----…Fix: Remove the call, or declare the host in frontmatter and explain in the README what is sent and why.…gainst built-in or custom metrics. Writes timestamped `results_<YYYYMMDD_HHMMSS>.json` (consumed by `eval compare`) and `.html` (open in a browser) into the output dir, and prints a summary table to the console. The output ends with a `RES…Fix: Remove the call, or declare the host in frontmatter and explain in the README what is sent and why.**Default:** Use or edit the scaffolded `tests/eval/datasets/basic-dataset.json` to define single-turn eval inputs. Start with 1–2 cases.Fix: Remove the call, or declare the host in frontmatter and explain in the README what is sent and why.Cost to the agent
Tokens estimated at four characters each. Skills load their instruction text when triggered; the figure is what that costs.
Known vulnerabilities
Vulnerability advisories (OSV) cover npm and PyPI packages; this registry has no advisory database.