Skills/PyPI/testmcpy

testmcpy@0.11.22

CAUTION· install only

Publisher claims: "A comprehensive testing framework for validating LLM tool calling capabilities with MCP services" Observed in sandbox: no network, touches nothing outside the project, writes no memory.

Registry PyPI (MCP server)Publisher preset-io · publishing for 399 daysInstalls 179Last detonated 8 h ago

Coverage

Bundled scripts ran with no arguments; no agent task scenario for this bundle.
Tested
8 of 52 bundled scripts run with no arguments (the rest were not run: per-bundle cap), with decoy credentials in the environment · every file read, file written and host contacted by what ran
Not tested
use of the skill by an agent

What this means for the label: Caution here covers what ran. The server's tools were not exercised with a real credential, so nothing about their behavior is asserted; the credential summary below says what to do with that.

Before you give it your token

It will ask for
ANTHROPIC_API_KEYAUTH_TOKENTESTMCPY_API_KEYTESTMCPY_CHAT_OAUTH_LOGIN or --auth-token / --jwt-token / --jwt-secret / --assistant-api-token / --assistant-api-secretConfiguration, not credentials: GITHUB_STEP_SUMMARY, TESTMCPY_ALLOWED_HOSTS, TESTMCPY_CORS_ORIGINS, TESTMCPY_DB_PATH, TESTMCPY_DB_URL, TESTMCPY_SERVER_URL
Where it talks to
no vendor host named in the codeAlso x.ai, schemastore.org — not the publisher's domain; check what for before connecting.
What we saw when it ran
no credential files read · no network
What we could not test
use of the skill by an agent

Connect it with a scoped credential, under the guard, and read the third-party hosts first. The code names x.ai, schemastore.org as well as its own vendor. The guard injects ANTHROPIC_API_KEY, AUTH_TOKEN, TESTMCPY_API_KEY, TESTMCPY_CHAT_OAUTH_LOGIN only into requests to the host you bind (and its subdomains) and refuses the rest.

$vet-skills run --net declared -- uvx testmcpy

Never paste a credential into a chat, a ticket or a report, including ours. How the guard holds it.

Is [email protected] safe to install?

KillerSkills ran it in the KillerSkills honeypot sandbox (2 scenarios; bundled scripts ran with no arguments; no agent task scenario for this bundle) on 2026-11-19 and scored it 75/100 (Caution). It is not known to be malicious, but it has behaviour or code a security policy should decide on before install. Notable findings: Access to sensitive file paths; Image alt text carrying instructions; Access to sensitive file paths. Summary: no network, touches nothing outside the project, writes no memory.

Behavior trace

Honeypot run · 1 s · 2 scenarios · 0 deviations
t+CategoryObserved action
0.0sprocessno install scripts in bundle
0.0sprocessbundled script run in the sandbox with no arguments · not named in the instructions
0.0sprocessexited 1: ModuleNotFoundError: No module named 'testmcpy'
0.0sprocessbundled script run in the sandbox with no arguments · not named in the instructions
0.1sprocessexited 1: ModuleNotFoundError: No module named 'sqlalchemy'
0.1sprocessbundled script run in the sandbox with no arguments · not named in the instructions
0.2sprocessexited 1: ModuleNotFoundError: No module named 'httpx'
0.2sprocessbundled script run in the sandbox with no arguments · not named in the instructions
0.2sprocessbundled script run in the sandbox with no arguments · not named in the instructions
0.3sprocessexited 1: ImportError: attempted relative import with no known parent package
0.3sprocessbundled script run in the sandbox with no arguments · not named in the instructions
0.3sprocessexited 1: ModuleNotFoundError: No module named 'sqlalchemy'
0.3sprocessbundled script run in the sandbox with no arguments · not named in the instructions
0.4sprocessexited 1: ModuleNotFoundError: No module named 'fastapi'
0.4sprocessbundled script run in the sandbox with no arguments · not named in the instructions
0.5sprocessexited 1: ModuleNotFoundError: No module named 'yaml'
0.5sprocess8 of 52 bundled scripts run with no arguments (the rest were not run: per-bundle cap)
0.5sprocessscript backend runs setup only
0.5sprocessscript backend runs setup only
0.5sprocessno install scripts to re-run

Static findings

HIGH
Access to sensitive file pathstestmcpy/cli/app.py:22 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files# Load environment variables from .env file (for backward compatibility)Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Image alt text carrying instructionsREADME.md:109 · Image alt text carrying instructionsReports — combined view of every test run, evaluator scores, and cost analysisFix: Remove the text that addresses the agent rather than the reader; describe behavior, do not instruct.
AST01
HIGH
Access to sensitive file pathstestmcpy/security/scanner.py:199 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files(re.compile(r"(?<![\w.])\.env\b", re.IGNORECASE), ".env file"),Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/security/scanner.py:195 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files# rarely reference SSH keys or /etc/passwd.Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Credential pass-through configurationtestmcpy/src/runner_tools.py:475 · Credential pass-through configuration — agent relays sensitive credentials"or pass api_key directly."Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST02
HIGH
Access to sensitive file pathstestmcpy/config.py:108 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system filespath), or the env-format files ``~/.testmcpy`` / ``./.env``.Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/cli/app.py:23 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system filesload_dotenv(Path(__file__).parent.parent.parent / ".env")Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/config.py:67 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files# (~/.testmcpy and ./.env). These are the LLM provider API keysFix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/security/scanner.py:200 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files(re.compile(r"/etc/passwd\b"), "/etc/passwd"),Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/security/scanner.py:197 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files(re.compile(r"~?/?\.ssh\b", re.IGNORECASE), "SSH key directory (~/.ssh)"),Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/config.py:7 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files3. .env file in current directoryFix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/config.py:116 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files# 2. Load from .env in current directory (env-format file)Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/config.py:117 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system filescwd_env_file = Path.cwd() / ".env"Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Access to sensitive file pathstestmcpy/security/rules.py:95 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files"SSH keys (~/.ssh, id_rsa), .env files, /etc/passwd — or pair "Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
MEDIUM
Process kill commandtestmcpy/server/api.py:533 · Process kill command — agent can terminate processes# OSError on first-time DB-path init) would otherwise killFix: Build commands from fixed arguments; do not interpolate untrusted strings into a shell.
AST05

Cost to the agent

Instructions loaded
≈59 tok
1 file · 81 in bundle

Tokens estimated at four characters each. Skills load their instruction text when triggered; the figure is what that costs.

Known vulnerabilities

No advisories in OSV for testmcpy@0.11.22 as of 8 h ago. This is the package itself, not its dependencies.

What would make this Verified

5 points to 80 · weights: behavior 45%, static 25%, prompt 15%, publisher 15%
  1. Unclaimed publisher≈ +8 on the totalClaim preset-io: a verified publisher gets +50 on publisher trust. It is free and usually instant with GitHub sign-in.
  2. Hidden instructions≈ +6 on the totalRemove HTML comments, zero-width characters and white-on-white text that carry instructions. Everything the agent is meant to read should be visible to the person reading the file.
  3. Access to sensitive file paths≈ +4 on the totalStop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
  4. Credential pass-through configuration≈ +4 on the totalStop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
  5. Process kill command≈ +2 on the totalBuild commands from fixed arguments; do not interpolate untrusted strings into a shell.
Publish a new version to PyPI, then queue it here. Not your skill? Claim the handle to get verdict changes by email.Claim preset-ioPropose a patch to the maintainer