Skills/PyPI/mcpbr

mcpbr@0.14.0

CAUTION· startup only

Publisher claims: "Model Context Protocol Benchmark Runner - evaluate MCP servers against software engineering benchmarks" Observed in sandbox: no network, touches nothing outside the project, writes no memory.

Registry PyPI (MCP server)Publisher greynewell · publishing for 301 daysInstalls 56Last detonated 4d ago
What would fix thisWatchDispute verdict

Coverage

Startup only; authenticated tool behavior not tested.
Tested
server startup over stdio (initialize sent) · every file read, file written and host contacted by what ran
Not tested
the tool list (the server exited before answering) · what the tools do when called with real credentials

What this means for the label: Caution here covers what ran. The server's tools were not exercised with a real credential, so nothing about their behavior is asserted; the credential summary below says what to do with that.

Before you give it your token

It will ask for
ANTHROPIC_API_KEYBRAVE_API_KEYDASHSCOPE_API_KEYGOOGLE_API_KEYMY_API_KEYOPENAI_API_KEYSUPERMODEL_API_KEY or --smtp-password / --slack-bot-token / --github-tokenConfiguration, not credentials: MCPBR_AUDIT_HMAC_KEY, MCPBR_LOG_LEVEL, MCPBR_THEME, PYTHONHASHSEED
Where it talks to
mcpbr.org (its own)Also python.org, shields.io, nodesource.com, no-color.org, claude.com, githubusercontent.com, google.dev, alibabacloud.com, aliyuncs.com, google.com, microsoft.com — not the publisher's domain; check what for before connecting.
What the honeypot saw
no credential files read · no network
What we could not test
the tool list (the server exited before answering) · what the tools do when called with real credentials

Connect it with a scoped credential, under the guard, and read the third-party hosts first. The code names python.org, shields.io, nodesource.com, no-color.org, claude.com, githubusercontent.com, google.dev, alibabacloud.com, aliyuncs.com, google.com, microsoft.com as well as its own vendor. The guard injects ANTHROPIC_API_KEY, BRAVE_API_KEY, DASHSCOPE_API_KEY, GOOGLE_API_KEY, MY_API_KEY, OPENAI_API_KEY, SUPERMODEL_API_KEY only into requests to mcpbr.org (and its subdomains) and refuses the rest.

$vet-skills run --secrets ANTHROPIC_API_KEY=mcpbr.org,BRAVE_API_KEY=mcpbr.org,DASHSCOPE_API_KEY=mcpbr.org,GOOGLE_API_KEY=mcpbr.org,MY_API_KEY=mcpbr.org,OPENAI_API_KEY=mcpbr.org,SUPERMODEL_API_KEY=mcpbr.org -- uvx mcpbr

Never paste a credential into a chat, a ticket or a report, including ours. How the guard holds it.

Is [email protected] safe to install?

KillerSkills ran it in the KillerSkills honeypot sandbox (2 scenarios; startup only; authenticated tool behavior not tested) on 2026-11-10 and scored it 64/100 (Caution). It is not known to be malicious, but it has behaviour or code a security policy should decide on before install. Notable findings: Credential pass-through configuration; Credential pass-through configuration; Credential pass-through configuration. Summary: no network, touches nothing outside the project, writes no memory.

Behavior trace

Honeypot run · 21 s · 2 scenarios · 0 deviations
t+CategoryObserved action
0.0sprocessno install scripts in bundle
0.9sprocessdependency install exit 0
21sclaimedstartup only: exited (0) before answering · or pip3 install mcpbr For more information, visit: https://github.com/greynewell/mcpbr ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
21sprocessscript backend runs setup only
21sprocessno install scripts to re-run

Static findings

HIGH
Credential pass-through configurationsrc/mcpbr/providers.py:267 · Credential pass-through configuration — agent relays sensitive credentials"or pass api_key parameter."Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST02
MEDIUM
File type combined with upload actionsrc/mcpbr/cli.py:741 · File type combined with upload action — agent may exfiltrate local data files# Create README.txt in output directory with finalized config valuesFix: Remove the call, or declare the host in frontmatter and explain in the README what is sent and why.
AST07
MEDIUM
File permission modification commandsrc/mcpbr/harnesses.py:886 · File permission modification command — agent changes file ownership/permissionsawait env.exec_command(f"chown mcpbr:mcpbr {prompt_file}", timeout=5)Fix: Request the narrowest permission that works; replace wildcards with the exact paths or hosts.
AST05
MEDIUM
Python subprocess execution detectedsrc/mcpbr/gpu_support.py:41 · Python subprocess execution detectedresult = subprocess.run(Fix: Build commands from fixed arguments, never from interpolated strings; prefer an API client over shelling out.
AST05
MEDIUM
Process kill commandsrc/mcpbr/harness.py:679 · Process kill command — agent can terminate processes# Force-kill the container immediately so blocking executor threadsFix: Build commands from fixed arguments; do not interpolate untrusted strings into a shell.
AST05
MEDIUM
npx invocation of non-well-known packagebin/mcpbr.js:6 · npx invocation of non-well-known package — may execute untrusted code* This wrapper provides npm/npx access to the mcpbr CLI tool,Fix: Pin versions and sources; vendor anything fetched at install time.
AST06
MEDIUM
File type combined with upload actionsrc/mcpbr/cli.py:2339 · File type combined with upload action — agent may exfiltrate local data filesmcpbr compare run1.json run2.json run3.json --output-html comparison.htmlFix: Remove the call, or declare the host in frontmatter and explain in the README what is sent and why.
AST07
HIGH
Credential pass-through configurationsrc/mcpbr/providers.py:424 · Credential pass-through configuration — agent relays sensitive credentials"or pass api_key parameter."Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST02
HIGH
Credential pass-through configurationsrc/mcpbr/providers.py:99 · Credential pass-through configuration — agent relays sensitive credentials"or pass api_key parameter."Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST02
HIGH
Access to sensitive file pathssrc/mcpbr/env_expansion.py:10 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files"""Load environment variables from a .env file.Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Destructive file system command detectedsrc/mcpbr/docker_env.py:505 · Destructive file system command detected&& rm -rf /var/lib/apt/lists/*Fix: Build commands from fixed arguments; do not interpolate untrusted strings into a shell.
AST05
MEDIUM
File type combined with upload actionsrc/mcpbr/cli.py:95 · File type combined with upload action — agent may exfiltrate local data filesmcpbr compare run1.json run2.json --output-html comparison.htmlFix: Remove the call, or declare the host in frontmatter and explain in the README what is sent and why.
AST07
HIGH
Access to sensitive file pathssrc/mcpbr/env_expansion.py:16 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system filesdotenv_path = Path(".env")Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
MEDIUM
File type combined with upload actionsrc/mcpbr/cli.py:2340 · File type combined with upload action — agent may exfiltrate local data filesmcpbr compare *.json --output-markdown comparison.mdFix: Remove the call, or declare the host in frontmatter and explain in the README what is sent and why.
AST07
HIGH
Privileged sudo commandbin/mcpbr.js:110 · Privileged sudo command — agent may execute with elevated privileges• Ubuntu: sudo apt install python3.11Fix: Build commands from fixed arguments; do not interpolate untrusted strings into a shell.
AST05
MEDIUM
File permission modification commandsrc/mcpbr/docker_env.py:949 · File permission modification command — agent changes file ownership/permissionsf"chown -R mcpbr:mcpbr {env.workdir}"Fix: Request the narrowest permission that works; replace wildcards with the exact paths or hosts.
AST05
HIGH
Access to sensitive file pathssrc/mcpbr/config.py:1078 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system filesf"2. Add it to a .env file in the current directory\n"Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Privileged sudo commandsrc/mcpbr/docker_env.py:849 · Privileged sudo command — agent may execute with elevated privileges"apt-get install -y -qq curl ca-certificates gnupg sudo && "Fix: Build commands from fixed arguments; do not interpolate untrusted strings into a shell.
AST05
HIGH
Credential pass-through configurationsrc/mcpbr/providers.py:192 · Credential pass-through configuration — agent relays sensitive credentials"or pass api_key parameter."Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST02
MEDIUM
File permission modification commandsrc/mcpbr/docker_env.py:948 · File permission modification command — agent changes file ownership/permissions"chown -R mcpbr:mcpbr /workspace && "Fix: Request the narrowest permission that works; replace wildcards with the exact paths or hosts.
AST05
HIGH
Collapsed <details> block carrying instructionsREADME.md:837 · Collapsed <details> block carrying instructions<summary>Options and examples</summary> | Option | Short | Description | |--------|-------|-------------| | `--dry-run` | | Show containers that would be removeFix: Remove the text that addresses the agent rather than the reader; describe behavior, do not instruct.
AST01
HIGH
Access to sensitive file pathssrc/mcpbr/env_expansion.py:13 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system filesdotenv_path: Path to .env file. If None, looks for .env in current directory.Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
HIGH
Collapsed <details> block carrying instructionsREADME.md:196 · Collapsed <details> block carrying instructions<summary>Prerequisites</summary> - Python 3.11+ - Docker (running) - `ANTHROPIC_API_KEY` environment variable - Claude Code CLI (`claude`) installed - Network aFix: Remove the text that addresses the agent rather than the reader; describe behavior, do not instruct.
AST01
MEDIUM
Process kill commandsrc/mcpbr/harness.py:851 · Process kill command — agent can terminate processes# Force-kill the container immediately so blocking executor threadsFix: Build commands from fixed arguments; do not interpolate untrusted strings into a shell.
AST05
HIGH
Privileged sudo commandbin/mcpbr.js:84 · Privileged sudo command — agent may execute with elevated privileges• Ubuntu: sudo apt install python3.11Fix: Build commands from fixed arguments; do not interpolate untrusted strings into a shell.
AST05
HIGH
Access to sensitive file pathssrc/mcpbr/config.py:1052 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system files# Load .env file if it exists in the current directoryFix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
MEDIUM
File permission modification commandsrc/mcpbr/docker_env.py:947 · File permission modification command — agent changes file ownership/permissions"chown -R mcpbr:mcpbr /home/mcpbr && "Fix: Request the narrowest permission that works; replace wildcards with the exact paths or hosts.
AST05
HIGH
Access to sensitive file pathssrc/mcpbr/config.py:1031 · Access to sensitive file paths — SSH keys, AWS credentials, env files, or system filesAutomatically loads .env file from current directory if it exists.Fix: Stop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
AST05
MEDIUM
Python subprocess execution detectedsrc/mcpbr/resource_limits.py:255 · Python subprocess execution detectedresult = subprocess.run(Fix: Build commands from fixed arguments, never from interpolated strings; prefer an API client over shelling out.
AST05
MEDIUM
npx invocation of non-well-known packagesrc/mcpbr/tutorial.py:258 · npx invocation of non-well-known package — may execute untrusted code" [green]command:[/green] npx # Executable to run\n"Fix: Pin versions and sources; vendor anything fetched at install time.
AST06
MEDIUM
File permission modification commandsrc/mcpbr/harnesses.py:937 · File permission modification command — agent changes file ownership/permissionsawait env.exec_command(f"chown mcpbr:mcpbr {env_file}", timeout=5)Fix: Request the narrowest permission that works; replace wildcards with the exact paths or hosts.
AST05
MEDIUM
File permission modification commandsrc/mcpbr/harnesses.py:987 · File permission modification command — agent changes file ownership/permissionsawait env.exec_command(f"chown mcpbr:mcpbr {mcp_json_path}", timeout=5)Fix: Request the narrowest permission that works; replace wildcards with the exact paths or hosts.
AST05
MEDIUM
Python subprocess execution detectedsrc/mcpbr/tutorial.py:852 · Python subprocess execution detectedresult = subprocess.run( # noqa: S602 -- tutorial validation runs user-defined shell commands by designFix: Build commands from fixed arguments, never from interpolated strings; prefer an API client over shelling out.
AST05
MEDIUM
npx invocation of non-well-known packagesrc/mcpbr/languages.py:78 · npx invocation of non-well-known package — may execute untrusted codecompile_command="npx tsc {file}",Fix: Pin versions and sources; vendor anything fetched at install time.
AST06
MEDIUM
Python subprocess execution detectedsrc/mcpbr/storage/cloud.py:108 · Python subprocess execution detectedreturn subprocess.run(Fix: Build commands from fixed arguments, never from interpolated strings; prefer an API client over shelling out.
AST05

Cost to the agent

Instructions loaded
≈59 tok
1 file · 81 in bundle
Tool list per prompt
≈0 tok
0 tools · 0 per tool
Handshake
no reply
launch to tools/list, stdio
Undocumented tools
0
description under 20 chars · avg 0

Tokens estimated at four characters each. The tool list is re-read by the agent on every turn, so it is the number to compare between servers that do the same job: ranked across the index.

Known vulnerabilities

No advisories in OSV for mcpbr@0.14.0 as of 4d ago. This is the package itself, not its dependencies.

What would make this Verified

16 points to 80 · weights: behavior 45%, static 25%, prompt 15%, publisher 15%
  1. Unclaimed publisher≈ +8 on the totalClaim greynewell: a verified publisher gets +50 on publisher trust. It is free and usually instant with GitHub sign-in.
  2. Hidden instructions≈ +6 on the totalRemove HTML comments, zero-width characters and white-on-white text that carry instructions. Everything the agent is meant to read should be visible to the person reading the file.
  3. Credential pass-through configuration≈ +4 on the totalStop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
  4. Access to sensitive file paths≈ +4 on the totalStop enumerating or reading credential locations; take secrets from a declared environment variable the user sets on purpose.
  5. Destructive file system command detected≈ +4 on the totalBuild commands from fixed arguments; do not interpolate untrusted strings into a shell.
  6. Privileged sudo command≈ +4 on the totalBuild commands from fixed arguments; do not interpolate untrusted strings into a shell.
Publish a new version to PyPI, then queue it here. Not your skill? Claim the handle to get verdict changes by email.Claim greynewellPropose a patch to the maintainer