CLI Tool
Run human QA tests from your terminal with the Runhuman CLI.
What is the CLI?
The Runhuman CLI is a command-line interface for managing QA tests, projects, templates, schedules, and API keys. It provides both interactive and JSON output modes, making it perfect for local development, CI/CD pipelines, and automation.
Why use the CLI?
- Fast and efficient terminal-based workflow
- JSON output mode for CI/CD integration
- Local configuration management
- Interactive prompts for common tasks
- Complete control over all Runhuman features
Installation
# Install globally
npm install -g runhuman
# Or use with npx (no installation required)
npx runhuman --help
Requirements: Node.js 18+
Quick Start
# 1. Login (opens browser for OAuth)
runhuman login
# 2. Create your first test
runhuman job create https://google.com \
-d "Search for 'recursion' and confirm the tester experiences a joke about recursion"
# 3. Check the status
runhuman job status <jobId>
# 4. Get results
runhuman job results <jobId>
Command Structure
Job commands live under the runhuman job namespace (job create, job status, job wait, job results, job list, job watch, job artifact, job share, job unshare, job create-issue). The flat shims (runhuman create, runhuman status, runhuman wait, runhuman results, runhuman list, runhuman watch) still work but are deprecated — invoking them prints a one-line warning and they will be removed in a future release. New scripts and CI configurations should use the job form.
All other top-level commands keep their plural-noun form: orgs, projects, keys, templates, schedules, github, transfers, notes, billing, config, plus init, login, logout, whoami. Singular aliases (org, project, key, etc.) are accepted everywhere.
Truncated IDs
All commands accept truncated ID prefixes, similar to git short commit hashes. Instead of typing a full UUID or hex ID, use just the first few characters:
# Full ID
runhuman job status 712e42ad-8f3b-4a1c-9d5e-1234567890ab
# Truncated prefix — resolves to the same job
runhuman job status 712e
# Hyphens are ignored during matching
runhuman job status 712e42ad8f3b
How it works:
- The CLI fetches your resources and finds all IDs that start with your prefix
- If exactly one matches, it resolves to the full ID automatically
- If multiple match, you’re asked to provide more characters
- Matching is case-insensitive and hyphen-insensitive
Full ID required for destructive operations:
For safety, projects delete, keys delete, schedules delete, transfers cancel, and projects transfer --to-org require the full resource ID. If you provide a short prefix, the error message shows what it would have matched so you can re-run with the full ID.
# This will be rejected with a helpful error
runhuman projects delete proj_a
# Use the full ID for destructive operations
runhuman projects delete proj_abc12345def67890abc12345def67890
Global Flags
Every command accepts the same four global flags:
-j, --json— emit machine-readable JSON to stdout (otherwise pretty-printed output)-q, --quiet— suppress success and info messages (stderr deprecation/error output is preserved)-y, --yes— skip confirmation prompts on destructive operations--no-color— disable ANSI color codes (also honored fromRUNHUMAN_NO_COLOR=1)
-f, --force is reserved for override-a-guard semantics, not confirmation-skip. Today it only appears on templates create, where it means “overwrite an existing template with the same name.”
-j, --json is the single output-mode flag. The CLI does not offer a -o, --output / --format choice — pretty output for humans and JSON for scripts are the two supported shapes. This also keeps -o bound to --organization on every command that has one.
JSON envelope
Every command that emits JSON uses the same envelope:
// Success
{
"success": true,
"data": <payload>, // see below
"pagination": { // only on list commands
"total": 42,
"limit": 20,
"offset": 0,
"hasMore": true
},
"warnings": ["..."], // optional, non-fatal server-side warnings
"timestamp": "2026-04-24T12:34:56.789Z"
}
// Failure
{
"success": false,
"error": {
"message": "...",
"code": "OPTIONAL_ERROR_CODE",
"details": { /* shape varies */ }
},
"timestamp": "2026-04-24T12:34:56.789Z"
}
Payload conventions:
- List commands (
projects list,job list,orgs list, etc.) put their items underdata.itemsand pagination on the envelope. Neverdata.jobsordata.projects— alwaysdata.items. - Show / create / update commands put the entity directly as
data(e.g.data: { id, name, ... }). - Delete / archive / cancel commands return a structured action payload such as
data: { id, deleted: true }. The envelope already carriessuccess: true;datanever echoes it.
A generic script can parse any CLI command’s output without inspecting which command was run:
runhuman projects list --json | jq '.data.items[].id'
runhuman projects show <id> --json | jq '.data.id'
runhuman projects delete <id> --json | jq '.data.deleted'
Core Commands
Job Management
Create and manage QA test jobs. All job commands live under the runhuman job namespace; the legacy flat shims (runhuman create, runhuman status, runhuman wait, runhuman results, runhuman list, runhuman watch) still resolve but emit a deprecation warning and will be removed in a future release.
job create [url]
Create a new QA test job. Aliases: runhuman create (deprecated).
# Basic usage
runhuman job create https://google.com \
-d "Search for 'recursion' and confirm the tester experiences a joke about recursion"
# With a named template (resolved from repo → project → built-in)
runhuman job create https://google.com --template "Find Bugs"
# With a local template file (.md with YAML frontmatter)
runhuman job create --template-file .runhuman/templates/smoke-test.md
# Specify device class
runhuman job create https://google.com -d "Test mobile layout" --device-class mobile
# Synchronous (wait for result)
runhuman job create https://google.com -d "Test search" --sync
# With custom output schema
runhuman job create https://google.com -d "Test" --schema ./schema.json
# With inline schema
runhuman job create https://google.com -d "Test" --schema-inline '{"passed":{"type":"boolean"}}'
# With an MDForm results template (free-form structured report)
runhuman job create https://google.com -d "Test" --results-template ./report.mdform
# With metadata
runhuman job create https://google.com -d "Test" --metadata '{"env":"staging"}'
# Multi-repo context (PRs/issues consolidated across repos)
runhuman job create https://staging.example.com \
-d "Smoke-test the checkout flow" \
--github-repos volter-ai/runhuman,volter-ai/runhuman-action \
--pr-numbers 3260,3293
# JSON output for scripting
runhuman job create https://google.com -d "Test" --json
# Minimal output (only job ID)
runhuman job create https://google.com -d "Test" --quiet
# With tester pool requirements
runhuman job create https://google.com -d "Test mobile layout" \
--required-devices ios,android \
--required-languages english \
--require-apk-install
Options:
-d, --description <text>- Test instructions for the human tester-t, --template <name>- Use a template by name (resolved from repo.runhuman/templates/, project templates, then built-ins)--template-file <path>- Path to a local.mdtemplate file (YAML frontmatter + markdown body). See Templates for the format.--duration <minutes>- Target test duration in minutes (1–60)--device-class <class>- Device class:desktop,mobile, orboth(default:both)--schema <file>- Path to JSON schema file for structured output--schema-inline <json>- Inline JSON schema string--results-template <file>- Path to an MDForm template; the AI fills it to produceresults.text. Alternative to--schema.--results-template-inline <markdown>- Inline MDForm content--metadata <json>- Metadata for tracking (JSON string)--github-repo <owner/repo>- Single GitHub repo for context--github-repos <repos>- Multi-repo context (comma-separatedowner/repo)--pr-numbers <numbers>- Comma-separated PR numbers (triggers AI test-plan consolidation across repos)--issue-numbers <numbers>- Comma-separated issue numbers (triggers AI test-plan consolidation)--no-check-testability- Skip the AI testability gate (gate runs by default when PRs/issues are provided)--github-token <token>- GitHub token for PR/issue context (fallback:GITHUB_TOKENenv)--additional-validation-instructions <text>- Extra LLM validation instructions--auto-create-github-issues- Auto-create GitHub issues from AI-extracted findings (requires--github-repoor--github-repos)--auto-create-github-issues-repo <owner/repo>- Target repo for auto-created issues (must be in--github-repos)--slack-notification-json <json>- Slack notification config override (JSON)--email <pref>- Per-job email-on-completion override:inherit(default — defers to project, then account-level Job updates pref),always, ornever--required-devices <devices>- Required tester devices (comma-separated:ios,android,pc,mac)--required-languages <languages>- Required tester languages (comma-separated:english,spanish)--require-social-videos- Require tester capable of recording social-media-style videos (Max+ tier)--require-apk-install- Require tester capable of installing APK files (Max+ tier)--max-extension-minutes <minutes>- Total extension time (default: 15, 0 to disable)--max-extension-count <count>- Number of extensions (default: 3, 0 to disable)-s, --sync- Wait for result before exiting--wait <seconds>- Max wait time in sync mode (default: 300)-p, --project <id>- Project ID (or use default from config)
Example schema file:
{
"searchWorks": {
"type": "boolean",
"description": "Does the search return results?"
},
"jokeAppears": {
"type": "boolean",
"description": "Does the recursion joke appear?"
},
"issues": {
"type": "array",
"description": "List of any bugs or issues found"
}
}
job status <jobId>
Check the status of a test job. Aliases: show, get, info, plus the deprecated flat runhuman status.
runhuman job status job_abc123
runhuman job status 712e # truncated ID
runhuman job status job_abc123 --json
Options:
-p, --project <id>- Scope search to a specific project-o, --organization <id>- Scope search to a specific organization-j, --json- Output as JSON
Shows the current status — one of pending, queued, waiting, working, completed, error, incomplete, expired, rejected — plus tester pool requirements, extension counters, and a dashboard link.
job wait <jobId>
Wait for a job to complete with live status updates. Alias: runhuman wait (deprecated).
runhuman job wait job_abc123
runhuman job wait 712e --timeout 300 # truncated ID
Options:
-p, --project <id>- Scope search to a specific project-o, --organization <id>- Scope search to a specific organization--timeout <seconds>- Maximum wait time (default: 600)-j, --json- Output as JSON
job results <jobId>
Display detailed test results, with optional rich-data sections. Alias: runhuman results (deprecated).
runhuman job results job_abc123
runhuman job results job_abc123 --json # For scripting
runhuman job results job_abc123 --schema-only # Only extracted schema data
runhuman job results job_abc123 --raw # Raw tester response
# Add rich data sections
runhuman job results job_abc123 --transcript --console-logs
runhuman job results job_abc123 --all
Options:
-p, --project <id>- Scope search to a specific project-o, --organization <id>- Scope search to a specific organization--schema-only- Show only extracted schema data--raw- Show raw tester response without processing--transcript- Include the voice transcript--console-logs- Include captured console logs--network- Include captured network requests--events- Include user-interaction events--key-moments- Include key-moments timeline--conversation- Include the AI ↔ tester conversation history-a, --all- Include every rich-data section above-j, --json- Output as JSON
Rich data sections (transcript, console logs, network, events, key moments, conversation) are paid-tier gated server-side; ungated requests return a clean “subscription required” error.
job list [filter]
List your test jobs with optional status filtering. The [filter] values are the same chip labels shown on the web Jobs page, so CLI and web vocabulary stay aligned. Alias: runhuman list (deprecated).
# List all jobs
runhuman job list
# Filter by status (positional argument, case-insensitive)
runhuman job list Completed
runhuman job list pending
# Filter by multiple labels at once
runhuman job list "Pending,Working,Completed"
# With pagination
runhuman job list --limit 20 --offset 40
# Filter by project, or scope to an entire org
runhuman job list --project proj_abc123
runhuman job list --org org_abc123
# JSON output
runhuman job list --json
Arguments:
[filter]- Chip label (comma-separated, case-insensitive):all,Pending,Queued,Working,Completed,Cancelled,Expired,Failed. Admins may additionally pass fine-grain labels that mirror the Jobs page “Show more” row (Invalidated,Cancelled by tester,Admin terminated,No capacity,Never started,Overtime,Incomplete,Error,Untestable,Other).
Options:
-p, --project <id>- Filter by project ID (or use default from config)--org <id>- Scope to all projects in an organization (mutually exclusive with--project)-n, --limit <number>- Maximum number of jobs (default: 20)--offset <number>- Pagination offset (default: 0)-j, --json- Output as JSON
job artifact <jobId> <type>
Download a single raw session artifact. Useful for piping transcripts, console logs, or network data into your own tools without parsing the full results envelope.
# Print the transcript
runhuman job artifact job_abc123 transcript
# Pipe raw console logs into jq
runhuman job artifact job_abc123 console-logs --json | jq '.data.consoleMessages[]'
# Pull just the structured-output payload
runhuman job artifact job_abc123 structured-output --json | jq '.data.result'
Arguments:
<jobId>- Job ID<type>- One of:structured-output,transcript,console-logs,network-requests,conversation,events,key-moments
Options:
-j, --json- Output as JSON (raw artifact shape wrapped in the standard envelope)
Rich artifacts are paid-tier gated server-side; ungated requests return a clean “subscription required” error.
job share <jobId> / job unshare <jobId>
Toggle public sharing for a job. share enables and prints the public URL; unshare disables and revokes any existing public link.
runhuman job share job_abc123
runhuman job unshare job_abc123
job create-issue <jobId> --index <n>
File one extracted finding from a completed job as a GitHub issue. Mirrors the web UI’s per-finding “Create Issue” button. Submits the finding unedited; use the web UI or gh issue edit if you need to change the title or body.
# File the first extracted finding to the job's default GitHub repo
runhuman job create-issue job_abc123 --index 0
# Override the target repo
runhuman job create-issue job_abc123 --index 0 --repo owner/repo
Required options:
--index <n>- Zero-based index intojob.extractedIssues
Options:
--repo <owner/repo>- Override the job’s default GitHub repo-j, --json- Output as JSON
job watch [patterns...]
Watch local files and auto-create test jobs on changes. Pattern, debounce, template, description, and URL defaults can also be persisted under watch in .runhumanrc. Alias: runhuman watch (deprecated).
# Watch source files
runhuman job watch "src/**/*.tsx" --url https://google.com
# Watch multiple patterns
runhuman job watch "src/**/*.tsx" "src/**/*.css" --url https://google.com
# With template and custom debounce
runhuman job watch "src/**/*" --url https://google.com --template "Search Test" --debounce 5000
# Check whether watch is running
runhuman job watch --status
# Stop the existing watch process
runhuman job watch --stop
Options:
-u, --url <url>- URL to test (required for new jobs)-t, --template <name>- Template to use for auto-created tests-d, --description <text>- Default test description--debounce <ms>- Debounce delay in milliseconds (default: 2000)--ignore <patterns...>- Patterns to ignore--stop- Stop existing watch process--status- Check whether watch is running
job delete <jobId>
Delete a job. Admin-only — customers cannot delete their own jobs (jobs must run through to a terminal state for billing and tester-coordination integrity). Use the dashboard’s archive flow instead for organizing your own job list.
runhuman job delete job_abc123 --force
Options:
-f, --force- Skip confirmation prompt-j, --json- Output as JSON
There is intentionally no job cancel command; once a tester has been engaged, jobs run through to completion. The backend cancel route is restricted to admin and tester callers.
Authentication
Manage your authentication and account.
login
Authenticate with Runhuman. Opens your browser for OAuth by default.
# OAuth login (opens browser)
runhuman login
# Login with API key (skip browser)
runhuman login --token <your-api-key>
# Print auth URL without opening browser
runhuman login --no-browser
Options:
--token <token>- Login with an API key instead of OAuth--no-browser- Print the auth URL instead of auto-opening your browser
The OAuth flow sets your default project automatically. Credentials are stored in ~/.config/runhuman/credentials.json with restricted permissions.
logout
Clear your authentication credentials.
runhuman logout
whoami
Show full context for the current session: who you are signed in as, which organization and project the CLI will use by default, where each default came from (flag / env / project config / global config / inferred), and which API host you’re talking to.
runhuman whoami
runhuman whoami --json
Displays:
- Account email + user ID
- Active organization (and source of the default)
- Active project (and source of the default)
- API URL the CLI is configured against
- Warnings when the configured project doesn’t belong to the configured organization, or when no default project can be resolved
Use this command when a job command picks the “wrong” project — whoami shows you exactly which config layer is winning so you can override it with --project, RUNHUMAN_PROJECT, runhuman config set project, or .runhumanrc.
Organizations
Manage your organizations. Projects, billing, and API keys are all scoped to organizations.
orgs list
List all organizations you belong to.
runhuman orgs list
runhuman orgs ls # alias
runhuman orgs list --json
Options:
-n, --limit <number>- Number of results (default: 20)-j, --json- Output as JSON
orgs show <organizationId>
Show detailed information about an organization.
runhuman orgs show org_abc123
runhuman orgs info org_abc123 # alias
runhuman orgs get org_abc123 # alias
Displays:
- Organization ID
- Name and slug
- Creation date
orgs balance <organizationId>
Check your organization’s billing balance.
runhuman orgs balance org_abc123
runhuman orgs balance org_abc123 --json
Displays:
- Current balance (USD)
- Billing status
- Estimated tests remaining
orgs projects <organizationId>
List all projects in an organization.
runhuman orgs projects org_abc123
runhuman orgs projects org_abc123 --limit 10
Options:
-n, --limit <number>- Number of results (default: 50)-j, --json- Output as JSON
orgs switch <organizationId>
Set the default organization for CLI commands.
runhuman orgs switch org_abc123
runhuman orgs use org_abc123 # alias
runhuman orgs switch org_abc123 --global # all directories (default)
Options:
-g, --global- Set as global default instead of local (default: true)
orgs members <organizationId>
List members of an organization, including their role, status (active / suspended), and join date. The org owner is marked with *.
runhuman orgs members org_abc123
runhuman orgs members org_abc123 --json
Options:
-j, --json- Output as JSON
orgs invite <organizationId> <email>
Invite a member to an organization via Clerk. Roles use the 3-role model: admin, contributor, viewer.
runhuman orgs invite org_abc123 person@example.com
runhuman orgs invite org_abc123 person@example.com --role admin
Options:
--role <role>- One ofadmin,contributor,viewer(default:contributor)-j, --json- Output as JSON
orgs remove <organizationId> <userId>
Remove a member from an organization. Alias: orgs delete.
runhuman orgs remove org_abc123 user_xyz789
runhuman orgs remove org_abc123 user_xyz789 --yes # skip confirmation
Options:
-y, --yes- Skip confirmation prompt
orgs invitations list|revoke
Manage pending invitations created by orgs invite. Group alias: orgs invites.
# List pending invitations for an org
runhuman orgs invitations list --organization org_abc123
# Revoke a specific invitation
runhuman orgs invitations revoke inv_xyz789 --organization org_abc123 --yes
Options (both subcommands):
-o, --organization <id>- Organization ID (defaults to current org context)-y, --yes- Skip confirmation onrevoke-j, --json- Output as JSON
orgs usage [organizationId]
Show usage analytics for an org: total jobs, total cost, per-project breakdown, and over-time / by-hour / by-source series. Defaults to the current org context if no ID is supplied.
# Default: last 30 days
runhuman orgs usage
# Preset window
runhuman orgs usage --period 7d
runhuman orgs usage --period all
# Specific month
runhuman orgs usage --year 2026 --month 4
# Pipe the full response into your own tool
runhuman orgs usage --json | jq '.data.byHour'
Options:
--period <window>-7d|30d|90d|all(mutually exclusive with--year/--month)--year <y>/--month <m>- Specific time window (--monthrequires--year)-j, --json- Full response JSON, includingoverTime,byHour, andbySourcearrays
Default rendering shows the summary plus the top 5 projects by job count. --json returns the full series for scripting.
Projects
Organize your tests into projects. Projects belong to organizations.
projects list
List all your projects across all organizations.
runhuman projects list
runhuman projects ls # alias
runhuman projects list --json
Options:
-n, --limit <number>- Number of results (default: 20)--offset <number>- Pagination offset (default: 0)-j, --json- Output as JSON
Shows each project’s name, organization, project ID, and creation date.
projects create <name>
Create a new project in an organization.
runhuman projects create "My App" --organization org_abc123
runhuman projects create "My App" --organization org_abc123 --default-url https://google.com
runhuman projects create "My App" --organization org_abc123 --set-default
Options:
-o, --organization <id>- Organization ID (required)--default-url <url>- Default URL for tests--github-repo <owner/repo>- Link a GitHub repository--github-repos <repos>- Link GitHub repositories (comma-separated owner/repo)--set-default- Set as default project after creation
projects show <projectId>
Show details of a specific project, including its organization and balance.
runhuman projects show proj_abc123
runhuman projects info proj_abc123 # alias
runhuman projects get proj_abc123 # alias
Displays:
- Project ID, name, and settings
- Organization name and ID
- Organization balance
- Default URL and GitHub repos (if set)
projects update <projectId>
Update a project’s settings.
runhuman projects update proj_abc123 --name "New Name"
runhuman projects update proj_abc123 --default-url https://google.com
runhuman projects update proj_abc123 --add-repos owner/repo
runhuman projects update proj_abc123 --remove-repos owner/old-repo
Options:
-n, --name <name>- New project name-u, --default-url <url>- New default URL--add-repos <repos>- Add GitHub repositories (comma-separated owner/repo)--remove-repos <repos>- Remove GitHub repositories (comma-separated owner/repo)
projects switch <projectId>
Set the default project for CLI commands.
# Set as global default (all directories)
runhuman projects switch proj_abc123
runhuman projects use proj_abc123 # alias
# Set as local default (current directory only)
runhuman projects switch proj_abc123 --global false
Options:
-g, --global- Set as global default instead of local (default: true)
projects transfer <projectId>
Transfer a project to another organization. The --to-org flag requires the full organization ID for safety.
runhuman projects transfer proj_abc123 --to-org org_target123
runhuman projects transfer proj_abc123 --to-org org_target123 --force
Options:
--to-org <organizationId>- Target organization ID (required, full ID required)-f, --force- Skip confirmation prompt-j, --json- Output as JSON
If the target organization requires approval, the transfer enters a pending state. Otherwise, it completes immediately.
projects delete <projectId>
Delete a project. Requires the full project ID for safety.
runhuman projects delete proj_abc123 --force
Options:
-f, --force- Skip confirmation prompt-j, --json- Output as JSON
Transfers
Manage project transfer requests between organizations.
transfers list
List pending incoming and outgoing transfer requests.
runhuman transfers list
runhuman transfers ls # alias
runhuman transfers list --json
transfers accept <transferId>
Accept an incoming project transfer.
runhuman transfers accept xfer_abc123
transfers reject <transferId>
Reject an incoming project transfer.
runhuman transfers reject xfer_abc123
transfers cancel <transferId>
Cancel an outgoing transfer you initiated.
runhuman transfers cancel xfer_abc123
API Keys
Manage API keys for organizations.
keys list
List all API keys for an organization.
runhuman keys list --organization org_abc123
runhuman keys ls --organization org_abc123 # alias
runhuman keys list --organization org_abc123 --show-keys
Options:
-o, --organization <id>- Organization ID (required)--show-keys- Display full unmasked keys (use with caution)-j, --json- Output as JSON
keys create <name>
Create a new API key for an organization.
runhuman keys create "Production Key" --organization org_abc123
runhuman keys create "CI/CD Key" --organization org_abc123
Options:
-o, --organization <id>- Organization ID (required)
Important: The full key is only shown once. Save it immediately.
keys show <keyId>
Show details of a specific API key.
runhuman keys show key_abc123
runhuman keys show key_abc123 --show-key
Options:
--show-key- Show full API key (default: masked)
keys delete <keyId>
Delete/revoke an API key. Requires the full key ID for safety.
runhuman keys delete key_abc123 --force
runhuman keys revoke key_abc123 --force # alias
runhuman keys rm key_abc123 --force # alias
Options:
-f, --force- Skip confirmation prompt-j, --json- Output as JSON
Templates
Create reusable test templates. Templates are scoped to projects.
templates list
List templates for a project with optional filters. Fetches every template across all pages before filtering, so results never miss matches on later pages.
runhuman templates list --project proj_abc123
runhuman templates ls --project proj_abc123 # alias
# Filters
runhuman templates list --project proj_abc123 --search checkout
runhuman templates list --project proj_abc123 --device mobile
runhuman templates list --project proj_abc123 --device mobile --device desktop # union
runhuman templates list --project proj_abc123 --has-schema --include-defaults
# Sort
runhuman templates list --project proj_abc123 --sort duration --direction desc
Options:
-p, --project <id>- Project ID (required, or use default from config)--search <text>- Filter by name or description (case-insensitive substring match)--device <class>- Filter by device class (desktop|mobile). Repeatable for a union.--has-schema- Only show templates that define an output schema--include-defaults- Include unmodified system default templates (hidden by default)--sort <field>- Sort bynameorduration(defaults toname)--direction <dir>- Sort directionascordesc(defaults toasc)-j, --json- Output as JSON
templates create <name>
Create a new template. If a template with the same name already exists, the CLI handles the conflict based on mode:
- With
--force: silently overwrites the existing template - Interactive terminal (no
--force): prompts you to confirm the overwrite - Non-interactive / JSON mode (no
--force): exits with code 7 and suggests using--force
# Create a new template
runhuman templates create "Search Test" \
--project proj_abc123 \
-d "Search for 'recursion' and confirm the joke appears" \
--schema ./search-test-schema.json
# Overwrite an existing template with the same name
runhuman templates create "Search Test" \
--project proj_abc123 \
-d "Updated description" \
--force
# In CI/CD, --force avoids interactive prompts
runhuman templates create "Nightly Smoke Test" \
--project proj_abc123 \
-d "Run nightly smoke test" \
--force --json
Options:
-p, --project <id>- Project ID (required, or use default from config)-d, --description <text>- Template description--duration <minutes>- Default test duration in minutes (1-60)--device-class <class>- Default device class (desktop/mobile/both)-S, --schema <path>- Path to output schema JSON file--max-extension-count <n>- Number of duration extensions allowed mid-job (0 disables; API default 3)--max-extension-minutes <m>- Total extension time in minutes (0 disables; API default 15)-f, --force- Overwrite an existing template with the same name without prompting
templates show <templateId>
Show template details.
runhuman templates show tmpl_abc123 --project proj_abc123
Options:
-p, --project <id>- Project ID (required, or use default from config)
templates update <templateId>
Update a template.
runhuman templates update tmpl_abc123 --project proj_abc123 --name "New Name"
runhuman templates update tmpl_abc123 --project proj_abc123 -d "Updated description"
Options:
-p, --project <id>- Project ID (required, or use default from config)-n, --name <name>- New template name-d, --description <text>- New description--duration <minutes>- New target duration in minutes (1-60)--device-class <class>- New default device class-S, --schema <path>- Path to new output schema JSON file--max-extension-count <n>- Number of duration extensions allowed mid-job (0 disables; API default 3)--max-extension-minutes <m>- Total extension time in minutes (0 disables; API default 15)
templates delete <templateId>
Delete a template.
runhuman templates delete tmpl_abc123 --project proj_abc123 --force
Options:
-p, --project <id>- Project ID (required, or use default from config)-f, --force- Skip confirmation prompt
Schedules
Create and manage recurring test schedules. Schedules are scoped to projects and run a template on a repeating cadence.
schedules list
List all schedules for a project.
runhuman schedules list --project proj_abc123
runhuman schedules ls --project proj_abc123 # alias
runhuman schedules list --project proj_abc123 --json
Options:
-p, --project <id>- Project ID (required, or use default from config)-n, --limit <number>- Number of results (default: 20)--offset <number>- Pagination offset (default: 0)-j, --json- Output as JSON
schedules create <name>
Create a recurring test schedule.
# Daily at 9:00 AM ET
runhuman schedules create "Daily Smoke Test" \
--template tmpl_abc123 \
--project proj_abc123 \
--frequency daily \
--time 09:00 \
--timezone America/New_York
# Weekly on Mon, Wed, Fri at 2:30 PM
runhuman schedules create "Weekday Regression" \
--template tmpl_abc123 \
--frequency weekly \
--weekdays mon,wed,fri \
--time 14:30
# Monthly on the 1st at 8:00 AM
runhuman schedules create "Monthly Full Suite" \
--template tmpl_abc123 \
--frequency monthly \
--day-of-month 1 \
--time 08:00
# One-time on a specific date
runhuman schedules create "Launch Day Test" \
--template tmpl_abc123 \
--frequency once \
--date 2026-05-01 \
--time 06:00
# Biweekly on Tue and Thu
runhuman schedules create "Sprint Review" \
--template tmpl_abc123 \
--frequency biweekly \
--weekdays tue,thu \
--time 10:00
Options:
-t, --template <id>- Template ID to run (required)-p, --project <id>- Project ID (required, or use default from config)--frequency <freq>- Frequency:daily,weekly,biweekly,monthly,once(default:weekly)--time <HH:MM>- Time of day in 24h format, quarter-hour only::00,:15,:30,:45(default:09:00)--timezone <tz>- IANA timezone string (default:America/New_York)--weekdays <days>- Comma-separated weekdays for weekly/biweekly (e.g.,mon,wed,fri)--day-of-month <n>- Day of month for monthly schedules (1–28, or0for last day)--date <YYYY-MM-DD>- Specific date for one-time schedules--url <url>- URL override (overrides the template’s default)--auto-github-issues- Auto-create GitHub issues from results-j, --json- Output as JSON
Frequency requirements:
weeklyandbiweeklyrequire--weekdaysmonthlyrequires--day-of-monthoncerequires--datedailyneeds only--timeand--timezone
schedules show <scheduleId>
Show details of a specific schedule.
runhuman schedules show sched_abc123 --project proj_abc123
runhuman schedules info sched_abc123 # alias
runhuman schedules get sched_abc123 # alias
Options:
-p, --project <id>- Project ID (required, or use default from config)-j, --json- Output as JSON
schedules update <scheduleId>
Update a schedule’s settings.
# Change frequency and time
runhuman schedules update sched_abc123 --frequency daily --time 08:00
# Change weekdays
runhuman schedules update sched_abc123 --weekdays mon,tue,wed,thu,fri
# Change template
runhuman schedules update sched_abc123 --template tmpl_new123
# Pause or resume
runhuman schedules update sched_abc123 --status paused
runhuman schedules update sched_abc123 --status active
# Clear URL override
runhuman schedules update sched_abc123 --url ""
Options:
-p, --project <id>- Project ID (required, or use default from config)--name <name>- New schedule name-t, --template <id>- New template ID--frequency <freq>- New frequency--time <HH:MM>- New time of day--timezone <tz>- New timezone--weekdays <days>- New weekdays--day-of-month <n>- New day of month--date <YYYY-MM-DD>- New date (for one-time schedules)--url <url>- New URL override (use""to clear)--auto-github-issues <bool>- Auto-create GitHub issues (true/false)--status <status>- Set status:activeorpaused-j, --json- Output as JSON
schedules delete <scheduleId>
Delete a schedule. Requires the full schedule ID for safety.
runhuman schedules delete sched_abc123 --project proj_abc123 --force
Options:
-p, --project <id>- Project ID (required, or use default from config)-f, --force- Skip confirmation prompt-j, --json- Output as JSON
schedules pause <scheduleId>
Pause a schedule (shortcut for update --status paused).
runhuman schedules pause sched_abc123
schedules resume <scheduleId>
Resume a paused schedule (shortcut for update --status active).
runhuman schedules resume sched_abc123
schedules history <scheduleId>
List execution history for a schedule.
runhuman schedules history sched_abc123
runhuman schedules executions sched_abc123 # alias
runhuman schedules history sched_abc123 --limit 50
Options:
-p, --project <id>- Project ID (required, or use default from config)-n, --limit <number>- Number of results (default: 20)--offset <number>- Pagination offset (default: 0)-j, --json- Output as JSON
Shows each execution’s status, linked job ID, scheduled time, and actual execution time. Statuses include completed, skipped_insufficient_tokens, skipped_template_deleted, and failed.
GitHub Integration
Connect GitHub repositories and test issues.
github link <repo>
Link a GitHub repository to your project.
runhuman github link owner/repo
runhuman github link owner/repo --project proj_abc123
Options:
-p, --project <id>- Project ID (or use default from config)
Saves the repository to your project config (.runhumanrc).
github repos
List all GitHub repositories accessible to an organization.
runhuman github repos --organization org_abc123
runhuman github repos --organization org_abc123 --search "my-app"
runhuman github repos --organization org_abc123 --json
Options:
-o, --organization <id>- Organization ID (required)-s, --search <query>- Filter by repository name-j, --json- Output as JSON
github issues <repo>
List issues for a repository.
runhuman github issues owner/repo
runhuman github issues owner/repo --labels bug,needs-testing
runhuman github issues owner/repo --state closed
Options:
-l, --labels <labels>- Filter by comma-separated labels-s, --state <state>- Filter by state: open, closed, all (default: open)-j, --json- Output as JSON
github test <issueNumber>
Create a test job for a specific GitHub issue.
runhuman github test 42 -r owner/repo -u https://google.com
Options:
-r, --repo <owner/repo>- Repository (or use default from config)-u, --url <url>- URL to test (required)-t, --template <name>- Use a template by name-s, --sync- Wait for result
github bulk-test
Test multiple GitHub issues at once.
runhuman github bulk-test -r owner/repo -u https://google.com
runhuman github bulk-test -r owner/repo -u https://google.com -l needs-testing
runhuman github bulk-test -r owner/repo -u https://google.com -n 5
Options:
-r, --repo <owner/repo>- Repository (or use default from config)-u, --url <url>- URL to test (required)-l, --labels <labels>- Filter issues by comma-separated labels-s, --state <state>- Filter by state: open, closed, all (default: open)-t, --template <name>- Template name to use-n, --limit <number>- Maximum number of issues to test (default: 10)
github installation refresh
Force-refresh a GitHub App installation’s repo and permissions list. Use this after granting the Runhuman GitHub App access to new repositories if they aren’t visible to github repos yet — instead of waiting for the periodic resync, this triggers an immediate fetch.
runhuman github installation refresh --installation 987654321 --organization org_abc123
Required options:
--installation <id>- GitHub installation ID
Options:
-o, --organization <id>- Organization ID (defaults to current org context)-j, --json- Output as JSON
Billing
User-scoped reads. Purchase flows and plan changes stay in the web UI — use billing portal to get there.
billing balance
Show credit balance for the authenticated user’s primary organization (override with -o).
runhuman billing balance
runhuman billing balance --organization org_abc123
Options:
-o, --organization <id>- Organization override (defaults to user’s primary org)-j, --json- Output as JSON
billing subscription
Show the active subscription: tier, billing cycle, renewal date, plus any active add-ons.
runhuman billing subscription
Options:
-o, --organization <id>- Organization override-j, --json- Output as JSON
billing portal [--open]
Return a one-time URL to the Polar customer portal (payment method, invoices, cancellation). Pass --open to launch it in your browser.
# Print the URL
runhuman billing portal
# Open it in the browser
runhuman billing portal --open
Options:
-o, --organization <id>- Organization override--open- Open the portal URL in your default browser instead of printing it-j, --json- Output as JSON
Notes
Tester knowledge-base entries — credentials, environment notes, setup instructions, known issues, and other context to surface to testers when they pick up jobs in your project. Notes commands require a paying subscription tier; ungated callers receive a clean “subscription required” error. Alias: note.
notes list
List notes scoped to an organization or project. Internal notes (created with --internal) appear only to admins and testers, never to customer end-users.
runhuman notes list --organization org_abc123
runhuman notes list --organization org_abc123 --project proj_abc123
runhuman notes list --organization org_abc123 --tags credentials,environment
Options:
-o, --organization <id>- Organization ID (required)-p, --project <id>- Scope to a project--tags <tags>- Filter by comma-separated tags-j, --json- Output as JSON
notes create
Create a new note. Provide the body inline, from a file, or pipe it from stdin via --body-file -.
runhuman notes create \
--organization org_abc123 \
--title "Staging credentials" \
--body "Username: tester Password: see 1Password" \
--tags credentials
# Read body from a file (use "-" for stdin)
runhuman notes create -o org_abc123 --title "Setup steps" --body-file ./setup.md --tags setup
Options:
--title <text>- Note title (required)--body <text>- Note body (inline)--body-file <path>- Read body from a file (or-for stdin)--tags <tags>- Comma-separated tags. One of:credentials,environment,setup,procedures,known-issues,contacts,general-o, --organization <id>- Organization ID (required)-p, --project <id>- Optional project scope--internal- Mark as internal (admin / tester visibility only)-j, --json- Output as JSON
notes show <noteId> / notes update <noteId> / notes delete <noteId>
Standard CRUD on a single note. update accepts the same --title, --body, --body-file, --tags, and --internal flags as create. delete accepts -f, --force to skip the confirmation prompt.
Configuration
Manage CLI configuration.
config get <key>
Get a configuration value.
runhuman config get apiUrl
runhuman config get project
config set <key> <value>
Set a configuration value.
runhuman config set project proj_abc123
runhuman config set color false
runhuman config set apiUrl https://runhuman.com
# Set globally vs locally
runhuman config set project proj_abc123 --global
Options:
--global- Set globally (not project-specific)
Common settings:
apiUrl- API base URLproject- Default project IDcolor- Enable/disable colored output (true/false)
config list
List all configuration values.
runhuman config list
Options:
--show-secrets- Show API keys (default: masked)-j, --json- Output as JSON
Shows the full configuration hierarchy: global config and project config with source information.
config reset
Reset configuration to defaults.
# Reset project config only
runhuman config reset --project --force
# Reset global config
runhuman config reset --global --force
# Reset everything
runhuman config reset --all --force
Options:
--project- Reset project config--global- Reset global config--all- Reset all config--force- Required to confirm reset
Project Setup
Initialize and manage project setup.
init
Interactive project initialization wizard.
runhuman init
runhuman init --name "My App" --url https://google.com --yes
Options:
-n, --name <text>- Project name-u, --url <url>- Default URL--github-repo <owner/repo>- GitHub repository--github-repos <repos>- GitHub repositories (comma-separated owner/repo)-y, --yes- Skip all prompts (use defaults)
Creates a .runhumanrc file in your project directory with default settings.
For file-watching auto-test mode, see job watch under Job Management.
Configuration System
The CLI uses a layered configuration hierarchy (highest to lowest priority):
1. CLI Flags (Highest Priority)
runhuman job create --project proj_abc123 https://google.com
2. Environment Variables
export RUNHUMAN_API_KEY="your_key"
export RUNHUMAN_API_URL="https://runhuman.com"
export RUNHUMAN_PROJECT="proj_abc123"
export RUNHUMAN_ORGANIZATION="org_abc123"
export RUNHUMAN_DEFAULT_DEVICE_CLASS="desktop"
export RUNHUMAN_NO_COLOR="true"
Available environment variables:
RUNHUMAN_API_KEY- API authentication keyRUNHUMAN_API_URL- API base URLRUNHUMAN_PROJECT- Default project IDRUNHUMAN_ORGANIZATION- Default organization IDRUNHUMAN_DEFAULT_URL- Default test URLRUNHUMAN_DEFAULT_DURATION- Default test duration in minutesRUNHUMAN_DEFAULT_DEVICE_CLASS- Default device class (desktop,mobile, orboth)RUNHUMAN_NO_COLOR- Disable colored output (set to"true")
3. Project Config
Create .runhumanrc in your project root (or run runhuman init):
{
"defaultUrl": "https://google.com",
"defaultDuration": 5,
"defaultDeviceClass": "desktop",
"githubRepos": ["owner/repo"]
}
4. Global Config
Stored in ~/.config/runhuman/config.json:
{
"apiUrl": "https://runhuman.com",
"project": "proj_abc123",
"color": true
}
Created automatically on first runhuman login.
5. Built-in Defaults (Lowest Priority)
Default values are used when nothing else is configured.
CI/CD Integration
The CLI is designed for seamless CI/CD integration with JSON output mode.
GitHub Actions
name: QA Test
on: [push]
jobs:
qa-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Runhuman CLI
run: npm install -g runhuman
- name: Run Human QA Test
env:
RUNHUMAN_API_KEY: ${{ secrets.RUNHUMAN_API_KEY }}
run: |
runhuman job create ${{ env.PREVIEW_URL }} \
--description "Test main user flow" \
--template "search-test" \
--sync \
--json > result.json
- name: Check Result
run: |
if [ "$(jq -r '.result.passed' result.json)" = "true" ]; then
echo "QA test passed"
exit 0
else
echo "QA test failed"
jq '.result' result.json
exit 1
fi
GitLab CI
qa-test:
stage: test
script:
- npm install -g runhuman
- |
runhuman job create $STAGING_URL \
--description "Search for 'recursion' and confirm the joke appears" \
--sync \
--json > result.json
- |
if [ "$(jq -r '.result.passed' result.json)" = "true" ]; then
exit 0
else
exit 1
fi
CircleCI
version: 2.1
jobs:
qa-test:
docker:
- image: node:18
steps:
- checkout
- run:
name: Install CLI
command: npm install -g runhuman
- run:
name: Run QA Test
command: |
runhuman job create https://google.com \
--template search-test \
--sync \
--json
Jenkins
pipeline {
agent any
stages {
stage('QA Test') {
steps {
sh 'npm install -g runhuman'
sh '''
runhuman job create ${STAGING_URL} \
--description "Test search" \
--sync \
--json > result.json
'''
script {
def result = readJSON file: 'result.json'
if (!result.result.passed) {
error("QA test failed")
}
}
}
}
}
}
Advanced Usage
Parallel Test Execution
Create multiple jobs and wait for all to complete:
#!/bin/bash
# Create jobs
JOB1=$(runhuman job create https://google.com -d "Test search" --json | jq -r '.data.jobId')
JOB2=$(runhuman job create https://google.com -d "Test images" --json | jq -r '.data.jobId')
JOB3=$(runhuman job create https://google.com -d "Test maps" --json | jq -r '.data.jobId')
# Wait for all to complete
runhuman job wait $JOB1 --json &
runhuman job wait $JOB2 --json &
runhuman job wait $JOB3 --json &
wait
# Get results (truncated IDs work here too)
runhuman job results $JOB1
runhuman job results $JOB2
runhuman job results $JOB3
Template-Based Testing
Use templates to avoid repeating the same parameters on every job. Templates can come from three sources, checked in this order: your repo (.runhuman/templates/), project templates, and built-in templates.
# Use a named template (resolved from repo → project → built-in)
runhuman job create https://google.com --template "Find Bugs"
# Use a local template file
runhuman job create --template-file .runhuman/templates/smoke-test.md
# Create a project template via CLI
runhuman templates create "Search Test" \
--project proj_abc123 \
-d "Search for 'recursion' and confirm the joke appears" \
--schema search-test-schema.json
# Use the project template
runhuman job create https://google.com --template "Search Test"
For details on the template format and available sources, see the Templates documentation.
Watch Mode for Local Development
Automatically test on file changes:
# Watch source files, auto-test on changes
runhuman job watch "dist/*.html" \
--url https://google.com \
--template "Search Test" \
--description "Auto search test"
When a file changes, a new test job is created automatically after the debounce period. See job watch for the full flag list.
JSON Output Mode
All commands support the -j, --json flag for machine-readable output. Every JSON response uses the standard envelope (success, data, optional pagination, timestamp, etc.) — see the Global Flags section above for the full shape and the parsing conventions.
Example:
runhuman job create https://google.com -d "Test search" --sync --json
Output (success):
{
"success": true,
"data": {
"jobId": "job_abc123",
"status": "completed",
"result": {
"passed": true,
"explanation": "All tests passed successfully",
"data": {
"searchWorks": true,
"jokeAppears": true
}
},
"costUsd": 0.18,
"testDurationSeconds": 120
},
"timestamp": "2026-04-26T12:34:56.789Z"
}
Exit codes:
0- Success1- General error / ambiguous ID / full ID required2- Authentication error3- Not found / no match for ID prefix4- Validation error5- Timeout error7- Conflict (e.g.,templates createwith a duplicate name and no--force)
Troubleshooting
Authentication Errors
# Error: Unauthorized (401)
# Solution: Login again
runhuman logout
runhuman login
Network Timeouts
# Error: Connection timeout
# Solution: Check API URL and network
runhuman config get apiUrl
Rate Limiting
# Error: Rate limit exceeded (429)
# Solution: Wait and retry, or check your organization balance
runhuman orgs balance org_abc123
Ambiguous ID
# Error: "7" matches multiple jobs
# Solution: Provide more characters
runhuman job status 712e # more specific prefix
Full ID Required
# Error: Full project ID required for deletion
# Solution: Use the full ID shown in the error message
runhuman projects delete <full-id-from-error> --force
Config File Issues
# Reset config if corrupted
runhuman config reset --all --force
# Or manually delete
rm -rf ~/.config/runhuman
Missing Dependencies
# Reinstall CLI
npm uninstall -g runhuman
npm install -g runhuman
# Or use npx (no install needed)
npx runhuman --help
Full Command Reference
Job (10 commands, namespace: runhuman job)
runhuman job create [url]- Create QA testrunhuman job status <jobId>- Check job status (aliases:show,get,info)runhuman job wait <jobId>- Wait for completionrunhuman job results <jobId>- Show results (with optional rich-data sections)runhuman job list [filter]- List jobsrunhuman job watch [patterns...]- Auto-create tests on file changesrunhuman job artifact <jobId> <type>- Download a raw session artifactrunhuman job share <jobId>/unshare <jobId>- Toggle public sharingrunhuman job create-issue <jobId> --index <n>- File an extracted finding as a GitHub issuerunhuman job delete <jobId>- Delete a job (admin-only)
Deprecated flat shims (still work, hidden, emit a warning): runhuman create, runhuman status, runhuman wait, runhuman results, runhuman list, runhuman watch.
Authentication (3 commands)
runhuman login- Login via OAuth (or--tokenfor API-key auth)runhuman logout- Clear stored credentialsrunhuman whoami- Show user, default org/project, and where each default came from
Organizations (10 commands)
runhuman orgs list- List organizationsrunhuman orgs show <id>- Show organization detailsrunhuman orgs balance <id>- Check billing balancerunhuman orgs projects <id>- List organization projectsrunhuman orgs switch <id>- Set default organizationrunhuman orgs members <id>- List membersrunhuman orgs invite <id> <email> [--role admin|contributor|viewer]- Invite a memberrunhuman orgs remove <id> <userId>- Remove a memberrunhuman orgs invitations list|revoke- Manage pending invitationsrunhuman orgs usage [id]- Usage analytics (jobs, cost, time series)
Projects (7 commands)
runhuman projects list- List projectsrunhuman projects create <name>- Create project (requires--organization)runhuman projects show <id>- Show project (includes org details)runhuman projects update <id>- Update project (incl.--add-repos/--remove-repos)runhuman projects switch <id>- Set default projectrunhuman projects transfer <id>- Transfer to another organization (full ID required for--to-org)runhuman projects delete <id>- Delete project (full ID required)
Transfers (4 commands)
runhuman transfers list- List pending transfersrunhuman transfers accept <id>- Accept incoming transferrunhuman transfers reject <id>- Reject incoming transferrunhuman transfers cancel <id>- Cancel outgoing transfer (full ID required)
API Keys (4 commands)
runhuman keys list- List keys (requires--organization)runhuman keys create <name>- Create key (requires--organization)runhuman keys show <keyId>- Show key detailsrunhuman keys delete <keyId>- Delete key (full ID required)
Templates (5 commands)
runhuman templates list- List templates (requires--project, supports filter/sort flags)runhuman templates create <name>- Create template (use--forceto overwrite)runhuman templates show <id>- Show templaterunhuman templates update <id>- Update templaterunhuman templates delete <id>- Delete template
Schedules (8 commands)
runhuman schedules list- List schedules (requires--project)runhuman schedules create <name>- Create recurring schedule (requires--template)runhuman schedules show <id>- Show schedule detailsrunhuman schedules update <id>- Update schedule settingsrunhuman schedules delete <id>- Delete schedule (full ID required)runhuman schedules pause <id>- Pause a schedulerunhuman schedules resume <id>- Resume a paused schedulerunhuman schedules history <id>- List execution history
GitHub (6 commands)
runhuman github link <repo>- Link repositoryrunhuman github repos- List repos (requires--organization)runhuman github issues <repo>- List issuesrunhuman github test <number>- Test a single issue (requires--url)runhuman github bulk-test- Test multiple issues (requires--url)runhuman github installation refresh- Force-refresh a GitHub App installation’s repo/permissions list
Billing (3 commands)
runhuman billing balance- Show credit balancerunhuman billing subscription- Show active subscriptionrunhuman billing portal [--open]- Get the customer portal URL
Notes (5 commands, paying tier)
runhuman notes list- List notes for an org/projectrunhuman notes create- Create a note (--title+--body/--body-file)runhuman notes show <id>- Show a noterunhuman notes update <id>- Update a noterunhuman notes delete <id>- Delete a note
Configuration (4 commands)
runhuman config get <key>- Get config valuerunhuman config set <key> <value>- Set config value (--globalfor global)runhuman config list- List all configrunhuman config reset- Reset config (--project,--global, or--all)
Setup (1 command)
runhuman init- Initialize project (creates.runhumanrc)
Next Steps
| Topic | Link |
|---|---|
| REST API documentation | API Reference |
| GitHub Actions integration | GitHub Actions |
| Practical examples | Cookbook |
| Full technical spec | Reference |
| npm package | npmjs.com/package/runhuman |
| Source code | GitHub |