Documentation
Welcome to run(human) - The simplest way to add human verification to your automated testing workflow.
run(human) is a human-in-the-loop QA testing API that lets you call a simple endpoint and get back real human test results. Perfect for when automation isn't enough and you need actual human judgment.
What is run(human)?
run(human) connects your application to a pool of real human testers who can verify functionality that's difficult or impossible to automate:
- Visual verification - Does this layout look correct?
- User experience - Is this form easy to use?
- Subjective judgment - Does this content make sense?
- Complex workflows - Can a user complete this multi-step process?
How It Works
- You call our API with a URL, task description, and output schema
- A human tester receives the job, performs the test, and describes what they found
- AI extracts structured data from their response and returns it to you
Key Features
- Synchronous API - Simple request/response, just like any other API
- Custom Schemas - Define exactly what data you need back
- Natural Language - Testers describe in plain English, AI extracts structured JSON
- Fast Turnaround - Average response time under 2 minutes
- Flexible Pricing - Pay only for execution time ($0.0018 per second)
Use Cases
- Pre-deployment smoke tests in CI/CD pipelines
- Verifying visual changes after CSS updates
- Testing complex authentication flows
- Validating form submissions and error messages
- Checking cross-browser compatibility
- Verifying accessibility features
Quick Start
Get started with run(human) in under 5 minutes.
1. Get Your API Key
First, sign up and create an API key from your dashboard:
- Go to https://runhuman.com/signin
- Sign in with your email
- Navigate to the "API Keys" section
- Click "Create New API Key" and give it a name
- Copy your API key (shown only once!)
Your API key is sensitive! Store it securely in environment variables, never commit it to version control.
2. Make Your First API Call
Here's a simple example testing a button click:
const response = await fetch('https://runhuman.com/api/run', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${process.env.API_KEY}`
},
body: JSON.stringify({
url: 'https://example.com',
description: 'Click the "Get Started" button and verify it works',
outputSchema: {
buttonWorks: {
type: 'boolean',
description: 'Did the button click work successfully?'
},
errorMessage: {
type: 'string',
description: 'Any error message that appeared (or "None")'
}
}
})
});
const result = await response.json();
console.log(result);
// { status: 'completed', result: { buttonWorks: true, errorMessage: 'None' } }
3. Understand the Response
The API returns a JSON object with the test results:
{
"status": "completed",
"result": {
"buttonWorks": true,
"errorMessage": "None"
},
"costUsd": 0.0036,
"testDurationSeconds": 2
}
- costUsd: Exact cost in USD with full precision (duration × $0.0018/second)
- testDurationSeconds: Time spent by tester (rounded up using Math.ceil)
- Costs are never $0 unless the tester never actually worked on it
- Any partial second counts as a full second
You've just run your first human-verified test. The tester clicked the button, described what happened, and our AI extracted the structured data you requested.
Next Steps
Authentication
All API requests require authentication using API keys.
API Keys
run(human) uses API keys to authenticate requests. Include your API key in the Authorization
header:
Authorization: Bearer YOUR_API_KEY
Creating API Keys
- Sign in to your dashboard
- Navigate to "API Keys"
- Click "Create New API Key"
- Give your key a descriptive name (e.g., "Production", "Staging")
- Copy the key immediately (it's only shown once)
- Never commit API keys to version control
- Store keys in environment variables or secrets management
- Use different keys for development, staging, and production
- Rotate keys regularly
- Revoke compromised keys immediately
Managing API Keys
You can manage your API keys from the dashboard:
- View keys - See all your API keys and when they were last used
- Revoke keys - Disable a key immediately if compromised
- Delete keys - Permanently remove unused keys
Example with Environment Variables
Store your API key in an environment variable:
# .env file
RUNHUMAN_API_KEY=your_api_key_here
Then use it in your code:
const API_KEY = process.env.RUNHUMAN_API_KEY;
const response = await fetch('https://runhuman.com/api/run', {
headers: {
'Authorization': `Bearer ${API_KEY}`
},
// ...
});
Core Concepts
Understanding the key concepts of run(human).
Jobs
A job is a single test request. When you call the API, you create a job that gets assigned to a human tester.
Job Lifecycle
- pending - Job created, waiting for a tester to claim it
- in_progress - A tester has claimed the job and is working on it
- completed - Tester finished, AI extracted the data successfully
- failed - Job failed due to an error
- expired - Job wasn't completed in time (10 minute timeout)
Output Schemas
An output schema defines what data you want extracted from the tester's response. It's a JSON object where each field has:
type
- The data type (string, boolean, number)description
- What you want the tester to check
{
"buttonWorks": {
"type": "boolean",
"description": "Did the button click work?"
},
"loadTime": {
"type": "string",
"description": "How long did the page take to load?"
}
}
Natural Language Processing
Testers don't provide JSON - they describe what they see in natural language:
Tester's response: "I clicked the button and it worked great! The page loaded in about 2 seconds. Everything looks good. done"
Extracted data:
{
"buttonWorks": true,
"loadTime": "about 2 seconds"
}
The "done" Keyword
Testers must end their final message with the word "done" to indicate they're finished. This prevents premature completion if the tester has more information to share.
The AI will not mark a job as complete unless the tester's message ends with "done", even if all required data is provided. This ensures the tester has finished their full assessment.
Pricing Model
You're charged based on execution time - the time from when a tester claims the job until they finish:
- Rate: $0.0018 per second of execution time
- No charge for time spent waiting for a tester to claim the job
- Example: A 30-second test costs $0.054
API Reference
Complete reference for all run(human) API endpoints.
Base URL
https://runhuman.com/api
Request Format
All requests must:
- Use HTTPS
- Include
Content-Type: application/json
- Include
Authorization: Bearer YOUR_API_KEY
- Send data as JSON in the request body
Response Format
All responses are JSON with this structure:
{
"status": "completed" | "failed" | "pending" | "in_progress",
"result": { /* extracted data */ },
"costUsd": 0.0036, // Exact cost (only when completed)
"testDurationSeconds": 2, // Duration in seconds (only when completed)
"error": "error message" | null
}
HTTP Status Codes
Code | Meaning | Description |
---|---|---|
200 |
OK | Request successful |
201 |
Created | Resource created successfully |
400 |
Bad Request | Invalid request parameters |
401 |
Unauthorized | Missing or invalid API key |
404 |
Not Found | Resource not found |
429 |
Too Many Requests | Rate limit exceeded |
500 |
Server Error | Internal server error |
POST /api/run
Create a test job and wait for completion (synchronous).
This is the recommended endpoint for most use cases. It's a simple synchronous API that waits for the test to complete and returns the result.
Creates a test job and waits for it to complete (up to 10 minutes).
Request
Headers
Header | Value |
---|---|
Content-Type |
application/json |
Authorization |
Bearer YOUR_API_KEY |
Body Parameters
Parameter | Type | Required | Description |
---|---|---|---|
url | string | Required | The URL to test |
description | string | Required | What you want the tester to do |
outputSchema | object | Required | Schema defining the data to extract |
Example Request
const response = await fetch('https://runhuman.com/api/run', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${API_KEY}`
},
body: JSON.stringify({
url: 'https://example.com/login',
description: 'Test login with email: test@example.com, password: demo123',
outputSchema: {
loginSuccessful: {
type: 'boolean',
description: 'Did the login succeed?'
},
errorMessage: {
type: 'string',
description: 'Any error message shown (or "None")'
},
redirectedToDashboard: {
type: 'boolean',
description: 'Was user redirected to dashboard after login?'
}
}
})
});
const result = await response.json();
Example Response
{
"status": "completed",
"result": {
"loginSuccessful": true,
"errorMessage": "None",
"redirectedToDashboard": true
},
"costUsd": 0.0054,
"testDurationSeconds": 3,
"error": null
}
Timeout
The request will wait up to 10 minutes for completion. If the job doesn't complete in time, you'll receive a timeout response:
{
"status": "expired",
"result": null,
"error": "Job timed out after 10 minutes"
}
Pricing
Simple, transparent pricing based on actual usage.
Pay Per Second
You're only charged for the time a human tester spends actively working on your job - from the moment they claim it until they say "done".
What You Pay For
- ✅ Execution time - Time from claim to completion
- ❌ Not waiting time - Time before a tester claims the job
- ❌ Not failed jobs - No charge if the job fails
Example Costs
Test Duration | Cost |
---|---|
30 seconds | $0.054 |
1 minute | $0.108 |
2 minutes | $0.216 |
5 minutes | $0.540 |
Volume Discounts
Contact us at hey@runhuman.com for enterprise pricing and volume discounts.
Support
We're here to help!
Get Help
- Email: hey@runhuman.com
- GitHub: Open an issue
- Status: Check system status
Response Times
- Critical issues: Within 1 hour
- General support: Within 24 hours
- Feature requests: Within 1 week