HumanEval · Verified Data

HumanEval
Code Generation Benchmark

Official original HumanEval.jsonl.gz, project README, MIT license text, and SHA-256 manifest. Please use an isolated environment before running generated code.

164 Evaluation TasksOriginal GZIP DataMIT License
164
Verified Records
1
Verified File or Split
0.05 MB
Delivered ZIP Size
ZIP
Private Download After Purchase

Real Data Preview

164 Python programming tasks. The chart below is based on actual delivered file statistics, not sample questions; the complete data is available for download after purchase.

Verified group counts for HumanEval data files

How Code Tasks Are Evaluated

The entry function in this original record is has_close_elements. Use the prompt to generate the function, then use the test in the same record to check the output; canonical_solution is the reference implementation and should not be used as input to the model under test. Place model-generated code in an isolated sandbox before execution.

Original Package Sample · test · 164 Records
{
  "task_id": "HumanEval/0",
  "entry_point": "has_close_elements",
  "prompt": "from typing import List\n\n\ndef has_close_elements(numbers: List[float], threshold: float) -> bool:\n    \"\"\" Check if in given list of numbers, are any two numbers closer to each other than\n    given threshold.\n    >>> has_close_elements([1.0, 2.0, 3.0], 0.5)\n    False\n    >>> has_close_elements([1.0, 2.8, 3.0, 4.0, 5.0, 2.0], 0.3)\n    True\n    \"\"\""
}

What's Included

Official original HumanEval.jsonl.gz, project README, MIT license text, and SHA-256 manifest. Please use an isolated environment before running generated code.

01

Official test split only

Official test split only; does not include a model runtime environment or automated evaluation service.

02

Each record includes task_id, prom

Each record includes task_id, prompt, entry_point, canonical_solution, and test.

03

Original package and attribution

Includes original files, source information, license text or publisher license statement, and SHA-256 manifest.

Delivery and Source

After purchase, obtain a private ZIP download link valid for 600 seconds through the platform's Dataset app. Anonymous visitors cannot access the original files.

Files and Verification

Filename: humaneval-reviewed-attributed.zip
Size: 48,948 bytes
SHA-256: 801fb5e51f7b2e330ad7a6196346749e06558a00f31bdca1a19e51bf56417fb4

License and Attribution

MIT. Please comply with the attribution, reposting, and applicable licensing terms specified by the original publisher.
View the original license text or publisher statement

Source Data

View the original release page. The public preview displays only one verified sample from the original package and aggregate statistics; the complete data must be downloaded after purchase.

View the HumanEval Dataset

Review the delivery scope and licensing terms, then select the standard-price plan; private file downloads can only be requested after purchase is complete.