Sample library
Open the task. Inspect the work.
Three original synthetic demonstrations, with task instructions, workspace files, reference outputs, and evaluation criteria you can explore and download.
Data Samples
Explore the Data
A task is more than a prompt. Explore the files, expected artifacts, and evaluation criteria that make a model’s work measurable.
Audit a legacy data pipeline
A small analytics pipeline has lost its owner. Inspect the four-file workspace and produce a machine-readable audit without running the scheduled jobs.
The assignment
- 01
Inventory every provided file using an absolute /workspace path.
- 02
List existing file-to-file dependencies and raw-to-derived table lineage.
- 03
Report the missing scheduled job with a file-and-line evidence reference.
- 04
Submit audit.json with exactly inventory, dependency_edges, table_lineage, and findings. Array order does not matter.
Start with the actual inputs.
Select a file to inspect the workspace and its output contract.
Workspace files
0 2 * * * python3 /workspace/jobs/load_orders.py30 2 * * * python3 /workspace/jobs/send_digest.py Work you can inspect.
Reference artifacts show what a successful submission contains.
Reference outputs
{ "inventory": [ "/workspace/config/warehouse.json", "/workspace/crontab", "/workspace/jobs/load_orders.py", "/workspace/schema.json" ], "dependency_edges": [ { "from": "/workspace/crontab", "to": "/workspace/jobs/load_orders.py" }, { "from": "/workspace/jobs/load_orders.py", "to": "/workspace/config/warehouse.json" } ], "table_lineage": [ { "source": "raw_orders", "target": "daily_orders", "operation": "INSERT" } ], "findings": [ { "type": "missing_scheduled_job", "path": "/workspace/jobs/send_digest.py", "evidence": "/workspace/crontab:2" } ]} Evaluation contract
Define success before the run.
Complete inventory
20 ptsAll four files appear once, using the required absolute paths.
Dependency graph
25 ptsBoth existing-file edges are present; the missing job is not treated as an existing node.
Table lineage
20 ptsraw_orders → daily_orders is correctly identified as an INSERT operation.
Evidence-backed finding
25 ptsThe missing send_digest.py job is tied to line 2 of the crontab.
Output contract
10 ptsThe JSON contains exactly the four specified top-level keys.
Deterministic validator included. Run it locally against a submitted audit; this preview shows the reference artifact, not a model run.
python3 grader.py reference/audit.jsonLOCAL CHECKThese synthetic demonstrations use fictional inputs and reference outputs. They do not represent customer projects or measured model performance.
Start a conversation