No description
| .env.example | ||
| .envrc | ||
| .gitignore | ||
| add_task_estimates_to_samples.py | ||
| analysis.ipynb | ||
| create_onet_database.sh | ||
| data_enrichment.ipynb | ||
| enrich_task_ratings.py | ||
| evaluate_llm_time_estimations.ipynb | ||
| flake.lock | ||
| flake.nix | ||
| legacy.ipynb | ||
| pyproject.toml | ||
| README.md | ||
| uv.lock | ||
Presentation
Notebooks
- data enrichment - contains the code to gather things from the O*NET data, BLS's OEWS database (unused for now), Barnett's data...
- prompt evaluation - the playground used to evaluate change in hyperparameters (system prompt, user prompt, schema, model...)
- analysis - code to generate the graphs in the paper
- legacy - if there are some missing pieces, it's worth looking in there.
Running the non-notebook code
To re-run everything, you need python and uv up and running, if you use have nix installed, run
nix develop .#impure
and then uv run ... as requested in the notebooks.
If some things are missing, email dorn@xfe.li, I'm usually reactive.
Copy .env.example to .env and fill in OPENAI_API_KEY. The total run and experiments cost less than <10$.