Apertus Engineer: Evaluations — ETH Zürich
NewCHF 73'500 - 111'500
ETH Zürich · Zurich (ZH)
- Location
- Zurich
- Contract
- fixed-term
- Posted
- Yesterday
SalaryCHF 73'500 - 111'500
Role overview
Apertus Engineer: Evaluations
100%, Zurich, fixed-term print Drucken
We are seeking a skilled engineer to join the Apertus evaluation effort.
- Apertus Engineer: Evaluations
- 100%, Zurich, fixed-term print Drucken
Additional details
- 100%, Zurich, fixed-term
- This role requires strong Python engineering, hands-on LLM evaluation experience, and the ability to work collaboratively in a research-focused environment. Project background
- The role requires someone who is comfortable working in an HPC environment and collaborating with researchers and infrastructure engineers.
- The engineer will own the evaluation codebase and pipelines that inform training decisions and releases. Evaluation infrastructure
- Debug evaluation failures, regressions, and inconsistencies across backends Benchmark coverage
- Integrate new benchmarks as the field evolves, working with our academic collaborators to onboard the benchmarks they create, and validate that metrics and harness implementations are trustworthy Comparative and third-party evaluation
- Work closely with the engineers focused on safety, deployment, and community needs, and integrate the evaluations they create into the shared pipeline
- MSc or PhD in Computer Science, Data Science, Artificial Intelligence, Machine Learning, or a related field.
Notes and original content
- This role requires strong Python engineering, hands-on LLM evaluation experience, and the ability to work collaboratively in a research-focused environment.
- Project background
- Job description
- The engineer will own the evaluation codebase and pipelines that inform training decisions and releases.
- Evaluation infrastructure
- Debug evaluation failures, regressions, and inconsistencies across backends
- Benchmark coverage
- Integrate new benchmarks as the field evolves, working with our academic collaborators to onboard the benchmarks they create, and validate that metrics and harness implementations are trustworthy
- Comparative and third-party evaluation
- Exceptional
