Explore Tasks
Develop and evaluate an LLM-based application against an agreed evaluation bar
The owner receives an LLM-based application (prompted, retrieval-augmented or fine-tuned) with an evaluation set, grounding and safety results against the agreed bar, data minimisation and known failure modes.
2jobs do it
3jobs help with it
40skills it needs
Does itHelps with it
- Reading and tracing unfamiliar code
- Reviewing human- and AI-generated code changes for correctness and risk
- Sampling and reviewing a team's work against standards
- Diagnosing misleading data visualisations
- Verifying reports and dashboards before and after release
- Verifying AI-assisted outputs against sources
- Disclosing AI contribution to professional work
- Testing competing explanations with discriminating evidence
- Independently checking analytical conclusions and claims
- Escalating unresolved risks with evidence and clear ownership
- Giving specific developmental feedback on data and AI work
- Judging the materiality of defects and deficiencies for the intended use
- Designing AI-assisted workflows with bounded tools and human authority
- Bounding and securing AI systems' tool and data access
- Designing and running evaluations of AI systems
- Classifying AI uses against regulatory obligations
- Presenting and defending evidence-based recommendations to decision-makers
- Decomposing one's own tasks for AI assistance
- Protecting sensitive data in AI-assisted work
- Judging whether and how to use AI for a task
- Evaluating fitness and full cost of AI-assisted work
- Analysing error patterns and failure modes in models and AI systems
- Evaluating responsible AI use risks
- Assessing privacy risk through data protection impact assessments
- Assessing supplier dependency, data-handling and exit risk
- Rating risks against appetite and choosing proportionate treatment
- Understanding and interrogating data and application APIs
- Writing maintainable code for analytical and data work
- Building bounded, secure data-service APIs
- Designing and evaluating retrieval for AI systems
- Writing evidence-led findings and auditable recommendations with their limits
- Specifying and iterating AI task instructions
- Building data protection by design into data products and pipelines
- Gating releases on evidence with rollback ready
- Setting and reviewing action thresholds for scores and model predictions
- Training reproducible machine learning models
- Evaluating ML generalisation and fitness
- Monitoring model fitness and drift
- Assessing data-driven fairness and bias
- Setting and applying a release and rollback bar for models and AI systems
Paths that teach the skills it needs
Skill Path
Data Literacy for Business Decisions
Judge whether a business claim from a table, KPI, chart or dashboard really holds up.
- Modules
- 5 open
- Practice data
- Raket Parts
Skill Path
Responsible AI Governance and Assurance
Govern and assure AI across its lifecycle including third-party AI and redress.
- Modules
- 5 open
- Practice data
- Raket Parts
Coming soonApplied AI at WorkMachine Learning: from Data to Validated Models and DecisionsDesigning Controlled Automations and AI-assisted Workflows