Lead Data Scientist
Accountable for the modelling standards a data science team works to and for reviewing other data scientists' models before they are used.
4tasks in this job
72skills it uses
7tools named
The skills this job leans on most
- 1Training reproducible machine learning modelsNeeded in 3 of 4 tasks
- 2Evaluating ML generalisation and fitnessNeeded in 2 of 4 tasks
- 3Choosing an analytical method to fit the question and dataNeeded in 2 of 4 tasks
ToolsMLflowPythonscikit-learnJupyter (Notebook / JupyterLab)Git / GitHubR (with RStudio / Posit)Optimizely
All 4 tasks, where people get stuck and all 72 skills
Tasks
- Prepare a model or AI system release decision with evaluation evidence, monitoring and rollback
- Develop and document a fit-for-purpose analytical or ML model
- Evaluate statistical and causal evidence for an intervention's impact
- Set and maintain working standards for a team's analytical, reporting and modelling work products
Where people get stuck
- Sampling and reviewing a team's work against standards
- Designing and running evaluations of AI systems
- Analysing error patterns and failure modes in models and AI systems
All skills
- Choosing an analytical method to fit the question and data
- Evaluating ML generalisation and fitness
- Training reproducible machine learning models
- Analysing error patterns and failure modes in models and AI systems
- Analysing event sequences, states and durations
- Applying data classification and handling rules
- Assessing data source credibility and fitness for a question
- Assessing data-driven fairness and bias
- Assigning and operating data ownership and stewardship
- Bounding and securing AI systems' tool and data access
- Building decision-focused reports and dashboards
- Calculating and interpreting percentages, rates and ratios
- Choosing and justifying missing-data treatments
- Combining tables safely: cardinality, fan-out and conservation of totals
- Communicating sensitive or unwelcome analytical findings
- Comparing groups
- Constructing clear and honest charts
- Controlling disclosure risk in shared data and outputs
- Decomposing one's own tasks for AI assistance
- Defining the analysis population: filters, nulls, dates and time zones
- Describing distributions with summaries and plots
- Designing and interpreting hypothesis tests
- Designing and running evaluations of AI systems
- Designing efficient and reproducible investigations
- Designing implementable change plans
- Diagnosing capability gaps from work evidence and planning development
- Diagnosing incorrect query and calculation results
- Diagnosing misleading data visualisations
- Disclosing AI contribution to professional work
- Eliciting the decision behind an analytical request
- Enforcing and auditing data access
- Escalating unresolved risks with evidence and clear ownership
- Establishing and declaring data grain
- Estimating quantities and differences with intervals
- Evaluating and improving chart accessibility
- Evaluating learning transfer and performance outcomes
- Explaining technical work to accountable non-specialists
- Exploring data to find patterns that survive validation
- Fitting, checking and interpreting inferential regression models
- Formulating answerable data questions
- Gating releases on evidence with rollback ready
- Giving specific developmental feedback on data and AI work
- Governing shared data definitions
- Identifying and prioritising critical data elements
- Independently checking analytical conclusions and claims
- Interpreting averages, weighted measures and mix effects
- Interpreting correlation without inferring cause
- Interpreting trends and seasonality
- Investigating anomalies
- Judging the materiality of defects and deficiencies for the intended use
- Judging whether a change exceeds normal variation
- Judging whether and how to use AI for a task
- Mapping stakeholders' decisions, constraints and needs
- Measuring campaign outcomes with valid denominators
- Monitoring model fitness and drift
- Preparing analysis-ready datasets reproducibly
- Profiling data
- Reading and tracing unfamiliar code
- Reading charts and dashboards correctly, including their state
- Recognising misleading numerical claims
- Reviewing human- and AI-generated code changes for correctness and risk
- Selecting charts for evidence and decisions
- Setting and applying a release and rollback bar for models and AI systems
- Setting and reviewing action thresholds for scores and model predictions
- Standardising and cleaning data under agreed rules
- Testing competing explanations with discriminating evidence
- Testing, implementing and sustaining measured improvements
- Verifying AI-assisted outputs against sources
- Verifying reports and dashboards before and after release
- Writing evidence-led findings and auditable recommendations with their limits
- Writing maintainable code for analytical and data work
- Writing verifiable requirements and acceptance criteria
Paths to start
Skill Path
Exploratory Data Analysis and Pattern Discovery
Describe a distribution honestly and decide which unusual records need checking before anyone acts.
- Modules
- 5 open
- Practice data
- Raket Parts
Tool Path
SQL for Analysis
Answer real business questions with SQL, from your first query to finding why a number moved.
- Modules
- 5 open
- Runs in
- Your browser
Coming soonMachine Learning: from Data to Validated Models and Decisions
6tasks in this job
98skills it uses
8tools named
The skills this job leans on most
- 1Sampling and reviewing a team's work against standardsNeeded in 6 of 6 tasks
- 2Training reproducible machine learning modelsNeeded in 5 of 6 tasks
- 3Designing and running evaluations of AI systemsNeeded in 3 of 6 tasks
- 4Analysing error patterns and failure modes in models and AI systemsNeeded in 3 of 6 tasks
- 5Evaluating ML generalisation and fitnessNeeded in 3 of 6 tasks
ToolsMLflowWeights & BiasesOptimizelyPythonscikit-learnJupyter (Notebook / JupyterLab)+ 2 more
All 6 tasks, where people get stuck and all 98 skills
Tasks
- Monitor a deployed model or AI system in use and recommend retraining, recalibration, change or retirement
- Prepare a model or AI system release decision with evaluation evidence, monitoring and rollback
- Design an ethical decision-relevant experiment
- Develop and document a fit-for-purpose analytical or ML model
- Evaluate statistical and causal evidence for an intervention's impact
- Set and maintain working standards for a team's analytical, reporting and modelling work products
Where people get stuck
- Evaluating ML generalisation and fitness
- Choosing an analytical method to fit the question and data
- Setting and applying a release and rollback bar for models and AI systems
All skills
- Analysing error patterns and failure modes in models and AI systems
- Designing and running evaluations of AI systems
- Evaluating ML generalisation and fitness
- Sampling and reviewing a team's work against standards
- Training reproducible machine learning models
- Analysing event sequences, states and durations
- Applying data classification and handling rules
- Applying research ethics and participant consent
- Assessing data source credibility and fitness for a question
- Assessing data-driven fairness and bias
- Assigning and operating data ownership and stewardship
- Bounding and securing AI systems' tool and data access
- Building data protection by design into data products and pipelines
- Building decision-focused reports and dashboards
- Calculating and interpreting percentages, rates and ratios
- Choosing an analytical method to fit the question and data
- Choosing and justifying missing-data treatments
- Classifying AI uses against regulatory obligations
- Combining tables safely: cardinality, fan-out and conservation of totals
- Communicating quantified uncertainty
- Communicating sensitive or unwelcome analytical findings
- Comparing groups
- Constructing clear and honest charts
- Controlling disclosure risk in shared data and outputs
- Decomposing one's own tasks for AI assistance
- Defining measurable outcomes and leading indicators
- Defining the analysis population: filters, nulls, dates and time zones
- Describing distributions with summaries and plots
- Designing and appraising samples
- Designing and interpreting hypothesis tests
- Designing and quality-checking survey instruments
- Designing decision rights and dispute routes for data and AI work
- Designing decision-oriented research with customers, users or employees
- Designing efficient and reproducible investigations
- Designing enforceable data governance policies
- Designing experiments
- Designing implementable change plans
- Diagnosing capability gaps from work evidence and planning development
- Diagnosing incorrect query and calculation results
- Diagnosing misleading data visualisations
- Disclosing AI contribution to professional work
- Eliciting the decision behind an analytical request
- Enforcing and auditing data access
- Escalating unresolved risks with evidence and clear ownership
- Establishing and declaring data grain
- Estimating and challenging causal effects
- Estimating quantities and differences with intervals
- Evaluating and improving chart accessibility
- Evaluating employee experience evidence responsibly
- Evaluating experimental results and integrity
- Evaluating learning transfer and performance outcomes
- Evaluating whether people understood, adopted and sustained a change
- Explaining technical work to accountable non-specialists
- Exploring data to find patterns that survive validation
- Fitting, checking and interpreting inferential regression models
- Formulating answerable data questions
- Gating releases on evidence with rollback ready
- Giving specific developmental feedback on data and AI work
- Governing shared data definitions
- Identifying and prioritising critical data elements
- Independently checking analytical conclusions and claims
- Influencing change without formal authority
- Interpreting averages, weighted measures and mix effects
- Interpreting correlation without inferring cause
- Interpreting customer feedback and qualitative evidence
- Interpreting trends and seasonality
- Investigating anomalies
- Judging the materiality of defects and deficiencies for the intended use
- Judging whether a change exceeds normal variation
- Judging whether and how to use AI for a task
- Judging whether uncertainty changes the decision
- Mapping stakeholders' decisions, constraints and needs
- Measuring campaign outcomes with valid denominators
- Monitoring model fitness and drift
- Preparing analysis-ready datasets reproducibly
- Presenting and defending evidence-based recommendations to decision-makers
- Profiling data
- Reading and tracing unfamiliar code
- Reading charts and dashboards correctly, including their state
- Recognising and resolving ethical conflicts in data and AI work
- Recognising misleading numerical claims
- Reviewing human- and AI-generated code changes for correctness and risk
- Selecting charts for evidence and decisions
- Setting and applying a release and rollback bar for models and AI systems
- Setting and reviewing action thresholds for scores and model predictions
- Standardising and cleaning data under agreed rules
- Statistical power and sample size
- Testing competing explanations with discriminating evidence
- Testing, implementing and sustaining measured improvements
- Verifying AI-assisted outputs against sources
- Verifying benefits realisation and attribution
- Verifying reports and dashboards before and after release
- Weighing conflicting evidence by design strength
- Working through stakeholder conflict over evidence and responsibility
- Writing evidence-led findings and auditable recommendations with their limits
- Writing maintainable code for analytical and data work
- Writing testable work standards and review checklists
- Writing verifiable requirements and acceptance criteria
All tools
- MLflow
- Weights & Biases
- Optimizely
- Python
- scikit-learn
- Jupyter (Notebook / JupyterLab)
- R (with RStudio / Posit)
- Git / GitHub
Paths to start
Skill Path
Exploratory Data Analysis and Pattern Discovery
Describe a distribution honestly and decide which unusual records need checking before anyone acts.
- Modules
- 5 open
- Practice data
- Raket Parts
Tool Path
SQL for Analysis
Answer real business questions with SQL, from your first query to finding why a number moved.
- Modules
- 5 open
- Runs in
- Your browser
Skill Path
Leading Analytics and AI Change
Lead adoption, diagnose learning needs, coach and assure team capability.
- Modules
- 5 open
- Practice data
- Raket Parts
Coming soonMachine Learning: from Data to Validated Models and Decisions
6tasks in this job
101skills it uses
9tools named
The skills this job leans on most
- 1Sampling and reviewing a team's work against standardsNeeded in 6 of 6 tasks
- 2Designing and running evaluations of AI systemsNeeded in 4 of 6 tasks
- 3Analysing error patterns and failure modes in models and AI systemsNeeded in 4 of 6 tasks
- 4Training reproducible machine learning modelsNeeded in 4 of 6 tasks
- 5Evaluating ML generalisation and fitnessNeeded in 3 of 6 tasks
ToolsMicrosoft Power BIMicrosoft PowerPointMLflowWeights & Biasesscikit-learnPython+ 3 more
All 6 tasks, where people get stuck and all 101 skills
Tasks
- Monitor a deployed model or AI system in use and recommend retraining, recalibration, change or retirement
- Evaluate an AI use case for suitability, value and risk before adoption
- Prepare a model or AI system release decision with evaluation evidence, monitoring and rollback
- Develop and document a fit-for-purpose analytical or ML model
- Evaluate statistical and causal evidence for an intervention's impact
- Set and maintain working standards for a team's analytical, reporting and modelling work products
Where people get stuck
- Evaluating ML generalisation and fitness
- Choosing an analytical method to fit the question and data
- Setting and applying a release and rollback bar for models and AI systems
All skills
- Analysing error patterns and failure modes in models and AI systems
- Designing and running evaluations of AI systems
- Evaluating ML generalisation and fitness
- Sampling and reviewing a team's work against standards
- Training reproducible machine learning models
- Analysing event sequences, states and durations
- Applying data classification and handling rules
- Applying research ethics and participant consent
- Assessing data source credibility and fitness for a question
- Assessing data-driven fairness and bias
- Assigning and operating data ownership and stewardship
- Bounding and securing AI systems' tool and data access
- Building data protection by design into data products and pipelines
- Building decision-focused reports and dashboards
- Calculating and interpreting percentages, rates and ratios
- Choosing an analytical method to fit the question and data
- Choosing and justifying missing-data treatments
- Classifying AI uses against regulatory obligations
- Combining tables safely: cardinality, fan-out and conservation of totals
- Communicating quantified uncertainty
- Communicating sensitive or unwelcome analytical findings
- Comparing groups
- Constructing clear and honest charts
- Controlling disclosure risk in shared data and outputs
- Decomposing one's own tasks for AI assistance
- Defining measurable outcomes and leading indicators
- Defining the analysis population: filters, nulls, dates and time zones
- Describing distributions with summaries and plots
- Designing and appraising samples
- Designing and interpreting hypothesis tests
- Designing and quality-checking survey instruments
- Designing decision rights and dispute routes for data and AI work
- Designing decision-oriented research with customers, users or employees
- Designing efficient and reproducible investigations
- Designing enforceable data governance policies
- Designing experiments
- Designing implementable change plans
- Diagnosing capability gaps from work evidence and planning development
- Diagnosing incorrect query and calculation results
- Diagnosing misleading data visualisations
- Disclosing AI contribution to professional work
- Eliciting the decision behind an analytical request
- Enforcing and auditing data access
- Escalating unresolved risks with evidence and clear ownership
- Establishing and declaring data grain
- Estimating and challenging causal effects
- Estimating quantities and differences with intervals
- Evaluating and improving chart accessibility
- Evaluating employee experience evidence responsibly
- Evaluating experimental results and integrity
- Evaluating learning transfer and performance outcomes
- Evaluating whether people understood, adopted and sustained a change
- Explaining control types, dependencies and compensating controls
- Explaining technical work to accountable non-specialists
- Exploring data to find patterns that survive validation
- Fitting, checking and interpreting inferential regression models
- Formulating answerable data questions
- Gating releases on evidence with rollback ready
- Giving specific developmental feedback on data and AI work
- Governing shared data definitions
- Identifying and prioritising critical data elements
- Independently checking analytical conclusions and claims
- Influencing change without formal authority
- Interpreting averages, weighted measures and mix effects
- Interpreting correlation without inferring cause
- Interpreting customer feedback and qualitative evidence
- Interpreting trends and seasonality
- Investigating anomalies
- Judging the materiality of defects and deficiencies for the intended use
- Judging whether a change exceeds normal variation
- Judging whether and how to use AI for a task
- Judging whether uncertainty changes the decision
- Mapping stakeholders' decisions, constraints and needs
- Measuring campaign outcomes with valid denominators
- Monitoring model fitness and drift
- Preparing analysis-ready datasets reproducibly
- Presenting and defending evidence-based recommendations to decision-makers
- Profiling data
- Reading and tracing unfamiliar code
- Reading charts and dashboards correctly, including their state
- Recognising and resolving ethical conflicts in data and AI work
- Recognising cybersecurity threats to data work
- Recognising misleading numerical claims
- Recognising personal data and privacy duties
- Reviewing human- and AI-generated code changes for correctness and risk
- Selecting charts for evidence and decisions
- Setting and applying a release and rollback bar for models and AI systems
- Setting and reviewing action thresholds for scores and model predictions
- Standardising and cleaning data under agreed rules
- Statistical power and sample size
- Testing competing explanations with discriminating evidence
- Testing, implementing and sustaining measured improvements
- Verifying AI-assisted outputs against sources
- Verifying benefits realisation and attribution
- Verifying reports and dashboards before and after release
- Weighing conflicting evidence by design strength
- Working through stakeholder conflict over evidence and responsibility
- Writing evidence-led findings and auditable recommendations with their limits
- Writing maintainable code for analytical and data work
- Writing testable work standards and review checklists
- Writing verifiable requirements and acceptance criteria
All tools
- Microsoft Power BI
- Microsoft PowerPoint
- MLflow
- Weights & Biases
- scikit-learn
- Python
- Git / GitHub
- R (with RStudio / Posit)
- Optimizely
Paths to start
Skill Path
Exploratory Data Analysis and Pattern Discovery
Describe a distribution honestly and decide which unusual records need checking before anyone acts.
- Modules
- 5 open
- Practice data
- Raket Parts
Tool Path
SQL for Analysis
Answer real business questions with SQL, from your first query to finding why a number moved.
- Modules
- 5 open
- Runs in
- Your browser
Skill Path
Leading Analytics and AI Change
Lead adoption, diagnose learning needs, coach and assure team capability.
- Modules
- 5 open
- Practice data
- Raket Parts
Coming soonMachine Learning: from Data to Validated Models and Decisions
About the role
Accountable for the modelling standards a data science team works to and for reviewing other data scientists' models before they are used. Owns validation and documentation conventions across several projects and challenges misuse of models with stakeholders. Leads this work without line management, where a Data Scientist owns the models for one problem.