Example · Data Scientist
58% of your week as a data scientist is still yours.
AI now fits and tunes standard models. Judging whether a result holds up stays with you.
Make my planA typical week at the middle pace. Your plan uses your own tasks.
Is data science a dying field?
In a typical week at the middle pace, 38% of this work needs your judgment and 20% is work you do using AI. AI could do about 42%. Fitting and tuning standard models got cheap this year. AI agents now win medals in most Kaggle-style contests, and data platforms ship agents that profile, split, tune and report a model from one sentence.
Read more
That is the part of the job most people were hired for, and it is the most exposed. About 43% of your week sits there.
What holds is the work that fails without anyone noticing: experiment design, causal claims, model validation, and the newer job of proving an AI feature is good enough to ship. Those are expensive to check and depend on facts nobody wrote down. The plan moves you upstream, from reading tests after launch to reviewing them before, and builds a record of what the tools got wrong.
The week, task by task
Who does each hour of a typical 40-hour week.
See every task
Needs your judgment.
- Model validation and monitoring: leakage, backtests, drift, does it still predict8%
- Experiment design: unit of randomization, power, metrics, guardrails, when to stop7%
- Causal questions from messy data: did the launch cause it, what confounds it7%
- Scoping the ask with the person who has the decision, or declining it6%
- + 2 smaller tasks8%
AI does the slow parts. You still own it.
- A/B test readouts: lift, variance reduction, significance, the summary8%
- Evaluating AI and LLM features: test sets, rubrics, judge agreement, regressions7%
- Reconciling numbers that disagree; knowing which table and definition to trust6%
AI could do most of it.
- Pulling and preparing data for a clear question: SQL, joins, cleaning, feature tables12%
- Fitting, tuning and comparing standard models (the AutoML part of the job)12%
- Writing analysis code, notebooks, functions and refactors9%
- Charts, standard reports, write-ups and first-draft slides7%
- + 1 smaller task3%
Why some tasks still need a person
Designing experiments: randomization unit, power, guardrails, stopping rules
This is hard to verify. A bad design is discovered weeks later, after the launch decision, and the cost is a wrong product call nobody reverses.
Answering causal questions from observational data
This rests on context. Whether a launch caused the lift depends on how the data was generated: a promo nobody logged, a region that rolled out early. Those facts live in people's heads.
Validating and monitoring models for leakage and drift
This is hard to verify. A leaked feature looks like a great model. Catching it means knowing which columns could not exist at prediction time.
Evaluating AI and LLM (large language model) features
This is hard to verify. A model can grade outputs, but someone has to show the grader agrees with people and that the test set looks like real traffic.
Reconciling numbers that disagree
This depends on context. Nobody has written down which table finance trusts or why the other one double-counts.
Good to know
What is hard about this field, and what you can do about it.
The hard truth
Most of the models you build may never ship.
A lot of data science work ends at the proof of concept. Companies scrap close to half of AI pilots before production, and more are abandoning most of their AI projects.
Sources (2)
- 42% of companies abandoned most of their AI initiatives in 2025, up from 17% in 2024. S&P Global Market Intelligence, via CIO Dive, 2025-03
- The average organization scrapped 46% of AI proofs of concept before they reached production. S&P Global Market Intelligence, via CIO Dive, 2025-03
What's next after data scientist?
Plan A is the best move inside the work you do now.
Grow in the job you have
The main focus. Most weekly sessions build toward it.
- Each week
- 4 sessions
- Ready in
- About 3 months of reviews on real tests
Why it fits
Why you. You already run readouts and you already see the bad designs. Being asked too late is the problem, so move the moment you are asked.
The gap. You need a habit of writing test plans in advance, checks for uneven traffic splits and early peeking, and the words to say no early.
The trade-off. You will build fewer models and your output will look smaller on any dashboard of shipped work. Some managers reward volume.
2 other moves that fit
- Move into AI and LLM evaluationReady in about 3 to 6 months alongside your job
- Move into regulated modeling in banking, insurance or health (the hedge)Ready in about 6 to 12 months
What's a plan B for data scientists?
Plan B is meant to be different from this job. It was picked because skills from this role carry over.
Get ready for a new kind of job
A second option. A newer job AI is creating that these skills already fit.
You already put confidence intervals on noisy metrics and distrust a number until you know how it was made, which is the core of judging an AI grader.
- Each week
- 1 session
- Outlook
- Hiring now
- Pay
- $150.4K to $385K a yearTwo postings: Apple (Oct 2026, base pay) and Perplexity (now closed).
2 other options
- AI evaluation specialistYour habit of building holdout sets and checking a model against them carries straight into writing test sets and rubrics for AI systems.
- AI value and ROI analystYou design experiments and measure lift, which is exactly how a company should judge whether an AI project paid off.
In your own plan you can switch to any of the 47 roles.
The 13 weeks
Small steps that add up. One plan for both: most sessions build Plan A, one a week builds Plan B.
By week 13 you will have
AFor Plan A
- A catch log of agent and test errors
- A one-page memo asking to own pre-launch review
- A published write-up of your AI testing
BAnd for Plan B
- A regression table with a significance check
- A published eval write-up for your portfolio
- 1Measure your week
- 2Check an AI agent's work
- 3Practice spotting leakage
- 4Review past experiments
You finish withA one-page summary of common test design mistakes
- 5Plan tests in advance
- 6Do your first test review
- 7Test a causal claim
- 8Monitor a live model
You finish withA one-line rule for when to review the model
- 9Test your company's AI feature
- 10Check your track record
- 11Offer review hours
- 12Write and send your memo
- 13Build proof you can take with you
You finish withOne application for an experimentation or AI testing role
Three courses from the plan
- Occupational Outlook Handbook: Data ScientistsBLS · Free
Yes, once. The growth projection is the counterweight to the layoff headlines.
- O*NET 15-2051.00 Data ScientistsO*NET OnLine · Free
Optional. Useful for checking which of your tasks the official description still lists.
- The Effect: An Introduction to Research Design and CausalityNick Huntington-Klein · Free online
Yes, Weeks 5 and 7. It is the best free source for the causal questions that hold.
Checked 2026-10-15.
The fuller picture
Risks beyond AI: moderateBudget and layoff-cycle risk
Budget cycles and layoff waves move data science headcount more than any model does. When a company misses its numbers, analytics and research teams are easy to trim, and tech layoffs ran past 108,000 by mid-May 2026. Against that, the government still projects much-faster-than-average growth for the occupation.
Where this could be wrong
- Benchmarks overstate fit to real work. MLE-bench, an AI test built from Kaggle contests, uses clean Kaggle tasks with a fixed metric, while real model work starts with a vague ask and messy joins, so fitting may be less exposed than scored here.
- Databricks' 77.1% internal benchmark for Genie Code has no published method. We used it only as a sign of direction.
- Your company may already have a strong experimentation team. If design review is owned, the experiment design move shrinks and AI evaluation becomes the lead.
- Evaluation work could fold into ML engineering titles, leaving fewer data scientist evaluation seats than the Apple and Perplexity postings suggest.
Also matched: data scientist, junior data scientist, product data scientist. Research checked 2026-10-15. Page updated .