SDLC in the AI Era

Lifecycle phase

What changes in Test

Every change this guide lists for the Test phase, across all roles. Open one to see before and now, with sources, on the role page.

Product Manager / Owner
- Defining quality for AI featuresA study: LLM judges need human checks; Husain and Shankar suggest expert grading
Business Analyst
- Handing acceptance criteria to testingThoughtworks: AI test cases from stories tended to miss non-functional needs
UX/UI Designer
- Defining what good output looks likeNN/g's Elman: designers should define 'good' so partners can evaluate the model
Software Architect
- Checking that the codebase still matches the intended architectureThoughtworks: agents can speed up drift; some of its teams add LLM checks
Software Developer / Engineer
- Automated testingWillison: tests help agents; in one study, many agent PRs had no test changes
QA Engineer / Tester
Security Engineer
- Getting scanner findings fixedGitHub: evaluate each Autofix suggestion; it may fail to fix or add flaws
- Finding vulnerabilitiesAI finds vulnerabilities; Anthropic argues verifying and patching is the limit
Data Engineer / Analyst
- Testing and monitoring pipelinesIn dbt's survey, validation investment isn't keeping pace with AI output
- Choosing and evaluating an AI query toolTwo text-to-SQL benchmarks had high annotation error rates that moved rankings
Scrum Master / Agile Coach
- Protecting the Definition of DoneReview AI-generated code as rigorously as a teammate's, and don't skip tests