All notes

Automating living systematic reviews with AI: what works

A step-by-step map of one living review cycle: what AI can do at each step, how strong the evidence is, and what still needs a human.

By George Burchell

One living review cycle as a loop of steps: search, deduplicate, screen, extract, assess bias, analyse and decide. Each step is marked as automated, AI with a human check, or a human decision.

Can AI automate a living systematic review? Partly. AI can take on much of the repeat work in each living cycle, such as screening new records and drafting extraction, and the evidence is strongest for screening. People still decide what gets included and whether the conclusions change. The Cochrane Handbook's chapter on living systematic reviews says very little fully automated study selection software has been properly evaluated.

If you are new to living reviews, start with our guide to what a living systematic review is and how to run one. This post maps one cycle step by step.

Why living reviews need automation

A living review repeats the same work on a schedule. The Handbook says searches should generally run at least every three months, and monthly cycles are common. Each cycle brings new records to deduplicate and screen, and sometimes new studies to extract, assess and analyse.

The Handbook is frank about the cost. Evaluations have found that a substantial share of living reviews had no updates, or late ones, after the first version. The reasons include lost funding, waning commitment and the burden of screening search results. That is the case for automation. The work is repetitive, and after the first version you have a good dataset to check a tool against.

The Handbook makes that second point directly. Once the baseline review is done, its screening decisions describe the review's scope, which is "an ideal scenario for developing, validating, and using machine learning". You can train a review-specific classifier on those decisions, or write and test prompts for a large language model against them. It also warns that records used to build a prompt should not then be used to validate it.

The automation map

One row per step of a living cycle. "Current evidence" sums up the sources listed under References. "In AI-SR today" describes only what AI Systematic Review ships now. Living reviews (coming later): where a step only makes sense across cycles, the answer is "Not yet".

Step What AI can do Current evidence Human checkpoint that stays In AI-SR today
Re-run the search Re-run saved searches on a schedule; database alerts do this without AI Progress is slow: many databases do not support automated searching. Open databases such as PubMed may give enough coverage for some reviews (Handbook 21.3.1.3) Review the strategy at least yearly and when criteria change Builds a PubMed search that runs only after you approve it. Scheduled re-runs across cycles: not yet
Deduplicate against earlier cycles Match new records against the library from every earlier cycle, and link new reports to studies already included Handbook 21.3.1.3 and 21.3.2 describe the task; we found no evaluation specific to living cycles Confirm that a new report really belongs to an existing study Deduplicates the library within a review. Across cycles: not yet
Title and abstract screening First-pass screening with a classifier or prompts validated on the baseline review The strongest evidence of any step, but from small studies (Homiar 2025; Fuller-Tyszkiewicz 2025; Handbook 21.3.2) Uncertain records go to a person; check recall before trusting exclusions AI screening with include, maybe or exclude; uncertain records queue for your decision
Full-text screening Find full texts and screen them against the criteria Handbook 21.3.2 says it is "now being implemented in some LSR workflows"; little formal evaluation A person decides uncertain full texts Screens publicly available full text or a PDF you upload
Data extraction Draft extraction for new studies, checked against the baseline No tool rigorously evaluated for fully automated extraction from full texts; semi-automated tools exist (Handbook 21.3.3); one preprint reports high accuracy (Cao 2025) Confirm every extracted value Extracts values with page quotes; you confirm the schema and review flagged values
Risk of bias Gather evidence for each domain; some tools predict a judgement No tool rigorously evaluated for fully automated risk of bias (Handbook 21.3.3) The judgement stays with a person Assistants gather evidence per domain; you record each judgement
Re-analysis Re-run the meta-analysis with new data Standard methods; repeated testing needs thought (Handbook 21.3.5) Check new data before it is pooled Pools only rows you approved, within one review. Re-analysis across cycles: not yet
Decide whether to publish an update Estimate whether new evidence is likely to change conclusions Thresholds should be set in advance (Handbook 21.3.4) Always a human decision Not yet
Report (PRISMA-LSR) Draft the change log, status and flow diagram Reporting items defined in PRISMA-LSR (Akl 2024) Authors sign off each version Drafts a manuscript from approved artefacts, with stale sections flagged. Versioned PRISMA-LSR reports: not yet

Our sister sites go deeper on two rows: automating literature searches and data extraction tools.

Re-screening in each cycle

Screening is where most living-review effort goes, and where the evidence is best.

Homiar et al. (2025) developed prompts for a general-purpose large language model to screen titles and abstracts for a living review on interventions for anhedonia. They refined the prompts on 1,045 records, then tested them on the remaining 11,939 baseline records and on two later updates. The prompts kept every study that was finally included after full-text screening. Sensitivity for records that humans included at the title and abstract stage varied from 58% to 100% across updates, and the simulated workload reduction was 65–85%. The authors describe the result as high sensitivity with moderate specificity, and note that it comes from a single review with one model, so it needs testing on more reviews.

Fuller-Tyszkiewicz et al. (2025) report on LitQuest, software that keeps a saved screening model for updated searches. Across seven reviews, about 36% of each search's records needed human screening before reaching the software's stop point, ranging from 22% to 53% depending on the review. The test used the initial searches. The saved model for updates is how the software is designed to support living reviews.

Two cautions from the Handbook apply to both. Evaluations so far are small and use different models on different datasets, so general conclusions are hard to draw. And if each update brings only a few records, building and checking a model or prompt may not be worth the effort or the risk of missing a study. For how AI screening compares with manual screening in general, see Study Screener's guide to AI screening.

When eligibility criteria change

Living reviews change their criteria over time. The Handbook notes that many COVID-19 living reviews first included non-randomised studies and later limited themselves to randomised trials. A change like that affects decisions you have already made.

PRISMA-LSR item L2 asks you to describe and justify every method change since the previous version, to say "no changes" when there are none, and to state whether the changes were applied to studies already included. Changes to the screening process, including starting to use an automation tool, count as method changes. Item L3 asks you to report any study whose eligibility changed. The Handbook adds that before each update you should record protocol changes in a table, and that widening the criteria may need a one-off search of earlier years.

A practical rule for re-screening follows from this:

  • Criteria narrowed: re-check everything you included under the old criteria.
  • Criteria widened: re-check what you excluded, and search earlier years for studies the old search would have missed.
  • Either way: log the change, the reason and what you re-checked in the cycle log.

AI Systematic Review has one shipped behaviour that fits this. If the eligibility criteria change while a screening run is in progress, the run pauses. You then choose to cancel it, or to continue under the new criteria, which screens the earlier AI decisions again. Decisions a person made are kept, with a warning to check them against the new criteria.

Fully automated reviews: what the 2025 headlines did and didn't show

In June 2025, Cao et al. posted a preprint describing otto-SR, a workflow built on large language models for screening, data extraction and risk of bias. Against graduate-level human reviewers, it reported 96.7% screening sensitivity (humans 81.7%) and 93.1% extraction accuracy (humans 79.7%). The authors then used it to reproduce and update 12 Cochrane reviews covering 146,276 citations. It found 114 eligible studies against the original authors' 64. All discrepancies were checked by two humans, and the updated meta-analyses changed statistical significance in three reviews. This is a preprint and has not been peer reviewed. When we checked on 21 November 2026, its DOI record listed no published version.

Nature's news story on the preprint ran under the headline "AI slashes time to produce gold-standard medical reviews — but sceptics urge caution", with the summary that "a fully automated system is still some way off".

Both points hold. The results are a strong signal that screening and extraction can be automated much further. They also come with dual human review of every discrepancy, which is the human checkpoint this post keeps returning to. For an opinion piece on the trade-off, see what AI systematic reviews get wrong about automation.

What still needs a human

  1. Deciding whether the review should be living. That takes the four Cochrane criteria, including whether your team can sustain it.
  2. Approving the search strategy and reviewing it each year. Controlled vocabulary has to be looked up, not invented.
  3. Uncertain screening decisions, and checking that automated exclusions are not losing studies.
  4. Confirming extracted data before it is pooled.
  5. Risk of bias judgements. No fully automated tool has been rigorously evaluated.
  6. Deciding whether new evidence changes the conclusions, against thresholds set in advance.
  7. Logging method changes and signing off each version.

The Handbook also asks for the planned use of automation tools to be stated in the protocol and registration, and to follow RAISE guidance on responsible AI in evidence synthesis.

How AI-SR fits

AI Systematic Review runs one review through seven stages, each with a human checkpoint, on PubMed and publicly available full text in the current V1. Every important decision keeps its provenance, so you can inspect it, replay work and see which outputs went stale after an upstream change. Artefacts are versioned. Screening pauses when eligibility criteria change during a run. Living reviews (coming later). You can try the demo or join the waitlist.

References

All links checked on 21 November 2026.

  1. Hodder RK, McDonald S, Synnot A, et al. Chapter 21: Living systematic reviews. In: Cochrane Handbook for Systematic Reviews of Interventions, version 6.5.1. Cochrane, 2026. cochrane.org
  2. Homiar A, Thomas J, Ostinelli EG, et al. Development and evaluation of prompts for a large language model to screen titles and abstracts in a living systematic review. BMJ Ment Health. 2025;28(1):e301762 (published 22 July 2025). doi:10.1136/bmjment-2025-301762
  3. Fuller-Tyszkiewicz M, Jones A, Vasa R, et al. Artificial intelligence software to accelerate screening for living systematic reviews. Clin Child Fam Psychol Rev. 2026;29(2):191–199 (online 18 April 2025). doi:10.1007/s10567-025-00519-5
  4. Akl EA, Khabsa J, Iannizzi C, et al. Extension of the PRISMA 2020 statement for living systematic reviews (PRISMA-LSR): checklist and explanation. BMJ. 2024;387:e079183 (published 19 November 2024). doi:10.1136/bmj-2024-079183
  5. Cao C, Arora R, et al. Automation of systematic reviews with large language models. medRxiv preprint, posted 13 June 2025; not peer reviewed. doi:10.1101/2025.06.13.25329541
  6. Pearson H. AI slashes time to produce gold-standard medical reviews — but sceptics urge caution. Nature (news), 19 June 2025. nature.com

Frequently asked questions

Can AI fully automate a living systematic review?

Not today. The Cochrane Handbook says very little entirely automated study selection software has been empirically evaluated, and no tool has been rigorously evaluated for fully automating full-text extraction or risk of bias. AI can take on repeat work; people still decide.

Which steps of a living review can AI do today?

Screening new records has the most evidence, because the first version of the review gives a set of decisions to test against. Deduplication, full-text retrieval and drafting extraction can also be automated, with checks. Deciding whether to publish an update stays human.

How accurate is AI screening when a review is updated?

It depends on the review. In one living review, prompts for a general-purpose language model kept every finally included study across two updates, but title and abstract sensitivity varied from 58% to 100%. Validate on your own baseline before relying on it.

Do I still need two human reviewers?

Plan for human screening and decisions unless your protocol justifies otherwise. The Handbook asks for planned tool use to be stated up front and to follow RAISE guidance, and PRISMA-LSR asks you to report any change to the screening process.

How often should a living review's search be re-run?

At least every three months, according to the Cochrane Handbook, and monthly is common. Match the interval to how fast new studies appear, and report any change to it.

Keep decisions inspectable

AI Systematic Review keeps agent choices, source quotes, and human checkpoints visible from search through manuscript.

Join the waitlist

More notes