AI tools for systematic reviews: where they help and where a person must decide

Systematic reviews are slow because they are exhaustive. In an analysis of 195 completed reviews registered on PROSPERO, the average project took 67 weeks from registration to publication, and only about 3 records in every 100 retrieved were finally included. Reading thousands of records that turn out to be irrelevant, and copying data out of papers by hand, are the jobs that most AI tools are aimed at.

This guide sorts the help into three tiers by how far you can trust it, goes through the stages of a review, and sets out five steps for adding an AI tool without weakening the review. SynthGap is our own tool and works at the topic-choice stage only, so this is a guide from a vendor, not an independent comparison.

Three tiers of trust

A useful question to ask of any AI step is what a mistake would cost. That gives three tiers.

Tier 1, helps you work: removing duplicates, ranking records for screening, managing references, correcting language. A mistake mostly costs time, and you can see it.
Tier 2, makes or suggests judgements: deciding whether a study is eligible, rating risk of bias, extracting numbers, grading the certainty of evidence. A mistake changes the review, so these need a pilot, a check by a person and a line in your methods.
Tier 3, writes text and references: summaries, an introduction, plain-language text. Every claim and every reference needs checking against its source, because fluent text can be wrong.

What the evidence says so far

The evidence is uneven. Tools that rank records during screening, such as ASReview, rest on years of simulation work, and a key risk is stopping too early. For language models doing the screening, published evaluations report anything from under half of the relevant records found to nearly all, depending on the review, the model and the prompt, so a pilot on your own topic is not optional.

For extraction, a 2024 proof-of-concept study reported about 96% agreement with human extraction on 160 data elements, with missed items the commonest error, while other evaluations found lower and more variable accuracy. For search strings, ChatGPT-written Boolean queries in a 2023 study were precise but retrieved fewer of the relevant studies than the comparison methods, and some subject headings were wrong. For references, a 2023 test found that 55% of those written by GPT-3.5 and 18% of those written by GPT-4 were fabricated. Newer models are likely to do better, but one invented reference in a published review is one too many.

Let SynthGap run the first pass

Enter an area. SynthGap proposes candidate topics, checks each against existing systematic reviews, and lists the primary studies behind it. Start on the Free plan: no card needed, and every run shows every topic it finds.

Stage by stage

Question and protocol: AI can suggest candidate topics and show which already have reviews; you write the question and the protocol. See using AI to find a review topic and the three tests for a topic.
Search: AI can draft and translate search strings, and an information specialist should review them. See AI and the search strategy.
Screening: ranking records and acting as a second screener are the best studied uses. See AI screening for systematic reviews.
Full texts: AI does not remove paywalls. Plan how you will get the papers you cannot open, and say in the review how many you could not obtain.
Data extraction: faster with AI, but every number that enters the pooled analysis still needs checking. See AI data extraction for meta-analysis.
Risk of bias: tools such as RobotReviewer suggest ratings for randomised trials. Treat them as a second opinion, not a judgement.
Writing: language editing is low risk. Drafted claims and references are not, so check each against its source.

Adding an AI tool to a review in five steps

Decide in the protocol: name the step, the tool, and why you think it is safe for this review.
Pilot it: run it on a sample that you have also done by hand, and compare the results.
Fix the rules in advance: thresholds, the stopping rule, and who settles a disagreement between a person and the tool.
Keep a log: the tool, version, date, settings and prompts, and what it decided.
Report it: PRISMA 2020 asks for details of automation tools in items 8, 9 and 11. See how to report AI use in a systematic review.

Where SynthGap fits

SynthGap sits at the start. It finds candidate topics in an area, checks them against existing systematic reviews and lists the studies behind each one, so that you can judge whether a topic is new and feasible before you write a protocol. It does not run your search, screen your records or extract your data, and its output is a set of candidates to verify, not a verdict.

Sources

The figures and rules on this page come from these sources.

Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 2017;7:e012545.
Flemyng E, Noel-Storr A, Macura B, et al. Position statement on artificial intelligence (AI) use in evidence synthesis across Cochrane, the Campbell Collaboration, JBI and the Collaboration for Environmental Evidence 2025. Campbell Systematic Reviews 2025;21:e70074.
Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
Lieberum JL, Toews M, Metzendorf MI, et al. Large language models for conducting systematic reviews: on the rise, but not yet ready for use - a scoping review. Journal of Clinical Epidemiology 2025;181:111746.
Gartlehner G, Kahwati L, Hilscher R, et al. Data extraction for evidence synthesis using a large language model: a proof-of-concept study. Research Synthesis Methods 2024;15:576-589.
Konet A, Thomas I, Gartlehner G, et al. Performance of two large language models for data extraction in evidence synthesis. Research Synthesis Methods 2024;15:818-824.
Wang S, Scells H, Koopman B, Zuccon G. Can ChatGPT write a good Boolean query for systematic review literature search? SIGIR 2023.
Walters WH, Wilder EI. Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports 2023;13:14045.
van de Schoot R, de Bruin J, Schram R, et al. An open source machine learning framework for efficient and transparent systematic reviews. Nature Machine Intelligence 2021;3:125-133.
Gartlehner G, Affengruber L, Titscher V, et al. Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial. Journal of Clinical Epidemiology 2020;121:20-28.

Questions

What is the best AI tool for systematic reviews?

There is no single best tool, because the steps differ and so does the evidence for each. Choose per step, prefer tools with published evaluations on reviews like yours, and pilot before you rely on one. The twelve questions in the guide to choosing a tool help you compare them.

Can AI replace the second reviewer?

The usual standard for a full review is two people working independently. The 2025 position statement of Cochrane, the Campbell Collaboration, JBI and the Collaboration for Environmental Evidence notes that single-reviewer abstract screening, common in rapid reviews, carries an estimated 13% risk of missing a relevant study, and that AI as a second reviewer could reduce that risk, provided it is justified, overseen by a person and reported.

Do I have to say that I used AI?

If it made or suggested judgements, such as eligibility, risk of bias, data extraction or the certainty of the evidence, yes: the position statement says to report it, and PRISMA 2020 asks for details of automation tools. Language editing generally does not need listing, but check the journal's policy.

Will an AI tool find every relevant study?

No tool does, and no search does. Check recall against studies you already know are relevant, search more than one source, and follow the citations of the key papers.

Keep reading

General methodological guidance, not medical advice. SynthGap's suggestions are AI-generated from the published literature, so verify them before registering a protocol.