Product
Litigation
- Early Case Assessment
- Evidence Analysis
- Web Search
- Document Review
- Discovery at Scale
- Production Setup
- Connectors
- Dictation
- Deep Research
- Legal Research (Upcoming)
Transactional
Solutions
Litigation
- Early Case Assessment
- Investigations & Subpoenas
- Trial Preparation
- Enterprise eDiscovery
- Cloud eDiscovery
Transactional
Who it’s for
Resources
Learn
Customers
Events & CLE
“Defensible” is the word every e-discovery vendor puts on the slide and almost nobody defines. It gets used as if it were a certification, or a feature you could switch on, or a property that some software has and other software lacks. It is none of those things.
The most useful answer anyone has given came from a sitting federal magistrate judge, in a law review article, before generative AI existed. In “Defensible” by What Standard?, 13 Sedona Conf. J. 217 (2012), the Hon. Craig B. Shaffer of the District of Colorado worked the question backward from the Federal Rules and arrived at something no vendor can sell you.
Short answer: A defensible process is one that can withstand an after-the-fact challenge from the other side. It is measured against reasonableness, not perfection, and it has to satisfy four things at once: the method must fit the data, be proportionate to the case, be demonstrably reliable in terms you can put a number on, and be explainable to a client, an adversary, and a judge who may know less about the technology than you do. Software helps with the first and the third. You own the second and the fourth.
1.The standard is reasonableness, not perfection
Start where Judge Shaffer starts, which is the Rules rather than the technology. Rule 34 requires a party to undertake reasonable efforts to identify and produce responsive, non-privileged material in its possession, custody, or control. Rule 26(g) makes counsel certify, by signature, that the response was formed after a reasonable inquiry and is neither unreasonable nor unduly expensive given the needs of the case. Rule 26(b)(2)(C) requires the court to limit discovery whose burden outweighs its likely benefit. Rule 1 asks for the just, speedy, and inexpensive determination of the action.
Not one of those provisions asks for a perfect production. As Pension Committee put it, courts do not hold discovery to a standard of perfection, and Shaffer builds the point out with an analogy to Freedom of Information Act litigation, where an agency’s search “need not be perfect, only adequate,” and adequacy is measured by “the reasonableness of the effort in light of the [requester’s] specific request.” His conclusion: a party should not be required to show that a search methodology is infallible or that it is certain to find every potentially responsive document.
District Judge Carter said the same thing when he affirmed the predictive coding protocol in Da Silva Moore v. Publicis Groupe: no review tool “guarantees perfection,” and “there are risks inherent in any method of reviewing electronic documents.” That includes the method most lawyers still treat as the safe default. Linear human review is not the baseline of accuracy; it is just the baseline of habit. The research Shaffer cites — Grossman and Cormack in 2011, Roitblat, Kershaw, and Oot in 2010 — had already found that technology-assisted review matched or beat exhaustive manual review on recall and precision at a fraction of the effort.
The defendants in Global Aerospace put numbers on that gap when they moved for a protective order approving predictive coding: linear first-pass review of their collection would run 20,000 hours and two million dollars and surface roughly sixty percent of the relevant documents; keyword search would find about twenty percent; predictive coding could reach upwards of seventy-five percent for a fraction of the cost. The Virginia court approved the use of predictive coding — in a very short order, without substantive discussion, and expressly without prejudice to a later challenge to the completeness of the production. Which is the whole point. Permission to use a method is not a finding that your use of it was reasonable.
2.Judge Shaffer’s four criteria
Here is the part worth pinning above your desk. Given the requirements of the Federal Rules, Shaffer wrote, a defensible search protocol should satisfy four criteria.
- 1. It must be functional. The methodology has to be commensurate with the quantity of potentially responsive ESI and with all the pertinent data types, repositories, and custodians. The focus is the “fit” between the technology and the collection being searched. As Shaffer puts it, “[i]t would be a mistake to believe that one search methodology would be optimal for all cases and all data collections.”
- 2. It must be reasonable against the proportionality factors. Measured under Rule 26(b)(2)(C) and Rule 26(g) — the fit between what the search costs and what the litigation is worth. Manual review may be perfectly reasonable for a small set of documents and antithetical to Rule 1 for several terabytes.
- 3. It must be demonstrably reliable. In terms of recall and precision, or other appropriate metrics. Shaffer concedes this one is hard — picking the right metrics challenges counsel and the court alike — and tells producing parties to anticipate reliability challenges and build quality assurance checks through identification, collection, review, and production.
- 4. It must be readily understandable to multiple audiences. The client, opposing counsel, and the court. “Defensibility ultimately depends upon counsel’s ability to communicate in a clear and comprehensible way the technical and qualitative aspects of the methodology and the rationale for its selection.”
Notice what those four criteria are not. None of them is satisfied by a logo on a slide, a SOC 2 report, or a vendor’s assurance that its model is state of the art. Two of them — proportionality and explainability — are lawyer work that no platform can do for you. The other two require the platform to give you something specific: control over how the method is applied to this collection, and numbers you can defend.
Shaffer’s advice on the fourth criterion is worth quoting because it cuts against instinct. Counsel should approach defensibility “from an educational, rather than adversarial perspective,” assisting the court rather than advocating for a zero-sum result. Most judges are on a steep learning curve with search methodology. “In that respect, candor is counsel’s ally.”
3.You get graded after you have already spent the money
The structural problem with defensibility, Shaffer observes, is that it is nearly always evaluated post hoc. The court is asked to assess a process only after a party has made the critical technological decisions, spent considerable time and money, and moved through most of the pretrial process. “[T]he true measure of a defensible e-discovery process is the ability to withstand an after-the-fact challenge by the opposing party.”
That has a practical consequence people miss. Because the challenge comes later, the thing that gets evaluated is not your process but the record of your process. Decisions you made for good reasons and never wrote down are, at the hearing, indistinguishable from decisions you made carelessly. You are building that record from the first custodian interview onward, whether or not you intend to.
It also means the cheapest defensibility available to you is the kind you buy early. Shaffer lays out three procedural postures, in descending order of comfort: negotiate a protocol at the Rule 26(f) conference and have the court adopt it; move for a protective order under Rule 26(c) and get early approval; or wait, and find yourself defending unilateral decisions in a Rule 37 motion at the end of discovery. In the third posture, as he dryly notes, you are no longer seeking permission — you are seeking approval, or forgiveness.
4.Who has to prove what
A large part of Shaffer’s article is a proposal about burdens, and it is worth understanding because it frames what a challenge to your methodology actually looks like.
In an ordinary motion to compel, once the requested material is facially relevant, the party resisting discovery carries the burden. Shaffer argues that framework is wrong when the motion attacks the producing party’s search methodology, because Rule 34 does not specify how a party must search, and Sedona Principle 6 puts the producing party in the best position to choose its own methods. Applying the usual paradigm, he warns, “simply invites a protracted evidentiary hearing that may devolve into a battle between competing vendors advocating for their own software product.”
His proposed sequence runs in three steps:
- The challenger goes first. The requesting party must make a threshold showing: relevance under Rule 26(b)(1), reasonableness under the proportionality factors, and evidence that the producing party actually failed to produce responsive material because of the methodology used, or that the protocol is demonstrably likely to miss responsive documents. Suspicion is not enough.
- Then you defend the method. The burden shifts to the producing party to show the protocol was reasonable in the circumstances — prepared to defend its technological decisions “with affidavits or other equivalent information from persons with the requisite qualifications and experience,” explaining the rationale, demonstrating that the method was appropriate for the task, and showing that it was properly implemented.
- Then the burden goes back. Once reasonableness and reliability are shown, the ultimate burden of persuasion should fall on the requesting party, which must show either bad faith — for example, search terms or a seed set chosen knowing they would omit obviously responsive documents — or that the protocol is so statistically or scientifically unreliable as to cause actual and substantial prejudice.
Two guardrails matter here. A methodology “should not be subject to challenge simply because experts or opposing parties have their own preferred approach, or have vague, unsubstantiated suspicions that documents or ESI are missing.” And courts should resist the invitation to second-guess in hindsight: borrowing from the Supreme Court, even if alternative methods existed, “it does not follow that the search as conducted was unreasonable.”
One caveat worth stating plainly: this burden-shifting sequence is a proposal in a 2012 journal article by a magistrate judge, not a rule and not binding precedent. Its value is as a map of what a court will want to see from you. Cite the Rules and the case law; use the framework to organize your own preparation.
5.Reliable, not correct: Rule 702 and Daubert
Courts have split on whether Rule 702 governs a fight over search methodology. In United States v. O’Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008), Magistrate Judge Facciola held that the selection of search terms “is clearly beyond the ken of a layman” and that a challenge to them must rest on evidence meeting Rule 702. In Da Silva Moore, Magistrate Judge Peck wrote that “Rule 702 and Daubert simply are not applicable to how documents are searched for and found in discovery.”
Shaffer’s read is that the two positions are not actually irreconcilable, because both judges are asking the same underlying question: will this protocol produce results that are reliable and consistent with the Federal Rules? A debate over the literal applicability of Rule 702, he suggests, “may be more distracting than helpful.”
He also makes a point about the gatekeeping function that lands differently in a discovery dispute than at trial. Quoting the Eleventh Circuit: “There is less need for the gatekeeper to keep the gate when the gatekeeper is keeping the gate only for himself.” On a motion to compel there is no jury to protect; the judge can hear the expert and give the opinion whatever weight it deserves. That should lower the temperature on procedure. It does not lower the bar on substance — expert testimony about how and why a protocol was chosen will often be necessary, though it “should not provide an opportunity for the opposing party to eviscerate the time and cost savings the new technologies were designed to produce.”
The line to carry forward is the standard Rule 702 itself applies: the court’s job is not to decide whether the method is correct, but whether it is reliable. As one court put it, “[t]he requirement of reliability is lower than the standard of correctness.” A proponent should be able to show that the selected tool or method “has adequately and accurately collected or captured responsive documents and ESI.”
6.The most defensible protocol is the one you did not pick alone
Shaffer’s strongest claim is also his simplest: “The most ‘defensible’ search methodology is one [that] has been jointly adopted by the parties and endorsed by the court.”
Rule 26(f)(3) is written in mandatory terms and requires the parties to address ESI issues before the Rule 16 scheduling conference. Treating that conference as a perfunctory exercise is a strategic error, and courts have said so with increasing bluntness. In DeGeer v. Gillis, the court criticized both sides — one for refusing to disclose its custodians and search terms, the other for ignoring repeated requests to propose any — and observed that “[s]electing search terms and data custodians should be a matter of cooperation and transparency among parties and non-parties.” In Covad Communications v. Revonet, where the plaintiff ignored requests for suggested search terms and then attacked the terms used, the court held it “unfair to allow Covad to fail to participate in the process and then argue that the search terms were inadequate.”
The Sedona Conference put the sharpest version in The Case for Cooperation: because knowledge of the producing party’s data is asymmetrical, refusing to help design a protocol that the data holder knows will surface responsive documents “could be tantamount to concealing relevant evidence.”
There are teeth behind this. A party that does not participate in good faith in developing a discovery plan can be made to pay the other side’s fees under Rule 37(f); the same goes for being substantially unprepared at the Rule 16(b) conference under Rule 16(f). And Shaffer suggests the reverse should also be true — fee-shifting under Rule 37(a)(5) ought to be “unjust” where a prevailing movant rebuffed the producing party’s good-faith attempts to negotiate a protocol in the first place.
7.What generative AI changed, and what it did not
Shaffer wrote in 2012, when the frontier was predictive coding and the fight was over whether courts would allow it at all. That fight is over. Since Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125 (S.D.N.Y. 2015), it has been described as black letter law that where the producing party wants to use TAR for document review, courts will permit it.
Quinn Emanuel’s survey of AI in defensive document discovery traces what has happened since, and the technical shift matters for defensibility in a specific way. Classic TAR — whether TAR 1.0, where reviewers code a seed set and the model then predicts across the corpus, or TAR 2.0 continuous active learning, where the model refines as reviewers work — learns from examples. It struggles with unique or unusual documents precisely because it has no exemplar to compare them to.
Generative AI does not work that way. It takes a written prompt that reads much like a document review protocol, which means it needs no coded training set, and it can explain itself. That last property is the one lawyers should care most about:
- It produces narrative reasoning. The system can say why it coded a document responsive or not. That surfaces patterns of miscoding and lets you revise the prompt before you scale — and it feeds directly into Shaffer’s fourth criterion, explainability to the client, the adversary, and the court. Predictive coding never gave you that.
- It does not depend on exemplars. The seed-set problem, and the discovery fights over seed sets, do not arise in the same form.
- It extends past responsiveness. The same pass can flag privilege, identify PII, categorize by issue, and surface documents that matter to case strategy rather than only to compliance.
- But it is not iterative once it runs. As Quinn Emanuel puts it, generative AI review “is not an iterative process once the prompts have been run over the corpus.” TAR 2.0 improved while the review happened. Generative AI largely does not.
That last point is the single most important operational change, and it moves Shaffer’s third criterion — demonstrable reliability — earlier in the project. With continuous active learning, you could partly validate as you went. With generative AI, your validation has to be front-loaded: test the prompts, measure the results, and revise, before you turn the thing loose on the collection. Rigorous pre-deployment testing is not a best practice here. It is the entire quality control regime.
8.What courts require in exchange: transparency
Permission to use the technology came with a price, and the price has been consistent. As one court summarized in In re Insulin Pricing Litigation, 2025 WL 1112837 (D.N.J. Apr. 11, 2025), “courts have mandated some level of transparency and validation of TAR methodologies.” The line runs back years:
- Progressive Cas. Ins. Co. v. Delaney, 2014 WL 3563467 (D. Nev. 2014). Cases approving technology-assisted review “have required an unprecedented degree of transparency.”
- Moore v. Publicis Groupe, 287 F.R.D. 182 (S.D.N.Y. 2012). Transparency “allows the opposing counsel (and the Court) to be more comfortable” with the process.
- In re Diisocyanates Antitrust Litig., 2021 WL 4295729 (W.D. Pa. 2021). “Transparency transcends cooperation. It requires that they disclose information sufficient to make those discussions meaningful.”
- Kaye v. N.Y.C. Health & Hospitals Corp., 2020 WL 283702 (S.D.N.Y. 2020). The court required disclosure of the collection criteria, the name of the continuous active learning software, the review workflow, and how the party intended to validate the results.
Seed sets have been the exception rather than the rule: in In re Biomet M2a Magnum Hip Implant Products Liability Litigation, 2013 WL 6405156 (N.D. Ind. 2013), the court declined to order disclosure of the seed set, though it urged the producing party to reconsider its refusal.
Which leaves the live question of 2026. Are generative AI prompts discoverable? Quinn Emanuel frames it as genuinely open: a prompt might be treated like a search term, which parties routinely exchange; like a seed set, which is sometimes withheld; or like attorney work product, which is rarely shared at all. A prompt drafted by a senior lawyer encoding the theory of the case looks a great deal like work product. It also functions, mechanically, exactly like a search term.
No one should want to litigate that question by surprise, in a Rule 37 motion, after the review is finished. The practical answer is Shaffer’s answer, thirteen years early: raise it at the Rule 26(f) conference or in ESI protocol negotiations, and get an agreement on the record.
9.Defensibility is also an ethics question
Rule 26(g) is not the only signature you are putting on this. ABA Formal Opinion 512, issued July 29, 2024, addressed generative AI directly, and four obligations bear on document review:
- Competence, Rule 1.1. Lawyers must acquire a reasonable understanding of the benefits and risks of the tools they use, or draw on the expertise of others, and must apply an appropriate degree of independent verification or review before relying on the output.
- Confidentiality, Rule 1.6. Use versions of the software with privacy protections that keep prompts and documents out of model training — protecting confidentiality, privilege, and work product. Where the data physically sits, and who else can reach it, is part of this analysis.
- Supervision, Rules 5.1 and 5.3. Managerial lawyers must establish clear policies on permissible use; supervisory lawyers must ensure compliance, including training on the ethical and practical use of these tools and their risks.
- Client communication, Rule 1.4. Lawyers should reasonably consult with the client about the means used to accomplish the client’s objectives, and should consider whether the circumstances warrant consulting the client about generative AI use specifically.
On top of that sits a patchwork of jurisdiction-specific guidance that continues to shift state by state. Check your own.
10.What a defensible generative AI review actually looks like
Put the 2012 criteria and the 2026 technology together and the practice falls out fairly cleanly.
- Fit — know what the tool cannot read. Identify immediately which files the platform cannot process, images, spreadsheets, audio, very large datasets, and which languages it cannot review. Then build a documented separate workflow for those. An excluded category you never inventoried is the gap opposing counsel will find, and it goes straight to Shaffer’s first criterion.
- Proportionality — write down the arithmetic. What linear review of this collection would cost in hours and dollars, what the AI-assisted path costs, and why the choice is reasonable given the amount in controversy and the issues at stake. This is the Rule 26(g) record, and it is a paragraph, not a project.
- Reliability — measure before you scale, not after. Because generative AI review does not improve mid-run, validation belongs up front: run the prompts against a statistically defensible sample, score precision, recall, and F1 against attorney-coded ground truth, read the model’s narrative reasoning on the documents it got wrong, revise the prompts, and re-measure. Keep every version and every score.
- Explainability — put a senior lawyer’s name on it. Quinn Emanuel’s first recommendation is to have an experienced attorney on the case own prompt development, analyze the reasoning, revise, and validate the metrics before scaling — the same role a partner plays in writing a review protocol. That person is also the affiant if this is ever challenged, which is the point.
- Cooperation — disclose early. Raise generative AI use at the Rule 26(f) conference or during ESI protocol negotiations, before the money is spent. An agreed protocol is not just cheaper than a fight; it is a better answer to the question the court will eventually ask.
11.The file you should be able to hand a judge
If defensibility is a record rather than a product, then the deliverable is a file. Both sources converge on roughly the same contents, and it should exist before anyone asks for it:
- The tool. Specific platform, version, and settings used.
- The prompts. The complete history, including every iteration and the reason for each revision.
- The validation. Methodology and metrics — precision, recall, F1 — with sample sizes and how ground truth was established.
- The humans. Who supervised, what they reviewed, what they changed, and what training they had.
- The exclusions. Every category of document the tool could not process, and the alternate workflow that covered it.
- The negotiation. What was disclosed to opposing counsel, when, and what was agreed.
Read that list against Shaffer’s framework and it is exactly what a producing party would need to carry the second step of his burden-shifting sequence: affidavits from qualified people explaining the rationale, showing the method was appropriate for the task, and showing it was properly implemented.
12.So: what does “defensible” mean?
It means a process that a reasonable person would call reasonable, applied to this collection, at a cost proportionate to this case, with reliability you can put a number on, explained clearly enough that a judge who has never heard of your platform can follow it — and documented well enough that the explanation still works two years later, under cross-examination, in front of an adversary who is looking for the gap.
Software is a real input to that. A platform that cannot show you its reasoning, cannot produce validation metrics, or cannot tell you which documents it silently skipped makes the four criteria harder to satisfy. A platform that does all three makes them easier. Neither one hands you the answer.
“Technologically advanced tools, however ‘cutting edge’ they may be,” Shaffer quoted The Sedona Conference as saying in 2009, “will not yield a successful outcome unless their use is driven by people who understand the circumstances and requirements of the case, as guided by thoughtful and well-defined methods, and unless their results are measured for accuracy.”
Nothing about generative AI has changed that sentence. It has only raised the stakes on the last clause.
13.Frequently asked questions
Is “defensible” a legal term of art?
No. It appears nowhere in the Federal Rules. It is shorthand for a process that can withstand an after-the-fact challenge, and the Rules supply the actual standards it has to meet — reasonable efforts under Rule 34, reasonable inquiry under Rule 26(g), and proportionality under Rule 26(b)(2)(C).
Can a vendor make my review defensible?
Not on its own. A platform can supply fit to your data and reliability metrics you can defend. Proportionality analysis and the ability to explain the methodology to a client, an adversary, and a court are lawyer work. Two of Judge Shaffer’s four criteria simply cannot be outsourced.
Do I have to disclose that I used AI for document review?
Courts have consistently required a degree of transparency about technology-assisted review, and several have ordered disclosure of the software used and the validation plan. Whether generative AI prompts themselves are discoverable is still unsettled. Raise it at the Rule 26(f) conference rather than letting it surface in a motion.
Is human linear review the safe choice?
It is the familiar choice, not the safe one. No method of reviewing electronic documents is risk-free, and published research has found technology-assisted review matching or exceeding exhaustive manual review on recall and precision. On a large collection, insisting on linear review can itself raise proportionality problems under Rule 1 and Rule 26(b)(2)(C).
What is the single highest-leverage step?
Negotiating the protocol at the Rule 26(f) conference and having the court adopt it. In Judge Shaffer’s formulation, the most defensible search methodology is one that has been jointly adopted by the parties and endorsed by the court.
This article summarizes and quotes two sources: Hon. Craig B. Shaffer, “Defensible” by What Standard?, 13 Sedona Conf. J. 217 (2012), and Quinn Emanuel Urquhart & Sullivan LLP, The Evolving Landscape of AI in Defensive Document Discovery. Judge Shaffer’s burden-shifting framework is a proposal advanced in a journal article, not binding authority. Case law and ethics guidance in this area continue to develop, and jurisdiction-specific obligations vary. This is general information about legal technology and discovery practice, not legal advice for any particular matter.
