Adapting When the Scope Changes

0:05 – Michael Gibeault

Welcome to TCDI Talks, where we highlight the people and ideas driving innovation in legal services and technology. I’m your host, Michael Gibeault, and today we’re talking about something every review team has experienced, but not everyone handles well. That’s scope changes.

So, one minute everything’s on track, moving along at a steady pace, and then the next minute your entire strategy is under pressure following a shift. The instinct in these moments is often to just throw more people at the problem. But as Jennifer Andres explains in a recent article, “How to Handle Scope Changes in Document Review,” that approach can create more issues than it solves.

Jenn is the Vice President of the Military Spouse Managed Review program, or MSMR, at TCDI, and she brings a practical, experience-driven perspective on how teams can adapt and stay in control when things change quickly.

Thanks for joining us today, Jenn.

1:13 – Jennifer Andres

Great. Thanks, Michael. It’s nice to be here.

Why Scope Changes are Inevitable (1:15)

1:15 – Michael Gibeault

Well, listen, Jenn, you open your article by saying scope changes are basically inevitable in document review. Why do you think so many teams still get caught off guard by them?

1:27 – Jennifer Andres

Yeah. So, it’s really a matter of when a project comes through, you plan for what you know. You can’t plan for what you don’t know, so, it makes sense to just plan for the volume and the deadline that you currently know about.

And then when something changes, it’s really about shifting to make it work. It’s about coordination and communication to do what you can to accommodate those scope changes. But really, it’s take what you know, plan the right team size, plan based on a throughput you expect to be hitting, and that’s where you have to start.

1:57 – Michael Gibeault

Well, in your article you paint a very real picture. Everything is on track one minute, then scope expands and deadlines shrink at the same time. In your experience, Jenn, how often does that actually happen?

2:11 – Jennifer Andres

So, it’s not every project, and they aren’t all the, you know, the catastrophic “everything is on fire.” You know, you’ve got both the massive, time-intensive, you know, additional documents plus the reduction in how long you have to complete them.

But most projects do suffer from some sort of scope change that needs to be accommodated. And usually where I see the bigger ones that require a lot more planning and, you know, really concern over how to handle those are the big ones. You know, you already have 500,000 documents, you already have a million documents, and you’re already like, “Argh, how are you going to get these done in this short amount of time?”

Those are the ones where when those scope changes happen, you kind of feel it more and it needs a lot more planning to make those happen. And unfortunately, those do happen still enough that it’s not completely uncommon when they come up.

3:06 – Michael Gibeault

Well, Jenn, one of the most interesting points you make is that adding more reviewers can actually make things worse. Why is that?

3:15 – Jennifer Andres

Yeah. So, that’s a great question. The first inclination is let’s add bodies. I can see why that’s the first instinct, but it’s not always the right one.

First, you need to think about, “Well, what do I have, and what have I done so far compared to what am I going to get, and what do I need to get done in the amount of time that I have left?”

And all of those questions are really going to be important to deciding if you really need to add team members. And that’s because when you add team members, you’re adding additional things that need to happen. It’s not just a matter of finding resources and adding them to the project. It’s a matter of getting them ramped up, which does take time.

They’re going to go slower than the current team that’s been on it for longer. You have to get them calibrated. They have to review all the feedback that you’ve gotten on the project and get a pretty good grasp on it in a shorter amount of time than the current team already had to do so. But then you also have to make them consistent.

It is a lot easier to make a team of ten consistent than it is a team of 50. So, if you just throw bodies at a project and you end up with a team of 50 people, trying to make 50 people consistent instead of ten people consistent, it’s going to take a lot more time and cost to be able to do that.

4:30 – Michael Gibeault

Well, speaking of cost, Jenn, what are the hidden costs of scaling a review team too quickly?

4:36 – Jennifer Andres

So, we just touched on one, and that’s definitely going to be that QC component. You’re going to have a lot more QC to make sure that you are being consistent. You know, with document review, obviously we’re concerned with accuracy, but consistency is also another main component that we have.

So, we’re going to do, you know, additional QC to make sure that we’re being consistent for those major calls: responsiveness, privilege, confidentiality. And that’s going to take time.

Then there’s also the PM time to it. The PM is going to be coordinating those new resources starting, getting them acclimated to the protocol, coordinating with the QC team to make sure they’re getting feedback and getting calibrated.

You know, all those things that the PM is going to have to do to quickly to bring those folks up to speed, and then getting to that average throughput that your current team is already getting to.

5:25 – Michael Gibeault

So, Jenn, in the article, you talk about stepping back before reacting. Let’s walk through how to actually do that. You start with the team capacity and schedules. What do people tend to misunderstand about the capacity planning?

5:42 – Jennifer Andres

So, that’s a that’s a great point. It’s not as easy as just saying, “Find ten more people, and add them to the project.” You have to find ten people who are available as you need them to be for the project.

I can easily find ten people who are, you know, available and able to work on the project, but do they have an upcoming vacation where they’re going to be off a couple of days? Do they have commitments where they work 20 hours a week instead of the 40 I would maybe expect from them?

If I’m not planning for all the different nuances that could be a part of their time each day, which is what I based the throughput on, I’m going to be off and still potentially miss my deadline.

6:23 – Michael Gibeault

Jenn, I liked your point that not all data is created equal. How do new or more complex data sources change the game mid-review?

6:34 – Jennifer Andres

They can end up having a pretty significant impact. You know, for example, you can be on a planning call, and it could be, “Oh, we’ve got 10,000 documents and you know, you’ve got two weeks to get it done. You can even use a pretty small team to knock those out.”

But if you’re not asking the right questions, and you don’t understand that out of those 10,000 documents, maybe 9,000 are Excel’s, which do end up taking more time to get through than something like an email or an image-based file, you’re not going to account for that right, with your throughput, and you’re going to be off your deadline target.

So, it is important to understand all the different data types that you have and understand which ones generally will come with a slower throughput than others so that you can account for that with your team size and your deadline planning.

7:17 – Michael Gibeault

You talk about executing a smarter strategy. Once you understand the impact, the next step is adjusting intelligently, not just working harder. So, you emphasize reducing review volume before increasing speed. Why is that the best first move?

7:35 – Jennifer Andres

Yeah. So, that’s going to be important because it’s always going to work out better if you can reduce what you have to do rather than increase the number of people it’s going to take to get that same amount of work done.

I kind of think of it as like a to-do list. If I have my daily to-do list and I have 20 items on it, and I’m looking at it that day, and I’m saying, “There’s no way I’m going to get through all 20 of these.” Okay.

Do I have somebody come on and help me with my to-do list, which may take more time to explain what I need done and how it needs to be done? Or do I go through my to-do list and say, “What can I take off that doesn’t really have to be done?”

If I can do that and reduce what needs to be done, I can still get it done and not have to worry about adding a team member. But if I can’t, I’m now adding team members, I’m now adding in those additional times, costs, all those other things that if I can just reduce what has to be done, I might not have those hidden costs that we’ve talked about, all those other things.

8:29 – Michael Gibeault

You also call out overly complex workflows. Where do you typically see teams overcomplicating things?

8:37 – Jennifer Andres

So, there’s a couple of different instances where we see overly complex workflows. And I think the two big ones we generally see are where counsel is trying to plan for something they think the opposing party might do.

For example, if there’s, you know, a motion out on whether certain documents have to get produced but it hasn’t been ruled on yet, they’re going to want to keep those non-responsive for now, but they want to flag them in case they have to go back.

So, usually that’s an issue tag, and that’s typically a nonstandard workflow where you have a non-responsive document getting an issue tag. And that tends to slow reviewers down, because they think if there’s an issue tag, it should be responsive. And those tend to kind of drive down the pace a little bit.

Whereas, you know, going back and maybe doing a secondary review for those if that comes about, could be a simplified workflow.

The other option, or the alternative that we see to that are family consistent reviews. Everybody likes to keep, you know, families complete when it comes to productions. And we don’t argue against that at all. We think that they should be family complete, especially where that’s what’s been agreed to. But we still prefer the simplified version of on-the-face coding.

And that’s for a couple of reasons. You know, where you have complex families. I’ve seen projects where there’s maybe 20 documents in a family. And if you get through the first 8 or 9 and they’re all non-responsive, it’s easy. But when you get to 10 or 11, and they start being responsive, now you have to go back through all the other documents you previously coded “non-responsive” to fix them and make them consistent using that workflow.

Additionally, where you have QC and you’re doing targeted searches, let’s say we got feedback that X type of documents should be non-responsive. Well, when you’re running that QC search, you can’t just look at that one document, because it may be marked responsive as a family member of something that’s responsive. So it, you know, it increases your QC time and cost as well to have a workflow like that.

10:37 – Michael Gibeault

Well, Jenn, let’s talk about technology. Where do tools like active learning and AI actually make the biggest impact in these situations?

10:47 – Jennifer Andres

So, they impact how we can get through the responsive documents quicker. You know, active learning, it can prioritize the responsive documents so that we’re getting through those first and getting those into production. Same for our QC and counsel QC. It’s just getting them those documents first.

For AI, it’s taking the first crack at coding so that we can then have some QC or validation run across them. And it’s taking care of larger volumes of documents in a shorter amount of time.

11:15 – Michael Gibeault

So, Jenn, you end your article by saying, “You can’t control how scope evolves, but you can be ready for it.” What does being ready actually look like?

11:27 – Jennifer Andres

It looks like not panicking because it happens, and it’s how you respond to it that matters. It’s knowing that it could very well happen. You’ve got to understand that it’s coming, and when it does, think about what you’ve done, what you have left to do, what can your current team accomplish versus what’s the request?

And then if it’s something where your current team can’t make it work, see what data analysis you can do. See what efficiencies you can offer or what can be brought to the table, and kind of think about those things in planning a secondary strategy call. You know, at the outset of every project, you’re going to have that scoping or planning call where you talk about the volume, the deadline, the workflow.

Well, now you need another one. Things have changed, and you need to get aligned. And everybody needs to understand the current situation versus the additional request. And on that call, be prepared to talk about strategy. It’s going to have to change. It’s going to have to evolve just as the scope did. You know, offer solutions, even if it’s a simplified workflow.

You know, “Hey, we got twice the number of documents that we had, but one week less. We need to just do responsive and privileged tagging. We can go back and do issue tagging later. That’s what’s really slowing the team down.” Or, you know, “We can tag for needs redaction, go back and do those documents secondarily and ask for an extension for redaction documents.”

It’s having those discussions with the counsel or client and really getting an understanding of how you can still be efficient and effective in getting what absolutely needs to be done in that right amount of time.

13:01 – Michael Gibeault

Jenn, where does human expertise still matter most, even with AI in the mix?

13:08 – Jennifer Andres

So, there should always be a human validation component. You know, we’re seeing with the courts these days that GenAI is not “out of the box” ready to go. You should have this human component going in and validating, QCing, refining.

Really making sure that the AI is working as intended, and where it’s not, prompting, and rerunning, and calibrating so that you can get it to the point where it’s working as expected. And then you’re running those targeted searches in case it’s, you know, misunderstanding some things here and there.

Anything that you’re finding that it’s maybe off a little bit, it should be more of a QC on a corrected set of criteria and then using the human expertise for that.

13:51 – Michael Gibeault

Jenn, thanks so much for joining us today and sharing your perspective. One of the biggest takeaways for me is that scope changes themselves aren’t the problem, it’s how teams respond to them. Taking a step back to reassess the capacity in the technology can make a huge difference between a project that spirals and one that stays on track.

And I think your point really stands out: You can’t control how scope evolves, but you can be ready for it.

14: 21 – Jennifer Andres

Absolutely.

14:22 – Michael Gibeault

If you’d like to keep up with what’s next at TCDI, visit tcdi.com or connect with us on LinkedIn. Thanks again for joining us, and we’ll see you next time on TCDI talks.