0:05 – Michael Gibeault

Hi, I’m Michael Gibeault with TCDI and I’m your host of TCDI Talks. And this is Episode 12, “Human-In-The-Loop: People-Centric AI,” with my colleague Caragh Landry. She’s our Chief Legal Process Officer. Welcome, Caragh.

0:21 – Caragh Landry

Hi, Michael.

0:23 – Michael Gibeault

So, Caragh, you wrote this amazing article, about people-centric AI, and why the human-in-the-loop matters in eDiscovery. So in your article, it puts a strong emphasis on being people-centric. In the age of AI, why has that been such a foundational principle at TCDI?

0:45 – Caragh Landry

I mean, it’s a foundational issue for TCDI for many reasons. It’s the same way we’ve always focused on process, not just technology. You know, AI is great and it’s changing things at lightning speed. And we’re all getting used to it and all learning what it can and can’t do. But it’s not about the technology.

It’s about the process of solving the problems for our clients. So we talk human-in-the-loop. There is no loop without the humans, right? Like machines and computer learning are making things faster and more consistent for us. But we’re still dictating how it works. We’re dictating the workflow. We’re putting, you know, AI where it needs to be, where it’s going to have the greatest impact.

So humans-in-the-loop, you know, we’re dictating the process. We’re validating the results. We’re giving it a nudge in the direction that we want it to go. And without that nudge and without that validation, you’re not going to get what you want. And that’s just, you know, it’s a huge focus at TCDI to make sure that clients understand that and that they understand that they’re law firms still have a really important role to play when it comes to, you know, legal and ethical concerns and decisions and process and workflow.

2:04 – Michael Gibeault

So what are some of the most critical touch points in the process-driven AI workflow, where human oversight makes the biggest impact?

2:14 – Caragh Landry

So we think about it the same way we think about our normal processes, which is, you know, use the technology, again, where it can improve on speed and accuracy and consistency. But where humans still have the most input is when it comes to context and when it comes to direction and decision making.

Right? So if you think about a non-AI process for document review, right? You’ve got council making review guidelines that explain what it is that you’re looking for. You then have reviewers getting initial training and then going in and doing document review. Well, reviewers are only going to be as good as they understand the protocol and the direction and guidance to be.

They need direction throughout the entire review process, right? That’s supervision of counsel. AI works the same way. And these AI workflows that we’re talking about, the biggest impact is after you tell AI what you want it to do is going in and deciding, is it getting it right, or does it need a push in this direction or push in that direction?

What’s it not getting right? And that’s, I mean, that’s all about context and that’s legal decision making. And the computers, the machines, the AI, the LLMs, they’re going to only get it so right. And they need that validation, that guidance and that push from the humans and that’s where I think we see that human-in-the-loop has the biggest role is after initial training, is then guiding it to the answers that you’re looking for.

Does that make sense?

3:58 – Michael Gibeault

Absolutely.

3:59 – Michael Gibeault

And in your article you mentioned TCDI’s AI-powered SMART Data processing and document review. How exactly does the collaboration between AI and human reviewers play out at TCDI?

4:15 – Caragh Landry

So one of the areas that we found we struggle with, humans struggle with the most, is bad text. We can only do so much with technology if the text is bad.

So you’re talking about images. We’re talking about handwriting. We’re talking about really bad scans. Like a lot of our clients, their data goes back decades, right? I mean, you think back to, I guess I should actually say even more than decades at this point, right? You know, I always think 20 years ago was the 80s, and it’s not 20 years ago, it’s the 2000s, right? But we see a lot of documents that are from the 70s and the 80s and the 90s, and they’re copies and copies of scans of old documents. So the text just is not readable. And when we talk about SMART Data, that’s really an area that we’ve focused on in the last couple of years, in addition to, you know, autonomous review and summarization and investigation, you know, because AI is really good at that.

And that’s what a lot of people are focusing on. But the area that we’re focusing on now is getting the data or the text to be better, so that all of these AI tools can actually use the text. So we’re talking, images, right? Summaries of what the images are, descriptions of what the images are. You know, we’ve got a person playing basketball and it’s an image. You know, tell me it’s a person playing basketball so that when I search for the term “basketball,” that document comes back, when historically on an image, nothing would come back.

Same thing with, you know, poor scans, like doing smarter OCR or better OCR, makes all of our searches work. It makes them work better, and it makes AI able to then contextually give us more insight into the documents. Foreign language is another area that we always struggle with. You know, we have to bring in foreign language reviewers, which luckily at TCDI, we have a nice, stable of foreign language reviewers.

But that’s still time consuming and it’s timely. And now that we have all these AI tools, we want to be able to use them. So AI is doing really neat things with changing the idea of translation. You can obviously use translation, and AI powered translation is much better than machine translation historically has been. Like, it’s getting better.

It’s focusing, not just on the words, but on the context and the meaning. So it used to be that you do machine learning, or you do machine translation, and you’d get very stunted text. Now you’re getting spoken text. You’re getting, you know, English versions of the way people actually talk, which is really interesting. But AI is taking it even further where now with these tools and our SMART translation tools and our SMART technology, you don’t even need to translate documents.

You can ask them in English, it’ll read them in a foreign language, and then it will return your results in English. So, as an English-speaking reviewer, I don’t need to speak Spanish anymore. I don’t need to speak Chinese or Korean in order to find the documents that are important, or in order to get summaries or understand what the documents are about.

That might come later. Like I might need that for court. I might need that for depositions. But really, especially in investigations, to get an idea of the documents, being able to skip that step completely. It’s really changing the way that our reviewers are able to interact with documents and the information we’re able to get back to our clients.

So we call it SMART Data because it’s making us smarter and it’s making that document smarter, right? An image is a dumb document. A foreign language document to an only English-speaking reviewer is a dumb document. And anything we can do to make them smarter, usable, is really, you know, expediting the whole process for our clients and driving down costs at the same time.

8:11 – Michael Gibeault

Well, along those same lines, Caragh, can you share with us a success story or example where TCDI’s human-in-the-loop approach helped prevent a critical error or improved a case outcome?

8:25 – Caragh Landry

You know, it’s funny, we get that, I get that question a lot from clients, who want to know about our tools.

And my answers are always the same, it’s “When doesn’t it help?” right? There’s every example. Any time we’ve used AI in a project, it’s been better than not using AI. And we used to say the same thing about TAR, right? Like when TAR first came out in 2010, you know, when people started using in 2010, 2012, people were afraid of it.

They didn’t understand it. They didn’t get how it could help. And the response from, you know, from me back then was, “Well, just give it a try. And if it doesn’t work, we’ll go back to human, right?” The worst-case scenario is all human review. So, same thing with AI. Every project that we use AI in, every time we implement a GenAI workflow, it’s better. And it’s better because the partnership that we’re able to create with the AI tools gives us faster insights. It gives us more information. It makes us better at what we’re doing. And the consistency, I mean, it’s amazing. So, actual examples would be PII redaction. It’s a no-brainer. You know that consistency on PII redactions is so important.

And using these tools it’s always going to find the data. It’s always going to it’s not going to miss one. Like you know Social Security number a human could miss it. Technology is not going to miss it because it’s a pattern that it is designed to find.

Same thing with data breach review. Again, it’s looking for personal information. That’s a great example, right? You often have 15, 20, 30 days to actually do the project and notify people that there’s been a breach. It’s really hard for humans to do; it is not hard for AI to do. So those are two of our biggest success stories. But we’ve had success stories on the document review side.

Using it for QC where it finds all sorts of errors, right? Humans are error-prone. And really it’s across consistency, right? You see the same type of document 15 times. It’s really hard to code it exactly the same 15 times if you’re a human review team. And we put lots of you know, belts and suspenders in place.

We think our quality is really good from a human perspective. But it just, AI makes us so much better. And the success story is it helps us code those 15 documents exactly the same. And sometimes, even if you’re wrong on the 15 documents, if you’re consistent wrong, it’s more explainable and defensible than if just sometimes you’re wrong.

So AI really helps us do a good job in every scenario. I have yet to come across a project where I wouldn’t say it’s better. And I think that’s only going to, I think that’s only going to grow.

11:21 – Michael Gibeault

What makes TCDI’s approach to AI and eDiscovery different from the other legal tech providers out there, Caragh?

11:29 – Caragh Landry

That’s a good one, too. I would say that we are very cautious about where we use AI, where maybe everybody else is just throwing AI at solutions or places where maybe it doesn’t fit.

And that’s not to bash, you know, other people in the market, but AI is not a one size fits all. And I feel like there’s a lot of people saying, “oh, well, AI can do that,” or “let’s just use AI for that.” We don’t do that at TCDI. We start with the process, right? And we’ve had process for years, right?

The process doesn’t have to change because there’s new technology. It can change and it should evolve with the advent of new technology, but the basic process of how eDiscovery works hasn’t changed in the 29 years I’ve been working in eDiscovery, right? It’s gotten better. It’s gotten more streamlined. It’s definitely gotten cheaper. It’s not, you know, $3,600 a gig to process data anymore, right?

It’s not $12.50 to review a document. And technology, AI is making, you know, it’s just taking that further. So where I think we’re different is that we start with process and then we add technology, various technology, all sorts of technology where in the process it’s going to have the biggest impact. And I think from a TCDI perspective, that we have a lot of pride in that, we’ve found lots of technology that works. We’ve found not only how it works, but how it can work. And we’ve worked with our partners and with our internal development team to improve upon what technology can do. And then we put it in the places where it’s going to have the most impact, but we don’t put it everywhere and we don’t use the same technology. In most of our workflows today,we are using legacy AI. We’re using non-AI, and we’re using GenAI in a single process at various points for different purposes. Non-AI tools do really good, you know, do really well at searching. They do really well at standardized searching. Legacy AI just really does a really good job at finding patterns, at being consistent in finding those patterns.

So there’s opportunities for both of those. GenAI, I mean, it’s doing a great job of finding new patterns and being consistent in new ways. And summarization is a massive boon when it comes to AI. And that’s not something we’ve ever had before. So if you think about depositions, you can use non-AI to find custodial data.

You can use legacy AI to find similar documents to ones that you do like. And then you can use GenAI to put it all together and tell you, “What did this custodian say? What was their involvement? What questions should I ask them?” And that’s three, those are three different technologies that we employ on every, you know, every time we’re doing depo prep.

And so I think that’s unique, a unique way that TCDI is approaching AI as opposed to some of our peers out in the market.

14:52 – Michael Gibeault

Caragh, thanks for joining us today on TCDI Talks. For our viewers, if you’d like to learn more about TCDI’s human-in-the-loop approach to AI, check out Caragh’s article.

We’ll have it linked next to this video. Thanks for joining us.

15:08 – Caragh Landry

Thanks, Michael!