Two Terabytes

Two Terabytes in Two Weeks

By Susan Bright

The Challenge 

Deadlines and short time-frames are a fact of life in complex litigation. TCDI understands that, so we set up processes and procedures in preparation for almost anything. 

That "anything" happened in January a year ago when we received a phone call on Thursday from an attorney we have worked with for more than 15 years, asking if we were in a position to process a terabyte of data with review set to start that Monday.

He then told us their normal vendor had, understandably, told him it was impossible. However, court deadlines made it imperative that it happen, so we took on the challenge and assured him we would commit the resources necessary and would deliver. On Friday, "anything" began to happen.

Clearly, the volumes at hand and the timing imposed were daunting. However, with our technology and the drive of our people, we were confident that no one in the country could meet this challenge better than we would.

A Call to Action

The short time-frame for processing and loading this large volume of data for review was our primary challenge. Plus, when it arrived, the one terabyte of data we were expecting turned out to be TWO terabytes.

Within the prior year, we completed initial development of a processing environment which is controlled by our proprietary processing management product Discovery WorkFlow®. In addition to controlling the various standard and advanced processing applications, logging the processing actions on each data object and monitoring multiple jobs, Discovery WorkFlow also distributes the processing work across a large network of processing servers (soldiers). Electronically stored information (ESI) processing is largely CPU-dependent―-if you need something sooner, you put more soldiers on the job. The terabytes of data and strict time-frames involved in this particular project necessitated a great many additional "soldiers."

On Friday morning, a call was placed to our hardware supplier to order the added hardware we needed for expanding our processing server pool. After much discussion regarding the urgency involved, they were able to supply our needs very quickly. The additional soldiers needed were quickly put in place and, at our processing-peak, we had 120 soldiers working 24 hours per day.

With the deadline looming, staff from our Client Data Services, Project Management and Networking teams worked the shifts necessary to ensure we maintained peak operation 24 hours per day throughout the project. To augment communication and expedite decision making, we also hosted an attorney from the client firm onsite at TCDI.

On Monday, the review began on schedule. TCDI was able to train reviewers on ClarVergence®, our proprietary review and production application designed to efficiently and securely manage large data collections. To ensure a smooth start-up and transition, we also placed one of our project people onsite at the firm's Chicago location for three weeks. This TCDI resource was able to assist with user needs and support the firm's review administrators in creating and assigning review sets and generating reports.

Over the course of about 10 days, culling reduced the data volume from 2 to 1.2 terabytes, at which point it was fully processed and loaded for document review.

Productions began after the first week of review and, all things considered, ran relatively smoothly with close coordination between our production group and outside counsel. Within four weeks, the last of approximately 20 productions – totaling close to 80,000 documents – was made and the teams of people involved were finally able to breathe.

A Look Back

I would love to say that this entire project went smoothly. Realistically, projects this big, moving this fast always have issues. Due to the tireless efforts and teamwork on all sides of the project, however, no problems were insurmountable.

Friedrich Nietzsche's philosophy, "that which does not destroy me, makes me stronger," seems applicable to this project. From a technology perspective, TCDI was in a position to handle projects of this nature, but the magnitude of the requirements pushed us even further to build a more robust processing environment. Additionally, we've managed long hours and tight deadlines before, but this project served as a reminder of TCDI's commitment to achieving its clients' goals. We've become known for that, and we like it that way.