Blind Tests Reveal High Accuracy of
Data Science & Analytics

Blind Tests Reveal High Accuracy of

Rosemary Jackson  •  

Q: How can a client company be certain of the accuracy of the algorithms used by

A: “We use blind tests as a part of every process with every company,” Richard Gaut, CFO of answers with a quick confidence. “The client chooses the analog pads, meaning the areas that are most geologically similar to what they’re going to be developing. And the most similar from an engineering standpoint to how they’re going to develop that pad, meaning 2 benches or 3 benches or some other spacing. You find the most similar pad or most similar DSU in your basin and then we run blind tests on that.

If we’re able to blindly predict performance on the most similar assets then that gives people confidence that we’ll also be able to blindly predict where their drill rig is going next.

“Right,” Dr. Troy Ruths, Founder and CEO of adds. "This isn’t your typical blind test where you throw the data over and we throw the response back. We go through a rigorous and heavy technical workflow to constrain and input all of the information into the system.

“And we do this in such a way that I think every single customer that’s gone through the process has a moment where they say two things. There are two moments that I consistently see.

The first is, “Okay, that’s what I would have done.” People try to chase insights rather than certainty with AI. When someone becomes aware that has just done what they would have, there’s a realizationthey’ve finally found something in the industry that they can rely on.

The second is, “How did it know that?” And the system knows because it has collected all of the information, all of the data and synthesized it together. As humans we lose track of all the information that’s gone into the models. And because as humans we’re biased towards our most recent information intake, it makes us unreliable for remembering critical assumptions and information from a long time ago. It’s just the application, the rigorous application, the reliable application of all your data into a system of tested AI algorithms that makes it so certain.

“Those two moments are when people start having an emotional connection with the process. Certainty is an emotional understanding beyond the quantitative. We can talk a lot about accuracy numbers but at the end of the day accuracy is not an intuitive process.

Blind Pad Prediction: Permian

“For instance, I was watching the NBA and Steph Curry who is the best shooter in possibly the history of the NBA with a 90.7% free throw shooting average. If you came to me and said, Troy, do you want to put some money on Steph Curry making a shot? I’d say, oh yes, I will. And that’s at 91%,’s accuracy is consistently higher than that.

“If we’re talking about the accuracy of a blind prediction, we’re shooting free throws better than Steph Curry. It’s like watching Steph Curry shoot. When I see his arm motion, that’s how I would shoot. Then you see him make shots and you say, “How’d he make that?”He’s just doing the same motion every single time. He’s a very reliable shooter. He has high accuracy.

“I think it’s those two moments. There are easy parallels there. Our shooting motion is our process by which we constrain the system and we build trust in it. And then people are really surprised with the results. “How did do that? How did that go in?

”All of your knowledge goes into and gives you back results with certainty.

“I think that’s what makes certainty so interesting. Certainty isn’t quantifiable. It’s an emotional journey everyone has to go on. And there need to be these aha moments. First where a client says, that’s what I would do and second, how did they figure that out? As an industry, we’ve been scratching our heads about this and are you telling me we had this solution the whole time. That’s the way our clients feel, “You mean we had this solution the whole time? We just didn’t put it all together?”

“And the answer is yea. You are sitting on all the information you need. It’s just so big and so complex you need something to put it all together for you and be a reliable turn of the crank. Or a practiced free throw that you can bet your money or your pad on.”

114 Main Street Ste. 400
Houston, TX 77002
4-5609 Avenue du Parc
Montreal QC, H2V 4S8