Large oil companies such as Exxon, Chevron, Shell and BP turned up huge discoveries offshore in ultra-deep water with the help of better sensors and faster computers that allowed them to see once-hidden oil deposits.
Onshore, small drillers learned how to pull oil and gas out of previously inaccessible underground rock formations.
For most of the oil age, drillers have looked for large underground pools of oil and gas that were easy to tap. These pools had grown over millions of years as oil and gas oozed out of what is known as source rock. Source rock is a wide, thin layer of sedimentary rock — like frosting in the middle of a layer cake — that is interspersed with oil and gas.
An engineer named George Mitchell and his company, Mitchell Energy, spent years searching for a way to free natural gas from this source rock. He finally succeeded when he figured how to drill horizontally, into and then along a layer of source rock. That allowed him to access the gas throughout a layer of source rock with a single well. Then he used a process known as hydraulic fracturing, or "fracking" to create tiny cracks in the rock that would allow natural gas to flow into and up the well.
The United States, which was facing a gas shortage five years ago, now has such enormous supplies it is looking to export the fuel in large volumes for the first time.
The common wisdom in the industry was that the process Mitchell had invented for natural gas wouldn't work for oil. Oil molecules are bigger and stickier than gas molecules, so petroleum engineers believed it would be impossible to get them to flow from source rock, even if the rock were cracked by fracking. But Mark Papa, the CEO of a small oil and gas company called EOG Resources, didn't accept that.
"The numbers were too intriguing, the prize was so big," he remembered.
He thought there could be as much as a billion barrels of oil within reach in Texas, North Dakota and elsewhere — if only he could squeeze it out.
In 2003, he had a "eureka!" moment while poring over pictures of rock. Sections of a 40-foot-long column of source rock had been run through a CT scanner, the same type used to peer into the human body.
He saw something in the source rock sections the rest of the industry didn't know was there: a network of passageways big enough for oil molecules to pass through. Papa believed the passageways could act like rural roads for the oil to travel through. Fracking could then create superhighways for the oil to gather and feed into a pipe and up to the surface.
EOG began drilling test wells, and in 2005, Papa got some results from one in North Dakota that made him realize oil could flow fast enough to pay off.
"It was kind of like holy cow," he says. "My first thought was we need to replicate this, make sure it's not a freak result."
It wasn't. EOG snapped up land in a similar formation in South Texas known as the Eagle Ford Shale for $400 an acre when his competition thought it would never produce much oil. That land now goes for $30,000 per acre.
Papa thought the Eagle Ford might hold 500,000 barrels of oil. The Department of Energy now predicts it holds 3.4 billion. Some even expect 10 billion, which would make it the biggest oil field in U.S. history.
SMART DRILLS, RIGS THAT CAN WALK
But even after drillers figured out how to find oil and gas deep offshore and in onshore source rock, they still needed to develop technology that would make it economical.
At the tip of every oil or gas drill is a rotating mouth of sharp teeth that chews through rock. In the past, these drill bits could only dig straight down. Now they are agile enough to find and follow narrow horizontal seams of rock.
The drilling-services company Baker Hughes has designed a bit that can change directions underground, without having to be drawn back up to the surface, reducing drilling time by as much as 40 percent.