The data archival challenge

A variety of new, cost-effective technologies help E&P companies improve productivity, accelerate data recovery, and make better-informed decisions with the deluge of seismic information.

“Advanced disk-based solutions can enable better-informed decision making using the latest data all in one place.”
-Ashraf Atia NetApp Middle East and Africa Energy Industry Director

Seismic information continues to grow at staggering rates in petroleum and gas exploration and production (E&P). This digital tidal wave includes new pre- and post-stack volumes, as well as derivative attribute volumes. The challenge is magnified by requirements to support higher-fidelity subsurface imaging and enhanced interpretation and visualization technologies that can utilize increasingly larger datasets.

Data Archiving Challenges

Data managers in information-intensive scientific industries like this one understand the challenges that their rapidly growing data stores pose. Poor access to the high-quality, complete data they need to make informed decisions can cost millions of dollars. Many E&P geoscientists spend far too much of their time identifying, locating, and obtaining data they need for their mission-critical analyses.

Every dollar saved on archiving this data for seismic storage, processing, and interpretation-critical yet passive requirements-could be redirected to proactive computing. The latter includes improving data access for seismic processing and interpretation application for better reservoir characterization. This can facilitate real-time well operations modeling or global collaboration, which ultimately helps reduce risk and decrease cycle times.

The Legacy of Tape-Based Archiving

Tape technology has long been the data archival mainstay for E&P companies and will continue to be a requisite part of the storage hierarchy for many organizations for years to come, simply because of the vast amount of legacy data. Yet today, some forward-thinking companies are looking at active archiving (near-line or online archiving). They are evolving toward disk-based archiving solutions that can address the growing data management challenges associated with tape-only solutions.

At the core of this trend is the inability of tape to effectively fulfill the active archiving role and the slow pace of data location, access, and recovery. (Active archives are those established for storage management reasons and that may be needed back online within one to two years.) In addition to these limitations, the tape medium cannot ensure integrity of data over time and requires periodic tape remastering at high cost, unless a robust archiving solution is in play that automates the latter.

Figure 1) Storage capacity saved using digital storage with lossless compression technology.

Technology Enablers

"We now recommend and implement a variety of technologies that enable disk-based storage to complement or, in some instances, replace tape-based archiving in E&P applications," explains NetApp Middle East and Africa Energy Industry Director Ashraf Atia. For example, data compression technology for disk storage has come a long way. Today, lossless compression solutions significantly reduce the storage requirements for both pre- and post-stack seismic data, without introducing data integrity issues (see Figure 1).

A second example is data deduplication, which reduces the amount of data required to be physically stored by eliminating redundant information and replacing subsequent iterations of it with a pointer to the original (see Figure 2). A prime candidate for deduplication is "new project files" that copy files from existing projects.

A third example involves metadata. Over the lifetime of a typical archive, knowledge of who interpreted the project, the location of the project, as well as the important horizons, faults, and well information that help define the prospect fade over time, and may be lost. Archiving systems now exist that allow collection and use of this metadata. 

Disk technology is a fourth area of advancement. A RAID 6 implementation prevents data loss when two disk drives fail. This enables use of drives that provide economical storage density, such as low-cost SATA (Serial ATA) disks, for seismic data storage, while also providing the needed data protection. SATA drive capacities are expected to increase rapidly in the future. In addition, the disk storage system actively reports disk failure. No such technology exists to report a bad data block on tape as it sits in storage.

Data-class storage management is another evolving area. Traditionally, E&P companies have archived data according to the type of data (for example, field seismic data, pre-stack data, post-stack data, well logs, and so on). A more useful way to archive data is to use a tiered storage solution in which data is classified and archived according to the need for the data or patterns of its usage.

Recent cost analyses between online storage and near-line storage technologies, such as tape-based systems, show that online storage can be more cost effective than tape-based solutions over the lifetime of the seismic data. As technological advances in disk storage densities increase and effective deduplication and compression technologies become available, these new online disk-based solutions are becoming more cost effective.

Figure 2) Deduplication complements compression to reduce storage requirements.

Potential Business Benefits

"Advanced disk-based solutions can enable better-informed decision making using the latest data all in one place," explains Atia. They can enable E&P users to identify, locate, and retrieve data quickly, accelerating mission-critical decision making, reducing field costs, and improving utilization of field assets. At the same time, the solutions can improve geophysicist productivity due to easier access to data anywhere and anytime. The same technologies can also accelerate data recovery in the event of data loss, corruption, or disaster.

Advanced disk-based solutions can enable more efficient completion of asset sales, including transfer of digital assets. They can reduce the cost of transcription and remastering of tapes due to the transfer of some archiving to disk. This can also enable "platform protection," avoiding obsolescence of data formats, systems, operating systems, and applications. This benefit, as well as improved data integrity, can reduce information technology costs.

Looking forward, the solutions can provide flexibility and scalability to meet continued data growth and analysis needs. They provide an outsourcing option that offers choice for firms seeking to outsource data archiving functions.

NetApp, the NetApp logo, and Go further, faster are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries.