Big analog data and data acquisition

May 14, 2013 12:44 pm

An outlook by National Instruments to highlight some of the most pressing trends and challenges engineers building data acquisition systems face
Differentiation is no longer about who can collect the most data; it’s about who can quickly make sense of the data they collect. There was once a time when hardware sampling rates, limited by the speed at which analog-to-digital (A/D) conversion took place, physically restricted how much data was acquired. But today, hardware vendors have accelerated data collection rates so quickly and enabled engineers and scientists to break through rate and resolution barriers so rapidly that they have ultimately triggered a new wave of data consequences. Simply put, hardware is no longer the limiting factor in acquisition applications; the management of acquired data is the challenge of the future.
Advancements in computing technology — including increasing microprocessor speed and hard drive storage capacity — combined with decreasing costs for hardware, and software have provoked an explosion of data coming in at a blistering pace. In test and measurement applications in particular, engineers and scientists can collect vast amounts of data every second. For every second that the Large Hadron Collider at CERN runs an experiment, the instrument generates 40 terabytes of data. For every 30 minutes that a Boeing jet engine runs, the system creates 10 terabytes of operations information. For a single journey across the Atlantic Ocean, a four-engine jumbo jet can create 640 terabytes of data. Multiply that by the more than 25,000 flights flown each day, and you can understand the enormous amount of data being generated.
The technology research firm IDC recently performed a study on digital data, which includes the world’s measurement files, videos, music files and so on. This study estimates that the amount of data available is doubling every 2 years. The fact that data production is doubling every 2 years mimics one of electronics’ most famous Moore’s law. In 1965, Gordon Moore stated that the number of transistors on an IC doubled approximately every 2 years, and he expected the trend to continue for at least 10 years. Forty-eight years later, Moore’s law still influences many aspects of IT and electronics. If the production of digital data continues to mimic Moore’s law, success as an organisation will hinge on the speed at which acquired data can be turned into useful knowledge.
The big data phenomenon adds new challenges to data analysis, search, integration, reporting, and system maintenance that must be met to keep pace with the exponential growth of data. The sources of data are many. However, among the most interesting to the engineer and scientist is data derived from the physical world. This is analog data that is captured and digitised; thus, it can be called “Big Analog Data”. It is collected from measurements of vibration, RF signals, temperature, pressure, sound, image, light, magnetism, voltage and so on. Challenges unique to “Big Analog Data” have provoked three technology trends in data acquisition.
Contextual data miningThe physical characteristics of some real-world phenomena prevent information from being gleaned unless acquisition rates are high enough, which makes small data sets an impossibility. When the characteristics of the measured phenomena allow more information gathering, small data sets often limit the accuracy of conclusions and predictions in the first place. Consider a gold mine where only 20 per cent of gold is visible; the remaining 80 per cent is in the dirt where you can’t see it. Mining is required to realise the full value of the contents of the mine. This leads to the term “digital dirt,” meaning digitised data can have concealed value. Hence, data analytics and data mining are required to achieve new insights that have never before been seen.

Advertising

Your future advertising space? Our media data