Neelie Kroes said that some barriers currently prevent the maximum benefit from being derived from the masses of research data generated within the EU but said that the public and private sectors could "work together" to ensure "economies of scale" could be achieved.
"Put the data together, and the value of the whole is far more than the sum of its parts," Kroes said in a speech in Lithuania. "That's why we need economies of scale. Essential for analysing for meaningful, valuable outcomes. And essential if we are to compete globally. This kind of work needs huge processing power. But the highest performance computers are unaffordable by any member state, even the largest, acting alone. A piecemeal approach would be inefficient, and subscale. And as it stands, too many companies and public services don't fully benefit, from their own data or from others'. That's why we need to work together. And enjoy those economies of scale."
Language barriers, national borders, sector divides and differing legal frameworks and policies within EU member states mean there is no "coherent data ecosystem" within the EU currently, Kroes said.
The fact that a number of different private and public bodies are producing data sets in accordance with different standards, affecting "interoperability", and that the data is sometimes difficult to locate, adds to the "highly fragmented" nature of research data within the trading bloc, she added.
Kroes said that plans to ensure more public data is published, under new EU legislation, as well as the creation of a new "pan-European open data portal" would help improve access to information but called for a new 'big data' partnership to be formed between the public and private sectors.
"A European public private partnership in big data could unite all the players who matter," the Commissioner said. "Researchers, software makers, data intensive sectors, venture capitalists, and more. It could build on the work we are doing in cloud and high performance computing, on our legislation to open up public administrations, and on our action to promote open science and boost trust in data handling. It could steer financial resources: European, national, private.
Most of all, it could support new research and innovation, stimulate demand for new products and services, and cement Europe as a global player. Energising our economy, supporting our society, and generating jobs."
Using big data does not necessarily mean using personal information, but where it does there should be privacy safeguards in place, she said.
"For data that does concern people, we need firm and modern data protection rules that safeguard this fundamental right," Kroes said. "And we need digital tools to help people take control of their data, so that they know they can be confident to trust this technology. Then we have a virtuous circle, where technological progress, our legal framework, and our fundamental rights mutually support each other."
'Big data' is a term used to describe the vast generation of data and the possibilities presented through analysing the information through computers.
Online payment services giant PayPal recently said that big data could be used by regulators to combat fraud, whilst security software company McAfee has said big data could be used to improve data breach monitoring.
Visa also released a report in April which said that big data technology could be used to recognise behavioural patterns and the information used to verify the identity of individuals and sanction digital payments.
Earlier this autumn UK think tank Policy Exchange called on the Government to run procurements in a bid to find private sector companies able to utilise 'big data'. It said that the outsourcing contracts should only be agreed on a "payment-by-results basis" in order to help the Government avoid having to "invest significantly in big data technologies on the promise of uncertain future benefits".
