Every day, apps track our movements, shops record our purchases, and websites register our online activity. We generate billions of data points – but these numbers are meaningless without analysis; it takes powerful computers and clever processing techniques to make sense of huge data flows.
This “big data” revolution is as profound as any of its predecessors, making – our lives increasingly “readable” in remarkable detail. It is also transforming how we do science, form policies, run businesses and provide services like education and healthcare.

Of course, we all have concerns about how pervasive deep data analysis has become. How does it affect our liberty and privacy? The region’s scientists are considering these issues,, but Yorkshire must not lose sight of the enormous scientific, social and economic opportunities this data revolution will create.

David Willetts, Minister for Universities and Science, sees “big data” as one of the “Great Eight Technologies” critical to innovation-led UK growth. On a national scale, it could create 58,000 British jobs and boost the economy by £216 billion by 2017. According to Deloitte, public sector data alone is already worth £6.8 million a year.

And, in the context of Chancellor George Osborne’s call two weeks ago for a northern economic powerhouse to rival London. Big data is one of the fields in which the north, and specifically Yorkshire, is in a position to take on not only London, but the world.

Yorkshire is home to a digital and creative sector worth £3 billion with leading universities in this field: we are in a powerful position. In February, Willetts announced an extra £73 million to bolster the UK’s big data research activity. Research at Yorkshire’s universities accounted for over 15 percent of this figure.

The University of Leeds is combining two major projects in an £11-million Leeds Institute for Data Analytics (LIDA), due to open later this year.

One project will develop new ways to mine digital health records from consenting patients to fight disease. Scientists are already making progress: they have used data from past patients to inform a new blood cancer treatment and new ways of monitoring kidney transplants. The project will also develop crucial big data skills among researchers and clinicians.

LIDA will also host a national Consumer Data Research Centre, a new secure facility for research in the field. A new Master’s course is designed to address skill shortages in big data analysis.

Analysing consumer retail data provides fascinating insights, but social networks are another great “big data” source. Millions of tweets, status updates and Facebook ‘likes’ reveal a lot about what people think and how they behave.

grid heart simulationBut not everything you read online is reliable and rumours can spread rapidly. To search for the truth, researchers from the University of Sheffield are developing an automatic lie detector. The system will allow governments, emergency services, journalists and private sector companies to respond to rumours and allegations more rapidly.

Big data is a key focus of Sheffield’s Information School, the leading school of its kind in the UK. Its research pioneers the use of big data in journalism and citizen engagement.

It is important to remember, however, that not all big data is about people. At the University of York, atmospheric researchers generate huge datasets as they test air samples for organic molecules. An airborne aerosol may contain 10,000 different compounds so the data quickly adds up.

York has received funding to create new tools to store, visualise and analyse their atmospheric sampling data. The new infrastructure will combine massive datasets with state of the art tools to create an archive for virtual atmospheric chemistry. Here, big data may help answer one of our biggest questions: how can we protect our planet?

All these projects show that meeting the challenge of big data requires more than powering up some fancy computers. This information revolution, like those that went before it, will be built on skills and connections to drive innovation. In Yorkshire, all the components are in place: a thriving digital business sector, pioneering projects, plus cutting-edge research and training in our universities. Big data is poised to become big business.