In an Expert Focus article for Waterbriefing, Oliver Grievson, from Anglian Water, the Director of the Sensors for Water Industry Group and Group Manager Water Industry Process Automation & Control on LinkedIn explores whether “Big Data” can provide the answer to the water industry’s data crisis.

Oliver Grievson: The water industry collects a lot of data, not as much as some industries and more than others. This is a fact. An arguable figure that I use is somewhere in the region of 300 million pieces of operational data every day. If we look forward into the future where there is a potential for a much more data hungry water industry then it has been predicted that the water industry is in for a tough time. Big Data has been promoted as a solution to this problem.

The extent of the potential problem

Does the industry really face a problem? What amount of data are we actually talking about?

Operationally -300 million pieces of data per day which is mainly single numbers i.e. what is actually being measured whether it is flow, chemical concentration or something else entirely different. This generally doesn’t include the data that modern instruments produce that tells companies about sensor health. Being conservative this can multiply the amount of data between 10 times. The amount of data operationally becomes 3 billion pieces a day.

Customer usage data – If the UK goes to a point where the entire water industry is universally metered with Smart Metering there will be approximately 25 million water meters, for customers, in the UK. If the customer data is picked up at an hourly resolution in line with comments made by a water industry metering manager this is another 600 million pieces of daily data.

So is Big Data a potential answer to this problem?

Big Data – what is it all about?

Taking the Wikipedia definition of “Big Data”

“Big data is broad term for data sets so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, and information privacy. The term often refers simply to the use of predictive analytics or other certain advanced methods to extract value from data, and seldom to a particular size of data set.”

 It has a history of being used in a number of different industries from retail with companies such as Walmart which deals with 1 million customer transactions every hour to the Large Hadron Collider which records more data in a second than the global water industry records in an entire year.

It seems that “Big Data” has the potential to meet the water industry’s needs - or at least some of them.

What are the options for the water industry?

Taking the operational issue alone the water industry does have coping mechanisms for the amount of data that it collects. At its worst, at the current time, it collects data, stores it and normally when required retrieves it and manually analyses it. Not the most efficient way of doing things and a lot of the value of the data that is collected is lost. A step further on large treatment works is the data is used for automated control purposes and is used to give a more efficient mode of operation. What tends not to be done is the automated presentation of this data to give easy access to information about how the plant or system is running.

To look at an example of this – a large works which typically has 250 instruments and sensors record everything from flows, concentrations, valve position and levels on a fifteen minute basis, the eventual daily data set is in the region of 24,000 pieces of data a day. The question to ask is what information does the operator of that works actually need to see in order to do his job. The works has six primary tanks which each have a flow meter and a dry solids meter on the sludge line, these meters record 1,152 pieces of data a day. What the operator actually needs to see is one or maybe two pieces of information. The data to information ratio in this case is 572 to 1. This reduces the amount of information that is derived from the data to 42 numbers, a lot more manageable.

Another example is regulated flow monitoring on wastewater treatment works. At the current time there are approximately 3,500 installations across England & Wales. These flow monitors, under the MCERTS programme, record 123 million pieces of data every year. For regulatory purposes only the actually pieces of information that is required is 1 piece per site giving a data to information ratio of 35,000 to 1. Not to say that the 15 minute data is not useful, it is, but this does not fall into the area that “Big Data” would provide any further use than what is already done. 

This is what has been termed as, Small Information, a buzz word for what the industry has been doing, in some case well and in others not so well, for years. The key is to identify the information that is required from the different stakeholders in the business from managing director to operator and structure the data and information management system accordingly. This is normally the remit of the system integrators and the information system specialists and more about the organisation of the data to get the information that is necessary to inform the business.

Big Data - there are areas within the water industry where it is a natural fit

So where does this leave “Big Data?” There are areas within the water industry where it naturally fits in. On the customer data side, when smart metering becomes more prevalent, a huge amount of data will be collected. This could be used, in conjunction with mapping software and system models to map consumption maps in DMAs where there are spikes in consumption. This can be extended further using water company instruments within the network, monitoring spikes or drops in pressure or could inform where disturbances in the network cause potential customer issues in water quality. The more data that is collected the better the resolution of the operational models which allows the monitoring of the system to be limited to the resolution of the measurement instrument, often in the region of 10-15 seconds. In a lot of cases this could be changed giving near real time monitoring of the operational performance of the potable water network.

In the wastewater network there are areas where Big Data can be used to form the basis of the so called Smart Wastewater Network. Working on the same principles of the potable water network the complex nature of the wastewater network lends itself much more to the use of Big Data. Cross mapping hydraulic models of the network with meteorological data, sewer flow and level data together with sewer overflow data allows customer information informing Beach Alerts similar to what is done in companies some of the water companies including  Northumbrian Water who were recently nominated for the approach. This allows the presentation of the data to company operators (or potentially an automated system) to allow informed decision making in the here and now for operationally reactive work or with further analysis allows prediction of where problems can occur.

Is Big Data the key to solving the water industry’s impending data crisis that some would argue is already upon us - quite clearly no. Saying this though, along with defining the information that we actually need and along with “Small Information” it is quite clearly a part of the solution that needs further investigation as to how it can be utilised in the future.

This year’s Sensors for Water Industry Group conference will take place in Nottingham on 23^rd and 24^th September. Click here for more information

About the author:

Oliver Grievson BSc(Hons) MSc FCIWEM FIEnvSC MInstMC CEnv CSci C.WEM is the Flow Compliance & Regulatory Efficiency Manager at Anglian Water Services

Director of the Sensors for Water Industry Group

Group Manager, Water Industry Process Automation & Control