Technically Speaking: Data Analytics and Water
Advanced analytics have the potential to transform the water sector, if they are used across the business and are backed by leadership and the right expertise
by Mark Smith, Managing Director, WRc and Ian Dawes, Senior Business Analyst, WRc
Ofwat’s Periodic Review 2014 determinations have made one thing clear: water companies must become significantly more efficient, providing better service with less expenditure. Current working practices are mature and further improvements are unlikely to realise the efficiency savings required; a step change is needed.
In many industries there has been much talk about the benefits of Business Analytics and how competitive advantage can be gained by embracing rapid fact-based decision making. The growth of many of the most successful commercial organisations has been underpinned by Business Analytics with exemplar firms ranging from Google, Netflix and Amazon to Tesco and Wal-Mart. In the retail sector, offers can be targeted at customers individually based on tracked buying habits or stated likes and dislikes. Purchasing analytics can tell supermarkets how many extra burgers to order when the temperature is forecast to rise to 3oC on a weekend in summer. The success of predictive crime analytics has grabbed headlines and shown that advanced analytical principles are well established and can bring significant value in many different sectors. So could advanced analytics enable the efficiency gains required in AMP6 and is the water industry ready to embrace such techniques?
To investigate the potential benefits of advanced analytics to the water industry it is first necessary to understand the three different types of advanced analytics.
Descriptive: Analytical techniques are used to look at historical data to provide insights.
Predictive: Relationships from historic and / or current situational data is used to describe what could happen in the future.
Prescriptive: Analytics take predictive outputs and simulate and optimise intervention options to either directly make interventions or to tell operators exactly what to do in response to evolving situations.
There is a common misconception that advanced analytics relies on sophisticated and novel analytical techniques. The reality is that the analytical techniques are not necessarily novel but that they can be implemented in a business context. The application of the Capital Maintenance Common Framework in business planning means that water companies are familiar with a range of statistical techniques, such as Generalised Linear Modelling, survival modelling, Markov chains, reliability modelling, and so on.
This range of statistical methods can readily be used for advanced analytics alongside additional methods. The advanced aspect relates not only to the statistical methods used but to the ability to integrate these techniques with corporate datasets and make the results available in sufficient time to make an on-going difference to the operation of the business.
In recent years the water industry as a whole has invested in corporate IT systems. This has resulted in the creation of large corporate datasets that include customer contacts, work order data, asset inventory, and telemetry data. The quality of these datasets may not be ideal but experience has shown that valuable insights can still be gained. The more data is used, the more its value is realised resulting in improved data quality.
This data was created to manage many different business functions ranging from responding to incidents, paying contractors, business planning and monitoring levels of customer satisfaction. However there are significant opportunities to use advanced analytics to gain business insights that can improve efficiency thus increasing the return in the existing investment in corporate IT systems.
For network operations, there is immediate scope for efficiency improvement by analysing asset status data and rainfall forecasts. Models can be developed to predict water demand or the likely location of sewer network escape events. These impacts are directly related to AMP6 regulatory performance commitments so any improvement to existing operational practices has a direct financial benefit.
In business planning, advanced analytical techniques can be used to monitor the economic effectiveness and efficiency of planned maintenance projects. The models used to develop the business plan can be reconciled with observed asset performance to undertake thorough post-project appraisal and close the gap between asset planning and project delivery.
In general the relationship between asset interventions and customer satisfaction is poorly understood in the water industry. Advanced analytical models can be developed to describe this complex relationship. These concepts can be extended to demographic analysis to fully understand the variation between individual operational areas. These models can be used to tactically manage the SIM score, but in the longer term the modelled relationships can also inform operational strategy from a serviceability perspective.
These are examples of areas where recent project experience has shown that advanced analytics can be of significant benefit. There will be many other water company functions where advanced analytics can be of benefit. In the water industry advanced analytics is in its infancy, but there is considerable maturity in other sectors. In his 2007 book (ISBN 1422103323), Thomas Davenport, a leading exponent of analytical approaches, described four characteristics of an organisation that enable successful analytics: a) the organisation needs to have senior management commitment to the approach; b) analytics need to be used across multiple different business functions; c) there needs to be an enterprise-level analytical platform to enable this; and d) the organisation needs to have sufficient expertise in both statistical/mathematical and business terms.
The successful implementation of advanced analytical projects in many different sectors means that there are several software vendors with enterprise level software solutions. The discipline is at a state of maturity such that the technology research company Gartner have recently produced a magic quadrant report analysing the comparative capabilities of different advanced analytical platforms.
As stated earlier, water companies have recently developed large corporate datasets suitable for the development of advanced analytical solutions. Experience from other industries has noted several barriers to the effective deployment of advanced analytics.
There is a major difference between identifying a relationship and realising efficiency. The gap is frequently due to difficulties in changing existing business processes to take advantage of advanced analytical techniques. Effecting change can be a major undertaking requiring a carefully designed change programme. There is a short term risk that the change will not be fully embraced and a tendency to revert to the previous way of doing things. Senior level commitment is required to ensure change.
One evident danger in using advanced analytical techniques is in spurious correlation: since there is always a chance that any two datasets may correlate without there being any genuine underlying reason, managing a business on such correlations is clearly a dangerous thing to do. Another problem is where correlation arises as an artifice of the data collection process. This has been seen in a project that modelled the relationship between customer contacts and network interventions. The design of the incident forms required the user to assign one customer contact to each network intervention. Naturally, the analysis revealed that every intervention was associated with one customer contact but the true relationship could have been have been stronger or weaker.
When any relationship is modelled the appropriateness of the statistical technique, the data collection process and the underlying engineering knowledge should all be reviewed to sense check the results and ensure that the results describe genuine processes.
The development and application of analytical models is an evolving process. The analytical methods, underlying data quality and interpretation of the output must be understood and reviewed continually by an expert team so that continual improvements can be made. Failure to fully understand the models means there is a real danger that the business is being run on inappropriate assumptions.
In summary, advanced analytics have the potential to realise significant efficiency savings both during AMP6 and in the longer term. Water companies have invested in the systems to provide data for advanced analytics. However in the water industry this type of technique is in its infancy and to realise value requires significant change to well established business processes. To make the step change requires ambition and senior level commitment.
- Opinion: Improving Resilience in AMP7 Getting the right data to measure asset health and support resilience will be crucial for water companies in meeting their... Read More >
- Digging Deeper: Monitoring Water Quality in Real Time As the introduction of the revised Bathing Water Directive approaches like the incoming tide, Radio Data Networks (RDN)... Read More >
- Data science - putting it all together It's only when different data sources are pooled and cross-correlated that the full power of IoT analytics and artificial... Read More >
- Moving towards maintenance 4.0 Water utilities need to embrace smart asset management technologies but that is only part of the solution, writes Chris... Read More >
- A data-led approach to clearing FOG Water utilities have a major challenge working with local food businesses to prevent fats, oils and grease entering the... Read More >
- The robots aren't coming... they're already here Stantec lead design engineer Dan Causley looks at advances in technology and their role in boosting productivity in the... Read More >
- Opinion: A smarter way to read the water network Smart metering technology could be the key to closing water companies' knowledge gaps about usage in their network -... Read More >
- Smart thinking: The digital asset management revolution With many of the pieces now in place for a digital asset management revolution, AI and machine learning solutions are set... Read More >