the problem
The Virtual Power Plant (VPP) Demonstrations solution was designed to build capabilities for a large energy market company to receive periodic and on-request data from market participants, VPPs and service providers.
The initial available dataset from this trial was greenfield telemetry data from individual devices (solar panels and batteries). Cross-referencing this data with the enrolment data provided by the VPP to the client provided a rich dataset upon which analytics and reporting was then built on. A major challenge for the business was to be able to receive the data of a high standard of quality in order to be able to action insights.
the approach
Evolving business requirements to access the lowest level of data granularity (at five-minute records) posed challenges for presenting within Power BI as a dashboard, which was solved by leveraging the new technology, AtScale, underpinning the data presentation layer. This tool provided a platform to unify and aggregate data, enabling fast querying of large volumes of data. As the VPP trial commenced with one VPP, Tesla, as the trial grows, the volume of data received is expected to grow exponentially, therefore it is imperative to ensure the data is of the right quality.
the solution
The Telemetry Data Quality (DQ) View was designed to assess the accuracy of telemetry records received by the client (sent by VPP participants) and understand the extent of different types of missing and invalid values within each received telemetry record.
Each device was expected to send attributes for approximately 60 fields every 5 minutes. Where the telemetry data dashboard assessed received data against expected data, the DQ view applied rules to assess if attributes were within an acceptable range of data values. If attributes fell outside the acceptable range, a flag was applied which allowed analysts to drill down on not only missing data, but errors within received data.
the outcome
The Telemetry Data Dashboard enabled the VPP team to review and monitor the flow of device data into the client . The VPP team had a requirement to analyse the completeness and regularity of the data against the expected amount of data, and to drill down to device level to diagnose exactly what data may be missing and ultimately, why. The dashboard was designed to allow the VPP team to view metrics across selected time periods to establish trends.
This dashboard exceeded the client expectations and additional positive feedback was received from Tesla suggesting that they would also implement a similar tool to qualify their data. The dashboard and DQ view allowed the client analysts to identify where data had been provided in incorrect units, which would have been otherwise unnoticed, and allowed actionable communication with VPPs to correct specific data errors with precision.