Green Button gives you standardized utility data, but standardized does not mean flawless. Real feeds contain estimated readings, missing intervals, values that have not yet been validated, and the occasional time zone surprise. If you load that data straight into a model or a dashboard without checking it, small quality problems become confident, wrong conclusions. This article covers how Green Button describes data quality, where gaps come from, and how to validate a feed before you trust it.
The good news is that the ESPI data model was designed with quality in mind. It carries explicit flags for how good each reading is, so much of the checking can be automated once you know where to look.
Quality is a spectrum, not a yes or no
The Green Button Alliance is direct about this: accuracy comes in shades, with a vetting process that gives weight to any given reading. A value can start as raw and unvalidated, sit in an unknown state, or fail a check and be marked questionable. Over time it may be promoted to a validated or revenue quality status suitable for billing. Treating every reading as final ignores this lifecycle and mixes freshly measured data with numbers the utility itself has not confirmed.
defaultQuality and per reading ReadingQuality
ESPI records quality in two places. The ReadingType can set a defaultQuality that applies to all of its interval readings, which establishes a baseline expectation. Then any individual IntervalReading can carry its own ReadingQuality flag that overrides the default for that value. A reading can even hold more than one flag at once, for example marked as revenue quality and also normalized for weather. When you parse, read the ReadingType default first, then let per reading flags take precedence.
Where gaps and estimates come from
Missing and estimated data are normal in metering, usually from communication outages, meter swaps, or readings that failed validation. ESPI ReadingQuality can tag exceptional events such as power failures and estimations for missing measurements. Estimates are typically flagged by method, for example estimated using linear interpolation from surrounding readings, or estimated using a reference day drawn from historical patterns. These placeholder values are meant to be replaced with real measurements later, so a value that is estimated today may change in a future pull of the same period.
Time related gaps are easy to miss because nothing is technically absent. IntervalReadings carry a start time and a duration, so a true gap shows up as a missing block in an otherwise regular series rather than as a null. Daylight saving transitions are the classic trap: without applying the LocalTimeParameters, a series can appear to have a missing hour in spring and a doubled hour in fall that are really just conversion artifacts.
Duplicates deserve their own mention because they are easy to introduce by accident. Pulling overlapping date ranges, or combining a Download My Data export with a Connect My Data feed for the same meter, can produce two IntervalReadings covering the same start time and duration. If your loader sums usage without deduplicating on the meter and interval start, you will overstate consumption. Keying readings by UsagePoint plus start time makes overlaps visible and lets you keep the higher quality version when two disagree.
A practical validation checklist
Before Green Button data reaches analysis, it helps to run it through a consistent set of checks. The table below outlines a workable baseline:
| Check | What to look for | Why it matters |
|---|---|---|
| Schema | Document validates against the ESPI XSD | Catches structural problems before field level parsing |
| Scaling | value combined with powerOfTenMultiplier and unit of measure | Prevents readings off by orders of magnitude |
| Continuity | Every expected interval is present for the period | Surfaces true gaps from outages or meter changes |
| Quality flags | Count of raw, estimated, and questionable readings | Separates measured data from placeholders |
| Time | Timestamps converted using LocalTimeParameters | Avoids daylight saving artifacts and drift |
| Duplicates | Overlapping IntervalReadings for the same period | Stops double counting of usage |
Validating structure against the published ESPI schema is the cheapest first step. The rest are semantic checks that the schema alone cannot enforce, which is why they belong in your ingestion code.
Completeness expectations to measure against
Knowing how much data you should have makes gaps easier to detect. In Ontario, for example, Regulation 633/21 requires distributors to make usage information available for intervals of one hour or less and to provide at least 24 months of data where the customer has held the account that long. Expectations like these give you a concrete target: if a feed that should hold two years of hourly data returns far fewer intervals, that is a signal to investigate rather than a fact to accept.
A validation checklist you can run on any feed
Run these checks on every delivery, not only the first one. Each row is a check, why it matters, and what to do when it fails. None of it needs a specialist tool: it is arithmetic against the file you already received.
| Check | Why it matters | What to do when it fails |
|---|---|---|
| Interval count per day | A day short of its expected intervals is a gap, not a low consumption day | Flag the day, re-request it, and keep it out of comparisons until it fills |
| Reading quality flags | Estimated and missing reads move a monthly total without changing its shape | Count estimates per period and report that percentage next to the total |
| Period continuity | Overlapping or missing billing periods double count or drop consumption | Sort periods per account and assert each one starts where the last ended |
| Unit consistency | A cubic metre read treated as a gigajoule is wrong by more than an order of magnitude | Assert the unit per commodity on ingest and convert once, centrally |
| Account coverage | A silently dropped account looks like a saving | Compare the accounts present in each period against the expected list and alert on the difference |
| Totals against the bill | It is the only end to end check of the whole pipeline | Reconcile summed intervals to billed consumption for the same period |
Handling quality problems downstream
Once you can measure quality, decide policy deliberately. Some analyses tolerate estimated intervals, while billing or settlement work should lean on validated readings only. Keep the original flags with the data so a later consumer can see which values were estimated, and consider re fetching periods that were estimated when you first pulled them, since the utility may have replaced them with measured values. The aim is not to discard imperfect data but to know exactly how imperfect it is.
How EZGB supports clean data
EZGB (Easy Green Button Connector) collects Green Button data across many utilities through Download My Data and Connect My Data, and it standardizes the output so quality checks are consistent from one utility to the next. It preserves reading quality flags, applies the correct scaling and time conversion, and delivers structured results through an API or an ongoing feed, which makes gap detection and re fetching of estimated periods straightforward. Where a period only exists on a PDF bill, EZGB can digitize it so your series is more complete before you validate it.
Green Button was built to tell you not just what a meter recorded but how much to trust each number. Reading the quality flags, watching for gaps and estimates, and validating structure and time before analysis turns a raw feed into data you can defend. Quality checking is not the exciting part of energy analytics, but it is what keeps the exciting parts from being wrong.
Source: greenbuttonalliance.org