How to validate utility bill data extracted from PDF bills

Extracting data from a PDF utility bill is the easy half of the job. Whether the extraction is done by a person keying numbers into a spreadsheet or by software reading the document, the output is a set of fields that claim to describe a billing period: account number, service address, start and end dates, consumption, demand, charges, and a total. The hard half is proving that those fields are right before anyone builds a benchmark, an emissions inventory, or a budget on top of them.

This guide is a working checklist for validating utility bill data after extraction. It applies whether you digitize bills yourself, receive extracted data from a service, or use BillScan, the PDF bill digitization side of EZGB. The checks are ordered from cheapest to most involved, so you can decide how deep to go for a given portfolio.

Why extracted bill data needs its own validation step

A utility bill is a document designed for a human reader, not a data feed. Layouts change between utilities and between bill redesigns at the same utility. A single PDF can contain several accounts, several meters, a cancelled and rebilled period, an estimated read, or a credit that reverses a prior charge. None of this is visible in a clean-looking row of numbers once the extraction is done. Green Button data, by contrast, arrives in a standard XML format defined by the Green Button Alliance, which is one reason the two sources behave so differently downstream, a difference covered in Green Button vs PDF bill digitization.

The reporting tools that consume bill data are not forgiving. ENERGY STAR Portfolio Manager runs more than 70 data quality checks on a property's meters and flags gaps, overlaps, and values that look unusual. Natural Resources Canada requires at least 12 full consecutive calendar months of energy data covering all fuel types before a Canadian building can receive an ENERGY STAR score. Ontario's large building reporting program asks owners to enter data from January 1 through December 31 of the reporting year and to run the Data Quality Checker before submitting. If extracted bill data has a missing month, a duplicated period, or a wrong unit, those tools will either reject it or, worse, accept it and produce a wrong score.

Level 1: field-level checks on every bill

These checks run on each extracted bill in isolation. They catch the majority of extraction errors and cost almost nothing to automate.

Account and meter identifiers are present and well-formed. Each utility uses a consistent account number pattern. An account number that changes length or format between months on the same site is a red flag for a misread.
Billing period dates are valid and ordered. The end date must be after the start date, the period should be within the utility's normal cycle length, and neither date should fall in the future.
Consumption has a unit and the unit matches the commodity. Electricity in kWh, natural gas in cubic metres or gigajoules, water in cubic metres. A gas bill that reports kWh, or an electricity bill with no unit at all, usually means a column was misaligned during extraction.
Line items sum to the total. Add the extracted charges, taxes, credits, and adjustments and compare to the extracted amount due. A mismatch beyond rounding means a line was dropped or a credit was read as a charge.
Read type is captured. Bills mark reads as actual or estimated. An estimated read is not an error, but it needs to be flagged so that a later true-up bill is not treated as a consumption spike.
Multi-account and multi-meter bills are split correctly. A bill that covers several meters must produce one record per meter, not a single record with the totals of all of them.

Level 2: continuity checks across a site's history

Once each bill passes on its own, line the bills up for the same account and check the sequence. This is where most of the problems that Portfolio Manager would later flag are found.

No gaps. The start date of each bill should be the day after, or the same day as, the end date of the previous one, depending on the utility's convention. A gap of a month means a bill is missing from the set, not that consumption was zero.
No overlaps. Two bills covering the same days double count consumption. Overlaps often come from a cancelled bill and its replacement both being extracted.
Period length is stable. Most accounts bill on a monthly or bi-monthly cycle. A 5-day period followed by a 55-day period usually indicates a rebill or a meter change rather than real usage.
Consumption per day is plausible. Divide consumption by days in the period and compare to the trailing average. Large swings that are not explained by season, occupancy, or a known change deserve a look at the original PDF.
Demand does not exceed physical limits. A peak demand value several times the service size is almost always a misread of another field.

The team behind Green Button data quality gaps and validation applies the same continuity logic to interval data. The rules are the same; only the source differs.

Level 3: cross-checks against a second source

The strongest validation compares extracted bill data against something independent. Where a utility offers Green Button, the customer can authorize an ongoing feed or download an XML file, and the billing periods and consumption in that file should reconcile with the extracted bills for the same account. Where only bills exist, a second extraction pass, a spot check of a sample of PDFs by a person, or a reconciliation against the accounts payable ledger all serve the same purpose. The point is that a number checked against one independent source is worth far more than a number checked against itself.

Book a walkthrough at ezgb.ca if you want to see how EZGB combines both sources: bills digitized through BillScan alongside authorized utility connections and Green Button, standardized into one dataset and delivered by API, CSV, or SFTP.

Building the validation into your workflow

Validation works best as a gate rather than an audit. A practical workflow looks like this:

Extract each bill into a structured record with a link back to the original PDF page.
Run the Level 1 field checks automatically and route failures to a review queue with the source page attached.
Run the Level 2 continuity checks whenever a new bill is added to an account's history, not just at reporting time.
Reconcile against Green Button or another independent source on a schedule, and log every correction with a reason.
Only then push the data to Portfolio Manager, RETScreen, Power BI, or a CRM, and run the destination tool's own checker as a final step.

The order matters. Catching a misread unit at the field level costs seconds. Catching it after a year of ENERGY STAR scores have been published costs credibility. For buildings 100,000 square feet and larger in Ontario, reported information must be verified by a certified professional before submission, so the correction is not only embarrassing but also expensive.

What EZGB does with extracted bill data

EZGB (Easy Green Button Connector) collects consumption and billing data from utilities through authorized utility connections, Green Button Download My Data and Connect My Data, and PDF bill digitization through BillScan. Every extracted bill is validated with the field and continuity checks described above, kept linked to its source document, and standardized to the same units and fields used for Green Button data. The result is delivered by API, CSV, or SFTP into ENERGY STAR Portfolio Manager, RETScreen, Power BI, or a CRM, so the data that reaches your reporting tools has already been through the gate. How the Portfolio Manager side works is covered in Get utility data into ENERGY STAR Portfolio Manager.

Frequently asked questions

How accurate is automated extraction from PDF bills?

Accuracy depends on the bill layout, the quality of the PDF, and whether the extraction software has seen that utility's format before. Text-based PDFs generally extract more reliably than scanned images. Rather than trusting a headline accuracy figure, treat every extraction as unverified until it has passed field and continuity checks, and measure your own error rate on a sample of bills you check by hand.

Which fields matter most for energy reporting?

Billing period start and end dates, consumption with its unit, and the meter or account identifier are the fields that reporting tools depend on. Charges matter for cost tracking and budgeting. Demand matters for electricity accounts with demand charges. If you can only afford to validate a few fields carefully, start with those.

What should I do with estimated reads?

Keep them, flag them, and expect a correcting bill later. When the true-up arrives, the corrected consumption belongs to the periods it covers, not to the month the correction was billed. Tools that track energy use over time will otherwise show a false dip followed by a false spike.

Can I skip validation if I use Green Button instead of bills?

You still need continuity checks, since interval and billing feeds can have gaps and estimated values too, but you avoid the layout-related extraction errors that PDF bills introduce. Most portfolios end up using both, because not every account, commodity, or utility has Green Button available.

How far back should I validate?

At least the 12 consecutive months that a benchmarking score needs, and ideally the full history you intend to report or model. Errors in a baseline year propagate into every comparison made against it.

Sources

Source: energystar.gov