Measuring an Industry That Had Never Been Measured

The situation

Manufacturers in flooring, tile, cabinets, and countertops spend heavily to create demand.

Almost none of them own the last step of the journey that demand produces. That step belongs to an independent local dealer, and it usually happens on the dealer’s website.

I could not find a population-level measurement of that layer. Not from the manufacturers funding the demand, not from the trade press, not from anyone whose method I could read and check.

What existed instead was anecdote. A handful of dealer sites screenshotted into a slide. A vendor case study built on the customers who had already bought the vendor’s product. Both are evidence of something. Neither tells you what the layer looks like.

So the question I actually wanted to answer, which was whether the dealer layer was quietly losing the demand being handed to it, had no denominator.

Without one, every answer is an argument about which examples are representative, and that is an argument nobody wins. A dealer network can always produce its best sites. A vendor can always produce its worst ones. The only way past it is to measure enough of the layer that the median stops being a matter of opinion.

What the channel reported about itself

The dealer layer was described in the terms a manufacturer can already see: how many dealers there are, how many participate in a program, how much co-op spend they draw.

Every one of those describes the relationship. None of them describes what happens when a customer arrives.

That is the condition bad measurement hides in: the reported numbers are real, they are just about something else, and while volume is up nobody has a reason to ask what they leave out.

Answering the question meant measuring the sites themselves, at enough scale that the answer would be about the layer rather than about whichever sites I happened to pick.

That turned out to be an instrument problem before it was a research problem.

What I built

The pipeline has three layers: discovery, classification, and audit.

Discovery surfaces dealer candidates from search-based market queries and from a confirmed-dealer network. The search pool is messy by design. A query for flooring or tile or cabinets in a metro area returns actual dealers, designers, contractors, distributors, brand pages, dead sites, and businesses that used the word once on an About page.

Classification runs every candidate through a multi-stage pipeline that reads the site, evaluates the business against a four-category taxonomy, and records a confidence score with the result. The categories are dealer, adjacent, non-dealer, and uncertain.

Audit measures each confirmed site with Lighthouse through the PageSpeed Insights API, mobile and desktop separately, and stores every result with a timestamp so the dataset can answer questions about change rather than only about now. A fixed sample is retested periodically, because web performance is not stable from one run to the next and a research program that pretends otherwise is reporting its own noise as movement.

What I decided

Three decisions did more for the credibility of the result than anything in the measurement code.

The first was to keep the noise. Candidates are deduplicated, normalized, and stored before anything classifies them, so the discovery layer stays visible instead of being quietly cleaned until the population looks better than it is. A research population you cannot see the intake of is a population you have to take on faith.

The second was to treat uncertain as a real category rather than a failure mode. Single-page placeholders, sites that load with errors, parked domains, businesses that may or may not still exist: none of those get forced into a bucket to keep the counts tidy. They are excluded from population statistics and they stay counted as excluded. Forcing them would have introduced false confidence at exactly the point where the data is weakest.

The adjacent category does the same job from the other direction. Interior designers who specify materials without retailing them, contractors who install without selling, distributors, manufacturer brand pages: all of them are real participants in the trade and none of them is a dealer. Counting them would have inflated the population, and dropping them silently would have hidden how much activity sits next to dealer retail without being it. Naming the category keeps the shape of the channel visible in the data rather than only in my head.

The third was to measure mobile and desktop separately, on the grounds that they are not the same experience. That decision was made for correctness. It turned into the finding.

What changed

The dealer layer now has a measured median instead of an anecdote.

Across the verified population, the median mobile Largest Contentful Paint is 8.7 seconds. Google considers anything over four seconds poor. Fewer than 4% of measured dealer sites pass the mobile threshold.

On desktop, approximately 71% pass.

That gap is the whole finding, and it is also the explanation for why the problem had gone unmeasured. Everyone who evaluates a dealer site evaluates it on a laptop, on an office connection, where it is fine. The customer arrives on a phone, on whatever signal is in the parking lot, where it is not. The published measurement of that is The Last Mile.

Nobody was hiding this. The measurement that would have shown it was never taken, because taking it required an instrument nobody had built.

What it cost to learn

The expensive part was not the measuring. It was the classification, and it stayed expensive because the honest version keeps costing more than the convenient one.

A yes-or-no dealer test produces bad data fast: it pulls in too much noise, excludes too many real edge cases, and hides the uncertainty that should have stayed visible. Getting past that took an explicit taxonomy with inclusion and exclusion rules, a manually reviewed sample to check the classifier against human judgment, and rule changes driven by the cases that broke it. Dealers working out of a residential address with no showroom. Manufacturer direct-to-consumer brands that look like dealer sites until you weight local presence. Closed businesses whose sites still serve content.

None of that produced a number anyone will quote. All of it is why the numbers that do get quoted are defensible.

The second cost is ongoing, and it is a discipline rather than an expense. The pipeline keeps running, so the corpus keeps growing, which means the population a published paper measured is smaller than the population that exists today. A published paper states the population it actually measured, frozen, with the date it was measured on. Rewriting it upward when the corpus grows would restate the denominator the finding was computed over, which would make the finding unrecomputable and therefore worthless.

So the papers and the site are allowed to disagree, and each paper carries a dated note explaining why.

That is the part I would not have predicted at the start. Building the instrument was most of the work, and keeping it honest afterward is most of what is left.