How to read a technical claim about a building product

The most common way to be misled by a construction performance figure is not to be given a false one. It is to be given a true one, measured correctly, under conditions that have nothing to do with your building.

A material tested dry performs as tested — dry. A system tested as a flat sample performs as tested — flat, uninterrupted, with none of the penetrations, junctions and fixings that a real assembly contains. Neither test lied. The reader supplied the missing context, and the missing context was the whole question.

Six questions

These are worth asking of any performance claim, in roughly this order.

1. What exactly was measured?

Not "how did it perform" — what physical quantity, in what units. Thermal conductivity is not the same as installed thermal performance. Reaction to fire is not fire resistance. Water resistance is not waterproofness. Products are routinely described using the adjacent property rather than the measured one, because the adjacent property sounds better.

2. Under what conditions?

Temperature, moisture content, age, orientation, loading, duration. Building physics is deeply condition-dependent [1], and a figure quoted without its conditions is not a figure, it is a number.

3. On what — a sample or an assembly?

This is the question that catches out the most people. Materials are usually tested as materials; buildings are assemblies. Between the two sit thermal bridges, junctions, fixings, workmanship and the difference between a laboratory specimen and something installed by someone standing on a ladder in the rain.

The gap between component performance and as-built performance is a well-established subject of building research [2], and it is not a small correction.

4. Who ran the test, and who paid for it?

Not as an accusation. A manufacturer testing their own product to a published method is often the only entity that has run the test at all, and the result may be entirely sound. But it changes what the result is evidence of: it is strong evidence about that product under those conditions, and weak evidence about how that product compares to an alternative tested by someone else, under conditions that may differ in ways neither datasheet mentions.

5. Is a comparison being made against a stated criterion?

"Better performing" is not a claim until you know better at what, than what, and measured how. A comparison with two of those three missing is a marketing sentence wearing a technical costume.

6. What does the claim not cover?

Durability is the usual omission. A great deal of published performance data describes new material. Buildings are not new for very long, and the honest question — how does this behave at fifteen years, at thirty, after the first repair — frequently has no published answer at all.

Reading a research paper without reading a research paper

You do not need to be able to evaluate a study's statistics to get most of the value out of it. Four things carry most of the weight, and all four are usually in the abstract and methods:

• What was the sample? Ten specimens in a lab and ten thousand buildings in a national dataset support very different sentences.
• What was the comparison? A result with no control is a description, not a finding.
• How long did it run? Anything about durability, moisture or weathering measured over weeks is provisional.
• What do the authors themselves say the limitations are? Researchers are usually admirably explicit about this. It is also the section that press coverage most reliably drops.

That last point is worth stating plainly: when a study's conclusions are reported as more certain than the study's own limitations section allows, the exaggeration was almost always added after the research, by someone summarising it.

Why this matters more in construction than elsewhere

In most fields, acting on a slightly wrong technical claim produces a slightly wrong outcome that can be revised.

In construction, the claim gets built into something. It is covered by other layers, warrantied by people who relied on it, and discovered — if it is discovered — years later by someone doing unrelated work. The cost of a wrong number is not paid at the moment the number is read.

That asymmetry is the entire argument for reading claims carefully, and for a publication being explicit about which of its own sentences are measurements and which are readings of measurements.