Sampling is the process of selecting material to be tested. The process used for collecting test samples directly affects the conclusions that can be supported by the test results. Statisticians use the term “inference” to describe the extension of test results from a sample to a broader group, or population, of product. An appropriate sampling process allows the ensuing test results to be inferred to the desired group. Conversely, inference of the results beyond the group that can be justified by the sampling process will lead to inappropriate conclusions.
From time to time, roofing industry members collect shingle, roll goods, or other product samples to have their properties tested and publish the results. This may be for a magazine article, a product rating on a website, or a blog post, among other uses. The conclusions that may be reached from those results are dependent upon the process used to collect the product samples.
Before starting a product test program, consider the intended use of the results. This first step is essential to creating an appropriate sampling program. If the desire is to make broad statements about a product, the number of samples to be collected will be much larger than if a specific statement is to be made about a defined collection of product. Figuring out the question to be answered is critically important.
A key aspect of any sampling program is the statistical concept of a “lot.” One definition of a lot is found in ASTM E456 – Standard Terminology Relating to Quality and Statistics which states, “a definite quantity of a product or material accumulated under conditions that are considered uniform for sampling purposes.” Examples of a lot might include:
- All the shingles of the same brand and style in a distributor’s warehouse
- All the modified bitumen cap sheet rolls delivered to a job site for application on a roof
- All the packages of a specific ventilation product on a single truckload shipping from a roofing manufacturer
“Random” is another key concept in sampling. Grabbing the first five bundles or rolls available is not random sampling. Random selection requires that each unit (e.g., bundle, roll, package) in the lot has the same chance (i.e., statistical probability) of being selected. This includes the bundle on the bottom row of the least accessible pallet and the roll in the middle of the pallet.
Proper sampling is essential. If a carefully designed sampling plan is followed, the tester can have reasonable confidence that similar results would be obtained if different bundles or rolls were selected and tested from the same “lot” of material. Results from testing properly selected samples can be inferred over the entire lot of material.
Suppose five asphalt ply sheet rolls are purchased at a retail outlet, without being randomly selected, and are subjected to testing. Results from testing the rolls are applicable only to those five rolls. The tester cannot properly infer those results to the pallet or pallets from which the rolls were taken, to the stock of those rolls in the retail outlet’s warehouse, or to the specific brand and style of roll good, because the randomness requirement of the sampling methodology was not satisfied. All that can be concluded is that the five-roll sample met or failed to meet the criteria being evaluated.
As another example, consider a situation in which 10 bundles of shingles are purchased from a roofing distribution outlet and evaluated for one or more properties. What can be concluded from the results? As in the previous example, the sampling process determines the limitations. In this case, the sampling process permits the distributor to select the shingle bundles. If the distributor selects and ships the most easily accessible 10 bundles, the conclusions of the testing are constrained to the 10 bundles and should not be inferred to apply to any larger group or population of shingles. If the distributor selects the shingle bundles randomly from stock of that product in its warehouse, the associated test results may be inferred to apply to the stock in the warehouse, but not to a larger group.
Conclusion
When considering test results for shingles, roll goods, or any roofing product, remember to consider the sampling process. Use of proper sampling techniques ensures confidence in conclusions drawn from testing of product samples.
DISCLAIMER OF LIABILITY: This document was prepared by the Asphalt Roofing Manufacturers Association and is disseminated for informational purposes only. Nothing contained herein is intended to revoke or change the requirements or specifications of the individual roofing material manufacturers or local, state and federal building officials that have jurisdiction in your area. Any question, or inquiry, as to the requirements or specifications of a manufacturer, should be directed to the roofing manufacturer concerned. THE USER IS RESPONSIBLE FOR ASSURING COMPLIANCE WITH ALL APPLICABLE LAWS AND REGULATIONS.
Nothing contained herein shall be interpreted as a warranty by ARMA, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose or non-infringement. IN NO EVENT SHALL ARMA BE LIABLE FOR ANY DAMAGES WHATSOEVER, including special, indirect, consequential or incidental damages or damages for loss of profits, revenue, use or data, whether claimed in contract, tort or otherwise. Where exclusion of implied warranties is not allowed, ARMA’s liability shall be limited to the minimum scope and period permitted by law.
