The Continuous Improvement Fund (CIF), a partnership between the Association of Municipalities of Ontario (AMO), the City of Toronto, Stewardship Ontario (SO) and the Resource Productivity and Recovery Authority (formerly Waste Diversion Ontario – WDO), recently published an article on understanding the complexities of waste audits.
The article provides extols the virtues of waste composition studies including the insights gained into program operations, aid in directing promotion & education (P&E) resources and developing long-term waste management strategies. It also provides information on the correct sample size, frequency of sampling and distribution for a waste audit.
Below are some of the highlights from the original article.
How many samples should I take?
The challenge with waste audits is ensuring that an accurate representation of the waste being generated is obtained at minimal cost. Statistical analysis provides information on confidence levels and margins of error. Howe does that apply to waste composition studies? The confidence level and margin of error effectively represent a range, where if you repeat the same study, you can be ‘confident’ your results won’t change by more than the margin of error.
Factors Affecting Sample Size Determination
In waste composition audits, there is a broad range of materials that are sorted plus they vary in total amounts. So not only does the methodology need to consider whether the material is present, but also how prevalent it is as a proportion of the total sample composition. Not surprisingly, materials present in smaller quantities require more samples to achieve the same confidence level and margin of error as those that are more prevalent.
Additionally, there is a long list of factors that affect material generation and composition. Variables like household demographics, seasonality and program participation have a big impact on waste generation. In most cases, municipalities simply don’t have enough budget to develop a study that can consider all of the possible variables and achieve high confidence levels (i.e., > 90%) with low margins of error across the broad range of material typically present in the waste stream.
Trade-Offs
Recognizing that most municipalities have a limited budget, three key questions should be considered:
- How diverse is the population demographics?
- Are most residents provided with the same level of waste service?
- Are you looking for big picture trends or looking to target a specific material?
Available budget will ultimately dictate the number of samples that can be taken and the project team will have to decide how best to allocate them to examine the issues in question and address identified variables such as demographics. Obviously, the more consistent factors such as the waste service levels and population demographics are, the greater the data consistency will be and the higher the confidence level will be across a set number of samples.
Dealing with Demographics
For most municipalities, it will be more important to focus their efforts on getting the sample distribution across the community right, especially if the data is being used for program planning. Recognizing that many communities have distinct demographic groups, it’s typically easiest to divide a community based on income levels as a surrogate for demographic differences. This can be done by obtaining Stats Canada data on household income levels, and proportioning it out into Low, Medium and High Income. Alternatively, a more complex analysis can be done that considers multiple factors through an Analysis of Variance (ANOVA) test as outlined in CIF Project #1059: Residential Audit Sample Optimization Toolkit.
Coming Soon: New tool for determining confidence levels and sample size
In order to help municipalities determine their confidence level for a set number of samples, the CIF has hired Martin Lysy, Associate Professor of Statistics and Director of the Statistical Consulting and Collaborative Unit at the University of Waterloo (PhD in Statistics, Harvard University, 2012) to develop a tool and guidance document to provide municipalities with an assessment of the trade-offs between statistical accuracy and budget.
The tool relies on ballpark estimates of waste composition data that the CIF has collected, or users can specify from their own historical waste audits. Based on these inputs and user-specified margins of error and confidence levels, the tool will estimate the number of samples required. Users can also test different sample sizes to see the resulting confidence levels and margin of error to ensure they can meet budget constraints. Work is still under way to finalize this new tool but if you want more information contact Mike Birett at [email protected] or Neil Menezes at [email protected].
