Written by Calvin Lakhan, Ph.D, Co-Investigator: “The Waste Wiki” – Faculty of Environmental Studies at York University
I’m actually really glad that Jodi Tomchyshyn London found and shared the following study by RRS: Impact of EPR for PPP on Price of Consumer Packaged Goods.
In short, after undertaking a fairly comprehensive examination of jurisdictions across Canada (both with and without EPR legislation), the study concluded that EPR policy has no affect on packaging prices.
The University had an opportunity to review this study as part of some technical advice that we were providing to the state of Oregon (specifically surrounding the impacts of EPR). The RRS study directly contravened our own findings, and as a result, we wanted to better understand why.
I want to preface this by saying that the intent of this post is not to criticize or undermine the work that RRS has done. It was well researched, and I applaud them for wading into such a messy and controversial topic and attempting to provide some clarity. However, the point I do want to make in this post is helping stakeholders understand what they can and cannot do with data. Jodi had made a really good point about understanding the context surrounding data – we need to understand how that information was collected, analyzed, interpreted and presented. I couldn’t agree more…. which is why I sometimes cringe when I see the conclusions that people arrive at, due to a fundamental misunderstanding of what you can do with data.
Going back to the RRS study, regardless of how you feel about EPR and its potential impacts, it is critical that stakeholders understand that the RRS study has some methodological deficiencies, and as a result, leads to erroneous conclusions that cannot be supported by the data. This isn’t a question of opinion – given the way the study was designed, it is not possible for RRS to make any statements regarding the effect of EPR policy on packaging prices. Comparing costs across jurisdictions (even for like products and retailers) is not likely to yield any meaningful inferences with respect to the impact of EPR policies. There are literally hundreds of variables that affect the price of goods across localities (even for the same product and retailer). Demographics, infrastructure, relative purchasing power, proximity to markets, density of competing retailers etc. all effect price. In order for RRS to make the statements they did, they would have to control for all of these factors using statistical techniques such as multivariate regression to specifically isolate the effects of EPR on packaging prices. Given that many of these explanatory variables are collinear, they would also need establish controls for interdependency among explanatory variables.
While the above description may be a tad technical, the best way to look at it is that we are trying to compare identical systems, where the only variable being changed is the presence or absence of EPR programs. All other variables that can potentially impact a product’s price need to be controlled for. As far as I can tell, RRS made no attempts to control for interdependent variables and arrived at a conclusion that cannot be substantiated empirically. The only observation that can be made is that product prices differ from province to province, but provides no insight as to why they differ.
Given that my perspective may be seen as biased given that the university developed an alternative model, I would *strongly* encourage you to have a third party expert with a background in statistics and study design to review the RRS methodology. I am absolutely positive that they would reach an identical conclusion.
This is what is so potentially dangerous about attempting to interpret data without having a sound knowledge of how that data was collected and what you can do with it. In my career, I have countless anecdotes of stakeholders from all walks of life who draw the wrong conclusions, imply causality or infer relationships that simply aren’t there. When a misinterpretation of data leads to policy and legislation, the results can be catastrophic.
In the very first presentation I ever gave on the Waste Wiki, one of the slides says “Data without consideration of context or design does not tell us very much”. That message rings true more than ever today.
____________________________
About the Author
Calvin Lakhan, Ph.D, is currently co-investigator of the “Waste Wiki” project at York University (with Dr. Mark Winfield), a research project devoted to advancing understanding of waste management research and policy in Canada. He holds a Ph.D from the University of Waterloo/Wilfrid Laurier University joint Geography program, and degrees in economics (BA) and environmental economics (MEs) from York University. His research interests and expertise center around evaluating the efficacy of municipal recycling initiatives and identifying determinants of consumer recycling behavior.
