Laboratory testing is vital to the success of a regulated cannabis economy, however, there are myriad problems relating to Washington’s cannabis lab testing ranging from standardization, to regulation, to consumer perception. In this article, we’ll explore these issues at length, and discuss some potential solutions.
As you read, the question we encourage you to keep in mind is, “who do I trust?”
Inflated THC Percentages
Apologies in advance: though we do our best not to go too heavy on “overly scientific" terminology in our blogs, it’s tough to avoid in this section.
In the wake of pervasive reports that certain cannabis laboratories were inflating their potency numbers (THC and CBD), Leafly research scientist Nick Jikomes and his Harvard colleague Michael Zoorob thoroughly analyzed publicly-available lab data from the Washington State Liquor and Cannabis Board (WSLCB) to test whether or not the cannabinoid content of Washington’s marijuana varies systematically across testing facilities.
After categorizing the products by type— "chemotype I" being THC dominant, "chemotype II" being a mix of THC and CBD, and "chemotype III" being CBD dominant— they compared chemotype I data between the top 6 labs in Washington and found that the median THC percentages varied greatly, ranging from 17.7% to 23.2%. The two scientists then arranged the lab data in order from lowest to highest median THC levels for further analysis: Confidence Analytics had the lowest median, and Peak Analytics had the highest. After calculating effect sizes to better assess differences between labs, they found that a random measurement by Peak Analytics was 82% more likely to be higher than a random measurement taken by Confidence Analytics. The same phenomenon was observed for chemotype II and III (high CBD) flower, as well.
Jikomes and Zoorob then performed a series of regression analyses to account for plausible confounds (strain name, grower, and time of measurement) and observed that the average adjusted total THC level for Peak Analytics was higher than any other lab, and this held true for both flower and concentrates, across all 3 chemotypes. This analysis suggests that the observed differences between labs cannot be explained away by differences in producers, strain names, or the product types being evaluated.
This extensive study also demonstrated that average THC levels for flower have not increased since 2015, and that lab results with lower median THC levels are "in better agreement with independent measurements of cannabis flower from legal markets.” This statement should come as no surprise to those who are familiar with the work of Mahmoud A. ElSohly, a federally-employed marijuana researcher who we mentioned in our article entitled Lab Results: Our Love-Hate Relationship.
"Our analyses revealed clear, systematic differences in the results obtained by different testing facilities in Washington,” concludes the study, "It is crucial that precise standards are adopted by the industry to ensure that laboratories produce results that are reproducible across labs, independent of the exact testing method used, with the ultimate goal of reporting results that consumers can trust.”
Laboratories aren’t wholly to blame for problems in potency reporting, as one could point a finger at producers, retailers, and even consumers too. In the case of growers, some engage in a practice known as “lab shopping,” in which they send samples of the same strain to multiple labs then ultimately label their product with the highest numbers they receive. To phrase that another way, some growers would be inclined to choose a lab like Peak Analytics because the consistently high lab results will help them sell their inventory faster, easier, and at a higher price point.
I know that lab shopping is a real thing, because I’ve witnessed it firsthand. While working for a cannabis consulting group in 2015, my team was tasked with reviving a struggling outdoor grow, re-branding it, and developing attractive new products for the recreational market. After the new crop was harvested, my colleagues recognized that we needed to be able to move the buds quickly and at a high dollar amount (specifically so we could get some cash back to invest in new product lines), so they submitted several samples to 3 different laboratories to see which lab gave out the highest numbers. Testing Technologies consistently came back with a range of 27% to 30%— plus they were cheaper than the other labs, and they offered free pickup— so that’s who my company went with.
A recent report from Straight Line Analytics shows evidence indicating that lab shopping is rampant in Washington. "The businesses paying for the highest number of lab tests achieve, on average, reported potency levels 2.71% higher than do those businesses paying for the lowest number of lab tests,” writes Dr. Jim MacCrae. Here at Herbn Elements, our budtenders can tell you that just 1% more THC could be the deciding factor when a customer is choosing between 2 bags of weed at the same price point.
“Lab shopping shouldn’t exist, because it is a symptom of lab variability,” says Jeff Doughty, president of Capitol Analysis. “We already have standards that should prevent variations in lab results and proficiency testing that shows that the labs are capable of doing the testing.” Fair point; let’s talk about those standards.
Existing Lab Standards
American Herbal Pharmacopoeia began working on a cannabis monograph in 2011— essentially a detailed guidebook for lab testing marijuana— collaborating with Dr. ElSohly, Dr. Ethan Russo, and other respected cannabis scientists. According to monograph co-author Dr. Michelle Sexton, [the first part¹ of] the monograph was published in 2013, at the request of the WSLCB, specifically so they could use it as a guideline. "The cannabis monograph was endorsed by the WSLCB and adopted for quality control standard guidelines,” wrote Dr. Sexton in an email.
Since these laboratory standards were adopted at the start of our state's recreational marijuana framework, gradual changes were made to these requirements over the next few years, with the most significant changes occurring only within the last year or so. On the surface, it seems counterintuitive for the state to pick and choose which of these standards can be modified (if not eliminated), though Dr. Sexton makes a point in saying, “[the monograph itself] probably needs to be updated, as the industry has made quantum leaps since 2012."
Revised Safety Regulations
Lest we forget, the true purpose of testing cannabis is to ensure that the products are safe for human ingestion; we need labs to make sure that the product we’re inhaling isn’t contaminated with mold, residual solvents, or harmful pesticides. In September 2017, the state made these revisions to lab testing requirements:
- Screening for microbiological life (aerobic bacteria, yeasts and molds, and coliform) are no longer required
- Limits for enterobacteria are now 10x higher
- Limits for allowable residual solvents are now 10x higher (5,000 parts per million, up from 500 ppm)
Most of these changes were championed by a Washington trade organization made up of marijuana industry workers and business owners. Their reasoning was that “research and proposals out of Colorado… indicated that the approved solvents are not human health hazards even at concentrations 10x higher than previously allowed.”
Dr. MacRae of Straight Line Analytics strongly disagrees with this decision. From his perspective, these changes were less about consumer safety and more about increasing the bottom lines for stakeholders. "One has to hand it to the LCB to have come up with such a simple, almost LEAN way of increasing wholesaler revenue by $30 million dollars per year,” he wrote in an article entitled Putzing With Poisons. "All they had to do was lower the safety thresholds for regulated cannabis defined by rule. Easy peasy. Fail less product, sell more product, collect more taxes."
Comparing the reported lab failure rates from August 2017 to September 2017— the month that the new testing standards were adopted— Dr. MacRae found that 72% of the flower that passed in September would have failed if it were tested in August, and 91% of the concentrates passed that September would have failed in August. Perhaps all this product is safe to consume, but many people will undoubtedly wince after reading those statistics… it harms consumer confidence.
A Brief Recap of Washington Lab Drama
Officially, Peak Analytics was suspended in June 2017 due to their inability to adequately detect microbes like salmonella, E. coli, and mold. Unofficially, many people in the cannabis industry believe they were suspended due to their super obvious inflation of potency numbers (in one instance, Peak tested a sample of Blue Dream flower at 37.2%, which is hilarious in and of itself).
As far as we can tell, Peak Analytics doesn’t seem to be in business anymore; their social media presences have been inactive since July 2017, they did not answer our calls during their normal business hours, and we haven’t seen certificates of analysis (COA) from Peak in our shop for several months.
Testing Technologies was suspended in April 2017 after showing "consistent inaccuracies" and "blatant disregard for good laboratory practices as well as sound scientific methods.”
The owner of the lab was quick to blame his former science director, who was purportedly terminated 2 months prior to the audit due to flawed and inaccurate measurements he made. The former science director refutes the claim, saying that he was fired over an equity dispute, and that false potency percentages could be attributed to growers who allegedly submitted flower samples with that were spiked with kief or concentrates.
Testing Technologies has since resumed testing cannabis as of June 2017, and the polite lady who answered my call said that they’re doing their best to move past last year's mistakes and provide reliable measurements.
The oldest running laboratory in Washington, Analytical 360 was briefly suspended in April 2018 after RJ Lee Group (the third party lab accreditors utilized by the state) alleged that Analytical lacked validation for their testing methods and that they were calculating potencies incorrectly.
Written statements from Analytical 360 staff infer that the behavior of these third party auditors was less than professional, to put it politely. I highly recommend reading their testimonies for all the juicy drama— here’s one, here’s another, and here’s one more— and I’d like to share a snippet from Analytical co-founder John Brown’s statement that I found to be particularly interesting:
“When I explained to Tammie Mussitsch that Analytical 360’s customers want [an array of microbial analyses] on their COA reports [that is no longer required as of 8/31/2017], she commented to Lauren Potts that this may cause bad press for the WSLCB, as the media could get hold of a COA through a FOIA request and show that there is moldy marijuana being sold in the marketplace or that the WSLCB could be sued. Lauren Potts agreed that if it was known that samples that would have failed under previous rules for having too high of microbial counts are passing now, it could look bad for the WSLCB."
Analytical 360 got their suspension overturned after appealing to the Attorney General of Washington, and have resumed testing as of May 2018.
So who do you trust? Who do you blame? Even after wading through the ocean of source material that went into writing this behemoth of a blog article, it seems difficult to point a finger at any one organization, be it the WSLCB, the trade groups, the auditors, or the laboratories. It seems that all of these groups have fed into this problem one way or another, even if they approached it with the purest of intentions.
Where do we go from here? According to Jikomes and Zoorob, a first step may be to "require that cannabis labs, like other testing facilities, receive third-party accreditation of compliance with International Organization for Standardization (ISO) 17025 guidelines for testing and calibration laboratories.” Simply put, ISO 17025 is the standard used by testing laboratories all over the world; at no additional cost to the state, participating labs would conduct tests on each other several times a year, and if a lab's results are too far off from the rest, they would lose their accreditation. The idea of self-regulation may not sound ideal considering what we know about the current state of Washington’s cannabis testing laboratories, though some of the good actors have already demonstrated that they value accuracy and consumer safety, such as the members of the Washington Cannabis Laboratory Association and the Lab Transparency Project.
It’s not clear why Washington chose not to require ISO 17025 accreditation for cannabis testing laboratories from the beginning— like California did— but there’s a document² on the WSLCB website dated August 26, 2013 in which a government advisory organization called Botec Analysis emphasizes to the state that ISO accreditation takes years to achieve and entails a "considerable financial burden” to the laboratories. "In the interim,” reads the report, "the LCB may need to rely on a simplified provisional regime of Testing Requirements, Proficiency Testing and/or ring testing.”
Instead of requiring ISO accreditation, the WSLCB opted to create its own lab proficiency requirements based off the ISO 17025 list itself, even after Botec warned that “developing the protocols for all aspects of certification would impose a logistical and financial burden to the State,” and “would also slow the accreditation of laboratories."
With regards to pesticides, a 2016 document³ on the WSLCB website signed by Director Rick Garza states that the Washington State Department of Agriculture (WSDA) will test [specific] samples of marijuana products for the Liquor and Cannabis Board and “accredit the method under ISO 17025.” As far as we can tell, the state is not actively taking measures to hold other laboratories to the same standard.
As of now, Medicine Creek Analytics is the only lab in Washington state with ISO accreditation. It seems ironic that the labs in the Washington Cannabis Laboratory Association and the Lab Transparency Project are not [currently] accredited, but the process takes time, so perhaps they’re working on it.
. . . . . . . . . .
Congratulations on making it to the end of the article! At this point you may be feeling frustrated, confused, nihilistic, and/ or apathetic (and you’re not alone). I should make reassuring concluding remarks to help tie the article together and end this on a positive note, but there’s a lot to be bummed about here, and it’s likely that these issues won’t be resolved in a timely manner. When it comes down to it, this is a problem of consumer trust, and eroding consumer confidence could snowball into a big problem for Washington’s marijuana industry. My hope is that the powers that be will re-evaluate the situation sooner than later— and hopefully legalize [recreational] home cultivation so more consumers can have peace of mind— but I'm not holding my breath.
¹ The second part of the monograph pertains to the therapeutic use of cannabis, and it was never published.
² In the event that this document is removed from the WSLCB website, use this backup link.
³ In the event that this document is also removed from the WSLCB website, use this backup link.
Article by Ramsey Doudar; an in-house marketing and social media strategist at Herbn Elements. Ramsey's perspective is influenced by 2 years of budtending, 5 years as a cannabis industry marketing professional, and 10+ years of being a super picky medical patient.