The Billion Dollar Dashboards –
The Critical Role of Data Analytics
Quality for Clinical Research
in the Pharmaceutical Industry
The Need For 100% Reporting Accuracy In Clinical Research
The pharmaceutical industry research, develops, and produces drugs for use as medication for the human population, with the goal to cure them or alleviate symptoms. On average it takes 10-15 years and around $2 billion to develop one successful drug to put onto the market. The holy grail is that on average the revenue generated is 14.5 times more than the cost of the R&D investment. As you can imagine, the need for 100% accuracy at every stage of the process is critical in eliminating any economic or societal health risks. This article will explain the importance of Data Analytics Quality of the supply chain evidence in clinical research. You will learn the true risks of relying on poor data Analytics and the criticalness of governing the last mile of the data journey.
A Brief Introduction to Clinical Trials
When it comes to developing a drug, it has to go through a long series of strict clinical trials to test the treatment, find the right dosage, and look for any side effects.
- Phase 0: Pre-clinical research on animals to verify if the drug is efficient or not.
- Phase I: Test the drug on humans in good health to test the highest tolerated dose.
- Phase II: Test on humans who have the specific condition to check if it’s efficient.
- Phase III: Registrational trials to assess the benefits/risks to gain market approval.
- Phase IV: Observational trials are carried out and are performed once a drug has gained market approval.
During these phases, the U.S. Food and Drug Administration (FDA) agency plays an essential role in allowing the pharmaceutical company to continue monitoring the effects and approve the drug for human use. “FDA approval of a drug means that data on the drug’s effects have been reviewed by CDER, and the drug is determined to provide benefits that outweigh its known and potential risks for the intended population”. If after phase three, FDA researchers find a drug to be safe and to have a meaningful benefit/risk profile with a significant patient benefit when compared to currently marketed products, the FDA approves the new product for market authorization. This may seem a fairly simple, straightforward process for pharmaceutical companies to follow but there are also several regulations around these clinical trials, including regulations around the software used during the process, which will be the next focus of this article.
The True Risk of Poor Data Analytics in Clinical Research
At each phase of the clinical trials, the pharmaceutical company will send the FDA information on who they tested, the results, and the benefits and risks of the drug. The FDA then analyzes this information for each phase and if everything is acceptable, the company can move on to the next phase. We spoke to several large pharmaceutical companies and the following two use cases resonated the most:
Challenge #1: anticipate the market launch
“Can more efficient technology (e.g. continuous BI Report validation) get my drug product to market 6 months sooner?” A question often asked by Pharmaceutical Executives because gaining six months on an already lengthy timeline could make a massive difference to society.
Let’s take an example from one of the pharmaceutical companies we spoke to who send their clinical trial results via Tableau dashboards. Imagine in phase III, this company has already spent $2 billion and has been working on it for 15 years, the drug is great but in Tableau, there is a problem with the data (for example, a discrepancy occurred within the data journey). The FDA will then judge that the drug isn’t good enough to put on the market and so they’ll lose the investment money and any of the potential ROI. Conversely, if the Tableau dashboard says everything is great and wonderful and they put the drug out into the market but then there are some deaths, that would also have detrimental consequences.
Given the enormous economic and health risks, the FDA enforced a regulation for pharmaceutical labs to continuously test all computerized systems that are used to generate results. The FDA calls this System Dependability: “The sponsor should ensure and document that computerized systems conform to the sponsor’s established requirements for completeness, accuracy, reliability, and consistent intended performance.” This means pharmaceutical companies need to provide documentation on the software being used, how it’s being used with documentation on test plans and results, as well as evidence of software validation after any change (i.e. software upgrades or component replacements).
In the case of this company, the company uses computerized systems in the supply chain of delivering evidence supporting its position (e.g. new drug based on phase III of the clinical trials), they have to demonstrate and document full control of the evidence. This process is generally known as Functional Validation and applies to any evidence reporting. If it was a Tableau dashboard presenting a benefit/risk analysis, this would need to be validated and would be subject to regulatory audit. This ensures the integrity of decision-making given the high stakes both from a health and business perspective.
So, if we go back to the initial question asked by the Pharmaceutical Executives, “Can more efficient technology (e.g. continuous testing) get my drug product to market 6 months sooner?”, the answer could be yes, but unfortunately, this functional validation process in pharmaceutical companies is often very bureaucratic, and unagile and involves high costs due to the complexity of putting in place a master validation strategy (e.g. processes, requirements, test plans, training, test execution). Pharmaceutical companies have yet to realize the full potential of automated continuous testing, even though it would certainly mitigate risks, improve efficiency and agility and maintain traceability in the industry. And in this case, could reduce the timeline to get the drug onto the market six months sooner. According to one of the pharmaceutical companies, “automated testing at scale to predictively identify data integrity issues across large, complex studies should be an area of focus at pharmaceutical companies for years to come”.
Challenge #2: understand how the investment is used
When there is a potential new drug going through clinical trials, Pharmaceutical Executives will also want to know how the $2 billion investment will be used. For example, one question we were told they often ask is: “How much of the investment is allocated to inefficient processes to ensure Data Analytics quality?” One pharmaceutical giant we spoke to sends a PDF of charts coming from Tableau, which are equally subject to testing and validation. Albeit experts and highly invested in data quality (data collection and storage), they didn’t realize that data quality wasn’t enough and was less aware of its counterpart, Analytics Quality, once the data is presented in dashboards. They forgot the last mile of the data journey also needs to be governed. The Medical Lead who signs the documents for the FDA is accountable if the charts have errors.
What’s important to note is that the Tableau results or dashboards are sent to the FDA at the end of every phase to be able to move on to the next one, but in the meantime, the Data Manager and Data Scientist are analyzing clinical data in Tableau on a daily basis. This means a problem can arise during the 15 years of research but it is much more impactful in the end when sent to the FDA.
Moreover, aside from the obvious health risks, there is also a huge risk to the stock price as this increases after each research phase. For example, phase I has acceptable toxicity, phase II is positive with a good dose of efficacy and risk (value generally rises quite sharply), and phase III is positive where the product may require a market launch (price increases depending on market size and expected sales). Phase IV is much less impactful on the stock market because they already have real-life data and the product is already marketed with sales. So, if the company has to stop at phase 3 when the market was huge because of a chart error then the stock market price drops. As you can imagine, this is very damaging to the company.
Continuous Testing Success Elsewhere
Despite the Pharmaceutical industry being late in leveraging continuous testing, the IT industry has done a fantastic job in greatly improving outcomes in other industries thanks to this invention. For example, it encouraged Netflix to create Chaos Monkey; an in-house tool created to test the resilience of its IT infrastructure by causing failures in a real-world environment and thus verify that the computer system continues to function. You can discover the full story behind this idea here. This is a perfect example of how automated continuous testing is massively beneficial to companies to ensure everything is constantly running smoothly.
Testing Solutions for Clinical Research Data Analytics
It is clear that the need for absolute accuracy in the Pharmaceutical industry cannot be ignored. Pharmaceutical companies don’t simply have a duty to themselves but a responsibility to society to make sure the drugs entering the market aren’t dangerous. Wiiisdom provides automated dashboard testing solutions to eliminate the risk of poor Analytics quality in this industry’s supply chain evidence. Deloitte’s research on the future of biopharma R&D states “Advanced analytics and technologies will enable end-to-end automation of R&D, reducing timelines significantly”, highlighting only further the need for automation, including test automation, to ensure faster, more accurate, and reliable data.
Ensure Life-Critical Decisions Are Accurate From Day 1
Any errors in the dashboards provided by pharmaceutical companies to regulators will create a massive impact both economically and socially. Before anything, these companies don’t want to harm people; it would be a catastrophe to harm thousands of people by putting a drug on to the mark due to an error in one dashboard. So much is at stake in this industry and this is why it is critical to continuously test dashboards to ensure the data is always accurate and the best decisions can be made.
Whether that be for Tableau or other Analytics solutions, Wiiisdom helps integrate data Analytics testing into the rest of your quality processes so you can be sure your Analytics is reliable and trusted at all times. Get in contact with us today if you’d like to learn more.