Integrating Tableau Automated Testing
To Ensure Trusted Dashboards
At Red Hat, Inc.
Red Hat At A Glance
Red Hat is a leading provider of enterprise open source solutions, providing storage, operating system platforms, management products, and support, training, and consulting services. The company has been using Tableau Online (now Tableau Cloud) for the past two years with 15K+ users (of which 14K are viewers and 1K are creators), 5K+ workbooks, and 4K+ data sources. Due to having multiple business units such as HR, finance, and sales, using Tableau Online (now Tableau Cloud) allows them to bring all the business units together.
Red Hat is a data-driven organization because they need data that is trusted in order to make the most reliable decisions. At Wiiisdom, our goal is to help our customers make the best decisions, and this article will show you how Abhishek, BI Engineer at Red Hat, and his team implement Wiiisdom Ops for its Tableau automation testing to have trusted analytics.
Challenges Facing Red Hat In Ensuring Data Trust
To make the best business decisions, Red Hat needs to ensure dashboards shared with stakeholders display accurate information and perform as designed. Like many organizations, they leverage multiple data sources and data transformations before the data is published in a dashboard. As a result, outputs need to be tested to ensure the data can be trusted. Testing has always been an integral part of its culture but this made Red Hat start to test the data visualizations outputs as well. The last mile is the most critical even if organizations have got good data preparation and cataloging processes in place. In a nutshell, Red Hat, like many other Tableau customers, had the following challenges:
- Dashboard proliferation: no way to manually test everything
- Data Quality prevents the users to trust the data and therefore lowers user adoption
- Behind time deliveries to business because of a lack of agility
- Lack of resources: no resources available to manually test the dashboards
Abhishek and his team want to test the dashboards but by default, it is done manually but who likes testing when it’s manual? It’s error-prone, expensive, and not scalable. The problem lies there — people don’t want to manually test and end up not carrying out the testing at all. However, not testing impacts the business because:
- Data and dashboards are not trusted
- Liability issues arise because false information is communicated
- Lower user adoption
- Slower decision-making
This is exactly where, as Abhishek put it, “Wiiisdom Ops came to the rescue”.
Wiiisdom Ops To The Rescue
Before Red Hat discovered Wiiisdom Ops, their test cases were defined in a Word document or Excel spreadsheet and tested manually at certain intervals every week. This took up to a whole day to perform for each dashboard and it was difficult to onboard the development team to testing. They also carried out manual content migration.
Fortunately, Wiiisdom Ops came onto the scene and changed things for the better. Abhishek and his team now define test cases within Wiiisdom Ops where they can write individual cases easily for each dashboard. It only takes a few minutes to execute and test, and they no longer need to wait a long time to see if a dashboard shows regressions or displays inaccurate data. For their manual content migration tasks from development to production, Wiiisdom Ops automatically tests the data and dashboard and publishes it to production leveraging the CI/CD framework. If it passes, it gets published but if it doesn’t then they know something is wrong.
Let me explain in more detail the specific types of Tableau automation testing Red Hat carry out within our solution:
Is my worksheet returning the same data as in my database? SQL testing in Wiiisdom Ops automatically compares the data in Tableau with the data in the data source and will highlight any differences.
Functional testing tests the functionality of a dashboard, for example:
- Does my dashboard open?
- If I filter certain data, does the dashboard present the correct data?
In essence, it simulates user journeys to make sure dashboards are working properly and users are having a good experience.
For Abhishek, this is one of the best features in Wiiisdom Ops. Tolerance testing ensures that the team gets notified whenever data displayed in a dashboard is outside its predetermined range. Commonly, developers determine thresholds based on Business rules such as KPIs.
Tableau Online (now Tableau Cloud) upgrades cannot be controlled by Red Hat and so when the pre-imposed upgrades happen, some dashboards might break. Whenever an upgrade happens, Red Hat takes the most business-critical dashboards, takes a snapshot of these dashboards, and runs test cases against the same snapshot they took previously.
Wiiisdom Ops also makes it easier to prove, share and document testing results with the different parties involved.
What’s more, Red Hat leverages automation for its CI/CD leading to faster test case updates and the ability to schedule test cases. They use GitLab for its version control and GitLab Runner as a scheduler for Wiiisdom Ops. The image below shows Red Hat’s CI/CD dashboard testing process:
When Is Best To Test?
Now you’ve seen what types of Tableau automation testing Red Hat carries out, it’s equally as important to understand when is the best time to carry out these tests. For Red Hat, they do testing:
- After an upgrade or migration (Tableau Online – now Tableau Cloud) or a database migration
- After a change in data source or data set
- After a change in a workbook
- Before information is shared with stakeholders so that they can be sure to trust the data
- For random quality control e.g. at certain intervals
Red Hat’s pro tip: During the dashboard design, make sure to gather business rules and then plan testing scenarios so the testing is done properly. It’s always easier to plan testing scenarios during the design phase than once the dashboard is in production.
We also created a Tableau testing checklist so you know exactly what needs testing.
Who Benefits From Wiiisdom Ops At Red Hat
Besides higher dashboard quality at lower costs and a 2x productivity increase for the development team, Tableau automated testing with Wiiisdom Ops brings many benefits to Red Hat’s different company personas:
- Reduce time for testing once test case is done thanks to automation
- Faster and leaner development
- Trust their data and outputs
- Ability to leverage business rules via preset ranges
- Document and share testing results
“Wiiisdom Simply Works”
Red Hat prides itself as a data-driven organization because they trust its analytics with the help of automated testing with Wiiisdom Ops. By putting in place regular testing for specific scenarios, the organization can be sure every dashboard is working properly and the business users can trust the data that they are consuming.
Are you ready to follow in Red Hat’s footsteps? Discover Wiiisdom Ops today.