Scroll Top

Abstract

Companies make business-critical decisions every day based on data from their business intelligence systems. It is therefore essential that Business Intelligence (BI) reports display correct data in an efficient way that is interactive and easy to understand for business users. BI reports are usually validated manually, but there are other efficient ways of conducting testing which are more reliable and can reduce testing time. In this whitepaper, we explore alternatives to manual testing of Tableau dashboards and describe how software developers have been relying on automated testing via Continuous Integration and techniques like Test-Driven Development to improve software stability and reduce maintenance costs. We will explore how Wiiisdom Ops supports these practices for Tableau.

Why Test Dashboard Changes?

Business intelligence dashboards constantly evolve as business and reporting needs change. Such changes can potentially alter existing unrelated functionality and cause unintended reporting errors, which is why testing dashboard deployments is essential. In large organizations, it is not uncommon to have a high number of changes committed on any given day to a dashboard or a number of dashboards. These changes can only be effectively validated with automated testing. We can also use automated testing to ensure that our reports display correct data and, that our data refresh processes and manual modifications are executed correctly either by comparing the results with pre-set values or referencing data in the underlying database. Performing such test tasks in an automated fashion on a Continuous Integration server can greatly reduce costs and speed up delivery.

Background

Modern software development practices encourage making testing a fundamental part of the project. If a software project does not have reasonable test coverage it is considered as “legacy code”: any change to such a project is a potential risk and makes maintenance harder and more costly.

There are a number of software test categories but some of the most commonly used ones are:

  • Unit
  • Functional
  • Performance

 

Each software test category provides different values: in unit testing, it is possible to write simple and quick tests and validate low-level logic, while functional tests are higher level, take longer to write but give more confidence in the overall correctness of the system. Performance tests can help identify slowdowns and bottlenecks introduced by changes. It is common to use a combination of several test types in a software project to get optimal results.

Ideally, we should be able to use a similar approach and perform different kinds of tests on our Tableau dashboards to gain complete confidence in our changes before promoting the new version to our clients.

Business Intelligence Testing

Drawing parallels between BI dashboard and software testing can be helpful to understand important similarities, but there is a major difference with business intelligence systems: software testing tools and frameworks are typically designed for traditional programming languages and are not suitable for BI projects. Testing Business Intelligence applications is different from testing traditional transactional applications, as it requires a data-centric testing approach and you need to work with third-party BI tools, resulting in limited alternatives for applicable testing tools. In addition, the volume, variety, and complexity of the data make it difficult to create robust test cases, and specialized skills are required to execute the data validation and verification processes. Despite limited testing options with Tableau, we can still implement a full test suite to validate the accuracy and performance of our dashboards.