Rolling the Dice:
SAP BusinessObjects Regression Testing

Any change to your SAP BusinessObjects environment has the potential to impact your reports. The only way to rule this out is through regression testing — running the same report before and after a change, and comparing the results. Regression testing is critical because SAP BusinessObjects users must be able to rely on the accuracy of report outputs. The quality of data, however, is one of the four major challenges of BI according to our study on the BI trends of 2020. 

Organizations that don’t test their Webi, Crystal and Deski reports for regressions every time they make changes to their SAP BusinessObjects environments are rolling the dice. The lucky ones won’t be impacted by regressions. The unlucky ones will realize too late that they made key business decisions based on incorrect data. Most organizations will fall somewhere in between; they’ll eventually find and fix the regressions that matter, but not before paying a price in terms of lost productivity and trust.

Slow Lane

The typical SAP BusinessObjects customer tests for regressions manually, a process that is both tedious and risky. It may be accomplished by comparing reports in two browsers (Alt+tab), on two screens, or on two hard copies. Sometimes regression testing is performed by SAP BusinessObjects Administrators. Other times, the burden of testing falls on end-users. No matter who does the work, manual regression testing is both time and labor intensive, making it expensive and hard to accomplish on a wide scale. And the fact that it’s manual means it’s prone to human error.

Fast Track

360Bind is the 360Suite solution that automates regression testing. From a technical perspective, 360Bind is a web application that can be deployed on an SAP BusinessObjects web application server in a matter of minutes.

360Bind compares report results over time, and across environments and versions. It does this by generating two different instances of the same report using the same filters or prompts, and then comparing them and highlighting all differences.

What distinguishes 360Bind from other regression testing tools on the market is that it was designed exclusively for use with SAP BusinessObjects, and highlights differences in data (whether displayed in tabs or charts), execution times, structure, style, and images down to the pixel level. You can find more in depth information on our automated regression testing solution here. 

Lane Change

During my years as a BI consultant, I encountered many situations in which manual regression testing was both inefficient and ineffective. The following organizations switched from the slow lane to the fast track, and became 360Suite customers, after recognizing the value of 360Bind automated regression testing.

Case 1: Real Estate Company

A U.S. real estate company migrated SAP BusinessObjects to BI4.1 and manually tested select reports for regressions. After they contacted us for information about 360Bind, we offered to demo the product by testing one of their most important reports. The company was shocked and dismayed when 360Bind uncovered significant differences between report versions, with discrepancies totaling more than $200M! The real estate company calculated that it was spending 40+ hours on manual regression testing following every SAP BusinessObjects service pack (SP) upgrade, of which there were one or two per quarter, in addition to other changes. Not only was manual regression testing inefficient, but it also proved ineffective at finding all the errors. With 360Bind, what once took hours now takes minutes, and the real estate company has total confidence in the results.

Case 2: Chemical Company

A giant chemical company that runs SAP BusinessObjects on top of SAP Business Warehouse (BW) spent 250 man-days on manual regression testing after a migration and later realized they had uncovered only 5% of errors. They purchased 360Bind upon realizing it could have run the same tests in 15 days with 100% accuracy.

Case 3: U.S. City

A U.S. city relies on a large number of prebuilt SAP BusinessObjects reports and universes, provided by original equipment manufacturers (OEM). The city deploys new universes when they are delivered by vendors, without understanding the impact of the new universes on their existing SAP BusinessObjects reports. It invested in 360Bind after realizing that the software could be used to regression test their SAP BusinessObjects reports following every universe change.

Case 4: U.S. Government Agency

A U.S. government agency that manages sensitive data is prohibited by regulations from outsourcing regression testing and must allocate scarce resources to performing manual regression testing in-house. The agency can’t get around the need for in-house testing, but it can do the job faster and better with 360Bind.

Case 5: Bank

A bank placed the burden of manual regression testing on end-users every time it applied an SAP BusinessObjects service pack upgrade. Not only did this reduce worker productivity, but it also failed to control for quality because some end users searched for regressions more thoroughly than others. The bank realized there was no need to place the burden of regression testing on end-users when it could be accomplished by 360Bind — in less time and with more consistent results.

Winds of Change

Many of our customers purchase 360Bind in anticipation of an SAP BusinessObjects upgrade or migration. But many other kinds of changes (e.g., calculation engine changes) also have the potential to cause regressions that impact reports.

• Service packs
• Fix packs
• Patches
• Universe updates
• Database changes
• Operating system changes
• ETL modifications

Some of these changes happen behind the scenes, often without the knowledge of IT. That’s why successful organizations approach regression testing as an ongoing activity rather than a one-off event. Whether driven by frequent changes or quality assurance standards, ongoing regression testing is a smart business practice.

Best All-Around

360Bind comes out on top because it offers many benefits.

Cost Savings

360Bind automated regression testing eliminates the need for labor intensive and time-consuming manual regression testing. Time is money, and 360Bind saves both.

Reliability

360Bind automated regression testing identifies regressions with 100% accuracy and eliminates the human error associated with manual testing. To err is human — and unnecessary.

Risk Reduction

360Bind automated regression testing ensures data accuracy and consistency. Take risks, but not with data.

Security

360Bind automated regression testing keeps testing in-house, where it can be performed in a controlled environment. Play it safe with 360Bind.

The client use case demonstrates the value of
360Bind automated regression testing

Suite Spot

When testing identifies regressions, other 360Suite solutions can be deployed to tackle the issues.

360Eyes

360Eyes can identify the root cause of regressions and perform impact analysis to understand the full scope and scale of a problem.

360View

360View can fix errors identified with regression testing by updating reports in bulk.

360Vers

360vers can highlight the differences between different versions of the same universe.

Who Needs Luck?

Your SAP BusinessObjects investment is only as valuable as the information it generates. So don’t gamble with the accuracy of your SAP BusinessObjects reports. Switch from manual to automated regression testing to find 100% of regressions, and to ensure that your organization complies with regulations, communicates accurately with external stakeholders, and makes smart business decisions based on facts. 360Bind saves time and money, increases reliability, reduces risk, and enhances security associated with SAP BusinessObjects reporting. It’s a small investment with a very big pay off. Who needs luck when you have 360Bind?