BUSINESS OBJECTS PERFORMANCE TESTING

Performance Testing: What and Why

Performance testing is a broad term that encompasses many types of tests designed to check how a system performs under certain conditions. The goal of performance testing is to establish benchmarks related to responsiveness, stability, speed, resource usage, etc. A load test is a type of performance test that checks how a system performs under a heavy load. By steadily increasing the number of virtual concurrent users/actions, it is possible to identify the upper limit of a system. A stress test is a type of performance test that goes even further than a load test. It checks how a system responds and recovers when overloaded, or pushed beyond its upper limit.

Understanding how systems perform allows organizations to make informed decisions. IT departments are primarily concerned with performance; business units are primarily concerned with cost. When it comes to investing in hardware/platforms/environments, standard sizing guidelines are a good place to start, but they are based on artificial assumptions. Only by testing the performance of systems under actual or simulated conditions can organizations determine how to design and maintain environments that satisfy both IT and business by achieving an ideal balance between performance and cost. 

Limitations of Performance Testing Software

Performance testing software such as Apache JMeter and Micro Focus LoadRunner are excellent tools for conducting load and stress tests. They measure the performance of static and dynamic resources by simulating multiple virtual users logging on concurrently and performing basic actions like navigating, opening, and refreshing. Users have lots of flexibility in terms of inputs, but the software generates very little in the way of outputs and tests only parts of the Business Objects architecture.

Another limitation of performance testing software is that it tests performance only at the level of the web application server (e.g., Apache Tomcat). To go deeper and simulate complex business intelligence activities, users must create complex JMeter test plans. This is easier said than done because creating test plans requires a fundamental understanding of the principles behind both JMeter and the application being tested. In the case of Business Objects, the challenge is to create test plans that automatically correlate dynamic data with simulated users. Few resources exist to aid with this process. For example, a 2012 article, Introduction to Apache JMeter, posted to the SAP community Wiki, has not been updated to reflect changes to subsequent versions of Business Objects and or the real-life impact of these changes on creating test plans.

360Suite: The Best Performance Testing Software Companion

360Suite is a set of software solutions that enhance Business Objects. Several of the solutions, including 360Live, 360Eyes, and 360Cast, are ideal companions to performance testing software. 360Live offers real-time analysis of Business Objects platform performance indicators. 360Eyes offers insight into metadata. 360Cast schedules and burs reports, activities that are important to test because they stress the full Business Objects architecture and back end databases. By simulating the front end and reaching the back end, these 360Suite solutions expand and extend the ability of performance testing software to generate crystal-clear insight into the performance of Business Objects.

Whereas JMeter examines how the web application server performs, 360Suite goes further to examine the behavior of the Business Objects components. Specifically, 360Suite tests the performance of the SAP Business Intelligence platform (e.g., CMS, FRS, and multiple servers–Adaptive Processing, Web Intelligence, Crystal Reports, Adaptive Job and Connection), associated dependences (e.g., system database, audit database, FileStore, and web application server), and overall architecture (e.g., web layer and SAP Business Objects load balancing, clustering, and reporting databases).

A typical test scenarios is as follows. A customer tests performance with JMeter, steadily increasing the number of virtual concurrent users from 10 to 10,000 or more. The customer runs 360Live during the test and 360Eyes afterwards; together they generate outputs and determine if and at what point the number of users starts to impact any Business Objects component. Next, the customer tests performance with JMeter alongside 360Cast, which simulates the batch scheduling of reports. The customer steadily increases the number of scheduled reports from 100 to 1,000 or more, and again runs 360Live during the test and 360Eyes afterwards to determine if Business Objects is managing the queuing effectively and at what point the number of reports starts to impact any component. The customer also relies on 360Eyes to verify that the stress test actually occurred and analyze the impact on average session duration.

360Suite outputs are displayed in prebuilt Webi reports.

BOBJ Internal Load Balancing

Left: Confirmation of Stress Tests; Right: Impact on Session Duration (Unique to 360Eyes, this graph empowers organizations to adjust sizing, as needed, to achieve desired response times.)

CPU and Memory Load of BOBJ Component

CPU and Memory Load of All Servers Involved in Performance Test

Regular performance testing is important to validate the sizing and behavior of an environment. 360Suite, in combination with performance testing software, is the ideal solution to regularly test the performance of Business Objects and ensure that attributes like responsiveness, stability, speed, and resource usage continue to satisfy requirements. Overinvesting in hardware wastes money. Underinvesting compromises performance and user satisfaction. Business Objects performance tuning with 360Suite is the professional way to fine-tune resource allocation and achieve the optimal end-user experience at the lowest possible cost. 360Suite also generates a byproduct that is less quantifiable but no less important. By empowering IT to optimize the performance of Business Objects, 360Suite aligns the goals of IT and business.

360Suite is a suite of agile governance solutions for SAP BusinessObjects developed by Wiiisdom.

At Wiiisdom, we transform your Analytics landscape into a reliable place to make better, trusted decisions every day and maximize your data assets.

360Suite is a set of solutions to ensure quality, reliability, performance, and efficiency of SAP BusinessObjects through testing, auditing, monitoring, cataloging, and scheduling methodologies. 360Suite is designed for large organizations looking to mitigate data risks, automate operations, and is the solution of choice for any migration project.

Author: Patrick Perrier