Lire cet article en Français 
Documenting SAP BODS Metadata With 360Suite
Documenting anything in SAP Data Services and keeping track of what has changed is impossible within the solution today. Simply navigating manually is no longer enough for users. With all the reglementary requirements that exist today, being able to document changes is vital. We have recently launched a solution to do just this; 360Eyes for SAP Data Services, a solution that documents SAP BODS metadata. Here I’ll explain the importance of documenting and how our solution can benefit you.
Importance Of Documenting SAP Data Services
It’s important to understand why it’s necessary to document SAP BODS metadata especially for BI system developers and business users who will certainly benefit from this functionality. So, why is documentation so important?
For The Business as a Whole
From a Data Governance perspective, proper documentation ensures that all the inputs and outputs within the ETL processes are correctly managed from acquisition, analysis, preparation, cataloging, usage, and management. In order to guarantee trusted and up-to-date data that is always available to drive business decisions, it is important to be able to trace and audit the various data sources that serve as the foundation of a company’s analytics framework. Not only does it help in ensuring its quality once it’s delivered to users for consumption, but it also allows you to understand how it’s been cleaned, combined, enriched and how it needs to be secured.
Business users want to be informed
Business users want to understand how the facts are determined so they feel confident that their decisions are based on accurate information. They require documentation that provides higher transparency regarding the origin and processing of data. It allows them to verify the sources and the quality of the information provided, thus ensuring that the software and the data presented meets their requirements.
More operational systems lead to higher complexity
For BI system developers, an increasing number of operating systems to be integrated means that ETL processes grow larger and become more complex. When systems have been “inherited”, the first exercise is to reverse-engineer the ETL process flows in order to understand the provenance of data from the company’s numerous heterogeneous systems. Having documentation on which you can rely on when dealing with current systems helps better understand their structure. It makes it easier to integrate new data sources, to modify current dataflows, and create new ones, thus reducing the total development and deployment time.
Allows you to anticipate and fix issues
More often than not, ETL processes are not or poorly documented. Documentation should be an integral part of an organization’s design and conception, otherwise it becomes obsolete and no longer usable. When you know who uses the data, you can analyze the impact any changes to the data has in your environment. Understanding the ETL processes allows you to mitigate the risks associated when such changes are made. Furthermore, it allows you to identify performance issues due to poor design and take the necessary actions to fix these.
360Eyes For SAP Data Services
I mentioned above that we have a solution to document SAP BODS metadata; 360Eyes for SAP Data Services. Our solution provides the ability to perform Business Intelligence on ETL processes as they’ve been defined in SAP Data Services, allowing you to leverage SAP Data Services ETL metadata and understand exactly what’s going on in your ETL.
A great thing about it is that it is a zero-install solution; you simply unzip the installation package, configure access to the 360Eyes and SAP Data Services databases, and run the extraction. We retrieve metadata related to all the layers of your ETL process.
We extract from one or more SAP Data Services Local Repositories into an offline datastore and using an SAP BusinessObjects Universe and WebI documents we ship, you can use the available data that is stored in the 360Eyes for SAP Data Services datastore. The use of SAP BusinessObjects artifacts allows the WebI reports to be scheduled and sent to the ETL users to cater to their respective documentation requirements. 360Eyes for SAP Data Services also allows for self-service — you can create your own WebI documents to access 360Eyes for Data Services. You can also access historical usage information by tracking and comparing historical metadata via snapshots so that any updates made to ETL objects are exposed.
What can you document in 360Eyes for SAP Data Services?
You will now be able to document the following in 360Eyes for SAP Data Services:
- Projects
- Jobs
- Workflows
- Dataflows including Query mapping formula (Query transform as of the 2020.3 release, with more to come in subsequent releases)
- Datastores (Database): sources and target
- Scheduled ETL job history and details
- Successful and failed jobs
- Schedules start and end datetime
- Number of rows retrieved
- Duration to track performance
Further to the SAP BusinessObjects artifacts, you will be able to gather information about the relationships between the data embedded in the ETL process definition, as follows:
- Which Batch Jobs are contained in a Project
- Which Workflows belong to other Workflows and Job
- The Dataflows inside another Dataflow and Workflow
- The Source and Target Datastores within a Dataflow
- The Query Transform Mappings that have been defined within a Dataflow
- Orphan artifacts that are left behind when a Project is deleted, for example, Dataflows, Transforms, etc.
What’s more, in the SAP Data Services Designer, you can tag the description of objects with a marker such as #SENSITIVEDATA. This will allow you to track and audit the utilization of objects that contain sensitive information, such as a Social Security Number, that have gone through the ETL process, and are used by SAP BusinessObjects universes, Freehand SQL statements, and report fields.
The sample reports provided as part of the 360Eyes offering will allow you to identify this tagged metadata for documentation and analysis so that GRC requirements are respected.
The Next Step
Have you been wanting to document your SAP BODS metadata? Well, now you can! 360Eyes for SAP Data Services will allow you to document any changes made in your ETL and give you a better understanding of what’s going on. Documenting ETL processes will allow you to identify any performance problems and provide you higher transparency of your data. If you want to start documenting your SAP BODS metadata, let us know and one of our experts will help.
