License
If you want a license for the Data Lineage Metadata API we need the SID of your BW(/4HANA) or ERP, S/4HANA system. Or the 3-digit HANA DB ID of your HANA database.
Requirements
Two possibilities to connect
There are two ways how the Metadata API can connect to your system.
Connection via RFC to BW or ECC systems
Connection via HANA through SQL API to BW, ERP, or HANA System
You can decide how you want to connect. For the RFC approach you need to install a Function Module in your development system and transport it through your landscape (more is described here).
No Function Module needed since version v1.4.0 of the Metadata API and with SAP BASIS releases >=
7.40 SP27
7.50 SP23
7.51 SP13
7.52 SP09
7.53 SP07
7.54 SP05
7.55 SP03
7.56 SP01
The Data Lineage Metadata API can be started from any machine (Windows or Unix) with access to your SAP System or the underlying SAP HANA database.
Depending on the approach you have chosen, you will need a HANA, a BW, or an ERP user. Which rights are needed is described in the system-specific pages:
Installation
Windows
Download the latest version of the Data Lineage Metadata API and unzip it. Open the terminal or command line to work with the tool.
Unix
Download the zip and start the following command
mkdir DataLineage; tar -xvf <DataLineageFileName>.tar.gz -C DataLineage; chmod +x DataLineage/DataLineage.sh; chmod +x DataLineage/Core/DataLineage
Insert License
Two possible ways to insert a license for Data Lineage API exist.
We sent you a license file or a license hash. What to do with them is described in the subchapters below.
Insert hash
Windows
DataLineage.cmd insert-license -l <license hash>
Unix
./DataLineage.sh insert-license -l <license hash>
Using license file
Add the file license.lic into the Core folder.
Maintain Connection Data
Normal Connection properties
To maintain the connection data you have to change the appsettings.json file.
Per default the appsettings.json file is empty, but the two connection possibilities are shown (see below).
{ "BwErpSystemConfigurations": [ { "BwErpSid": "", "Languages": [ "", "" ], "Server": "", "SystemNumber": "", "Client": "", "User": "", "Prefix": "" } ], "HanaSystemConfigurations": [ { "BwErpSid": "", "HanaSchema": "", "SchemaBlacklist": [ "", "" ], "Languages": [ "", "" ], "Server": "", "Database": "", "SqlPort": "0", "UseSsl": "false", "User": "", "ConnectionString": "" } ], "SacSystemConfigurations": [ { "Url": "", "TokenUrl": "", "ClientId": "", "ExcludePrivateObjects": false } ] }
If you decided on the RFC connection you use the properties below BwSystemConfiguations
. If you decided on the HANA SQL connection you use the properties below HanaSystemConfiguations
.
It can contain multiple systems depending on what is maintained in our license (see below what it might look like).
Explaining the properties:
Property | Meaning | Connection | Example |
---|---|---|---|
BwSid | The System ID of your BW system | RFC & HANA | BI2, A4H, … |
Server | The host or IP address of your HANA database | RFC & HANA | hdb.mycompany.de |
SystemNumber | The system number of the BW system | RFC | 00, 03, … |
Client | The client of the BW system | RFC | 001, 800, … |
Prefix | The prefix of the function module for the RFC connection since v1.4.0 you can run the Metadata API without installing a function module. In that case you do not have to set it in appsettings file (more infos here). | RFC | Z_, /MYCOMP/Z_, … |
UseAlwaysGetString | DO NOT USE WITHOUT CONSULTNG BLUETELLIGENCE. | RFC | true, false |
Database | The database on your HANA tenant where the BW application is located | HANA | HN2, HDB, SYSTEMDB, … |
HanaSchema | The schema where the BW application is located | HANA | SAPBI, SAPA4H, SAPHANADB, … |
HanaSchemaBlacklist | Schemas which should not be analyzed during the process of getting metadata from HANA. It totally make sense to restrict it via the list at least for some technical SAP schemas. | HANA | ["_SYS_AFL", "_SYS_AUDIT", "_SYS_BI", "_SYS_BIC"] |
SqlPort | The SQL Port of your HANA DB. | HANA | 30213 |
UseSsl | If you have set up and want to use an SSL connection to your HANA database. | HANA | true, false |
Languages | List of Languages you want to export | RFC & HANA | [ "E", "D" ], [ "E" ], ["D", "C", "F"] |
User | User for the BW Data Lineage API. The user needs full select rights on the HANA schema of the BW system. | RFC & HANA | |
EncryptedPassword | No manual input. The password must be set via command line (see here). In that case a property EncryptedPassword is stored in the appsettings.json file. If you need to use a clear password, please use the connection string. | RFC & HANA | |
ErpComponentBlacklist | Blacklist of Application Components in of DDIC tables, views and CDS Entities | RFC & HANA | Recommended to start with a bigger blacklist in the beginning |
Url | URL of the SAC instance | SAC | https://mysacinstance.eu10.hcs.cloud.sap |
TokenUrl | Token URL of the SAC instance | SAC | https://mysacinstance.authentication.eu10.hana.ondemand.com/oauth/token |
ClientId | Client ID for the OAuth Authentication in SAC | SAC | |
EncryptedSecret | No manual input. The secret must be set with the first start of the API for SAC. | SAC | true, false |
ExcludePrivateObjects | Handle if the private SAC entities should be included or excluded | SAC | |
SplitFilesChunckSize | Chunck size to split the Details, Buildups, and BuildupDescriptions file to multiple files by a number of entities. 0 - default. Output 1 file | RFC & HANA | 0, 10000, 1000000 |
Complex connection properties (HANA connection only)
Instead of using the delivered content of the appsettings.json file, you can use a connection string if you need to set more properties for your HANA database. That is important if you use e.g. proxy servers and users.
For more detailed information you can visit the SAP guide.
Example of ConnectionString:
If you do not insert a password in the connection string you can also set an encrypted password via command line (see here).
Connection properties for upload
If you want to upload the file directly to a platform you need to maintain the parameters to upload your metadata.
Set Password
The following command in the command line sets an encrypted password into the appsettings.json file.
In the following examples, the set password is done for BW-/ERP-systems with an RFC connection. Here the parameter -s <SID> is used. If you have a HANA connection you have to set the password with parameter -b <HANA HDB>
.
Windows
DataLineage.cmd set-password -s <SID> -p <password>
Command for versions < 1.2.5
DataLineage.cmd set-password -s <SID> -h <HANA Schema> -p <password>
Unix
./DataLineage.sh set-password -s <SID> -p <password>
Command for versions < 1.2.5
./DataLineage.cmd set-password -s <SID> -h <HANA Schema> -p <password>
Run Application
The BW Data Lineage API can be started for one system.
To find all parameters you can use, try --help to get more information.
The following command is for examples only. The complete command list can be found here.
Windows
Run the application with the following command
DataLineage.cmd extract-all -s <SID>
Command for versions < 1.2.5
DataLineage.cmd extract-all -s <SID> -h <HANA Schema>
Unix
Run the application from the terminal
./DataLineage.sh extract-all -s <SID>
Command for versions < 1.2.5
./DataLineage.cmd extract-all -s <SID> -h <HANA Schema>
Plan a regular job
In most scenarios, it makes sense to pull the metadata of a system regularly. A recurring job should be created for this purpose. This chapter describes how to do it in Windows and Unix environments.
Note that this only works from versions > 1.4.0.
Windows
Open the Task Scheduler in Windows
Right Click > Create a task
In the general tab, you can give the task a name and should choose the second option “Execute independently of the user login”. You can also change the user here
In the tab Trigger, you have to set the interval in which the task should be scheduled
In the tab Actions, you have to insert the command which should be run.
Unix
Make sure DataLineage.sh has execution rights (chmod +x)
Edit the crontab file (crontab -e), adding the line for job
Example for a job every 5 minutes:
*/5 * * * * /<path>/DataLineage.sh extract-all -s <SID>
Make sure the job is in the jobs list after saving the file (crontab -l)
If the job doesn't run as expected, check the logs (grep "DataLineage" /var/log/syslog)