0

SAP Data Intelligence | How to get headers of CDS view in Gen 1 graph for Initia...

 1 year ago
source link: https://blogs.sap.com/2023/05/04/sap-data-intelligence-how-to-get-headers-of-cds-view-in-gen-1-graph-for-initial-load/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
May 4, 2023 1 minute read

SAP Data Intelligence | How to get headers of CDS view in Gen 1 graph for Initial Load

For the CDS view extraction from SAP S/4HANA we are going to use ABAP CDS Reader operator in DI. It is a standard operator to read data from a CDS view.

Problem description:

Using this operator, you will not see any headers in the file by default.

There are three versions of ABAP CDS Reader operator. The main difference here is the output data type. But independent of version you are using, the headers are still not visible in the file.This problem we are going to solve in this post.

Solution:

Among these three versions, only ABAP CDS Reader V2 has an information about fields in its output. The output data type of this operator is message. A message has a body and attributes. The body of the output deliveries data, and attribute some information about fields, last batch etc.There is no standard solution to get headers using Generation 1 graph, we must use a python script.

The pipeline looks like:

SAP%20DI%20Pipeline
  1. First operator is standard ABAP CDS Reader V2 operator.
  2. Second operator is python operator with input “input2” as message type and output “outData” as message type.
    from io import StringIO
    import csv
    import pandas as pd
    import json
    
    
    def on_input(inData):
        
        # Read Body 
        data = StringIO(inData.body)
        
        # Read Attributes
        var = json.dumps(inData.attributes)
        attributes = inData.attributes
        # Parse JSON
        result = json.loads(var)
        ABAP= result['ABAP']
        
        if len(ABAP) == 3:
            Field = ABAP['Fields']
            columns = []
            
            for item in Field:
                columns.append(item['Name'])
                
            df= pd.read_csv(data,index_col= False, names = columns)
    
            df_csv = df.to_csv(index = False, header= True)
        
            api.send('outData',api.Message(attributes =  attributes, body = df_csv))
            
        if len(ABAP) == 2:
        
            api.send('outData',api.Message(attributes =  attributes, body = ''))
        
    api.set_port_callback("input2", on_input)
  3. Third operator is standard write file operator as csv.
  4. Fourth operator is again python operator with input “input1” as message type and output “outData1” as message type.to read the last batch information.
    from io import StringIO
    import csv
    import pandas as pd
    import json
    
    
    def on_input(inData):
        
        # Read Attributes
        attr = inData.attributes
        result = attr['message.lastBatch']
        
        if str(result) == 'True':
            api.send('outData1','stop')
        
    api.set_port_callback("input1", on_input)​
  5. Last operator is Graph Terminator which will be terminated when last batch is true.

Conclusion:

Using the above approach, data is written into CSV file with headers and graph will be terminated. This can be helpful for initial load.

Hope you find the content of this blog helpful. Feel free to comment for further clarifications.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK