Service Virtualization

Expand all | Collapse all

Virtual service via Copybook

  • 1.  Virtual service via Copybook

    Posted Aug 22, 2017 09:34 AM
      |   view attached

    For a proof of concept in my company I'm trying to create a virtual service based upon a copybook and request/response files via MQ transport protocol.

     

    From the documentation I see that to accomplish this I need the copybook, req/rsp files and a payload mapping file.

    I have received the copybook and req/rsp files from the project team, but unfortunately they do not know/used a payload mapping file and are unable to provide me this.

     

    I have contacted CA Support to see whether it is possible to create a virtual service without this mapping file, but the feedback I received was that this is not the case.

     

    In the meantime I have been experimenting to create my own mapping file, but so far without succes (I'm not a mainframe/copybook expert).

     

    Whenever I'm running the wizard to create the virtual service (adding the copybook data handler), I'm getting a copybook parser error "java.lang.RuntimeException: Failed to parse C:\UserTemp\CA\SV\POC\Data\Copybook\cpb-req.txt" without any further info.

     

    Can someone on the community have a look at my copybook and mapping file to see what I'm doing wroing?

    I'm really stuck here and the POC we are doing is actually to proof that DevTest is able to do it.

     

    PS: I have included the files in the poc.zip

    Attachment(s)

    zip
    poc.zip   2 KB 1 version


  • 2.  Re: Virtual service via Copybook

    Posted Aug 24, 2017 05:44 AM

    Anyone that could help me on this?



  • 3.  Re: Virtual service via Copybook

    Posted Aug 24, 2017 01:16 PM

    You will get a better head start if you begin by using the Portal application.  Portal has behavior to create copybook bundles and provides an area for you to import your copybook.  Many of the items necessary to identify requests and responses can be configured there.

    Start by reviewing this link: Create Copybook Bundles - DevTest Solutions - 10.0 - CA Technologies Documentation  

     

    The mapping file can be a little complex depending on how many different copybooks are used to identify a given request and/or response.  Without seeing the format of the MQ messages, there is not a way that we can guide you by providing specific details.

     

    At a minimum, you need to know every copybook that is used for a request. In the mapping, you provide details as to how DevTest looks at an incoming request and identifies the copybook to map the request to. 



  • 4.  Re: Virtual service via Copybook

    Posted Aug 25, 2017 02:46 AM

    Hi Joel,

     

    Thank you for your feedback.

     

    I just tried to create a copybook bundle via the Portal, but it is stuck on parsing the imported copybook with an error "ParserException:[1,1] expecting: number88, number not88". Unfortunately I have no clue what this error exactly means. 

     

    Is it possible to try to reproduce this from the files I attached in the original post (req/resp, copybooks and mapping file) and see what might be wrong?

     

    The mapping file was created by myself, since they said don't use this at mainframe side.

     

    Kind regards,

    Dave



  • 5.  Re: Virtual service via Copybook

    Posted Aug 25, 2017 01:06 PM

    Let's have a look at your artifacts: req/res samples, copybooks and mapping file.

     

    Ransford



  • 6.  Re: Virtual service via Copybook

    Posted Aug 25, 2017 02:55 PM

    Hi Dave,

     

    I don't know a lot about copybook file syntax but it seems that the "(X)-" is causing a problem for cb2xml. When I remove those symbols the XML copybook file is generated without errors. However I wasn't able to progress beyond that because of this error, not sure if it's due to the modified copybook.

     

    2017-08-25 18:52:31,843Z (14:52) [AWT-EventQueue-0] DEBUG com.itko.lisa.vse.stateful.protocol.copybook.CopybookDataProtocol - Request text: '03031100CWF SSC SEPABDF     001FR7630004005300004050501032       17131AAKEOZSDG              73  B  17131AAAGVG9                                                    0FQMRBE   APPL.ALIAS.CETR07.REPLY                                         20170511102925               R                                                                                                                              **EOH***66C101401  2000000                                                0021001FR7630004005300004050501032       EURBNPAFRPP   0      02PANM1SIT-BDDF/DD/BDDF/OPEN/BRT/ACC1/Fran02PANM2ce                                 02PAAD1Rue de St Tropez 1                02PAAD3St Tropez 1                        02PAAD5FR                                 02MICIDFR57ZZZ671000                      02MICITPOPR                               02BSRF1DP015010617155155                  02LCID 991501                             02MICIDFR57ZZZ671000                      02MICITPOPR                               00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        00                                        000000000000000100EUR'
    2017-08-25 18:52:31,844Z (14:52) [AWT-EventQueue-0] DEBUG com.itko.util.copybook.MappingHelper - Found previously parsed mapping file, so using that.
    2017-08-25 18:52:31,844Z (14:52) [AWT-EventQueue-0] DEBUG com.itko.util.copybook.PayloadProcessor - Found payload mapping: TEST
    2017-08-25 18:52:31,844Z (14:52) [AWT-EventQueue-0] DEBUG com.itko.util.copybook.PayloadProcessor - Earliest copybook: copybook-req.txt
    2017-08-25 18:52:31,844Z (14:52) [AWT-EventQueue-0] DEBUG com.itko.util.copybook.CachingCopybookParser - Caching is disabled.  So, converting C:\Users\ransford.hewitt\OneDrive - Rogers Communications Inc\LISA\ProjectRoot\R817\Data\copybooks_poc\copybook-req.txt on demand.
    2017-08-25 18:52:31,858Z (14:52) [AWT-EventQueue-0] DEBUG com.itko.util.copybook.PayloadProcessor - Storage length of the payload: 3000
    2017-08-25 18:52:31,858Z (14:52) [AWT-EventQueue-0] DEBUG com.itko.util.copybook.PayloadHelper - Converting raw payload to an object using XUVR0303
    2017-08-25 18:52:31,859Z (14:52) [AWT-EventQueue-0] DEBUG com.itko.util.XMLUtils         - Reading XML string
    2017-08-25 18:52:31,860Z (14:52) [AWT-EventQueue-0] ERROR com.itko.lisa.vse.stateful.protocol.copybook.CopybookDataProtocol - The content of elements must consist of well-formed character data or markup.
    org.xml.sax.SAXParseException: The content of elements must consist of well-formed character data or markup.
     at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
     at com.itko.util.XMLUtils.newXMLDocument(XMLUtils.java:702)

    Attachment(s)

    zip
    mapping_poc.xml.zip   394 B 1 version
    zip
    copybook-req.txt.zip   709 B 1 version
    zip
    copybook-rsp.txt.zip   767 B 1 version


  • 7.  Re: Virtual service via Copybook

    Posted Aug 28, 2017 09:06 PM

    Hi,

    Yeah looks like Copybook fails in parsing due to element names having (), It works fine if we replace "(X)" with "X". However there seems to be some mismatch in the response copybook and the response sample shared. In the below Response copybook section

      06 X-AMS-RETURN-INFO OCCURS 6 TIMES.
    08 X-AMS-RETURN-CD PIC X(5).
    08 X-AMS-RETURN-CD-MSG
    PIC X(105).
    08 X-AMS-RET-IND PIC 9.
    08 X-AMS-PARTY-CD PIC 99.

     

    X-AMS-RET-IND is expecting Integer whereas the sample message has space. Just a wild guess one of the following might be the reason for the failure

    1. Response sample message is incorrect

    2. This field 06 X-AMS-RETURN-INFO   OCCURS 6 TIMES. occurrence might be depending on the "X-AMS-RETURN-IND" as below

      06 X-AMS-RETURN-INFO OCCURS 0 TO 6 TIMES
    DEPENDING ON X-AMS-RETURN-IND.

    If 2 option then similar approach may be required for below elements so as to parse the message

    04 X-AMS-SCANNING-INFO   OCCURS 3 TIMES.

    06 X-BSI-INFO          OCCURS 20 TIMES.

     

    Thanks,

    Udhaya



  • 8.  Re: Virtual service via Copybook

    Posted Aug 29, 2017 02:57 AM

    Thank you all.

     

    I have gone back to the mainframe team asking to provide a proper copybook.

    I will update this post once i have more info



  • 9.  Re: Virtual service via Copybook
    Best Answer

    Posted Aug 30, 2017 03:48 PM
      |   view attached

    I believe there may be several issues.

    1) A valid Cobol element name should not contain the "(" or ")" symbols, so the attached example removes those.

    2) Most Cobol copybooks begin in column 7. The fist 5 characters are reserved for line numbers and column 6, when it contains an "*" is considered a comment line.  So, I shifted your copybooks over.

    3) Your input data is in UTF format format. Many times when dealing with mainframe data, you need to apply an IBM037 codepage conversion to take care of EBCDIC characters. Your example payloads are clear text so I did NOT apply a codepage conversion. Your live data may need this conversion so you will need to check on that with the Dev team.

     

    Once I executed steps 1 & 2 above, I could get your copybook to parse into a bundle.

     

    The next step is to identify what value constitutes the identification of a request and maps it to a given copybook and similarly a response mapping. For sake of example, I used the incoming request queue name located in the payload as the value that aligns requests & responses to the copybooks. This is likely incorrect, but it was the only thing I could figure out by looking at the sample payload. 

     

    I generated a simple copybook mapping. Then took your R/R pairs and ran them through a fictitious R/R pair from JMS recorder. The attached DT project file has a fake VSI that you can look at to see an example mapping of the request document and bundle. I forgot to include a Copybook DPH on the response side, so the response looks like a response payload without the Cobol copybook mapping applied.  You can simply take the R/R pairs and apply this DPH to see how it looks in the response.

     

    Hopefully, this will get you moving in the right direction.

    Attachment(s)

    zip
    PoC.zip   24 KB 1 version


  • 10.  Re: Virtual service via Copybook

    Broadcom Employee
    Posted Aug 30, 2017 03:59 PM

    I think Joel means the first six columns are reserved for optional line numbers.  The seventh column is the comment indicator (anything non-whitespace).  It's confusing since copybook columns are 1-based.

     

    --Mike



  • 11.  Re: Virtual service via Copybook

    Posted Aug 30, 2017 04:02 PM

    good catch.  



  • 12.  Re: Virtual service via Copybook

    Posted Sep 05, 2018 03:01 AM

    hi Joel, I am  trying to add multiple transactions in one vsi  for copybook protocol, is it possible , do I have to update payload mapping file.