Clarity

  • 1.  Read Accented Characters in GEL script

    Posted Mar 31, 2011 03:17 AM
    Hi All,

    We have a GEL script that is reading data from a CSV file to XOG resource records to CA Clarity. The GEL Process reads data from CSV and stores it in a staging table. GEL further processes the records from staging table to XOG them into Clarity System.

    This CSV file has a resource with an accented character in its name - "é". The GEL process that is reading this data is interpreting the character as "?". The direct DB update (to staging table, not CA Tables :bashful: ) or even manual update via CA Clarity works fine.

    Can a GEL process read special characters such as "é" or does it need any special coding/configuration?

    We are using the below code to read CSV file stored on a shared drive from Clarity App Server:

    <core:catch var="errorVar">

    <f:readFile commentIndicator="#" delimiter="," embedded="false" fileName="${v_Tabseparatedfile}" var="input"/>
    </core:catch>

    ${v_Tabseparatedfile} is the variable storing an absolute path of CSV file.

    Appreciate any help regarding this issue.

    Thanks
    ~Dev


  • 2.  RE: Read Accented Characters in GEL script
    Best Answer

    Posted Mar 31, 2011 03:27 AM
    Your CSV file needs to be UTF-8, which it probably isn't if you exported it from excel. Excel uses WIN-1252 encoding, as do most other windows applications, but Clarity insists on UTF-8.

    Personally I have a custom jelly tag for converting files to UTF-8 so this stopped being a problem for me a long time ago, but I remember it being a right proper pain in the ***. Good luck.


  • 3.  RE: Read Accented Characters in GEL script

    Posted Mar 31, 2011 03:43 AM
    Mark,

    Thanks for your response.

    How do we convert the file to UTF-8? If possible, could you direct me how to achieve it or share your code please? :*)

    Thanks once again!

    ~Dev


  • 4.  Re: RE: Read Accented Characters in GEL script

    Broadcom Employee
    Posted Mar 03, 2016 03:06 AM

    5 years ago. I wonder, did you ever get the piece of code to convert Excel CSV to UTF-8 ?

    I have the same problem.



  • 5.  Re: Read Accented Characters in GEL script

    Posted Mar 03, 2016 03:44 AM

    If you do not have to use the created csv file directly you could open it in eg notepad and save as with UTF-8 encoding.



  • 6.  Re: Read Accented Characters in GEL script

    Broadcom Employee
    Posted Mar 04, 2016 04:34 AM

    Yes, that is our current workaround, but we have to avoid manual intervention.

    But it seems the only solution is using the java.io.* library.

    I would just have assumed, since Clarity only accepts utf-8, that there would have been a gel implementation to convert file encodings.