DX NetOps

  • 1.  CAPM API calls to Aggregator

    Posted May 31, 2018 11:42 AM

    We have an upcoming need to start making API calls to the aggregator for statistics on about 300 interfaces. We'll need basic metrics for each interface (bp/s in, bp/s out, errors, discards, etc). These calls will need to be made about every 5 minutes.

     

    My question is if anyone else has done something similar to this did you make individual calls for each interface or were you able to make a single call? Would you build a group with all the interfaces and just make a call to that group? Did you notice any measurable increase in utilization on the server - either at the application or OS level?



  • 2.  Re: CAPM API calls to Aggregator

    Posted Jun 01, 2018 07:40 AM

    We are using OpenAPI for exporting config- and stats-data of Devices, Interfaces and Components in different cycles (like "real-time" - every 5mins, daily, weekly...) 

    Do you use APM for monitoring the performance of CAPC, DA, DC, DR ? 

    And yes, we are always using groups as it makes life easier ;-) 



  • 3.  Re: CAPM API calls to Aggregator

    Posted Oct 22, 2018 05:03 AM

    Hey Matthias ..can you pls share your method of exporting? Is it a script which is in cron. I would like to know how is the access part handled in such cases, when the caller application is not web based and running out of CAPM system.

     

    Regards,

    Abhishek



  • 4.  Re: CAPM API calls to Aggregator

    Broadcom Employee
    Posted Jun 01, 2018 07:48 AM

    Hi David,

    yes, several customers are using this kind of OpenAPI queries.

    OpenAPI in general is described here: OpenAPI - CA Performance Management - 3.5 - CA Technologies Documentation including lots of examples.

     

    Regarding your use case:

    Correct, you would create a group with your interfaces of interest.

    The query could look like this:

    http://DA-HOST:8581/odata/api/interfaces?$format=text/csv&$expand=portmfs&$select=ID,AlternateName,portmfs/Timestamp,portmfs/im_BitsIn,portmfs/im_BitsOut,portmfs/im_Discards,portmfs/im_Errors&$filter=((groups/Name eq 'Interfaces of Interest'))

     

    This query will report the time series data according to the default interval (1h). If you want to run this every 5 min and only want to see most recent metric data, I would suggest to set the following:

    On the DA host, in the file
    /opt/IMDataAggregator/apache-karaf-2.4.3/etc/com.ca.im.odata.beans.ODataLimiters.cfg
    replace the default value for defaultRateTimeIntervalSecs from 3600 to 300 so it reads

    defaultRateTimeIntervalSecs=300

    It will take effect once you save the file.

     

    On a properly sized system, with the numbers you specified, I would not expect a noticeable increase of load.

     

    Hth,

    Lutz.



  • 5.  Re: CAPM API calls to Aggregator

    Broadcom Employee
    Posted Jun 01, 2018 09:56 AM

    David,

     

    As Lutz and Matthias both mentioned, groups are an integral way to get the grouping you want.  You can target the group and be done instead of trying to create logic like if the name contains xyz and ipaddress like 1.2.*.* etc.  It will save you a lot of time just to set up the group and use that.  Not to mention if you need to add to or take away from the pool of interfaces you just add or revoke group membership instead of altering the syntax of the query.

     

    Troy



  • 6.  Re: CAPM API calls to Aggregator

    Posted Oct 22, 2018 05:01 AM

    Hi Lutz,

     

    Thanks for a detailed response. I also have a similar requirement at my customer place, that they want to export number of metrics on certain device/interfaces at a regular frequency to an external DB.

    I went through the documents on the link you shared where they explained using querybuilder to build and test queries, and referred to the git-hub apps. I got the basic idea as these apps are webapps which get used in PM's dashboard pages.

    My query is, if my 'app' is not web based, but say a Java program, how would the access and credentials part work? Just to be on same page, I am talking about a Java based REST client program which will make http REST calls on this url, for instance

    (http://DA-HOST:8581/odata/api/interfaces?$format=text/csv&$expand=portmfs&$select=ID,AlternateName,portmfs/Timestamp,portmfs/im_BitsIn,portmfs/im_BitsOut,portmfs/im_Discards,portmfs/im_Errors&$filter=((groups/Name eq 'Interfaces of Interest')))

     

    and I will get a JSON response back.

     

    Much thanks!

    Abhishek



  • 7.  Re: CAPM API calls to Aggregator

    Broadcom Employee
    Posted Oct 22, 2018 09:44 AM

    Hi Abhishek,

    please see the following (from engineering). HTH.

     

    In most simple case it could be done using the code below, but I would recommend use Apache Olingo client library https://olingo.apache.org/doc/odata2/tutorials/OlingoV2BasicClientSample.html

     

    public static final String PASSWORD="admin";

    public static final String USERNAME="admin";

    public static final String METADATA = "$metadata";

    public static final String HTTP_METHOD_PUT = "PUT";

    public static final String HTTP_METHOD_POST = "POST";

    public static final String HTTP_METHOD_GET = "GET";

    public static final String HTTP_HEADER_CONTENT_TYPE = "Content-Type";

    public static final String HTTP_HEADER_ACCEPT = "Accept";

    public static final String APPLICATION_JSON = "application/json";

    public static final String APPLICATION_XML = "application/xml";

     

        public static HttpURLConnection connectToURL(URL url, String contentType, String httpMethod)

            throws IOException

        {

            HttpURLConnection connection = (HttpURLConnection) url.openConnection();

            String credentials = TestConstants.USERNAME + ":" + TestConstants.PASSWORD;

            String encoding = new sun.misc.BASE64Encoder().encode(credentials.getBytes());

            connection.setRequestProperty("Authorization", "Basic " + encoding);

            connection.setRequestProperty(TestConstants.HTTP_HEADER_ACCEPT, contentType);

            connection.setRequestMethod(httpMethod);

           return (connection);

        }

     

        public static HttpURLConnection readDataFromServer(URL url, String contentType)

                throws Exception

        {

            // Connect to url

            HttpURLConnection connection = connectToURL(url, contentType, TestConstants.HTTP_METHOD_GET);

           

            // Get the result

            int responseCode = ((HttpURLConnection) connection).getResponseCode();

            if(responseCode == HttpURLConnection.HTTP_OK)

            {

                // Buffer to read

                BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));

               

                StringBuilder response = new StringBuilder();

             

                // Just read off the data, but do not store for now.

                String inputLine;

                while ((inputLine = in.readLine()) != null)

                {

                        response.append(inputLine);

                       

                }

                in.close();

                System.out.println(response);

            }

            return(connection);

        }

     

     

      public static HttpURLConnection readXMLDataFromServer(URL url, String contentType)

            throws Exception

    {

        // Connect to url

        HttpURLConnection connection = connectToURL(url, contentType, TestConstants.HTTP_METHOD_GET);

        // Get the result

        int responseCode = connection.getResponseCode();

        if(responseCode == HttpURLConnection.HTTP_OK)

        {

            DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();

            DocumentBuilder db = dbf.newDocumentBuilder();

            InputSource is = new InputSource(connection.getInputStream());

           

            TransformerFactory transformerFactory = TransformerFactory.newInstance();

            transformerFactory.setAttribute("indent-number", 2);

            Transformer transformer = transformerFactory.newTransformer();

            transformer.setOutputProperty(OutputKeys.INDENT, "yes");

           

            //initialize StreamResult with File object to save to file

            StreamResult result = new StreamResult(new StringWriter());

            DOMSource source = new DOMSource(db.parse(is));

            transformer.transform(source, result);

            String xmlString = result.getWriter().toString();

            System.out.println(xmlString);

        }

        return(connection);

    }

     

        URL url = new URL(“http://hostname:8581/odata/api/devices?”);

        // Connect and read from the server

        HttpURLConnection connection = ODataUtil.readDataFromServer(url, APPLICATION_JSON);



  • 8.  Re: CAPM API calls to Aggregator

    Posted Oct 23, 2018 04:22 AM

    Thanks Lutz for superfast response....that helps a lot! So it is using basic authorization. I'll try it out.

     

    Regards,

    Abhishek