Layer7 API Management

  • 1.  Having an issue with Cassandra & multiple data centers

    Posted Sep 27, 2017 11:21 AM

    We are seeing an interesting behavior within our Cassandra connections. It appears as though Datacenter B's API gateways are preferring Datacenter A's Cassandra cluster.

     

    Our contact point configurations are as follows:
    Datacenter A Contact points:

    1. Datacenter A Cassandra
    2. Datacenter B Cassandra

    Datacenter B Contact points:

    1. Datacenter B Cassandra
    2. Datacenter A Cassandra
         

    We are unsure why the Datacenter B gateways are preferring Datacenter A Cassandra nodes. Any insight would be appreciated.

     

     

    Are the contact points ordering not respected? How are the contact points decided?



  • 2.  Re: Having an issue with Cassandra & multiple data centers

    Broadcom Employee
    Posted Sep 27, 2017 01:11 PM

    Hi,

    I am assuming you have a JDBC connection set for Cassandra usage. (Also it would be helpful if this is being used as an OTK backend to note that). 

     

    We would need to see how you have the JDBC connection defined to see the order it should go and what the timeout or latency causing the JDBC connection failover. This would all be in the JDBC connection string you define in the policy manager Tasks menu for JDBC.

     

    Thanks...



  • 3.  Re: Having an issue with Cassandra & multiple data centers

    Posted Sep 27, 2017 01:18 PM

    Our Cassandra connections are definitely leveraged by the OTK.

     

    You mentioned JDBC connections, the only one we have configured is the dummy one that we had to set up for the initial OTK install, it is not being used. (We use Tasks -> Data Sources -> Manage Cassandra Connections to manage our Cassandra connections, not JDBC)

    Our OAuth_Cassandra/MAG_OAuth_Cassandra connections are pretty simple, we have the Contact Points defined (IP list, comma separated), a Port specified (9042 default), Keyspace, Username/Password. Nothing else is configured.



  • 4.  Re: Having an issue with Cassandra & multiple data centers
    Best Answer

    Broadcom Employee
    Posted Sep 27, 2017 01:53 PM

    Sorry thats what I meant.

    But Is it in the same order.

    As far as the development specs it seems that 

    Clustered (2hosts) comma separated. It goes in the order defined. 

    Meaning if The order of the hosts in the cassandra connection is DataCenterA, DatacenterB. 
    A would be primary used then b. 


    So is the Order of Contact points the same on both A and B or did the IPs get reversed properly for the order of preference?



  • 5.  Re: Having an issue with Cassandra & multiple data centers

    Posted Sep 27, 2017 01:58 PM

    The IP listing is reversed. 

     

    Datacenter A has the A list first, Datacenter B has the B list first.
    While both have the opposite datacenter IPs as the last half of the contact points list.

    It was our understanding that the first contact points listed would have preference, but even when our Cassandra clusters are not fully engaged, we see connections span across data centers. We would actually prefer to be able to have a primary/backup list of IPs that only get used if the primary is down.