DX Unified Infrastructure Management

  • 1.  How to monitor NAS replication and NIS bridge synchronization?

    Posted Nov 23, 2017 05:54 AM

    Dear experts,

    could you advice me how to monitor NAS replication and NIS bridge synchronization? Particularly I'm interested in the situation during/after failback from HA/standby HUB back to primary HUB. I'd like to be sure that all alarms are replicated to transactionlog on primary HUB or even better all alarms are synchronized with NIS database (Oracle in my case) via NIS bridge which is enabled only on primary HUB. From my point of view, we can consider HA/DR procedure as finished, only when all alarms are fully synchronized.

    Kind regards,

    Marek Lukes



  • 2.  Re: How to monitor NAS replication and NIS bridge synchronization?
    Best Answer

    Broadcom Employee
    Posted Nov 23, 2017 07:27 AM

    Hi Marek,

    There is a callback on the NAS called repl_queue_info which you can report on with a short lua script and if it's a certain value, alarm, or you could send it to a QOS metric.

    Although note there is a standard alarm which is generated if the replication queue is large.

     

    It is also worth pointing out that if you failover to secondary and use the communities UMP_failover script (search ump_failover) to reconfigure your UMP to secondary hub, then there is an option to turn on the nis_bridge on the secondary. That way you don't have to wait for failback to primary to get alarms into the database. And you can run on the secondary as if it's the primary.

     

    Let me know if I've misunderstood your requirement.

     

    Hope that helps

    regards



  • 3.  Re: How to monitor NAS replication and NIS bridge synchronization?

    Posted Nov 24, 2017 01:48 AM

    Hi Rowan,
    you understood very well and I really appreciate your answer.

     

    I've tested callback repl_queue_info without parameter name and it provides some data - replication is running so everything seems to be synced. I'll retest it with NAS deactivated on HA HUB.

     

    But to be honest I'm really interested in the fact we could turn on nis_bridge on HA HUB after failover. We don't use UMP_failover script from communities, however we've written our script some time ago that, in fact, does the same - change UIM paths from primary to HA HUB in UMP configuration and restart UMP. As UNIX/Linux admins and being not so experienced in NAS/LUA scripting we've written the script in bash and start it regularly from cron.
    Do I understand it well that after UMP failover, i.e. UIM paths are changed, we could enable nis_bridge on HA HUB? In all documents it is recommended to disable it on HA HUB. Moreover, we were worried about alarm duplication in NIS database after failback from HA back to primary HUB. When NAS starts to replicate alarms from HA HUB back to primary HUB and primary HUB will write them once more to NIS database. Or am I wrong?
    Enabling nis_bridge would probably help us with two issues we're facing when UIM is running on HA HUB, they are
    -1) "green" UMPs, they show no alarm, probably due to the fact that trellis depends on nis_bridge
    -2) some NAS/LUA scripts which read data from NAS_* tables in NIS database don't work properly

     

    Kind regards,
    Marek Lukes



  • 4.  Re: How to monitor NAS replication and NIS bridge synchronization?

    Broadcom Employee
    Posted Nov 24, 2017 05:10 AM

    NIS_Bridge (in backup UIM's NAS) should be turned off as long as main UIM's NAS is active.

    It is reasonable when you turn on NIS_Bridge (in backup UIM's NAS) while main UIM becomes offline. 



  • 5.  Re: How to monitor NAS replication and NIS bridge synchronization?

    Posted Sep 24, 2018 05:00 AM

    Hi all,

     

    repl_queue_info can monitor the replication between primary and secondary Hubs, but will not monitor the nis_bridge sync.  Any ideas on how to monitor it?

     

    I'm thinking maybe a SQL query to ensure something is present in the alarm DB within the last few minutes maybe?



  • 6.  Re: How to monitor NAS replication and NIS bridge synchronization?

    Posted Sep 24, 2018 05:03 AM

    Just found a callback in the nis_server named _status, and the "messages" field increases as you run it



  • 7.  Re: How to monitor NAS replication and NIS bridge synchronization?

    Broadcom Employee
    Posted Sep 27, 2018 06:46 AM

    Suggest use nas probe log (loglevel=3 or more) to check errors of nis bridge thread.

    I have one customer using logmon probe to monitor nas.log