DX Application Performance Management

  • 1.  "APM DB Server Restoration" Affect on APM

    Posted Aug 29, 2018 10:14 PM

    So, here is my scenario...

     

    Our Unix Admin team were working with us on our monthly patching of our APM infrastructure (RHEL Servers). I brought down the entire APM Infra:

     

    WV (Dedicated Physical Server)

    MoM (Dedicate Physical Server)

    8 Agent Collectors (Dedicated Physical Servers)

    TCS (TIM) Collector (Dedicated Physical Server)

    4 TIMs (Dedicated Physical Servers

    Oracle APM DB (VM)

     

    Once everything was down, they started patching all the servers. The first server they started with was Oracle APM DB server. They couldn't access it and it seemed that the entire VM was just GONE (disappeared). The Unix Admin had taken a snapshot of that VM prior to patching and that was GONE (disappeared) also. That had no clue how this happened. They reached out to the Virtual Sys Admin team and couldn't get anywhere.

     

    So, after much troubleshooting........it was decided that the VM will have to be restored from the last backup. The went underway couple of hours ago.....here is the feedback from Unix Admin after the restore was completed

    ------

    "As far as I can tell, the restore of Oracle APM DB is finished but it doesn’t look right…

     

    It’s supposed to be a duplicate of the original vm but there are quite a few differences that don’t make any sense. For instance, the original has 2 network interfaces and the restoration hasn’t got any. The restoration also has a number of SCSI interfaces that the original doesn’t have and the amount or RAM and CPU don’t match between the 2, either.

     

    That said and considering there isn’t anyone in ECC over night to talk to about this, the complete restore is going to have to wait until the morning. I’m not sure what I’m looking at, can’t tell if it’s a sane restoration and need to know more before I start working with the restored vm. I really shouldn’t have to alter its hardware configuration at all (aside from documenting and updating its MAC addresses) and that looks like this restoration is going to require a lot of that."

    ------

     

    Now my questions are:

    1. Can I still bring up the APM infra w/o APM DB? If so, what's the downside?

    2. Cause for concerns if this restore happens w/ some setbacks?

    3. What other setbacks should I be looking for?

    4. Anything else anyone can think of?

     

    Thanks in advance for your help

     

    Cheers

    Manish



  • 2.  Re: "APM DB Server Restoration" Affect on APM

    Posted Aug 30, 2018 03:09 PM

    Bueller.............Bueller!?!?!?

     



  • 3.  Re: "APM DB Server Restoration" Affect on APM

    Broadcom Employee
    Posted Aug 30, 2018 03:16 PM

    Dear Manish:

    Converting to a discussion

     

    Can fellow community members help Manish?



  • 4.  Re: "APM DB Server Restoration" Affect on APM

    Posted Aug 30, 2018 03:22 PM


  • 5.  Re: "APM DB Server Restoration" Affect on APM

    Posted Sep 10, 2018 10:21 AM

    Hi M Parikh,

     

    What was the Rhel & APM version.....in your environment??

     

    Regards,

    Anuranjan Kumar



  • 6.  Re: "APM DB Server Restoration" Affect on APM

    Posted Sep 17, 2018 11:30 AM

    We ended up building a brand new VM and then recovering the VMDK to it. They were able to then restore the db backup from it. But the OS and VM are now brand new. Still no specifics other than boot sector corruption, on what really happened to the server.

     

    anuranjan.kumar

    APM v10.5.2 SP2

     

    Red Hat Enterprise Linux Server release 6.10 (Santiago)