DX NetOps

Expand all | Collapse all

Spectro Server shows the following exception. Nov 30 18:52:09 ERROR TRACE at CsDbAlarmRp.cc(84): CsDbAlarmRep::CsDbAlarmRep() restore_packed_data() failed. The spectroserver started to run but on connecting it gets terminated.

  • 1.  Spectro Server shows the following exception. Nov 30 18:52:09 ERROR TRACE at CsDbAlarmRp.cc(84): CsDbAlarmRep::CsDbAlarmRep() restore_packed_data() failed. The spectroserver started to run but on connecting it gets terminated.

    Posted Nov 30, 2016 08:43 AM
      |   view attached

    Nov 30 18:26:26 : E:/win32app/Spectrum/SS/SpectroSERVER.exe starting
    **************************************************************************

     

    CA Technologies
    One CA Plaza
    Islandia, NY  11749 USA
    Copyright (c) 2012 CA Technologies
    All Rights Reserved

     

    This software is protected by Copyright law and international treaties.  

     

    Unauthorized reproduction or distribution of this software may result in
    civil and criminal penalties.  

     

    This software incorporates the following issued U.S. patents
    and foreign patents, other patents pending:  
    5,261,044; 5,295,244; 5,436,909; 5,504,921;
    5,559,955; 5,590,120; 5,649,103; 5,666,481;
    5,696,486; 5,706,436; 5,734,642; 5,748,781;
    5,751,933; 5,751,965; 5,768,501; 5,777,549;
    5,793,362; 5,812,750; 5,822,305; 5,832,503;
    5,872,928; 5,907,696; 6,000,045; 6,014,697;
    6,049,828; 6,057,757; 6,064,304; 6,115,362;
    6,131,112; 6,205,563; 6,233,623; 6,243,747;
    6,255,943; 6,374,293; 6,373,383; 6,430,712;
    682272 AU; 681972 AU; 69132279.1 DE; 0737920 FR;
    0737920 GB; 685335 AU; 69132280.5 DE; 0737921 FR;
    0737921 GB; 711060 AU; 69608107.5 DE; 0838056 FR;
    0838056 GB; 674572 AU; 720061 AU; 700957 AU;
    699509 AU; 69611542.5 DE; 0865698 FR; 0865698 GB;
    702607 AU; 715918 AU; 720079 AU.  

     

    Subject to CA Spectrum license agreements enclosed with this software.  

     

    This notice shall be marked on any reproduction of this computer software,
    in whole or in part.

     


    **************************************************************************
    Version 9.2.2.008

     

    Nov 30 18:26:26 : Validating and loading SPECTRUM Intelligence Modules.
    Nov 30 18:26:28 : All SPECTRUM Intelligence Modules successfully loaded.

     

    Nov 30 18:26:28 : Validating database...
        reindexing table: ssAlarms
    Nov 30 18:26:33 :     ...done

     

    Nov 30 18:26:33 : Please wait. SpectroSERVER
         is loading landscape 0x500000 at precedence 10...

     


    **************************************************************************
    The following output is intended to facilitate debugging, please forward
    to CA technical support.  The files listed are not part of the customer
    installation of SPECTRUM.
    **************************************************************************

     

           Number of models loaded      : 23130

     

    Nov 30 18:27:10 : E:/win32app/Spectrum/SS/SpectroSERVER.exe
         is now ready on port 0xbeef...

     

    Nov 30 18:39:31 ERROR TRACE at VNM.cc(771): E:/win32app/Spectrum/SS/SpectroSERVER.exe is out of memory while allocating 1734702190 bytes.  Attempting to recover.
    Saved compact diagnostic file to 'E:\win32app\Spectrum\SS\support\SpectroSERVER_20161130_1839.dmp'

     

    0x101c2b4d libGlobl.dll!CsSymbolInfo::print_current_stack
    0x100a5d25 libsskrnl.dll!SearchManager::process_queued_work
    0x1058355c libPort.dll!Cs_new_handler
    0x10f5d4b3 mtsfm32.dll!mts_bad_free
    0x10bdb4f6 libssdbl.dll!CsDbMAttrRep::get_value
    0x1006e0a3 libsskrnl.dll!CsModel::get_int_attr_val
    0x10067058 libsskrnl.dll!CsModel::read_int
    0x1006b2a9 libsskrnl.dll!CsModel::read
    0x1006b79e libsskrnl.dll!CsModel::read
    0x107c3bb5 libgserv.dll!CsDModelIf::read
    0x107c4054 libgserv.dll!CsDModelIf::dispatch
    0x107be776 libgserv.dll!CsCoreIf::dispatch
    0x100576a2 libsskrnl.dll!CsMultiReqScheduler::dispatch_request
    0x10057947 libsskrnl.dll!CsMultiReqScheduler::process_requests
    0x100579dd libsskrnl.dll!CsMultiReqScheduler::process_requests
    0x1005444e libsskrnl.dll!CsLandscape::multi_model_request
    0x100814da libsskrnl.dll!CsModelDomainSrvc::readAttrValListOfModelsInt
    0x10081b35 libsskrnl.dll!CsModelDomainSrvc::readAttrValListOfModelsByIDs
    0x10001c7b libsskrnl.dll!CModelDomainItcM::processItcRequest
    0x108618b6 libitc.dll!ITC_Request_Parms::processRequest_IgnoreExceptions
    0x108619bc libitc.dll!ItcWorkQueue::process_work_item
    0x10e61fe9 libwkmgr.dll!CsWorkScheduler::do_work
    0x76dff476 kernel32.dll!CreateFiberEx

     

    Nov 30 18:39:33 ERROR TRACE at VNM.cc(807): E:/win32app/Spectrum/SS/SpectroSERVER.exe is out of memory while allocating 1734702190 bytes.  Scheduling shutdown.

     

    0x101c2b4d libGlobl.dll!CsSymbolInfo::print_current_stack
    0x100a5b89 libsskrnl.dll!SearchManager::process_queued_work
    0x1058355c libPort.dll!Cs_new_handler
    0x10f5d4d0 mtsfm32.dll!mts_bad_free
    0x10bdb4f6 libssdbl.dll!CsDbMAttrRep::get_value
    0x1006e0a3 libsskrnl.dll!CsModel::get_int_attr_val
    0x10067058 libsskrnl.dll!CsModel::read_int
    0x1006b2a9 libsskrnl.dll!CsModel::read
    0x1006b79e libsskrnl.dll!CsModel::read
    0x107c3bb5 libgserv.dll!CsDModelIf::read
    0x107c4054 libgserv.dll!CsDModelIf::dispatch
    0x107be776 libgserv.dll!CsCoreIf::dispatch
    0x100576a2 libsskrnl.dll!CsMultiReqScheduler::dispatch_request
    0x10057947 libsskrnl.dll!CsMultiReqScheduler::process_requests
    0x100579dd libsskrnl.dll!CsMultiReqScheduler::process_requests
    0x1005444e libsskrnl.dll!CsLandscape::multi_model_request
    0x100814da libsskrnl.dll!CsModelDomainSrvc::readAttrValListOfModelsInt
    0x10081b35 libsskrnl.dll!CsModelDomainSrvc::readAttrValListOfModelsByIDs
    0x10001c7b libsskrnl.dll!CModelDomainItcM::processItcRequest
    0x108618b6 libitc.dll!ITC_Request_Parms::processRequest_IgnoreExceptions
    0x108619bc libitc.dll!ItcWorkQueue::process_work_item
    0x10e61fe9 libwkmgr.dll!CsWorkScheduler::do_work
    0x76dff476 kernel32.dll!CreateFiberEx

     

    CA memory allocation error. Aborting.
    Nov 30 18:51:33 : E:/win32app/Spectrum/SS/SpectroSERVER.exe starting
    **************************************************************************

     

    CA Technologies
    One CA Plaza
    Islandia, NY  11749 USA
    Copyright (c) 2012 CA Technologies
    All Rights Reserved

     

    This software is protected by Copyright law and international treaties.  

     

    Unauthorized reproduction or distribution of this software may result in
    civil and criminal penalties.  

     

    This software incorporates the following issued U.S. patents
    and foreign patents, other patents pending:  
    5,261,044; 5,295,244; 5,436,909; 5,504,921;
    5,559,955; 5,590,120; 5,649,103; 5,666,481;
    5,696,486; 5,706,436; 5,734,642; 5,748,781;
    5,751,933; 5,751,965; 5,768,501; 5,777,549;
    5,793,362; 5,812,750; 5,822,305; 5,832,503;
    5,872,928; 5,907,696; 6,000,045; 6,014,697;
    6,049,828; 6,057,757; 6,064,304; 6,115,362;
    6,131,112; 6,205,563; 6,233,623; 6,243,747;
    6,255,943; 6,374,293; 6,373,383; 6,430,712;
    682272 AU; 681972 AU; 69132279.1 DE; 0737920 FR;
    0737920 GB; 685335 AU; 69132280.5 DE; 0737921 FR;
    0737921 GB; 711060 AU; 69608107.5 DE; 0838056 FR;
    0838056 GB; 674572 AU; 720061 AU; 700957 AU;
    699509 AU; 69611542.5 DE; 0865698 FR; 0865698 GB;
    702607 AU; 715918 AU; 720079 AU.  

     

    Subject to CA Spectrum license agreements enclosed with this software.  

     

    This notice shall be marked on any reproduction of this computer software,
    in whole or in part.

     


    **************************************************************************
    Version 9.2.2.008

     

    Nov 30 18:51:33 : Validating and loading SPECTRUM Intelligence Modules.
    Nov 30 18:51:39 : All SPECTRUM Intelligence Modules successfully loaded.

     

    Nov 30 18:51:40 : Validating database...
        reindexing table: ssAlarms
    Nov 30 18:52:09 :     ...done

     

    Nov 30 18:52:09 : Please wait. SpectroSERVER
         is loading landscape 0x500000 at precedence 10...

     


    **************************************************************************
    The following output is intended to facilitate debugging, please forward
    to CA technical support.  The files listed are not part of the customer
    installation of SPECTRUM.
    **************************************************************************

     

    Nov 30 18:52:09 ERROR TRACE at CsDbAlarmRp.cc(84): CsDbAlarmRep::CsDbAlarmRep() restore_packed_data() failed.Nov 30 18:52:09 ERROR TRACE at CsDbAlarmRp.cc(84): CsDbAlarmRep::CsDbAlarmRep() restore_packed_data() failed.       Number of models loaded      : 23130

     

    Nov 30 18:53:14 : E:/win32app/Spectrum/SS/SpectroSERVER.exe
         is now ready on port 0xbeef...



  • 2.  Re: Spectro Server shows the following exception. Nov 30 18:52:09 ERROR TRACE at CsDbAlarmRp.cc(84): CsDbAlarmRep::CsDbAlarmRep() restore_packed_data() failed. The spectroserver started to run but on connecting it gets terminated.

    Broadcom Employee
    Posted Feb 24, 2017 06:36 PM

    I did some research into previous support cases opened for the same CsDbAlarmRp errors seen in the VNM.OUT, here is a complete description of reason and resolution:

     

    ISSUE: One Spectrum server is down and can't be restarted. When restarting it, you get large amount of repeated error message in VNM.OUT

     

    ERROR TRACE at CsDbAlarmRp.cc(84): CsDbAlarmRep::CsDbAlarmRep() restore_packed_data() failed.May 04 05:32:10 ERROR TRACE at VNM.cc(846): /spectrum/SS/SpectroSERVER is out of memory. Continuing shutdown.

     

    Then, it will run out of memory and shutdown itself. I tried to restore from last known good copy, still the same issue, can't start Spectrum.

     

    ROOT CAUSE: My research indicates the following error is most likely caused by database corruption:

     

    May 04 05:30:19 ERROR TRACE at CsDbAlarmRp.cc(84): CsDbAlarmRep::CsDbAlarmRep() restore_packed_data() failed

     

    The most likely causes of Spectrum database corruption are:

    1. Anti virus software that touch open database files. We recommend to exclude the $SPECROOT directory

    2. Disk backup utilities that touch open database files. We recommend to exclude the $SPECROOT directory

    3. A bad disk drive

    4. Some other application hitting open database files

     

    RESOLUTION: Unless one of the above is seen to cause the database corruption, it is almost impossible to determine the root cause of the corruption. To recover, reinitialize to the legacy database and loading of the models on top of that resolves most database corruption.