CA Client Automation

Hi All,

We are experiencing low disk space issues on the scalability servers, From the domain manager we are sending around 10GB files per week across 1500 scalability server using deploy option(NOT STAGE option) So temporarily it is copying to \..\Programfiles\DSM\SD\tmp and executing the files to target location. This is working as expected.

Our L1 team used above Deploy option and distributed the packages to SS and some how files went to D:programfiles\CA\DSM\SD\ASM\Library\Activate. This is not expected as per CA design while using Deploy option.( We are sure that we didn't use stage option)

Can we delete the files manually from CA\DSM\SD\ASM\Library\Activate. I have tried deleting the files using an option from Domain manager->Control Panel->Scalability->Software staging library->removal scheduled job. Here there are no files and it is showing zero records.

Is it recommended to delete the .zip,.arc files from library folder or any other option to delete those files.

Thanks and Regards
Varun kumar

Hi Varun,

You should not delete directly from Library\activate folder.  In the case the package in activate references a staged package on the SS, the Library uses something called an NTFS junction in the file-system.  The NTFS junction maps a file system shortcut to the real package in the library.

The problem is, if you delete something in activate that has an NTFS junction, it will also delete the files from the volume of the software package in the .arc folder, in the SS staging library.  Which is not a desired outcome.  

The answer-- use "WinOffline.exe -cleanserver".  I'm pretty sure I provided you a copy recently.  The -cleanserver option will look to remove every folder in activate.  If it comes across an NTFS junction, it will unlink the junction in the file system first, so deleting the folder within does not affect the staging library on the SS.

In the scenario where the folder in activate was still needed for some active job, that's OK, software delivery is smart enough to replace it on-the-fly, without having to recreate the job container.  Trust me, I've tested at the time I was build the -cleanserver option.

/////////////////////////

Related topic.  Tomorrow I will be posting an updated WinOffline to the community that will introduce a -checklibrary and -cleanlibrary options.  It will perform a three-way sync between the library.dct file, contents of the library folder, and contents of the database.  This will allow you to remove erroneous or orphaned packages from the SS staging library and free up some space.

I have a few more quick tests to run, and I should have the updated WinOffline posted tomorrow with the new -checklibrary and -cleanlibrary options, with instructions.

Stay tuned,

Brian

HI Brian,

Thanks for the response, I will try this options and let you know the results.

Thanks and Regards

Varun kumar

HI Brian,

I Couldn't find option "WinOffline.exe -cleanserver" on winoffline.exe. Can you please share instruction on how to run this option.

Do you want me to run this command through command prompt ?

Thanks and Regards

Varun kumar

It is part of the "Patch Maintenance" functionality...

Note: This will recycle all ITCM services/processes.

Alternatively, yes, you could run directly from the command line:

WinOffline.exe -cleanserver

All the switches are documented in the GUI:

Also you can see them on command line:

WinOffline.exe /?

-Brian

Varun,

Latest WinOffline is available here:

Latest WinOffline -- 22 May 2018 

The posting has instructions for running a library cleanup.  Recommend you share the output of the -checklibrary switch, before running -cleanlibrary.  Refer to the instructions/examples in the posting.

Regards,

Brian

HI Brian,

I have created software Delivery package and deployed to one scalability server using following parameters.

Attaching the job output for your reference.

I want to delete files from DSM\SD\ASM\Library\Activate folder as well ? Any leads on this.

And also ITCA services get auto started post Win offline deployment? Found caf services are in stopped state.

Job output:-

=========

Thanks and Regards

Varun kumar

Hi Brain,

Adding to above, I have tried below options.

-cleanserver : This parameter is working as expected, Deleted all the files under activate folder.

-cleanlibrary : This parameter hasn't deleted folders, Below are the parameters provided in procedure options.

Any leads on this?

Thanks and Regards

Varun kumar

Hey Varun,

I was going to say, just add the -cleanserver switch, but I see you already found and added it

Coming to the -cleanlibrary switch, I observed the following error from the job output posted:

LibraryMgr|Error: Library file does not exist: D:\CA\DSM\SD\ASM\LIBRARY\library.dct

This means WinOffline did not find the library.dct at the path it expected to find it.

WinOffline determines the location of the SD Library from the output of this command:

ccnfcmda -cmd getparametervalue -ps itrm/usd/shared -pn ARCHIVE

Looking higher up in the job output, I can see it provided this output:

InitComstore|Software library: D:\CA\DSM\SD\ASM\LIBRARY

Can you verify the location of the "library.dct" file on the SS?

And also verify the output of the ccnfcmda command, even though it's in the output:

ccnfcmda -cmd getparametervalue -ps itrm/usd/shared -pn ARCHIVE

I'm thinking the SD library location might have changed, and the ITCM comstore needs to be updated to reflect the correct location.

If that's the case, you would run this command:

ccnfcmda -cmd setparametervalue -ps itrm/usd/shared -pn ARCHIVE -v "X:\Program Files (x86)\CA\DSM\SD\ASM\LIBRARY"

Just set the value to the proper location of the LIBRARY folder.

Let me know...

Regards,

Brain

Hi Brain,

I couldn't find Library.dct file on the SS.

Here is the output for command given.

Thanks and Regards

Varun kumar

If there's no library.dct file, then the scalability server is probably not working!  What happens if you send a test package to the SS, or a test package to an agent registered to the SS?

Have you checked other scalability servers to confirm if the library.dct file exits?

Hi Brian,

Scalability server is working & operational. We were able to send multiple packages to scalability server without any issues.

And also in our environment we don't use ITCA to send software packages to end points.

Yes, I Confirm you that library.dct file isn't available on SS.

Thanks and Regards

Varun kumar

Hey Varun,

I tested on my end, and removed the library.dct file from my scalability server.  When I send a software package, it does work OK.

Here is the entry from the log when it looks for the library.dct file:

Basically, it works around the problem, and downloads the package temporarily from the Domain Manager, for use with the job:

In a way, I guess this defeats the purpose of a scalability server, as what's the point of having one if you aren't staging packages there?

Nevertheless, when I staged a software package to the library, it actually created a new library.dct file.  And much to my surprise it rediscovered all the package that were already staged in the library, when it created the new dictionary file.

So if you are missing a library.dct file, just pick a package to stage to the SS.  When it's done, the library.dct file will be recreated automatically, since it's missing somehow.

Regards,

Brian

HI Brian,

We never stage packages to SS instead we deploy the packages(file copy) to SS directly. Would like to understand the reason why all the packages deployed got accumulated in Activate folder.

This is happening since a week. Is it a known issue ?

Regards

Varun kumar

They are there because you have sd jobs that are not completed.  Once the job is complete they go away.

Richard Lechner

Principal Engineering Services Architect

CA Technologies

Mobile: +1 703 655 7161 | Richard.Lechner@ca.com

<mailto:Richard.Lechner@ca.com>[CA]<http://www.ca.com/us/default.aspx>[Twitter]<http://twitter.com/CAInc>[Slideshare]<http://www.slideshare.net/cainc>[Facebook]<https://www.facebook.com/CATechnologies>[YouTube]<http://www.youtube.com/user/catechnologies>[LinkedIn]<http://www.linkedin.com/company/1372?goback=.cps_1244823420724_1>[Google]<https://plus.google.com/CATechnologies>[Google+]<http://www.ca.com/us/rss.aspx?intcmp=footernav>

Hey Varun, Rich's comment is correct.  When a software package is being deployed to one or more targets, a new folder inside of activate is created.

If the package is staged on the SS library already, the new folder in activate will contain an NTFS junction in the filesystem, to the location where the package files are stored in the library.

If the package is not staged on the SS library, a DTS job will be initiated to copy it, and the files will be stored directly in the activate folder (versus an NTFS junction to the library location).

In either case, once all software jobs that reference that software package are completed, sd_server will remove the activate folder.  For a staged package, it will unlink the NTFS junction, and then remove the folder in activate.  For a non-staged package, it will just remove the folder and all its contents.

-------------------------

That being said we are not tracking any bugs with this.  It works fine across all my lab environments, production CA, and I'm not seeing this from any other customers.  Given what you've told us already-- that somehow you don't have a library.dct file on the SS, things already seems strange about the environment.

When you say you "file copy" packages to the SS-- are you referring to the DTS job that ITCM performs to temporarily copy down the package, or are you doing something custom or outside the ITCM functionality?

Also, may I ask why you have the scalability server is you never intend to stage packages?

Hi Brain,

We use software Delivery  package & Copy procedure for file transfer to scalability servers,  In our environment scalability server is being used as a file share for that particular site, All end points which are in this site will get the file copied through external tools( We dont use ITCA software Distribution on end points) Our scope of work is to transfer the file until scalability servers only.

Coming to activate folder files queued. This is happening since a week and we were sending files through ITCA since one year. Once the deployment is completed we never had this files queued issue in past.

Regarding auto deletion of files post job complete, You meant to say that if we target 150 machines per job, all these 150 machines job should be completed to get deleted the files from activate folder on each server ? 

And also file transfer is completed on Scalability server, But files aren't deleted from activate folder.

Thanks for your cooperation & detailed explanation on this issue.Really appreciated .

Regards

Varun kumar

Hi Brain,

I just deleted the failed,active & waiting machines from the job then automatically the files in activate folder got deleted from the same job successful machines where status was Ok & green.

is this the product behaviour ?

what is the reason behind this ?

Thanks and Regards

Varun kumar

Hey Varun,

Yes, this is working as designed, and exactly as we described above. The library\activate folder is a temporary area that is associated with the active jobs.  Once there are no more active jobs pointing at a particular software package, it's folder is removed from activate.

So if you deploy a software package to 25 agents registered to the same scalability server, the package will be available in the library\activate folder until all 25 have executed the job, or until the job container is completed/terminated at the Domain Manager-- whichever comes first.

Starting from the beginning, when you deploy a software package from the domain manager, to a list of target agents, software delivery queries the database to generate a list of scalability servers that will be involved in the job.

Once it has the list of scalability servers, it queries the database records to find out if the package is already staged at each SS or not.  For those that do not have the package already staged, it initiates a DTS job, to temporarily store the package on the SS.  This is what creates the activate folder, and populates it with files.

For the scalability servers that already have the package stored in the staging library, the activate folder is still created, and it's populated with an NTFS junction (think of it as a shortcut), to the arc folder in the library that contains the package.

It's important to note that an NTFS junction is much more serious than a simple shortcut.  Deleting a "shortcut" will not delete the referenced file/folder on the other end.  However, deleting an NTFS junction (which Windows Explorer uses the same short icon for), will result in the file/folder referenced to be deleted on the other end.  This is why you never want to manually cleanup the activate folder.  It will actually erase software packages stored in the staging library!

Thus, whether the package is staged already in the SS library or not, a folder within activate is generated, and the source file for the package are either copied/extracted by software delivery, or an NTFS junction is created to reference the source files in the staging library.

Once the scalability server realizes that there are no more jobs (as their may be multiple referencing the same package), it will delete the subfolder in activate, as it's no longer needed.

I hope that clarifies further, but real understanding will come by observing the activate folder, and sending a test software package to agent(s) registered with that scalability server.

I still don't understand exactly what you're doing by having scalability servers without a staging library, and serving agents that will never receive a software job.  It sounds to me like your using the SS for some other purpose, outside of ITCM, maybe in conjunction with another tool.  But by the same token, if activate is getting populated with packages, you must be running some software jobs, to some ITCM agents that register with that SS.

Regards,

Brian

Hi Brain,

Thanks for the clarification once again. But i just have one more question on below.

"So if you deploy a software package to 25 agents registered to the same scalability server, the package will be available in the library\activate folder until all 25 have executed the job, or until the job container is completed/terminated at the Domain Manager-- whichever comes first."

Question : You meant to say the package will be available at Library/activate in all 25 agents machines reporting to the scalability server or just in the library/Activate folder of scalability server ? 

Thanks and Regards

Varun kumar

Actually both.  It will be a link in the activate folder until all agents have completed or the job timed out.  As the agent connects it will be transferred to the agent activate folder until the job on that agent is complete  failed or the job timed out.

Get Outlook for Android<https://aka.ms/ghei36>

Scalability server:

..CA\DSM\SD\LIBRARY\activate

for as long as any container(s) [unexecuted target(s)] are referencing it

Agent:

..\CA\DSM\SD\TMP\activate

for as long as the agent is executing the package

Of course the agent download method is a factor.

NOS = Package executed from SDLIBRARY$ share

NOS-less = Package downloaded to agent, and executed locally

Default is NOS

Fallback is NOS-less

NOS won't work unless the SDLIBRARY$ share is enabled.

Thus agents will automatically fallback to NOS-less in absence of SDLIBRARY$ share on SS.