is there anybody who run checks for testing agent functionality (not only agent alive)?
Short answer: Yes.
Long answer: We run a heartbeat job by generating a file with a timestamp on the server. Then we transfer that file to some of our agents with a JOBF, and for others we generate a file with a timestamp on the agent directly with a JOBS. Either way, the central Nagios server monitors all those files for "freshness".
This works extremely well and allows us to find hanging agents (those that don't function/don't process jobs, but still have an OS process and respond to the engine as if they were alive), and it also allows us to generate statistics on effective agent performance by analysing the various time stamps for that heartbeat job in the "AH" db table.
We designed a simple Testjob with commands like "whoiam", "uname -a", "ls" etc. an run this Testjob against an "ALL Linux Agents HOSTG" on demand.
Same thing with Windows Testjob.
Our approach is similar to what NilsHesselink608217 described. We built a workflow that tests all of our agents that we call SYSTEM.VALIDATIONS. We've expanded this workflow to also validate critical database connections, FTP connections, and the availability of several vendor products that our business depends upon.
We are a small shop with only a handful of agents, so we don't have a need to run this regularly. This workflow is extremely useful during DR testing.
Retrieving data ...