DevTest is a soft real-time distributed system, and, as such care must be taking when planning an installation or upgrade. This document aims to assist those wishing to avoid reliability, performance and communication issues when installing and configuring DevTest, and consolidates information found elsewhere within the DevTest documentation.
The information here should be used in conjunction with the release notes for the DevTest version being configured.
DevTest ships with an internal database (Derby) configured that allows for a single machine to be used for evaluation, demonstration or debugging purposes only.
The Derby database is not supported for extended use, use in a deployed system or use within a distributed system and must be replaced with an enterprise grade database.
- This enterprise database must be well maintained, and have a high bandwidth/low latency link to DevTest. Database maintenance must be arranged locally.
- It is recommended to ensure that schema level backups are performed as appropriate for your business needs – issues arising from database corruption as a result of database rather than DevTest issues will need to be recovered from a backup or via creation of a new schema.
- Databases that require periodic re-building of the indexes may, dependent upon workload, need to have the frequency of the re-build increased in order to allow proper functioning of DevTest.
- If a database exhausts its storage ("tablespace" or filesystem) then the state of the database will be inconsistent - it will not be possible to continue with a database in this state. A database restored to a new schema, or an empty schema must be provisioned for further use of DevTest.
- DevTest requires two database schemas – one use by the Enterprise Dashboard, and one by the Registry and other components. These schemas may reside on the same server instance, but must be separate schemas.
- The supported enterprise database types are DB2, Oracle, MySQL and Microsoft SQL Server. For supported versions please consult the release notes for the DevTest release that you are configuring.
- The DevTest product requires that schemas are configured to use UTF-8 character encoding. Should this restriction not be observed then it will not be possible to log in to DevTest.
- As of DevTest release 9.0, in-place database schema upgrades are supported – elevated database privileges such as DBA privilege may be required in order to perform such an upgrade.
- If DevTest is being installed rather than upgraded then new schemas must be provided. It is not possible to re-use an existing schema.
Please note that, once upgraded, it is not possible to revert to a previous version as the schema will no longer be compatible.
DevTest relies on good network communication being available at all times, and is dependent upon network performance
- No server component may have a network latency to the database that exceeds 20ms.
- No server component may have a network to the registry or portal that exceeds 20ms.
- The workstation may exceed this 20ms limit, but reliability and performance issues should be expected. If the Workstation round trip time exceeds 100ms then the configuration should be considered unsupported.
Any issues arising from network latency, bandwidth constraints or packet loss will need to be addressed by the local network engineering team or by re-engineering your system.
DevTest requires that hostname resolution work correctly at all times, in both a forward and reverse direction – that is, resolving a name to an IP address must function correctly, and then resolving that IP address must yield the original host.
NAT (Network Address Translation)
DevTest communicated via ActiveMQ, with messages containing network endpoint information. It is therefore not possible to use DevTest via NAT.
Virtual IP / Network Address Changes.
DevTest does not support the use of Virtual IP or any installation where the network address may change during the lifetime of a server component (Registry, Portal, Broker, Coordinator, Simulator or Virtual Service Environment).
It is necessary for DevTest to resolve a hostname internally to the same address as that perceived from outside the system upon which it is installed, and thus installations using virtual IP will encounter issues.
Since the resolution of the hostname is performed at component start a change of address of the running system is also not permitted.
The File System
DevTest writes real-time communication data and log files into a temporary directory. Given the real-time nature of this data, it is required that this temporary directory (commonly referred to as lisatmp) is situated on a local file system. Systems that have lisatmp on a remote file system will experience issues and are not supportable.
This constraint extends to installations that are present within a Virtual Machine (VM) – the disk image used by the VM must reside on local disking. It may be necessary to consult your system administrators in order to determine if this is the case as the disking may appear to be local even if the disk image is remotely hosted.
Connections to the System Under Test (SUT) or Virtual Service Environment
It is necessary to ensure that your Simulators have adequately fast and reliable communications with your SUT, and that the Virtual Service Environments (VSEs) have adequate connectivity to any systems that they depend upon (this includes JMS/MQ environments, live systems, databases used)
Any database connections required for test or virtual service execution must be established via the use of a Type 4 (pure Java) JDBC Connector.
Please consult the database vendor for limitations and configuration.
Integration with 3rd Party Products
DevTest provides interfaces, such as a REST API and Junit support, that allow integration with third-party products. When configuring these integrations CA Support can supply advice on the use of the interfaces, but cannot provide specific information on a given product.
Where possible it is advisable to exercise the DevTest interface independently of any product in order to determine where any issues lie.
Customisations / Addition Functionality
DevTest may be extended with the addition of customised code that adds such things as test steps and additional protocols to the product.
It is essential that such customisations are compiled against the version of DevTest in use – problems arising from code not compiled against the correct release may be unpredictable and not limited to the additional functionality.
Consult the owner of the customisation code to ensure that it has been correctly compiled for use in the release that you have.
Running Mixed Versions
It is not permitted to run mixed versions of the major components of DevTest – that is, the Registry, Portal, Broker, Coordinator, Simulator, Virtual Service Environment and Workstation must all be at the same release.
It is, however, permitted to connect to an Enterprise Dashboard that is of a later version than the other components, for instance
- Connecting a Registry from version 8.5 to an Enterprise Dashboard from 9.5.1 is permitted since the Enterprise Dashboard is of a later version than the connecting registry
- Connecting a 9.5.1 Registry to a 9.1 Enterprise Dashboard is not permitted since the Enterprise Dashboard is of an earlier version than the connecting registry
It is therefore recommended that the Enterprise Dashboard be the first component to be addressed during an upgrade.
Operating System Considerations.
When deploying DevTest it is necessary to consider not only the product but also the environment in which it is running – in addition to ensuring that the system has adequate physical resources there it is also necessary to consider the OS resources that may require adjustment.
Linux Hosts – Maximum Number of Open Files per Process
In particular, on Linux hosts, the maximum number of open files per process may, by default, be configured to a value suitable for interactive use rather than a value suitable for servers.
This value may be configured on a per-user basis, and may be ascertained by entering the command
at the shell prompt of the user that will be running DevTest.
Should the value returned be less than 10240, then it should be increased. Since the configuration of this value may be vendor specific, please consult the OS vendor’s documentation for instructions on how to increase this value.
Not increasing this value will have adverse effects upon DevTest, with the VSE, Simulator and database machines being most likely to encounter difficulties.
An ephemeral port is a networking term that refers to the network port used for the outgoing leg of a connection. Each end of a network connection is identified by its network address and its port – typically the destination port is known, and the source port is randomly allocated from the pool of ephemeral ports.
Consider a connection to a DevTest Registry from a Coordinator
The connection is to a known address/port pair, or example tcp://registry.my.net:2010/Registry
where registry.my.net corresponds to a known address, and the port is 2010.
However, this connection must originate from an address/port pair – in this case the coordinator machine will be known in advance, but the port will be randomly selected by the operating system.
If there are not enough ephemeral ports available to support all of the connections that are required for every process running on the system, then the ephemeral ports ranges can be extended.
The Internet Assigned Numbers Authority (IANA) recommends that the ephemeral port range be 49152 to 65535.
Many Linux Implementations use the range 32768 to 61000.
Microsoft Windows 7 / Server 2008 and later IANA range by default. Versions earlier than Vista/Server 2003 used 1024 to 5000
Apple OS X used the IANA range.
All modern operating systems allow the ranges to be changed, however, and, in the event that this has been done some issues opening sockets may be encountered.
Changing Ephemeral Port Allocations.
As would be expected, changing the ephemeral port is OS specific. A summary of how these may be adjusted on various OS types can be found here http://www.ncftp.com/ncftpd/doc/misc/ephemeral_ports.html. Should this document not provide the appropriate information for your specific operating system and version then it will be necessary to obtain the required instructions from the operating system vendor.