Bypassing AE limits in UC_SYSTEM_SETTINGS

Discussion created by Michael_Lowry on Jan 16, 2017
Latest reply on Mar 28, 2017 by Michael_Lowry
The Automation Engine has several system settings (in UC_SYSTEM_SETTINGS) that impose limitations on the amount of items (or size of file) that can be handled by certain requests.
AE system setting AE Java API impacted Message upon
exceeding limit
GENERIC_SEARCH_LIMIT SearchObject U04005935 Query OH & OFS tables instead.
MAX_EXPORT_COUNT ExportObject U04005884,
Split export into parts.
MAX_IMPORT_SIZE ImportObject U04001888,
Split input XML into parts.
GENERIC_ACTIVITIES_LIMIT ActivityList ? Query EH table instead.
GENERIC_STATISTICS_LIMIT TaskStatistics U04005662 Query AH table instead.

For example, we want to keep GENERIC_SEARCH_LIMIT set to a reasonably low number (2000 or so) so that searches performed by our hundreds of GUI users do not negatively impact the performance of the Automation Engine server. Ideally, we would like to impose a lower limit on ordinary users, but still retain the ability to retrieve full results for applications that interface with the AE. Unfortunately, there is no way to limit the size of user searches without also limiting the number of results that can be returned via the Java API SearchObject. There is currently no way to enforce one limit on the GUI, and a different limit on Java API calls. There is also no way to impose different limits on different users/user groups.

In this particular example, we decided to find a work-around: we opted to replace uses of the SearchObject class with equivalent SQL queries. (See the discussion entitled SQL query to list all objects (and folders) under a given folder.) This allowed us to fetch an unlimited number of results while still imposing a reasonably low GENERIC_SEARCH_LIMIT for normal users. Using SQL comes with downsides though: it goes against best application development practices and exposes us to the risk that these SQL queries will stop working after the AE DB structure is updated by a future version upgrade.

Has anyone else run into this before? What was your solution?