you can add the:
AX_JAVA_OPTIONS=2048m;export AX_JAVA_OPTIONS
line to the remote agents also and re-invoke the sosite then restart. This should allow you more head room on available memory.
Beyond that we would need to look at what is using all the memory on the server. As a general rule we recommend 2048m as a standard setting so if you have issues after upping to that on the agents I would open a ticket with support regarding.