Hi Mike,
I have seen sporadic issues caused due to memory issues, but checking on the current documentation, I found the following known issue which is fixed in R12.52 SP1 CR04, which could match with the behavior you described:
Defects Fixed in 12.52 SP1 CR04 - CA Single Sign-On - 12.52 SP1 - CA Technologies Documentation
Policy Server Generates a Core
Deadlock with the ACE initialization functions causes Policy Server to generate a core.
Just in case this helps you to solve your issue.
Best regards,
Albert.