Layer7 API Management

Workaround for AWS/RDS when AuditArchiver max tablespace is unknown 

Mar 22, 2019 03:00 AM

1. Background : 

AuditArchiver does a nice job of printing % of database space that is user.   It can also trigger archiving process, and also BYPASS option when the database is getting too full.  Explained here :  https://communities.ca.com/message/241782782?commentID=241782782#comment-241782782  and in the manual.

 

2. But this is not possible with AWS/RDS

But, it seems AWS /RDS does not give abiltiy to set the max size for the mysql db.   

https://forums.aws.amazon.com/thread.jspa?threadID=97879

 

For AWS /RDS then the Audit log then gives : 

2019-01-22T16:17:25.732+1100 WARNING 115 com.l7tech.server.audit.AuditArchiver: 2205: Audit Archiver error: Max innodb tablespace size not defined 

And does not run all those special benefits. 

 

For running out of database space Amazon suggest to alarm the AWS disk partition.

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MonitoringOverview.html

 

3. Long term solution.

Happily a solution is already coded, and will appear in latter release of the API Gateway, where if the max db size cannot be detected it will read a value from the global properties and use that as the max db space available, and then calculate the percentage used. 

 

4. Short term solution 

Meanwhile, we can kind-of simulate that same calculation with some policy code and a scheduled task. 

 

4.1 Design

We use a variable, if you want it can be read from a global cluster property, as the max-number-of-audit-rows.   

And then use an SQL query to connect to the audit databse and determine the current number of rows : 

select count(*) as "auditCount" from audit_main;

 

Comparing the two, we've used javascript to calculate a percentage, but XPATH or Maths tactical assertion could also be used.

 

Then we write the % Audit record usage to the audit log file. 

 

 

4.1 Test Policy

 

Attached is test policy to do the calculation : 

With jdbc query : 

 

And javascript : 

 

 

4.2 Test Run

When we call the policy: 

 

Then we get the calculated % result : 

And entry gets written to the audit log : 

 

4.3 Setting the policy up as a batch job

We create a "BackGroundTask" type policy fragment : 

With the policy being much the same : 

 

And we setup a scheduled task to tin it periodically (say every 20min) : 

 

 

 

I've included both sets of policy as attachments. 

 

Cheers - Mark

Statistics
0 Favorited
4 Views
2 Files
0 Shares
0 Downloads
Attachment(s)
zip file
test-log-policy.xml.zip   1 KB   1 version
Uploaded - May 29, 2019
zip file
batch-job-policy.xml.zip   1 KB   1 version
Uploaded - May 29, 2019

Related Entries and Links

No Related Resource entered.