When the data_engine queue had 35 Million items, the q_data_engine.sds file size was 35GB. Now that the queue is only at 5 million, the q_data_engine.sds file is still the same size. How do I shrink the queue file?
The queue file does not shrink until the queue is empty. I do not know for certain why this is, but I would guess this is for efficiency reasons. It is probably very efficient for the hub to only append data to the queue file until the queue is empty, at which point it can just delete (or truncate) the file. If it needed to rewrite the file when removing messages from the front of the queue, I think there would be a tremendous performance penalty. This is all speculation on my part, of course.
Good speculation though, Keith!
That's exactly why the hub behaves as it does. It would cost a tremendous amount of resources (while the hub is clearly very busy!) to do housekeeping like that. Waiting until the queue is empty then resetting the file to 0 bytes costs next to nothing however
There was an unusual spike in data being queued in the hub and the drive was almost full.The queues were finally unloading, but the system drive was still in danger of being filled, which is why I wanted to force the shrinking of the queue file.
Retrieving data ...