Support to rotate /core files

[thenamehasbeentake]

We encountered the problem not long ago.
#9885

bluefs _allocate unable to allocate memory and the osd pod keeps trying to start and crash with large core file, causing the disk to fill up. Insufficient system disk capacity lets pods to be evicted. Shall we use logrotate to rotate /core files?

[parth-gr]

Logrotate is nice feature to remove the old logs and just keep the new logs,

If your OSD is crashing just because of the space, either use log-rotate or increase its storage.
If there is another problem then storage free feel to report the issue with osd logs.

[thenamehasbeentake]

I means that osd pod will crash when starting like the issue above. Osd crashes and a core file is created. Then kubernetes pull osd pod up. And osd crashes repeatedly. Lots of core files are created after repeating this many times.

When the diskPressure has satisfied an eviction threshold, all pods on the node will be affected.

If we deploy the cluster without kubernetes, the osd will not be retried after it crashes, and there only one core file would be created.

[parth-gr]

yaa it's good practice to use the log rotate, To avoid excessive storage utilization

[travisn]

@thenamehasbeentake Please open a new github issue with these details so we can look into logrotate. Although IIRC the core files are in a system directory where rook may not have control to run logrotate. The files are in /core?

[thenamehasbeentake]

Core dump file is in /var/log/ceph as same as log file like this