Logs should not be allowed to grow indefinitely

[spinque]

Some DPUs can be very verbose. This combined with situations where many intermediate files are processed makes the log files grow quickly to tens of GB.
ODN is often deployed on VMs where large disk space is not expected to be necessary (certainly not for logs).
In this settings, it happens that pipelines fail because no space is left on device. Just making a DPU too talkative makes the whole system becomes vulnerable. This should never be allowed.

I suggest to make sure that log files can never grow larger than a (configurable) size. I would also suggest not to implement this as a scheduled log rotation mechanism, because that may be not sufficient (one single pipeline can already fill up the disk space before the log rotation comes in action)

[tomas-knap]

Some notes:
http://stackoverflow.com/questions/18083450/maximum-file-size-supported-in-log4j-fileappender
http://stackoverflow.com/questions/10447866/log4j-daily-rolling-catalina-out-without-restarting-tomcat

[tomas-knap]

See for backend: d6ce758

[tomas-knap]

Doc: https://grips.semantic-web.at/display/UNVI/Logging+in+UnifiedViews

[tomas-knap]

Solved by introducing max log size for backend, frontend, master app