Avoid storing a list of outputs to compute aggregated metrics at the end of the epoch.
Hi,
to my understanding, the current way of logging an aggregated metric at the end of an epoch requires implicitly to store the outputs of all steps in the epoch. There are tasks like semantic segmentation requiring to accumulate a confusion matrix over steps, e.g. in order to compute the mIoU metric. However, storing all those confusion matrices for the only sake of aggregating them at the end is a waste of resources (it might occupy gigabytes of memory). Do you have a preferred solution to sidestep the issue? It would be better to also offer the possibility of aggregating information while traversing the steps (without storing all outputs) and let the aggregated data to be accessible at the end of the epoch e.g. for logging.
Thank you.