Torch-Summary Integration to replace ModelSummary
🚀 Feature, Motivation, Pitch
Hello, I am the current maintainer of torch-summary, which is a rewrite of yet another torchsummary library. Currently, ModelSummary is a builtin PyTorch-Lightning feature, but it is limited in scope and support, especially in more complex models. The proposal here, seconded by @carmocca , is to deprecate/remove the current ModelSummary implementation in pytorch-lightning and instead move to torch-summary as a more sustainable and long-term solution to the problem of visualizing model details.
Implementations
There are several ways to implement this idea, all of which I am open to:


Using torch-summary as a required dependency for pytorch-lightning by adding it to requirements.txt and integrating it into applicable models. Lightning would use the current public API for torch-summary.


Bring the torch-summary code in-house as an upgrade to the existing code as a one time upgrade.


Adopt torch-summary (or a copy of the repository) into the PytorchLightning organization as a Lightning-specific repository that can be developed independently of the original project. (in other words, surfacing an API that only works for Lightning models). I hope that this would add additional contributors that will help the project grow to fit more diverse kinds of models.


Additional context
As the maintainer, I will say that this project is far from perfect, and there are many kinds of models that torch-summary does not yet fully support. e.g. functional layers, lists or dicts of tensors, other types of CUDA, etc. The project is currently only maintained by me, which is sustainable for the current user base, but not sustainable if Lightning users' needs eventually outpace the project.
However, I do think that separating the ModelSummary functionality from Lightning is important, and I think that expanding this feature is something that a lot of users would enjoy.
(Continuing discussion from #4521)