Lightweight Hyperparameter Datastructure
🚀 Feature
A simple and flexible way to store hyperparameters in a dict/Namespace-like object.
Motivation
Pitch
An object that behaves like this:
# just like Namespace
hparams = Hyperparameters(x=1, y=2)  
# or from a dict
hparams = Hyperparameters({"x": 1, "y": 2})  
# it could support nesting
hparams = Hyperparameters({"x": 1, "y": {"a": 3}}) 

# Namespace-like look up
x = hparams.x
a = hparams.y.a
# or like a dict
x = hparams["x"]
a = hparams["y"]["a"]

# we will have to check for invalid keys
hparams["batch_size"]  # ok
hparams["batch-size"]  # error
hparams["batch size"]  # error
Optional features:
# could support reading from files
# useful for checkpoint loading
hparams = Hyperparameters.from_yaml("file.yml")

# could support flattening/sanitizing nested structure
# useful for logging to TensorBoard
# or to make it pickleable
clean = hparams.flatten().sanitize()

# note that these internal methods will prevent these keys:
hparams.flatten = True  # Problem, because not hyperparameter!
Pro:

Behaves like dict and Namespace in one, so it will work out of the box with PL since internally we treat it as dict anyway (saving checkpoint).

Contra:

Any convenience methods/attributes we add to the class will collide with hyperparameters names. See example above.
Need to  spend extra effort making sure user does not pick bad dict keys

Considerations

"Hyperparameters" may be a too specific name for such a general datastructure.

Additional context
Discussed on slack and idea has popped up in other isses as well.
Related to #1841, #1735 and would solve #1737