Caching problem when deleting a dataset 
Thanks for reporting! Can you also share the error message printed in step 5? I did not store it at the time but I'll try to re-do a mwe next week to get it again I haven't managed to reproduce this issue using a [notebook](https://colab.research.google.com/drive/1m6eduYun7pFTkigrCJAFgw0BghlbvXIL?usp=sharing) that follows the steps to reproduce the bug. So, I'm closing it.

But feel free to re-open it if you have a better reproducer.
### Describe the bug

Pushing a dataset with n + m features to a repo which was deleted, but contained n features, will fail.

### Steps to reproduce the bug

1. Create a dataset with n features per row
2. `dataset.push_to_hub(YOUR_PATH, SPLIT, token=TOKEN)`
3. Go on the hub, delete the repo at `YOUR_PATH`
4. Update your local dataset to have n + m features per row
5. `dataset.push_to_hub(YOUR_PATH, SPLIT, token=TOKEN)` will fail because of a mismatch in features number


### Expected behavior

Step 5 should work or display a message to indicate the cache has not been cleared

### Environment info

- `datasets` version: 2.12.0
- Platform: Linux-5.15.0-88-generic-x86_64-with-glibc2.31
- Python version: 3.10.10
- Huggingface_hub version: 0.16.4
- PyArrow version: 11.0.0
- Pandas version: 2.0.0
