AttributeError on using multi gpu even on using ddp
🐛 Bug

I am getting attribute not found error on using multi gpu. The code works fine on using a single gpu. I am also using ddp as suggested. Here's a traceback.
Traceback (most recent call last):
  File "/home/sohigre/STL/stl_bert_trial_lightning.py", line 245, in <module>
    trainer.fit(model)
  File "/home/sohigre/anaconda3/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 952, in fit
    self.ddp_train(task, model)
  File "/home/sohigre/anaconda3/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_data_parallel.py", line 500, in ddp_train
    self.optimizers, self.lr_schedulers, self.optimizer_frequencies = self.init_optimizers(model)
  File "/home/sohigre/anaconda3/lib/python3.7/site-packages/pytorch_lightning/trainer/optimizers.py", line 18, in init_optimizers
    optim_conf = model.configure_optimizers()
  File "/home/sohigre/STL/stl_bert_trial_lightning.py", line 178, in configure_optimizers
    total_steps = len(self.train_dataloader()) * self.max_epochs
  File "/home/sohigre/STL/stl_bert_trial_lightning.py", line 109, in train_dataloader
    return DataLoader(self.ds_train, batch_size=self.batch_size,num_workers=4)
  File "/home/sohigre/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 594, in __getattr__
    type(self).__name__, name))
AttributeError: 'Abuse_lightning' object has no attribute 'ds_train'


Code sample
  def prepare_data(self):
    # download only (not called on every GPU, just the root GPU per node)
    df = pd.read_csv(self.filename)
    self.df_train, self.df_test = train_test_split(df, test_size=0.2, random_state=self.RANDOM_SEED)
    self.df_val, self.df_test = train_test_split(self.df_test, test_size=0.5, random_state=self.RANDOM_SEED)
    self.tokenizer = BertTokenizer.from_pretrained(self.PRE_TRAINED_MODEL_NAME)
    self.ds_train = AbuseDataset(reviews=self.df_train.comment.to_numpy(), targets=self.df_train.Score.to_numpy(),
                          tokenizer=self.tokenizer,max_len=self.max_len)
    self.ds_val = AbuseDataset(reviews=self.df_val.comment.to_numpy(), targets=self.df_val.Score.to_numpy(),
                          tokenizer=self.tokenizer,max_len=self.max_len)
    self.ds_test = AbuseDataset(reviews=self.df_test.comment.to_numpy(), targets=self.df_test.Score.to_numpy(),
                          tokenizer=self.tokenizer,max_len=self.max_len)

  @pl.data_loader
  def train_dataloader(self):
    
    return DataLoader(self.ds_train, batch_size=self.batch_size,num_workers=4)

Environment

CUDA:

GPU:

GeForce GTX 1080 Ti
GeForce GTX 1080 Ti
GeForce GTX 1080 Ti
GeForce GTX 1080 Ti


available:         True
version:           10.1


Packages:

numpy:             1.18.1
pyTorch_debug:     False
pyTorch_version:   1.5.1
pytorch-lightning: 0.8.4
tensorboard:       2.2.2
tqdm:              4.42.1


System:

OS:                Linux
architecture:

64bit



processor:
python:            3.7.6
version:           #1 SMP Debian 4.19.118-2+deb10u1 (2020-06-07)