How to inference on GPU?
❓ Questions and Help
Hi.
I have trained a Model with Trainer.fit(). Now I want to load the checkpoint at another place and preform inference. But I have no idea how to inference on GPU.  Where could I assign a GPU for my inference just like assigning a GPU before training:
trainer = pl.Trainer(max_epochs = cfg['n_epochs'],
                            callbacks=[checkpoint_monitor, lr_monitor],
                            gpus=1)

My inference code is (simplified version):
model = plModel.load_from_checkpoint('lightning_logs/version_0/checkpoints/model_epoch=17_val_loss=95.99.ckpt')
model.eval()
test_loader =test_dataloader()
        
correct_count = 0
nsample = len(test_loader)
for i, data in enumerate(test_loader):
    output = model(data['x'])
    label = data['label'].data.cpu().numpy()
    pred = nn.functional.softmax(output, dim=1)
    pred = np.argmax(pred.data.cpu().numpy(), axis = 1)
    correct_count += np.sum(pred == label)
accuracy = correct_count / nsample

What's your environment?

OS: [ Linux,]
Packaging [pip]
Version [1.1.0]