OptoLlama

Meet OptoLlama — a masked diffusion language transformer aimed to solve inverse design of multi-layer thin film structures.

Key Features

• Masked diffusion language model (MDLM)
• Support for reflectance, absorption and transmittance RAT spectra 📈
• Wave length from 300-2,000nm 💡
• State-of-the-art predictive performance for inverse material design 😎

Supporting Material

ArXiV Paper on MDLM: 📝 https://arxiv.org/pdf/2406.07524

Usage

Install Dependencies

python -m pip install torch
python -m pip install safetensors

Load Model Checkpoint

from safetensors.torch import load_file

model = OptoLlama()

safetensors_path = "optollama-model.safetensors"
state_dict = load_file(safetensors_path)
model.load_state_dict(state_dict)

Useful Information

Stat	Value
#Parameters	111,555,513
Best validation MAE	0.0140
top_p	0.9
top_k	5
Epochs trained	1,000
Best epoch	866
Batch size	256
n_blocks	6
n_heads	8
d_model	1,024
max_seq_length	20

Acknowledgements

This work is supported by the Helmholtz Association Initiative and Networking Fund through the Helmholtz AI platform, and the HAICORE@KIT grant.

---