jihyoung/M3C-retrieval

Enabling Chatbots with Eyes and Ears: An Immersive Multimodal Conversation System for Dynamic Interactions

[📜 Paper] [🖥️ Project Page] [📖 Dataset] [🤗 Model Weights]

_{Image Generated by DALL·E}

✅ TODO List

• Write documentation (README)
• Release M³C dataset
• Release dialogue module weight
• Release retrieval module weight
• Release training code
• Release inference code
• Release model self-chat code
• Launch Gradio demo for live chat

📚 Citation

@article{jang2025enabling,
  title={Enabling Chatbots with Eyes and Ears: An Immersive Multimodal Conversation System for Dynamic Interactions},
  author={Jang, Jihyoung and Bae, Minwook and Kim, Minji and Hakkani-Tur, Dilek and Kim, Hyounghun},
  journal={arXiv preprint arXiv:2506.00421},
  year={2025}
}