Sheikh Shafayat

About Me

Hi, I am an incoming PhD student at Max Planck Institute for Software Systems.

My current research interests include LLM reasoning (RLVR), AI creativity, and open-endedness. I did my undergraduate at KAIST where I worked on evaluation, multilinguality, and cultural NLP. Currently, I work as an intern with Abhilasha Ravichander.

A (partially) maintained CV of mine can be found here.

I am really open to meeting and chatting with new people. Feel free to send me an email if you want to chat about something (any paper, potential collaboration, a cool novel, or just about anything). If you happen to be in Germany, shoot me an email to grab a coffee or go bouldering together.

On my free time, I enjoy bouldering, running, or basking in the sunset with a novel.

Research Interests

NLP: LLM reasoning, algorithmic creativity, open endedness, multilinguality

News

[May. 2025] New Preprint: Can Large Reasoning Models Self-Train?
[Dec. 2024] New Preprint: A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls
[July. 2024] MultiFAct for Multilingual Factuality got accepted to COLM-2024
[May. 2024] BEnQA got accepted to ACL Findings
[Apr. 2024] MultiFAct got accepted to ICLR Global AI Cultures Workshop

Publications

Arxiv

Can Large Reasoning Models Self-Train?

Sheikh Shafayat*, Fahim Tajwar*, Ruslan Salakhutdinov, Jeff Schneider, Andrea Zanette

PDF Code Project Page Preprint

Arxiv

A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls

Sheikh Shafayat, Dongkeun Yoon, Woori Jang, Jiwoo Choi, Alice Oh, Seohyon Jung

PDF Preprint

COLM

Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScore

Sheikh Shafayat, Eunsu Kim, Juhyun Oh, Alice Oh

Accepted to COLM 2024

PDF

Arxiv

The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models

Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim, Dongkeun Yoon, Guijin Son, Yejin Cho, Sheikh Shafayat, Jinheon Baek, Sue Hyun Park, Hyeonbin Hwang, Jinkyung Jo, Hyowon Cho, Haebin Shin, Seongyun Lee, Hanseok Oh, Noah Lee, Namgyu Ho, Se June Joo, Miyoung Ko, Yoonjoo Lee, Hyungjoo Chae, Jamin Shin, Joel Jang, Seonghyeon Ye, Bill Yuchen Lin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo

PDF Code Accepted to NAACL 2025 (Best Paper Award)

ACL

BEnQA: A Question Answering and Reasoning Benchmark for Bengali and English

Sheikh Shafayat, HM Hasan, Minhajur Rahman Chowdhury Mahim, Rifki Afina Putri, James Thorne, Alice Oh

Accepted to ACL-Findings 2024

PDF Code

ACL

LangBridge: Multilingual Reasoning Without Multilingual Supervision

Dongkeun Yoon, Joel Jang, Sungdong Kim, Seungone Kim, Sheikh Shafayat, Minjoon Seo

Accepted to ACL-Main 2024

PDF Code

Current Readings

Here are a few latest books that I am reading/just finished. I am happy to receive book recommendations from anyone!

East of Eden (John Steinbeck): This is the most American book I have ever read. A fantastic two family saga relating war, violence, love, immigration, assimilation, capitalism, westward expansion, corruption and moral ambivalence. This book does a justice to the first generation of Chinese American experience. The characterization of Lee, a Chinese American, as one of the central moral pillars of the story was way ahead of its time (1952).
Nexus (Yuval Noah Harari): Very interesting read on the role of information technology in sustaining large scale human networks. My position on this book is somewhat mixed. While I find Harari’s thesis on how information networks and intersubjective reality shape human history to be absolutely brilliant, he seems to be deeply mistaken on the current state of AI progress and what it means by intelligent, decison making systems. I love the part I of the book, but part II? Not so much.
Kafka on the Shore (Haruki Murakami): This is my second Murakami book after Norwegian Wood. The writing style and some recurring themes are strikingly similar. Kafka on the Shore was a dreamy, melancholic, and soothing read that made me think about the role of memory, strength and human fragility. It is the memories we live by, live for and hold onto.
Flowers for Algernon (Daniel Keyes): This is such a tiny novella, yet so powerful. An intellectually disabled person goes through surgical procedure to 3 fold his IQ. The book is a daily reflection journal of the man. As a result of the surgery, he slowly becomes smarter in every page. But as he becomes smarter, he becomes more self-concious, less happy and less kind. As someone working in artificial intelligence, this novella made me think for a long time.