Biography
Siyu Yuan (员司雨) is a third-year Ph.D. student at Fudan University. She is devoted to making machines have human-like cognitive abilities and aligning autonomous generative agents with human cognition. Her research topics are mostly around cognitive science with generative agents, including (but not limited to)
- Cognitive Reasoning, especially on exploring the cognitive reasoning abilities of generative agents, including analogical reasoning, concept understanding, role-playing and belief exploration of language agents. The ultimate goal is to enhance the understanding of these agents about themselves and others, thereby enabling them to generate responses that align better with human cognition.
- Strategic Planning, especially on equipping generative agents with human-level planning capabilities revolving around constrained planning, tool invocation and multitasking planning.
- Knowledge Acquisition, especially on excavating knowledge based on generative agents, including concept acquisition, script generation, idiom construction, pun generation and analogy making. These aim to construct rich knowledge resources that can be effectively utilized.
(Download my resumé. The last update was on 2024-04.)
Interests
- Cognitive Science
- Applications of LLMs
- Writing Novels
Education
Ph.D., Statistics, 2021-2026 (estimated)
Fudan University
B.S., Bachelor of Data Science and big data technology, 2017-2021
Fudan University
News
- Apr. 2024 Can large language models understand puns? Check out our new pre-prints. We leverage three popular pun tasks to systematically evaluate LLMs’ capability of understanding puns.
- Apr. 2024 Check out two pre-prints on Role-playing Agents, which extend InCharacter! CROSS systematically evaluate LLMs’ capability on the character profiling task, i.e., summarizing profiles for characters from fictional works. LIFECHOICE investigate whether LLMs can predict characters’ decisions provided with the preceding stories in high-quality novels.
- Feb. 2024 Congratulations on EASYTOOL and TaskBench being accepted to ICLR 2024 Workshop on LLM Agents!
- Feb. 2024 Introducing TimeArena, a Time-Aware simulated textual environment for language agents to complete multiple tasks in the shortest time, which means simulating realistic temporal & resource constraints! Check out our project page for more details!
- Feb. 2024 InCharacter is out! A new method to test personality fidelity in Role-Playing Agents using psychological interviews. Play with InCharacter demo!
- Jan. 2024 Enhance LLM-based agents with EASYTOOL! Effortlessly convert complex, varied tool documentation into streamlined, unified tool instructions. Significantly improve performance and reduce token consumption!
- Dec. 2023 Gave a talk at Tencent AI Lab about Coscript. Thanks for the invitation!
- Dec. 2023 Congratulations on our paper IdiomKB being accepted to AAAI 2024! Our work focuses on creating a multilingual knowledge base for idioms with the help of Large Language Models (LLMs), aiming to improve idiomatic translation in smaller models.
- Dec. 2023 Join in EMNLP 2023, Singapore! Our work SCAR will be in the poster session!
- Nov. 2023 We released TaskBench, a benchmark for evaluating the task automation capabilities of large language models.
- Oct. 2023 Check out our Auction Arena! We explore how LLMs navigate the complex and dynamic environment of auctions! We introduce AucArena, a novel simulation environment to evaluate the planning and strategic abilities of LLMs. Play with arena demo and see if you can beat AI!
- Oct. 2023 Our paper SCAR on analogical reasoning got accepted at EMNLP 2023 (Findings)! See you in Singapore.
- Sept. 2023 Start Student Researcher Internship at Microsoft Research Asia, advised by Dr. Kaitao Song!
- July 2023 Our paper Coscript got an Outstanding Paper Award in ACL 2023 (top 1%)!
- July 2023 Gave a talk for Peking University Shenzhen Graduate School Shanghai Alumni Association.
- May 2023 Check out two pre-prints on Analogical Reasoning. AnalogyKB is a million-scale analogy KB derived from existing KGs, to enable machines to achieve analogical reasoning skills. SCAR is a new challenge for evaluating the structure abduction ability of LLMs for scientific analogies, which is essential for human-like analogical reasoning.
- May 2023 Two papers accepted to ACL 2023! One is Coscript on constraint language planning, and the other is KPCE on concept extraction through the lens of a Structural Causal Model.
- Jan. 2023 Start Student Researcher Internship at Bytedance, working with the great AILab!
- Oct. 2022 Our work Generative Entity Typing with Curriculum Learning got accepted at EMNLP 2022!
- Sept. 2022 Our work (CISPE) about emotion recognition in conversation has been reported in the ECML PKDD 2022 online!
Experience
Research Intern
Microsoft Research Lab Asia
Mentored by Dr. Kaitao Song and Dr. Kan Ren. Autonomous Agents with Planning and Tool Use.
Research Intern
ByteDance AI Lab
Mentored by Dr. Jiaze Chen and Dr. Changzhi Sun. Working on LLM Evaluation and Instruction Tuning on LLMs.
Research Intern
Brain Technologies Inc
Mentored by Dr. Charles Jankowski. Working on Symbolic Knowledge Distillation and LLM Prompt Engineering.
Student Researcher
Knowledge Works Lab at Fudan University
Working on Knowledge Generation and Knowledge Graph.
Awards
ACL 2023 Outstanding Paper Award
Outstanding Graduate Student of Shanghai Colleges and University
Outstanding Student Pacemaker of Fudan University
China National Scholarship
Featured Publications
Recent Publications
Quickly discover relevant content by filtering publications.
(2024).
A good pun is its own reword: Can Large Language Models Understand Puns?.
Preprint.
(2024).
Character is Destiny: Can Large Language Models Simulate Persona-Driven Decisions in Role-Playing?.
Preprint.
(2024).
Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works.
Preprint.
(2024).
InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews.
Preprint.
Gallery
