😎 About Me

I am Chengyou Jia (贾成铕), a Ph.D candidate in Xi’an Jiaotong University, major in Computer Science, under the supervision of Prof. Minnan Luo. I am also working closely with Prof. Xiaojun Chang. I am currently a visiting student in Singapore under the supervision of Pro. Ivor. Previously, I was a research intern at Shanghai AI LAB, supervised by Dr. Zhiyong Wu. Before starting my doctoral studies, I received my B.E. degree from Xi’an Jiaotong University in 2021.

I have authored several publications in top-tier conferences and journals, including AAAI, ACM-MM, IEEE TIP, IEEE TCSVT, among others. I also serve as a reviewer for esteemed conferences and journals like ECCV, AAAI, ICASSP, IEEE TIP and IEEE TNNLS.

Research Interests

I am working in the field of CV & Multi-modal. My current research interests and past experience can be summarized as follows:

• Controllable Image Generation: Text-to-image; Layout-to-image; Complex Scene Generation
• Multimodal learning: Multimodal for Open-set Recognition; Multimodal for Automated Agents
• Object Detection & Identification: Person Search, Person Re-Identification

🔥 News

• 2024.12:   Our paper [OS-Genesis] is recently released. 🎉🎉
• 2024.12:   One paper is accepted by IEEE TCSVT. 🎉🎉
• 2024.11:   Our papers [ChatGen, OS-Atlas, AgentStore] are recently released. 🎉🎉
• 2024.10:   Ended a fulfilling internship at Shanghai AI Lab and started as a visiting student in Singapore.
• 2024.07:   One paper is accepted by ACM-MM 2024. See you in Melbourne, Australia. 🎉🎉
• 2023.11:   Two papers are accepted by AAAI 2024. 🎉🎉
• 2023.09:   One paper is accepted by IEEE TIP. 🎉🎉

📖 Educations

• 2021.09 - 2026.06 (expected), M.S. + Ph.D Student, Computer Science, Xi’an Jiaotong University.  
• 2017.09 - 2021.06, B.S. in Computer Science, Xi’an Jiaotong University.

💻 Internships

• 2024.11 - Present, Research Intern @ CFAR, A*STAR. Focus on Multimodal Agents for Image Generation.
• 2024.03 - 2024.10, Research Intern @ Shanghai AI LAB. Focus on Multimodal Agents for OS (Operating System).
• 2022.12 - 2024.03, Research Intern @ SGIT AI Lab, State Grid Corporation of China. Focus on Controllable Image Generation.

📝 Selected Publications

Preprint

ChatGen: Automatic Text-to-Image Generation From FreeStyle Chatting 🔥🔥 [Preprint]
Chengyou Jia*, Changliang Xia*, Zhuohang Dang, Weijia Wu, Hangwei Qian, Minnan Luo

(* means equal contributions)

Code   Project Page   Datasets   Models  

Preprint

OS-ATLAS: A Foundation Action Model for Generalist GUI Agents 🔥🔥 [Preprint]
Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, Yu Qiao

Code   Project Page   Demo  

Preprint

AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistant 🔥🔥 [Preprint]
Chengyou Jia, Minnan Luo, Zhuohang Dang, Qiushi Sun, Fangzhi Xu, Junlin Hu, Tianbao Xie, Zhiyong Wu

Code   Project Page  

ACM-MM 2024

Generating Action-conditioned Prompts for Open-vocabulary Video Action Recognition [CCF-A]

Chengyou Jia, Minnan Luo, Xiaojun Chang, Zhuohang Dang, Mingfei Han, Mengmeng Wang, Guang Dai, Sizhe Dang, Jingdong Wang

AAAI 2024

SSMG: Spatial-semantic map guided diffusion model for free-form layout-to-image generation [CCF-A]

Chengyou Jia, Minnan Luo, Zhuohang Dang, Guang Dai, Xiaojun Chang, Mengmeng Wang, Jingdong Wang

IEEE TIP

Collaborative Contrastive Refining for Weakly Supervised Person Search [CCF-A]

Chengyou Jia, Minnan Luo, Caixia Yan, Linchao Zhu, Xiaojun Chang, Qinghua Zheng

🧑‍ Other Paper

• Preprint OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis
  Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, Ben Kao, Guohao Li, Junxian He, Yu Qiao, Zhiyong Wu

• IEEE TCSVT PSDiff: Diffusion Model for Person Search with Iterative and Collaborative Refinement [CCF-B]
  Chengyou Jia, Minnan Luo, Zhuohang Dang, Guang Dai, Xiaojun Chang, Jingdong Wang, Qinghua Zheng

• ICASSP 2023 Towards Real-time Person Search with Invariant Feature Learning [CCF-B]
  Chengyou Jia, Minnan Luo, Zhuohang Dang, Xiaojun Chang, Qinghua Zheng

• AAAI 2024 Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [CCF-A]
  Zhuohang Dang, Minnan Luo, Chengyou Jia, Guang Dai, Xiaojun Chang, Jingdong Wang

• IEEE TCSVT Disentangled representation learning with transmitted information bottleneck [CCF-B]
  Zhuohang Dang, Minnan Luo, Chengyou Jia, Guang Dai, Jihong Wang, Xiaojun Chang, Jingdong Wang

• IEEE TIP Disentangled Generation with Information Bottleneck for Enhanced Few-Shot Learning [CCF-A]
  Zhuohang Dang, Minnan Luo, Jihong Wang, Chengyou Jia, Caixia Yan, Guang Dai, Xiaojun Chang, Qinghua Zheng

• IEEE TCSVT Counterfactual Generation Framework for Few-Shot Learning [CCF-B]
  Zhuohang Dang, Minnan Luo, Chengyou Jia, Caixia Yan, Xiaojun Chang, Qinghua Zheng

• Preprint Disentangled Noisy Correspondence Learning
  Zhuohang Dang, Minnan Luo, Jihong Wang, Chengyou Jia, Haochen Han, Herun Wan, Guang Dai, Xiaojun Chang, Jingdong Wang

🎖 Honors and Awards

Honors

• Oct.2022   Outstanding Graduate Student of Xi’an Jiaotong University, Award.
• Jun.2021   Excellent bachelor degree thesis award (Top 1%), Award.
• Jun.2021   Outstanding Graduate Student of Xi’an Jiaotong University, Award.
• Sep.2019   Outstanding Student in Xi’an Jiaotong University
• Sep.2018   Outstanding Student in Xi’an Jiaotong University

Competition Awards

• Jun.2021   Top 1% in the TianChi Global Video Cloud Innovation Challenge, Video Object Segmentation Algorithm Challenge (8/2904).

🎖 Scholarships

• 2023.09   Freshman First Prize Scholarship (PhD)
• 2022.09   First-Class Fellowships for Graduate Students at Xi’an Jiaotong University, Fellowships.
• 2020.09   Computer Science Special Scholarship at Xi’an Jiaotong University, Fellowships.

💬 Academic Services

• Reviewer: ECCV, AAAI, ICASSP, IEEE TIP, IEEE TCSVT and IEEE TNNLS.