About Me

How to pronounce my name? Jiqun [Jee-chwen] Liu [Lee-oo]

I am an Assistant Professor of Data Science and Affiliated Assistant Professor of Psychology at the University of Oklahoma (OU). I hold a PhD in Information Science from Rutgers iSchool. I currently work with my students at the Human-Computer Interaction and Recommendation (HCIR) Lab at OU.

My research focuses on the intersection of Human-centered Information Retrieval, Machine Learning, and Cognitive Psychology, and seeks to apply the knowledge learned about boundedly rational users interacting with information in search and recommendation, user education and intelligent nudging. My recent projects focus on:

• Psychology-Informed Intelligent Information Systems: investigate the impact of users’ biases, heuristics, and expectations on human-AI interactions and develop useful, proactive IR, recommendation, and generative information systems.
• Bias-Aware, Ethical, and Socially Responsible AI: evaluate and enhance the FATE (fairness, accountability, transparency, and ethics) aspects of AI-assisted chatbots and personal assistants in critical application domains, such as health and medical information access, healthcare, K-12 education, online privacy and secruity, and business decision-making.

Our work is possible thanks to National Science Foundation, Microsoft, Data Institute for Societal Challenges, and OU VPRP. My recent CV can be found at here.

[GoogleScholar] [DBLP] [LinkedIn] [Twitter]

[Note to Perspective Students:] I am actively looking for self-motivated research and visiting students to join my Human-Computer Interaction and Recommendation group at OU and work on interesting cutting-edge problems in information retrieval and intelligent systems. Research opportunities are available at both undergraduate and graduate levels. I am especially interested in students with any of the following backgrounds and research interests:

• Human-Computer Interaction, Interactive Information Retrieval/Recommendation, UX Design/Research, Cognitive Psychology or Experimental Economics using quantitative or qualitative methods (or both);
• Machine Learning, Natural Language Processing (NLP), and Generative Artificial Intelligence (AI).
• Societal Impacts of Intelligent Information Systems, especially in healthcare and lifelong learning; Social and Community Informatics.

If you are interested, please email me at (jiqunliu@ou.edu) with your CV and a brief description of your previous research experiences & current research interests.

Recent News:

• Workshop: We are organizing the IWILDS’25 - The 5th Internation Workshop on Investigating Learning during Web Search at ACM WSDM 2025. Please join us in discussions on future learning in search and AI-enabled chat!
• Happy to serve as an Associate Editor for ACM Transactions on Information Systems!
• Award: Our proposal (with Dr. Yong Ju Jung) titled “Enhancing AI Literacy Through Maker-Based Learning with Generative AI” received Elfreda A. Chatman Research Award from ASIS&T SIG USE in 2024!
• Tutorial: We are organizing a tutorial on Evaluating Cognitive Biases in Conversational and Generative IIR at SIGIR-AP2024. Please access our tutorial materials through the ACM SIGIR slack channel: sigir24-searchunderuncertainty-tutorial!
• Full paper: “AI can be cognitively biased: An exploratory study on threshold priming in LLM-based batch relevance assessment” accepted by SIGIR-AP2024. Link to our arxiv paper here.
• Tutorial: We are organizing a tutorial on Testing, Mitigating, and Accounting for Cognitive Biases in Search Experiments at ACM SIGIR 2024. Please access our tutorial materials through the ACM SIGIR slack channel: sigir24-searchunderuncertainty-tutorial!
• Tutorial: We are organizing a tutorial on Modeling Search Interaction with Behavioral Economics at ACM CHIIR 2024. The tutorial website is online!
• Full paper “Characterizing and early predicting user performance for adaptive search path recommendation” accepted by ASIS&T 2023 and received 2023 SIG USE Best Information Behavior Conference Paper Award.
• Journal paper “Constructing and meta-evaluating state-aware evaluation metrics for interactive search systems” accepted by Information Retrieval Journal.
• Journal paper “Investigating the role of in-situ user expectations in Web search” accepted by Information Processing and Management.
• Full paper “A reference-dependent model for Web search evaluation: Understanding and measuring the experience of boundedly rational users” accepted by ACM Web Conference 2023.
• Full paper “A two-sided fairness framework in search and recommendation” accepted by CHIIR2023.
• New book Authored research book “A behavioral economics approach to interactive information retrieval: Understanding and supporting boundedly rational users” in production with Springer Nature. [book abstract]
• Journal paper “Toward Cranfield-inspired reusability assessment in interactive information retrieval evaluation” accepted by Information Processing and Management
• Full paper “Leveraging user interaction signals and task state information in adaptively optimizing usefulness-oriented search sessions” accepted and selected as a Vannevar Bush best paper nominee at JCDL2022.
• I will be serving as the Chair-elect for ASIS&T SIG AI in 2022!
• Grant: My project “CRII:III: A bias-aware approach to modeling users in interactive information retrieval” is funded by National Science Foundation (NSF)! The project’s homepage is here.
• Journal paper “Deconstructing search tasks in interactive information retrieval: A systematic review of task dimensions and predictors” accepted by Information Processing and Management
• Full paper “Interest development, knowledge learning, and interactive IR: Toward a state-based approach to search as learning” accepted by CHIIR2021
• I serve as a workshop co-Chair for iConference 2021.
• I serve as a co-Chair for ASIS&T AM 2020 SIG USE Symposium.
• Full paper “Investigating reference-dependence effects on user search interaction and satisfaction: A behavioral economics perspective” accepted by SIGIR2020.
• Book “Interactive IR user study design, evaluation, and reporting” published by Morgan & Claypool Publishers!

Grants & Awards

• Microsoft Research Award: Functional fixedness evaluation in human-large language model (LLM) interaction ($20,000, PI: Jiqun Liu)
• JFSF Summer 2024: Junior Faculty Summer Fellowship Award: Measuring potential harms of AI manipulation: An exploratory study, Dodge Family Colledge of Arts and Sciences, University of Oklahoma ($7,000, PI: Jiqun Liu)
• DISC Seed Fund 2023: Bias-aware evaluation of generative search engines: An exploratory study, The Data Institute for Societal Challenges (DISC), University of Oklahoma ($10,000, PI: Jiqun Liu)
• BFIP 2023: Bridge Funding Investment Program Award: Identifying and mitigating cognitive biases in generative-AI-assisted online learning, University of Oklahoma Office of the Vice President for Research and Partnerships ($33,918, PI: Jiqun Liu)
• JFF Summer 2023: Junior Faculty Fellowship Program Award: Toward a two-sided fairness framework for search and recommendation, University of Oklahoma Office of the Vice President for Research and Partnerships ($7,000, PI: Jiqun Liu)
• NSF REU 2022: Research Experience for Undergraduates (REU) supplemental award from the National Science Foundation (NSF) bias-aware user modeling in interactive IR project ($8,000, PI: Jiqun Liu)
• DISC Seed Fund 2022: Active learning method for fair and useful Learning to Rank (LTR) in information retrieval, The Data Institute for Societal Challenges (DISC), University of Oklahoma ($9,800, PI: Jiqun Liu, Co-PI: Chao Lan)
• JFSF Summer 2022: Junior Faculty Summer Fellowship Award: Assessing the reusability of session test collections in evaluating intelligent information retrieval systems, Colledge of Arts and Sciences, University of Oklahoma ($7,000, PI: Jiqun Liu)
• FIP 2022: Faculty Investment Program Award: Toward expectation-based effectiveness metrics for adaptive whole session information retrieval evaluation, University of Oklahoma Office of the Vice President for Research and Partnerships ($15,000, PI: Jiqun Liu)
• National Science Foundation (NSF): CRII:III: A bias-aware approach to modeling users in interactive information retrieval. ($174,959, PI: Jiqun Liu).
• JFF Summer 2021: Junior Faculty Fellowship Program Award: Learning task type and states for adaptively supporting users in complex search tasks, University of Oklahoma Office of the Vice President for Research and Partnerships ($7,000, PI: Jiqun Liu)

Publications

Book

• Liu, J. (2023). A behavioral economics approach to interactive information retrieval: Understanding and supporting boundedly rational users. Springer Nature. [book]
• Liu, J. & Shah, C. (2019). Interactive IR user study design, evaluation, and reporting. Synthesis Lecture on Information Concepts, Retrieval, and Services. Morgan & Claypool Publishers. [book]

Refereed Journal Article

• Zhang, Y. & Liu, J. (2024). Falling behind again? Characterizing and assessing older adults’ algorithm literacy in interactions with video recommendations. Journal of the Association for Information Science and Technology. (JASIST)
• Wang, B. & Liu, J. (2024). Understanding users’ dynamic perceptions of search gain and cost in sessions: An expectation confirmation model. Journal of the Association for Information Science and Technology. (JASIST) [Paper]
• Jung, Y. J. & Liu, J. (2024). Toward a conceptual framework characterizing the interplay of interest development, information search, and knowledge construction (ISK) in children’s learning. Aslib Journal of Information Management. Ahead-of-print. (AJIM)
• Markwald, M., Liu, J., & Yu, R. (2023). Constructing and meta-evaluating state-aware evaluation metrics for interactive search systems. Information Retrieval Journal. (IRJ) [Paper]
• Zhang,Y. & Liu, J. (2023). Deconstructing proxy health information-seeking behavior: A systematic review. Library and Information Science Research. 45(3): 101250. (L&ISR)
• Wang, B. & Liu, J. (2023). Investigating the role of in-situ user expectations in Web search. Information Processing and Management. 60(3): 103300. (IP&M)
• Jiang, T. & Liu, J. (2023). Reflection on future directions: A systematic review of reported limitations and solutions in interactive information retrieval user studies. Aslib Journal of Information Management. [Paper] (Aslib)
• Liu, J. (2022). Toward Cranfield-inspired reusability assessment in interactive information retrieval evaluation. Information Processing and Management. 59(5): 103007. (IP&M) [Paper]
• Liu, J. (2021). Deconstructing search tasks in interactive information retrieval: A systematic review of task dimensions and predictors. Information Processing and Management, 58(3): 102522. (IP&M) [Paper]
• Sarkar, S., Mitsui, M., Liu, J., & Shah, C. (2020). Implicit information needs as explicit problems, help, and behavioral signals. Information Processing and Management, 57(2): 102069. (IP&M) [Paper]
• Liu, J., Wang, Y., Mandal, S., & Shah, C. (2019). Exploring the immediate and short-term effects of peer advice and cognitive authority on Web search behavior. Information Processing and Management, 56(3), 1010-1025. (IP&M) [Paper]
• Liu, J. (2017). Toward a unified model of human information behavior: An equilibrium perspective. Journal of Documentation, 73(4), 666-688. (JDoc) [Paper]

Refereed Conference Paper

• Chen, N., Liu, J., Dong, X. Liu, Q., Sakai, T. & Wu, X-M. (2024). AI can be cognitively biased: An exploratory study on threshold priming in LLM-based batch relevance assessment. In Proceedings of the 2nd International ACM SIGIR Conference on Information Retrieval in the Asia Pacific. (SIGIR-AP) [Paper]
• Zhang, Y* & Liu, J. (2024). Where do older adults’ mental models for video recommender systems come from: A qualitative study. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries. (JCDL2024)
• Wang, B. & Liu, J. (2024). Cognitively biased users interacting with algorithmically biased results in whole-session search on debated topics. In Proceedings of the ACM SIGIR Conference on the Theory of Information Retrieval. (ICTIR2024)
• Wang, B., Liu, J., Karimnazarov, J., & Thompson, N. (2024). Task supportive and personalized human-large language model interaction: A user study. In Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval. 4 pages. Sheffield, UK. (CHIIR2024)
• Wang, X., Rahmani, H., Liu, J., & Yilmaz, E. (2023). Improving conversational recommendation systems via bias analysis and language-model-enhanced data augmentation. In Proceedings of the Findings of Empirical Methods in Natural Language Processing. (EMNLP 2023 Findings)
• Wang, B. & Liu, J. (2023). Characterizing and early predicting user performance for adaptive search path recommendation. In Proceedings of the Annual Meeting of the Association for Information Science and Technology. (ASIS&T2023) (SIG USE best information behavior paper award)
• Chen, N., Liu, J., & Sakai, T. (2023). A reference-dependent model for Web search evaluation: Understanding and measuring the experience of boundedly rational users. In Proceedings of the ACM Web Conference 2023. Austin, TX, USA. (TheWebConf2023)
• Liu, J. (2023). A two-sided fairness framework in search and recommendation. In Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval. Austin, TX, USA. (CHIIR2023)
• Lei, J., Bu, Y., & Liu, J. (2023). Information retrieval research in academia and industry: A preliminary analysis of productivity, authorship, impact, and topic distribution. In iConference 2023. (iConference2023)
• Liu, J. & Han, F. (2023). 5-4 ≠ 4-3: On the uneven gaps between different levels of graded user satisfaction in interactive information retrieval evaluation. In Proceedings of 56th Hawaii International Conference on System Sciences. Maui, HI. (HICSS2023) [Paper]
• Liu, J. & Han, F. (2022). Matching search result diversity with user diversity acceptance in Web search sessions. In Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. 5 pages. Madrid, Spain. (SIGIR2022) [Paper][Poster]
• Liu, J. & Shah, C. (2022). Leveraging user interaction signals and task state information in adaptively optimizing usefulness-oriented search sessions. In Proceedings of ACM/IEEE Joint Conference on Digital Libraries. 10 pages. Cologne, Germany. (JCDL2022) [Paper][Slides] (JCDL2022 Vannevar Bush best paper nominee)
• Brown, T. & Liu, J. (2022). A reference dependence approach to enhancing early prediction of session behavior and satisfaction. In Proceedings of ACM/IEEE Joint Conference on Digital Libraries. 5 pages. Cologne, Germany. (JCDL2022) [Paper]
• Liu, J. & Yu, R. (2021). State-aware meta-evaluation of evaluation metrics in interactive information retrieval. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management. Virtual event, Australia. (CIKM2021) [Paper]
• Liu, J. & Jung, Y. J. (2021). Interest development, knowledge learning, and interactive IR: Toward a state-based approach to search as learning. In Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval. Virtual event, Australia. (CHIIR2021) [Paper]
• Liu, J. & Han, F. (2020). Investigating reference dependence effects on user search interaction and satisfaction. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1141-1150). Xi’an, China. (SIGIR2020) [Paper]
• Liu, J., Sarkar, S., & Shah, C. (2020). Identifying and predicting the states of complex search tasks. In Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval (pp. 193-202). Vancouver, Canada. (CHIIR2020) [Paper]
• Liu, J., & Shah, C. (2019). Proactive identification of query failure. In Proceedings of the Association for Information Science and Technology, 56(1), 176-185. (ASIS&T 2019 Best student long paper award) [Paper]
• Liu, J., Mitsui, M., Belkin, N. J., & Shah, C. (2019). Task, information seeking intentions, and user behavior: Toward a multi-level understanding of web search. In Proceedings of the ACM SIGIR 2019 Conference on Human Information Interaction and Retrieval (pp. 123-132). Glasgow, UK. (CHIIR2019) [Paper] [Slides]
• Liu, J., & Shah, C. (2019). Investigating the impacts of expectation disconfirmation on web search. In Proceedings of the ACM SIGIR 2019 Conference on Human Information Interaction and Retrieval (pp. 319-323). Glasgow, UK. (CHIIR2019) [Paper]
• Mitsui, M., Liu, J., & Shah, C. (2018). How much is too much? Whole session vs. first query behaviors in task type prediction. In Proceedings of the ACM SIGIR 2018 Conference on Research and Development in Information Retrieval (pp. 1141-1144). Ann Arbor, MI. (SIGIR2018) [Paper]
• Mitsui, M., Liu, J., & Shah, C. (2018). The paradox of personalization: Does task prediction require individualized models? In Proceedings of the ACM SIGIR 2018 Conference on Human Information Interaction and Retrieval (pp. 277-280). New Brunswick, NJ. (CHIIR2018) [Paper]
• Mitsui, M., Liu, J., Belkin, N. J., & Shah, C. (2017). Predicting information seeking intentions from search behaviors. In Proceedings of the ACM SIGIR 2017 Conference on Research and Development in Information Retrieval (pp. 1121-1124). Tokyo, Japan. (SIGIR2017) [Paper]
• Liu, C., Liu, J., & Wei, Y. (2017). Scroll up or down? Using wheel activity as an indicator of browsing strategy across different contextual factors. In Proceedings of the ACM SIGIR 2017 Conference on Conference Human Information Interaction and Retrieval (pp.333-336). Oslo, Norway. (CHIIR2017) [Paper]
• Wang, Y., Liu, J., Mandal, S., & Shah, C. (2017). Search successes and failures in query segments and search tasks: A field study. In Proceedings of the 80th Annual Meeting of the Association for Information Science and Technology, 54(1), 436-445. (ASIS&T2017) [Paper]
• Li, Y., Liu, J. (2017). Chinese individual investors’ information-seeking behavior on the Web. (10 pages). Paper presented at iConference 2017, Wuhan, China. (iConference2017) [Paper]

Refereed Conference Poster

• Chen, N., Liu, J., Sakai, T., & Wu, X. (2023). Decoy effect in search interaction: A pilot study. In Proceedings of the 2023 Workshop on Evaluating Information Access (EVIA) at NTCIR-17. (NTCIR-17)
• Jung, Y. J., & Liu, J. (2022). Children’s interest, search, and knowledge: A pilot analysis of a STEM maker workshop. In Proceedings of the 85th Annual Meeting of the Association for Information Science and Technology. 59(1). (ASIS&T2022) (SIG USE best information behavior poster award)
• Wang, B., & Liu, J. (2022). Investigating the relationship between in-situ user expectations and Web search behavior. In Proceedings of the 85th Annual Meeting of the Association for Information Science and Technology. 59(1). (ASIS&T2022)
• Wang, B. & Liu, J. (2021). Extracting implicit search task states from explicit behavioral signals in complex search tasks. In Proceedings of the 84th Annual Meeting of the Association for Information Science and Technology. (ASIS&T2021)
• Wang, Y., Liu, J., Mandal, S., & Shah, C. (2018). Persuasion by peer or expert for Web search. In Proceedings of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing (pp. 225-228). Jersey City, NJ. (CSCW2018) [Paper]
• Liu, J., & Wang, Y. (2016). Information worth spreading: An exploration of information sharing from social Q&A to other social media platforms. In Proceedings of the 79th Annual Meeting of the Association for Information Science and Technology, 53(1), 1- 5. (ASIS&T2016) [Paper]

Refereed Conference Demo

• Mitsui, M., Liu, J., & Shah, C. (2018). Coagmento: Past, present, and future of an individual and collaborative information seeking platform. In Proceedings of the 2018 Conference on Human Information Interaction and Retrieval (pp. 325-328). New Brunswick, NJ. (CHIIR2018) [Paper]

Doctoral Consortia

• Liu, J. (2019). Characterizing the stages of complex tasks. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1445-1445). Paris, France. Mentors: Dr. Gareth Jones and Dr. Noriko Kando. (SIGIR2019)
• Liu, J. (2019). A reference-dependent model of search evaluation. In Proceedings of the ACM SIGIR 2019 Conference on Human Information Interaction and Retrieval (pp. 405-408). Glasgow, UK. Mentor: Dr. Diane Kelly. (CHIIR2019)

Service

Organizing and Chairing Activities

• 1st ASIS&T SIG/AI student research symposium, Organizer [Event Poster]
• SIGIR 2022 IWILDS Workshop Organizing Committee Member
• Chair-Elect, ASIS&T SIG AI 2022 -
• CIKM 2021 IWILDS Workshop Organizing Committee Member
• SIGIR 2021 Registration Chair
• iConference 2021 Workshop Co-Chair
• ASIS&T Research Engagement Committee Member 2020-2023
• ASIS&T 2020 SIG USE Symposium Co-Chair
• ASIS&T SIG USE Communications Officer 2017-2019

Grant Review Service

• Institute of Museum and Library Services (IMLS) Reviewer 2022
• National Science Foundation (NSF) Reviewer 2021

Journal Reviewer

• ACM Transactions on Information Systems
• Journal of the Association for Information Science and Technology
• Information Processing and Management
• Library and Information Science Research
• Journal of Information Science
• Aslib Journal of Information Management
• Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Program Committee Member

• ACM SIGKDD (2022)
• WSDM (2022, 2023)
• ACM SIGIR (2021, 2022, 2023, 2024)
• Senior Member, SIGIR Artifact Evaluation Committee (AEC) 2021-
• ACM CHIIR (2020, 2021, 2022, 2023)
• ASIS&T AM (2018, 2019, 2020, 2021)
• ALISE (2020)
• iConference (2020)
• SIGIR 2019 EARS workshop
• ACM Multimedia Conference 2019 SALMM workshop
• CIKM 2020 IWILDS workshop