Jacob Whitehill's research group

Areas

Group Members

• Jacob Whitehill - PI and Associate Professor, Department of Computer Science, and Learning Science & Technologies (LST) - jrwhitehill@wpi.edu


• Viet Anh Trinh - Post-doctoral Researcher - vtrinh@wpi.edu


• Xinlu He - PhD student (Data Science) - xhe4@wpi.edu


• Jiani Wang - PhD student (Computer Science) - jwang21@wpi.edu


• Yiwen Guan - PhD student (Computer Science) - yguan2@wpi.edu

Projects

• Developing New Scientific Instruments for Classroom Observation: A Multi-modal Machine Learning Approach, funded by NSF, headed at WPI.
• AI Institute for Student-AI Teaming (iSAT), funded by NSF, headed at CU Boulder.
• Hybrid Human-Agent Tutoring: Accelerating Middle School Math Achievement for Low Income Students, funded by Schmidt Futures, headed at CU Boulder.

Publications

2024

• Wang, J. Dudy, S., He, X., Wang, Z., Southwell, R., and Whitehill, J. "Speaker Diarization in the Classroom: How Much Does Each Student Speak in Group Discussions?". Educational Data Mining. PDF.
• He, X., Wang, J., Trinh, V.A., McReynolds, A., and Whitehill, J. "Tracking Classroom Movement Patterns with Person Re-Id". Educational Data Mining. PDF.
• Whitehill, J., and LoCasale-Crouch, J. "Automated Evaluation of Classroom Instructional Support with LLMs and BoWs: Connecting Global Predictions to Specific Feedback". Journal of Educational Data Mining. PDF.
• Southwell, R. Ward, W., Trinh, V.A., Clevenger, C., Clevenger, C., Watts, E., Reitman, J., D'Mello, S., and Whitehill, J. "Automatic Speech Recognition Tuned for Child Speech in the Classroom". International Conference on Sound, Speech, and Signal Processing (ICASSP). PDF.
• D'Mello, S., Biddy, Q., Breideband, T., Bush, J., Chang, M., Cortez, A., Flanigan, J., Foltz, P., Gorman, J., Hirshfield, L., Ko, M., Krishnaswamy, N., Lieber, R., Martin, J., Palmer, M., Penuel, W., Philip, T., Puntambekar, S., Pustejovsky, J., Reitman, J., Sumner, T., Tissesnbaum, M., Walker, L., and Whitehill, J. "From learning optimization to learner flourishing: Reimagining AI in Education at the Institute for Student‐AI Teaming (iSAT)". AI Magazine 45, no. 1. PDF.

2023

• Dai, Z., McReynolds, A., and Whitehill, J. "In Search of Negative Moments: Multi-Modal Analysis of Teacher Negativity in Classroom Observation Videos.". Educational Data Mining. PDF.
• Li, Z., He, X., and Whitehill, J. "Compositional clustering: Applications to multi-label object recognition and speaker identification". Pattern Recognition. PDF.

2022

• Whitehill, J., and Erfanian, A. "How to Give Imperfect Automated Guidance to Learners: A Case-Study in Workplace Learning". Artificial Intelligence in Education. PDF.
• Jiang, H., Li, Z., and Whitehill, J. "Can the Mathematical Correctness of Object Configurations Affect the Accuracy of Their Perception?". 1st CVPR Workshop on Vision Datasets Understanding. PDF.
• Zhang, Z., Cisneros, E., Lee, H., Vu, J., Chen Q., Benadof, C., Whitehill, J., Rouzbehani, R., Sy, D., Huang, J., Sejnowski, T., Jankovic, J., Factor, S., Goetz, C., Barbano, R., Perlmutter, J., Jinnah, H., Berman, B., Richardson, S., Stebbins, G., Comella, C., and Peterson, D. "Hold that pose: capturing cervical dystonia's head deviation severity from video". Annals of Clinical and Translational Neurology. PDF.

2021

• Ruiz, N., Yu, H., Allessio, D., Jalal, M., Joshi, A., Murray, T., Magee, J., Whitehill, J., Ablavsky, V., Arroyo, I., Woolf, B., Sclaroff, S., and Betke, M. "Leveraging Affect Transfer Learning for Behavior Prediction in an Intelligent Tutoring System". IEEE Automatic Face and Gesture Recognition. PDF.
• Jiang, H., Serpell, Z., and Whitehill, J. "Measuring the Effect of ITS Feedback Messages on Students' Emotions". AIED Workshop on Multimodal Artificial Intelligence in Education. PDF.
• Gupta, A., Menon, N., Lee, W., Rebelsky, W., Allesio, D., Murray, T., Woolf, B., Whitehill, J., and Arroyo, I. "Affective Teacher Tools: an Affective Class Report Card and an Affective Dashboard". Artificial Intelligence in Education. PDF.
• Liamthong, P, and Whitehill, J. "Text Representations of Math Tutorial Videos for Clustering, Retrieval, and Learning Gain Prediction". Educational Data Mining. PDF.
• Li, Z., and Whitehill, J. "Compositional embedding models for speaker identification and diarization with simultaneous speech from 2+ speakers". IEEE International Conference on Speech and Signal Processing (ICASSP). PDF.
• Kyriacou, H., Ramakrishnan, A., and Whitehill, J. "Learning to Work in a Materials Recovery Facility: Can Humans and Machines Learn from Each Other?". Learning Analytics and Knowledge (LAK). PDF.
• Li, Z., Mozer, M., and Whitehill, J. "Compositional Embeddings for Multi-Label One-Shot Learning". Winter Conference on Applications of Computer Vision (WACV). PDF.

2020

• Pham, M., Li, Z., and Whitehill, J. "How Does Label Noise Affect the Quality of Speaker Embeddings?". InterSpeech. PDF.
• Zylich, B., and Whitehill, J. "Noise-Robust Key-Phrase Detectors for Automated Classroom Feedback". IEEE International Conference on Speech and Signal Processing (ICASSP). PDF.
• Aguerrebere, C., Bulger, M., Cobo, C., Garcia, S., Kaplan, G., and Whitehill, J. "How Teachers Affect Students' Online Participation in EFL Courses in Uruguay". Computer-Based Learning in Context. PDF.
• Pham, M., Li, Zeqian, and Whitehill, J. "Toward Better Speaker Embeddings: Automated Collection of Speech Samples from Unknown Distinct Speakers". IEEE International Conference on Speech and Signal Processing (ICASSP). PDF.