Quantifying the Impact of Human-AI Supervisor Assemblages on Employee Performance: A Field Experiment
by
Xueming Luo
Founder/Director of Global Center on Big Data in Mobile Analytics
Charles Gilliland Distinguish
Chair Professor of Marketing, Strategy, and MIS
Fox School of Business
Temple University
Friday, April 24
10:30 – 12:00 pm | Zoom
Abstact:
Despite the promises of artificial intelligence (AI), there are concerns from both employees and managers about adopting AI at workplaces. Examining how firms can integrate AI into performance management systems (PMS), this research focuses on the impact of various human-AI supervisor assemblages on employees’ task performances and relations with human bosses. We utilize data from a field experiment on customer service employees in a fintech company who are randomly assigned to receive job performance feedback from human managers only, an AI bot only, or human-AI supervisory assemblages. A unique feature in our experiment is that the assemblages encompass a dual human-and-AI configuration (where employees receive feedback from both human managers and an AI bot in parallel) and a shadow-AI-human-face configuration (where employees receive feedback that is generated by an AI bot but delivered by human managers). The results suggest that relative to conventional human supervision, a dual human-and-AI design negatively impacts employee task performance, whereas a shadow-AI-human-face design positively impacts employee task performance. Explorations of the mechanisms support that a dual condition with both AI and human supervision in parallel leads employees to perceive more confused leadership and feedback, less learning from the feedback, and lower employee-manager relationship quality in a vicious cycle. In contrast, the shadow-AI design significantly improves employees’ perceptions of feedback accuracy and consistency, willingness to proactively seek feedback, and organizational commitment in a virtuous cycle. These findings suggest that firms should prudently design the human-AI supervisory assemblages. As a double-edged sword, AI-based PMS should be deployed in the shadows to empower human managers, rather than to displace or compete with them, to achieve higher worker productivity and healthier employee-manager relationships.