Field Experiment

Mar 17 – Sang-Pil Han – “AI Effectiveness, Task Difficulty, and Employee Income in the Gig Economy: When AI is the Default Service Provider Rather than Humans”

March 5, 2023 By Aleksi Aaltonen

Time: Friday, 17 March 2023, 10:30–12:00
Room: LW420

Abstract

Increasingly, artificial intelligence (AI) serves as a frontline operator, while humans perform backend operations. Despite growing firms in the gig economy employing a business model with AI as the default service provider, the literature is limited regarding the impact of such a model on employee income. To address such limitations, this research proposes an AI-first service framework, which asserts that AI initially attempts to solve tasks, but upon unsatisfactory service outcome, customers pay a fee for employee assistance with such tasks. To empirically investigate our proposed framework, we partner with an AI-first learning app, where AI and tutors are the default and on-request service providers, respectively. We use tutor-level observational data and find that as AI, the default service provider, becomes more effective, tutor income is mediated by changes in task volume and margin, but that the pathways differ depending on employee expertise. Findings from our granular data show that, as AI effectiveness increases, tasks that are difficult due to their broad coverage across topics are passed on to tutors such that low-expertise (vs. high-expertise) tutors accept fewer tasks. We also conduct a field experiment at the customer level to show that customers are willing to pay for extra assistance from tutors, despite receiving free service from AI, for tasks that are difficult due to in-depth knowledge required within a topic. We discuss the theoretical implications of our findings and practical ramifications to effectively manage and develop human competency in the era of an AI-driven economy.

Bio

Sang-Pil Han is an Associate Professor of Information Systems in the W. P. Carey School of Business at Arizona State University. His research focuses on artificial intelligence, digital platforms, and business analytics. His research has been published in top-tier academic journals such as Management Science, Management Information Systems Quarterly, Information Systems Research and Journal of Marketing, and featured in Harvard Business Review and BBC News. He has received grants from the Marketing Science Institute and Wharton Interactive Media Initiative, the NET Institute, the Wharton Customer Analytics Initiative, the Korea Research Foundation, the Hong Kong General Research Fund, as well as private companies. At ASU, he was a Co-Faculty Director for the Master of Science in Business Analytics program. He served as an Associate Editor at Information Systems Research. He advises a variety of organizations, including tech startups like Mathpresso, a leading AI-powered education platform, and RoundIn, an online golf learning platform, as well as non-profits like Simple Steps, a 501c3 organization that assists female immigrant talent in achieving their professional goals. In his spare time, he enjoys playing golf with his wife and two daughters.

April 10 – JaeHwuen Jung to present “The secret to Finding Love: A Field Experiment of Choice Structure in Online Dating Platform”

September 11, 2020 By Sezgin Ayabakan

The secret to Finding Love: A Field Experiment of Choice Structure in Online Dating Platform

JaeHwuen Jung

Assistant Professor
Department of Management Information Systems
Fox School of Business
Temple University

Friday, April 10

10:30 – 12:00 pm | Zoom

Abstact:

Online matching platforms require new approaches to market design since firms can now control many aspects of search and interaction process through various IT-enabled features. While choice structure—the size of choice set and the number of choices a platform offers to its customers—is one of the key design features of online matching platforms, its impact on engagement and matching outcomes remains unclear. In this study, we examine the effect of different choice structures on the number of choices and matches on the platform by conducting a randomized field experiment in collaboration with an online dating platform. Specifically, we design four treatment groups with different choice structures where users can only interact with other users in the same group, select users who are in a similar age range and live in the same geographical location, and randomly assign them to each treatment group. We find that providing higher choice capacities to male and female users have a different effect on choice behaviors and matching outcomes. Moreover, while increasing the choice capacity of male users yields the highest number of choices, increasing the choice capacity of female users is the most effective way to increase matching outcomes. Structural analysis further reveals the underlying mechanisms of choice behavior and matching results, suggesting that users significantly decrease the number of choices after receiving a choice from other users and the effect of the choice capacity on matching outcomes differs by gender. We further provide a counterfactual analysis that explores optimal choice structure design depending on the gender ratio of the online dating platform.

April 24 – Xueming Luo to present “Quantifying the Impact of Human-AI Supervisor Assemblages on Employee Performance: A Field Experiment”

September 11, 2020 By Sezgin Ayabakan

Quantifying the Impact of Human-AI Supervisor Assemblages on Employee Performance: A Field Experiment

Xueming Luo

Founder/Director of Global Center on Big Data in Mobile Analytics
Charles Gilliland Distinguish
Chair Professor of Marketing, Strategy, and MIS
Fox School of Business
Temple University

Friday, April 24

10:30 – 12:00 pm | Zoom

Abstact:

Despite the promises of artificial intelligence (AI), there are concerns from both employees and managers about adopting AI at workplaces. Examining how firms can integrate AI into performance management systems (PMS), this research focuses on the impact of various human-AI supervisor assemblages on employees’ task performances and relations with human bosses. We utilize data from a field experiment on customer service employees in a fintech company who are randomly assigned to receive job performance feedback from human managers only, an AI bot only, or human-AI supervisory assemblages. A unique feature in our experiment is that the assemblages encompass a dual human-and-AI configuration (where employees receive feedback from both human managers and an AI bot in parallel) and a shadow-AI-human-face configuration (where employees receive feedback that is generated by an AI bot but delivered by human managers). The results suggest that relative to conventional human supervision, a dual human-and-AI design negatively impacts employee task performance, whereas a shadow-AI-human-face design positively impacts employee task performance. Explorations of the mechanisms support that a dual condition with both AI and human supervision in parallel leads employees to perceive more confused leadership and feedback, less learning from the feedback, and lower employee-manager relationship quality in a vicious cycle. In contrast, the shadow-AI design significantly improves employees’ perceptions of feedback accuracy and consistency, willingness to proactively seek feedback, and organizational commitment in a virtuous cycle. These findings suggest that firms should prudently design the human-AI supervisory assemblages. As a double-edged sword, AI-based PMS should be deployed in the shadows to empower human managers, rather than to displace or compete with them, to achieve higher worker productivity and healthier employee-manager relationships.

Sep 13 – Siva Viswanathan to present “Designing Promotional Incentives to Embrace Social Sharing: Evidence from Field and Lab Experiments”

September 12, 2019 By Sezgin Ayabakan

Designing Promotional Incentives to Embrace Social Sharing:
Evidence from Field and Lab Experiments

Siva Viswanathan

Dean’s Professor of Information Systems and Digital Innovation & Co-Director of DIGITS
University of Maryland
Robert H. Smith School of Business

Friday, September 13

10:30 – 12:00 pm | Speakman 200

Abstact:

Despite the increasing connectivity between customers and the large volume of social shares supported by digital technologies, there is an absence of research systematically investigating how firms can design the promotional incentives that jointly consider their customers as both purchaser and sharer. In this study, we examine whether and how firms can take advantage of customers’ social connections and sharing motives to design novel incentives to engage customers in this social sharing era. In collaboration with a leading online deal platform, we conduct a large-scale randomized field experiment and two lab experiments to test the effectiveness of different incentive designs (varied by shareability and scarcity of promotion codes) in driving social sharing senders’ purchase and referrals. We find that different incentive designs have distinct impacts on senders’ purchases and further successful referrals. Specifically, providing senders with one non-shareable promo code significantly increases their purchase likelihood, but does not influence their referrals. In contrast, the senders who receive one shareable code are less likely to purchase themselves yet are more likely to make successful referrals. Surprisingly, the incentive design with two codes that has one non-shareable code and one shareable code increases neither the senders’ purchase nor their successful referrals. Very interestingly, we estimate that the one non-shareable promo code group derives the highest net revenue for the current experiment period, whereas the one shareable promo code group will derive the highest lifetime value from the new customers the incentive lures in. We further conduct two lab experiments on Amazon Mechanical Turk that replicate the field experiment’s findings and explore the underlying mechanisms of the observed relationships. We find that the exclusivity perception and social motives triggered by one promo code incentive designs mediate and explain their effect on sender’s purchase and successful referrals, respectively. Our study extends prior IS literature on social sharing that has focused on sharing information to the domain of sharing incentives, providing implications to firms on how to design promotional incentive that accommodates the dual role of customers as purchasers and sharers and sheds light on the motives underlying social sharing.

Link to the paper: Click here

Oct 14: Prabuddha De to speak on An Empirical Investigation of the Effects of Product-Oriented Web Technologies on Product Returns

September 10, 2011 By Sunil Wattal

Prabuddha De,

Accenture Professor of Information Technology and Professor of Management,

Krannert School of Management, Purdue University

October 14, 2011

Speakman Hall 200, 1000am – 1130am

Seminar Title : An Empirical Investigation of the Effects of Product-Oriented Web Technologies on Product Returns

Abstract

Internet retailers have been making significant investments in advanced technologies, e.g., zoom, alternative photos, and color swatch, that are capable of providing detailed product-oriented information and, thereby, mitigating the lack of “touch-and-feel,” which, in turn, is expected to lower product returns. However, a clear understanding of the impact of these technologies on product returns is still lacking. This study attempts to fill this gap by using several econometric models to unravel the relationship between product-oriented technology usage and product returns. Our unique and rich dataset allows us to measure technology usage at the product level for each consumer. The results show that zoom usage has a negative coefficient, suggesting that a higher use of the zoom technology leads to fewer returns. Interestingly, we find that the use of alternative photos increases the likelihood of returns. Perhaps more importantly, its use has a negative effect on net sales. Color swatch, on the other hand, does not seem to have any impact on returns. Thus, our findings show that different technologies have different effects on product returns. We provide explanations for these findings based on the extant literature. We also conduct a number of tests to ensure the robustness of the results.

Click here for a copy of the paper.