• Log In
  • Skip to main content
  • Skip to primary sidebar

MIS Distinguished Speaker Series

Temple University

You are here: Home / Archives for label acquisition

label acquisition

October 19 – Maytal Saar-Tsechansky to Present “More For Less: Adaptive Labeling Payments in Online Labor Markets”

October 11, 2018 By Jing Gong

More For Less: Adaptive Labeling Payments in Online Labor Markets

by

Maytal Saar-Tsechansky

Associate Professor

McCombs School of Business, University of Texas at Austin

Friday, October 19, 2018

10:30 AM – noon

Speakman Hall Suite 200

 

Abstract

In many predictive tasks where human intelligence is needed to label training instances, online crowdsourcing markets have emerged as promising platforms for large-scale, cost-effective labeling. However, these platforms also introduce challenges that must be addressed for these opportunities to materialize. In particular, it has been shown that different trade-offs between payment offered to labelers and the quality of labeling arise at different times, possibly as a result of different market conditions and even the nature of the tasks themselves. Because the underlying mechanism giving rise to different trade-offs is not well understood, for any given labeling task and at any given time, it is not known which labeling payments to offer in the market so as to produce accurate models cost-effectively. Effective and robust methods for dealing with these challenges are essential to enable a growing reliance on these promising and increasingly popular labor markets for large-scale labeling. In this talk I will first present a new data science problem, Adaptive Labeling Payment (ALP): how to learn and sequentially adapt the payment offered to crowd labelers before they undertake a labeling task, so as to produce a given (machine learning) predictive model performance cost-effectively. I will then present our approach to address the problem and a rich set of results that demonstrate its performance under a variety of market settings. We also show that the method is highly versatile and can acquire more labels of lower quality (and cost) under some market conditions, while pursuing fewer and higher quality labels in other settings. Overall, our method yields significant cost savings and robust performance; as such, it can be used as a benchmark for future mechanisms to determine cost-effective payments.

Bio

Maytal Saar-Tsechansky is an Associate Professor of Information, Risk and Operations Management at the McCombs School of Business, The University of Texas at Austin, and a co-founder of Sweetch — a mobile health startup firm. Her research focuses on developing machine learning (ML) and artificial intelligence (AI) methods to improve decision-making and to benefit people, organizations, and society. Most of her work aims to augment ML & AI by bringing to bear the problems that machine learning and AI inform in practice and the context in which learning itself occurs, with the goal of effectively dealing with the constraints and taking advantage of the opportunities presented in these environments. Her research integrates business, machine learning and artificial intelligence, and she has addressed challenges in different domains, including health care, smart electricity grid, fraud detection, finance, and emerging forms of work, such as online labor markets. Maytal received her Ph.D. from New York University’s Stern School of Business. Her research has been published in the Journal of Finance, Management Science, Information Systems Research, Journal of Machine Learning Research, and Machine Learning Journal, among other venues. Maytal’s research has been supported by both government and industry, including the National Science Foundation, SAP, and the Israeli Science Ministry. In recent years she has served on the editorial boards of the Machine Learning Journal, the Information Systems Research (ISR) journal, the INFORMS Journal on Computing, Decision Sciences, and she is a frequent Program Committee member in the premier machine learning, data mining, artificial intelligence, and Information Systems conferences. At McCombs, Maytal has developed and taught popular applied machine learning and data mining courses tailored to business students.

Tagged With: Adaptive Labeling Payments, crowdsourcing, label acquisition, Machine Learning, Maytal Saar-Tsechansky, Online Labor Markets, supervised learning, UT Austin

Primary Sidebar

RSS MIS News

  • AIS Student Chapter Leadership Conference 2025 April 17, 2025
  • Temple AIS wins at the 2024 AIS Software Innovation Challenge! January 15, 2025
  • 10 Week Summer Internship in CyberSecurity October 7, 2024
  • Volunteer for Cybersecurity Awareness Month October 7, 2024
  • MIS faculty awarded promotions June 17, 2024

Tags

AI amrit tiwana Artificial Intelligence blockchain boston college bots brian butler carnegie mellon univ crowd culture deception Deep Learning Design experiment Field Experiment financial technology georgia state georgia tech Healthcare Human vs AI information security Innovation Institutional Theory IT Outsourcing long tail Machine Learning machines Maryland media Online Communities platform privacy productivity Quasi-natural experiment recommender systems simulation Social Capital social media social network steven johnson technology adoption temple univ user generated content UT Dallas wharton

Archives

Copyright © 2025 Department of Management Information Systems · Fox School of Business · Temple University