<div dir="ltr"><div class="gmail-group-header" style="box-sizing:inherit;font-family:Mallory,Verdana,Arial,Helvetica,sans-serif;font-size:17px"><span class="gmail-field gmail-field-name-title gmail-field-type-ds gmail-field-label-hidden" style="box-sizing:inherit"><span style="box-sizing:inherit"><span class="gmail-odd gmail-first gmail-last" style="box-sizing:inherit"><h1 class="gmail-title" style="box-sizing:inherit;font-weight:300;padding:0px;font-feature-settings:"kern","liga","dlig";font-size:1.76471em;line-height:normal;font-stretch:normal;color:rgb(0,60,118);text-transform:uppercase;display:inline-block">ASHESH RAMBACHAN</h1></span></span></span>, <span class="gmail-field gmail-field-name-field-university gmail-field-type-text gmail-field-label-hidden" style="box-sizing:inherit;margin-left:5px"><span style="box-sizing:inherit"><span class="gmail-odd gmail-first gmail-last" style="box-sizing:inherit">Harvard University</span></span></span><div class="gmail-field gmail-field-name-field-abstract-title gmail-field-type-text gmail-field-label-hidden" style="box-sizing:inherit"><div class="gmail-field-items" style="box-sizing:inherit"><div class="gmail-field-item even" style="box-sizing:inherit;font-size:20px;font-weight:600;line-height:1.2;margin-bottom:1em;margin-top:0.5em">Title: Identifying Prediction Mistakes in Observational Data</div></div></div></div><div class="gmail-group-left" style="box-sizing:inherit;float:left;width:auto;padding-right:15.3125px;max-width:30%;font-family:Mallory,Verdana,Arial,Helvetica,sans-serif;font-size:17px"><div class="gmail-field gmail-field-name-field-image gmail-field-type-image gmail-field-label-hidden" style="box-sizing:inherit"><div class="gmail-field-items" style="box-sizing:inherit"><div class="gmail-field-item even" style="box-sizing:inherit"><img src="https://statistics.yale.edu/sites/default/files/styles/user_picture_node/public/asheshrambachan_headshot.jpg?itok=KXmv6ZUS" width="400" height="480" alt="" style="box-sizing: inherit; border: 0px; max-width: 100%; height: auto; vertical-align: bottom;"></div></div></div></div><div class="gmail-group-right" style="box-sizing:inherit;float:left;width:auto;max-width:65%;padding-left:22.9688px;font-family:Mallory,Verdana,Arial,Helvetica,sans-serif;font-size:17px"><div class="gmail-field gmail-field-name-field-event-time gmail-field-type-datetime gmail-field-label-hidden" style="box-sizing:inherit"><div class="gmail-field-items" style="box-sizing:inherit"><div class="gmail-field-item even" style="box-sizing:inherit;color:rgb(0,60,118);font-size:18px;line-height:1.4"><span class="gmail-date-display-single" style="box-sizing:inherit">Monday, February 07, 2022<span class="gmail-date-display-range" style="box-sizing:inherit;float:left;width:394.625px"><span class="gmail-date-display-start" style="box-sizing:inherit">4:00PM</span> to <span class="gmail-date-display-end" style="box-sizing:inherit">5:00PM</span></span></span></div></div></div><div class="gmail-field gmail-field-name-field-location gmail-field-type-location gmail-field-label-hidden" style="box-sizing:inherit"><div class="gmail-field-items" style="box-sizing:inherit"><div class="gmail-field-item even" style="box-sizing:inherit"><div class="gmail-location gmail-vcard" style="box-sizing:inherit"><div class="gmail-adr" style="box-sizing:inherit"><div class="gmail-street-address" style="box-sizing:inherit">Via Zoom: <a href="https://yale.zoom.us/j/91718936837">https://yale.zoom.us/j/91718936837</a></div></div></div></div></div></div><div class="gmail-field gmail-field-name-field-website gmail-field-type-link-field gmail-field-label-hidden" style="box-sizing:inherit"><div class="gmail-field-items" style="box-sizing:inherit"><div class="gmail-field-item even" style="box-sizing:inherit"><a href="https://scholar.harvard.edu/asheshr" style="box-sizing:inherit;text-decoration-line:none;outline:none;line-height:1.5;color:rgb(0,60,118);font-size:16px">Website</a></div></div></div></div><div class="gmail-group-footer" style="box-sizing:inherit;clear:both;padding-top:15px;font-family:Mallory,Verdana,Arial,Helvetica,sans-serif;font-size:17px"><div class="gmail-field gmail-field-name-body gmail-field-type-text-with-summary gmail-field-label-above" style="box-sizing:inherit"><div class="gmail-field-label" style="box-sizing:inherit;font-weight:bold">Information and Abstract: </div><div class="gmail-field-items" style="box-sizing:inherit"><div class="gmail-field-item even" style="box-sizing:inherit"><p style="box-sizing:inherit;margin:0px 0px 1em;padding:0px"><em style="box-sizing:inherit">Decision makers, such as doctors, judges, and managers, make consequential choices based on predictions of unknown outcomes. Do these decision makers make systematic prediction mistakes based on the available information? If so, in what ways are their predictions systematically biased? Uncovering systematic prediction mistakes is difficult as the preferences and information sets of decision makers are unknown to researchers. In this paper, I characterize behavioral and econometric assumptions under which systematic prediction mistakes can be identified in empirical settings such as hiring, pretrial release, and medical testing. I derive a statistical test for whether the decision maker makes systematic prediction mistakes under these assumptions and show how supervised machine learning based models can be used to apply this test. I provide methods for conducting inference on the ways in which the decision maker’s predictions are systematically biased. As an illustration, I apply this econometric framework to analyze the pretrial release decisions of judges in New York City, and I estimate that at least 20% of judges make systematic prediction mistakes about failure to appear risk given defendant characteristics.</em></p></div></div></div></div></div>