<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Arial, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-size: 22pt; color: rgb(40, 109, 192);"><a href="https://statistics.yale.edu/" id="OWA2791e56c-05c0-c8b3-788b-f744dab9eafa" class="OWAAutoLink" title="Department of Statistics and Data Science" style="color: rgb(40, 109, 192); margin-top: 0px; margin-bottom: 0px;"><img alt="Department of Statistics and Data Science" width="150" height="49" style="width: 150px; height: 49px; margin-top: 0px; margin-bottom: 0px;" data-outlook-trace="F:1|T:1" src="cid:0eb57de6-a2ae-4099-be34-ac97e0a7ec7d"></a></span><span style="font-size: 11pt; color: black;">
</span><span style="font-size: 22pt; color: rgb(40, 109, 192);"><b><a href="https://statistics.yale.edu/" id="OWAd04c3078-324c-f1d0-d9ef-45f51c26f235" class="OWAAutoLink" title="Home" style="color: rgb(40, 109, 192); margin-top: 0px; margin-bottom: 0px;">Department
of Statistics and Data Science</a></b></span></div>
<div id="Signature">
<p><span style="font-family: Arial, sans-serif; font-size: 11pt;"> </span></p>
<div style="font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
<span style="line-height: normal;">Zhuoran Yang</span>, Yale University</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.2; margin-top: 0.5em; margin-bottom: 1em; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
<img id="image_0" width="115" height="138" style="width: 115px; height: 138px; max-width: 100%;" src="https://statistics.yale.edu/sites/default/files/styles/user_picture_node/public/picture-2273-1693233160.jpg?itok=azUwtJuO"></div>
<div style="padding-right: 15.3125px; max-width: 30%;"></div>
<div style="padding-left: 22.9688px; max-width: 65%;">
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.4; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
Date: Monday, March 31, 2025</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.4; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
Time: 4:00PM to 5:00PM</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
Location: Kline Tower, 13th Floor, Rm. 1327 <span style="color: rgb(40, 109, 192);">
<u><a href="http://maps.google.com/?q=219+Prospect+Street%2C+New+Haven%2C+CT%2C+06511%2C+us" id="OWAe40944f5-b4ff-790f-fa39-88ab305b7cad" class="OWAAutoLink" style="color: rgb(40, 109, 192);">See map</a></u></span> </div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
219 Prospect Street</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
New Haven, CT 06511</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.2; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
Webcast option: https://yale.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=23d16765-e107-4f8d-992a-b233012bcdb3</div>
</div>
<div style="padding-top: 15px;">
<div class="elementToProof" style="text-align: left; text-indent: 0px; line-height: 1.2; background-color: rgb(255, 255, 255); margin: 0.5em 0px 1em; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
Title: Unveiling In-Context Learning: Provable Training Dynamics and Feature Learning in Transformers</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; padding-right: 15.3125px; max-width: 30%; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
Information and Abstract: </div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px 0px 1em; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
In-context learning (ICL) is a cornerstone of large language model (LLM) functionality, yet its theoretical foundations remain elusive due to the complexity of transformer architectures. In particular, most existing work only theoretically explains how the
attention mechanism facilitates ICL under certain data models. It remains unclear how the other building blocks of the transformer contribute to ICL. To address this question, we study how a simple softmax transformer is trained to perform ICL on two synthetic
tasks — (multi-task) linear regression and n-gram Markov chain. We show that transformer successfully learns these tasks in-context. More importantly, we will interpret the estimator represented by the learned transformer, show how transformers are trained
by gradient-based dynamics, and how features emerge during training. Our theory is further validated by experiments. </div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px 0px 1em; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
This is joint work with Siyu Chen, Xintian Pan, Heejune Sheen, and Tianhao Wang.</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; margin: 0px 0px 1em; font-family: Arial, sans-serif; font-size: 14pt; color: rgb(0, 0, 0);">
3:30pm - Pre-talk meet and greet teatime - 219 Prospect Street, 13 floor, there will be light snacks and beverages in the kitchen area.</div>
</div>
<p><span style="font-family: Arial, sans-serif; color: black;">For more details and upcoming events visit our website at
</span><span style="font-family: Arial, sans-serif; color: rgb(70, 120, 134);"><a href="https://statistics.yale.edu/calendar" id="OWA6776b274-6ae5-4594-40a0-5978a2655620" class="OWAAutoLink" style="color: rgb(70, 120, 134); margin-top: 0px; margin-bottom: 0px;">https://statistics.yale.edu/calendar</a></span><span style="font-family: Arial, sans-serif;">.</span></p>
<p><span style="font-family: Arial, sans-serif; font-size: 11pt;"> </span></p>
<p><span style="font-family: Arial, sans-serif; font-size: 18pt;">Department of Statistics and Data Science</span></p>
<p><span style="font-family: Arial, sans-serif; font-size: 9pt; color: black;">Yale University<br>
Kline Tower</span></p>
<p><span style="font-family: Arial, sans-serif; font-size: 9pt; color: black;">219 Prospect Street<br>
New Haven, CT 06511</span></p>
<p><span style="font-size: 11pt; color: rgb(70, 120, 134);"><a href="https://statistics.yale.edu/" id="OWA2dd66aef-6120-acd4-7b39-e0658bbde877" class="OWAAutoLink" style="color: rgb(70, 120, 134); margin-top: 0px; margin-bottom: 0px;">https://statistics.yale.edu/</a></span></p>
<p> </p>
</div>
</body>
</html>