<div dir="auto">Reminding everyone that we have lunch and a talk now on the 13th floor <br clear="all"><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">Sent from mobile phone</div></div></div><div><br></div><div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br>From: <strong class="gmail_sendername" dir="auto">Torres, Elizavette</strong> <span dir="auto"><<a href="mailto:elizavette.torres@yale.edu">elizavette.torres@yale.edu</a>></span><br>Date: Tue, Mar 25, 2025 at 12:39 PM<br>Subject: [Sds-faculty] [Sds-announce] S&DS Seminar: Blake Bordelon, 03/28/25, 12pm-1pm, KT 13th Floor, Rm. 1327, "Scaling Limits and Scaling Laws of Deep Learning"<br>To: <a href="mailto:sds-announce@mailman.yale.edu">sds-announce@mailman.yale.edu</a> <<a href="mailto:sds-announce@mailman.yale.edu">sds-announce@mailman.yale.edu</a>><br></div><br><br>
<div dir="ltr">
<div style="font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="font-size:22pt;color:rgb(40,109,192)"><a href="https://statistics.yale.edu/" id="m_2875317185798441140OWA20d5d453-690c-7000-de8c-1c5831398307" title="Department of Statistics and Data Science" style="color:rgb(40,109,192);margin-top:0px;margin-bottom:0px" target="_blank"><img alt="Department of Statistics and Data Science" src="cid:ii_195dd637c904e702b401" style="width:150px;max-width:100%"></a></span><span style="font-size:11pt;color:black">
</span><span style="font-size:22pt;color:rgb(40,109,192)"><b><a href="https://statistics.yale.edu/" id="m_2875317185798441140OWA13c226f5-2747-4f87-26f6-b700f8db2d40" title="Home" style="color:rgb(40,109,192);margin-top:0px;margin-bottom:0px" target="_blank">Department
of Statistics and Data Science</a></b></span></div>
<div id="m_2875317185798441140Signature">
<p><span style="font-family:Arial,sans-serif;font-size:11pt"> </span></p>
<div style="font-family:Arial,sans-serif;font-size:14pt;color:rgb(0,0,0)">
<span style="line-height:normal"><b>Blake Bordelon</b></span><b>, Harvard University</b></div>
<div style="text-align:left;text-indent:0px;line-height:1.2;margin-top:0.5em;margin-bottom:1em;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<img id="m_2875317185798441140image_0" width="115" height="138" style="width:115px;height:138px;max-width:1603px" src="https://statistics.yale.edu/sites/default/files/styles/user_picture_node/public/blake-bordelon.png?itok=v5PtFAeH"></div>
<div style="padding-right:15.3125px;max-width:30%"></div>
<div style="padding-left:22.9688px;max-width:65%">
<div style="text-align:left;text-indent:0px;line-height:1.4;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Date: Friday, March 28, 2025</div>
<div style="text-align:left;text-indent:0px;line-height:1.4;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Time: 12:00PM to 1:00PM</div><a href="https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g">
</a><div style="text-align:left;text-indent:0px;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Location In-Person: Kline Tower, 13th Floor, Rm. 1327 <span style="color:rgb(40,109,192)">
<u><a href="http://maps.google.com/?q=219+Prospect+Street%2C+New+Haven%2C+CT%2C+06511%2C+us" id="m_2875317185798441140OWA67dde3d1-145e-8b2e-eb4b-2180d8e9b7db" style="color:rgb(40,109,192)" target="_blank">See map</a></u></span> </div>
<div style="text-align:left;text-indent:0px;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<a href="https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g">219 Prospect Street</a></div>
<div style="text-align:left;text-indent:0px;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)"><a href="https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g">
New Haven,</a> <a href="https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g">CT</a> <a href="https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g">06511</a></div>
<div style="text-align:left;text-indent:0px;line-height:1.2;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Webcast Option: <a href="https://yale.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=3eba2c4e-d770-410c-bb8a-b233012bcef8" id="m_2875317185798441140LPlnk100198" target="_blank">
https://yale.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=3eba2c4e-d770-410c-bb8a-b233012bcef8</a></div>
</div>
<div style="padding-top:15px">
<div style="text-align:left;text-indent:0px;line-height:1.2;background-color:rgb(255,255,255);margin:0.5em 0px 1em;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Title: Scaling Limits and Scaling Laws of Deep Learning</div>
<div style="text-align:left;text-indent:0px;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Information and Abstract: </div>
<div style="text-align:left;text-indent:0px;margin:0px 0px 1em;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Scaling up the size and training horizon of deep learning models has enabled breakthroughs in computer vision and natural language processing. Empirical evidence suggests that these neural network models are described by regular scaling laws where performance
of finite parameter models improves as model size increases, eventually approaching a limit described by the performance of an infinite parameter model. In this talk, we will first examine certain infinite parameter limits of deep neural networks which preserve
representation learning and then describe how quickly finite models converge to these limits. Using dynamical mean field theory methods, we provide an asymptotic description of the learning dynamics of randomly initialized infinite width and depth networks.
Next, we will empirically investigate how close the training dynamics of finite networks are to these idealized limits. Lastly, we will provide a theoretical model of neural scaling laws which describes how generalization depends on three computational resources:
training time, model size and data quantity. This theory allows analysis of compute optimal scaling strategies and predicts how model size and training time should be scaled together in terms of spectral properties of the limiting kernel. The theory also predicts
how representation learning can improve neural scaling laws in certain regimes. For very hard tasks, the theory predicts that representation learning can approximately double the training-time exponent compared to the static kernel limit.</div>
<div style="text-align:left;text-indent:0px;margin:0px 0px 1em;font-family:Arial,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<b>Lunch at 11:30am in room 1307<br>
Talk at 12:00-1:00pm in room 1327A</b></div>
</div>
<div style="margin-top:1em;margin-bottom:1em;font-family:Arial,sans-serif;font-size:14pt">
<span style="color:black">For more details and upcoming events visit our website at
</span><span style="color:rgb(70,120,134)"><a href="https://statistics.yale.edu/calendar" id="m_2875317185798441140OWA43d4e0bc-3b62-247a-987f-7688b21b1f89" style="color:rgb(70,120,134);margin-top:0px;margin-bottom:0px" target="_blank">https://statistics.yale.edu/calendar</a></span><span style="color:rgb(0,0,0)">.</span></div>
<p><span style="font-family:Arial,sans-serif;font-size:11pt"> </span></p>
<p><span style="font-family:Arial,sans-serif;font-size:18pt">Department of Statistics and Data Science</span></p>
<p><span style="font-family:Arial,sans-serif;font-size:9pt;color:black">Yale University<br>
Kline Tower</span></p>
<p><span style="font-family:Arial,sans-serif;font-size:9pt;color:black"><a href="https://www.google.com/maps/search/219+Prospect+Street+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g">219 Prospect Street</a><br><a href="https://www.google.com/maps/search/219+Prospect+Street+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g">
New Haven, CT 06511</a></span></p>
<p><span style="font-size:11pt;color:rgb(70,120,134)"><a href="https://statistics.yale.edu/" id="m_2875317185798441140OWA4299706f-42e5-9799-10c3-292cb964539b" style="color:rgb(70,120,134);margin-top:0px;margin-bottom:0px" target="_blank">https://statistics.yale.edu/</a></span></p></div></div><div dir="ltr"><div id="m_2875317185798441140Signature">
<p> </p>
</div>
</div>
-- <br>
Sds-announce mailing list<br>
<a href="mailto:Sds-announce@mailman.yale.edu" target="_blank">Sds-announce@mailman.yale.edu</a><br>
<a href="https://mailman.yale.edu/mailman/listinfo/sds-announce" rel="noreferrer" target="_blank">https://mailman.yale.edu/mailman/listinfo/sds-announce</a><br>
-- <br>
Sds-faculty mailing list<br>
<a href="mailto:Sds-faculty@mailman.yale.edu" target="_blank">Sds-faculty@mailman.yale.edu</a><br>
<a href="https://mailman.yale.edu/mailman/listinfo/sds-faculty" rel="noreferrer" target="_blank">https://mailman.yale.edu/mailman/listinfo/sds-faculty</a><br>
</div></div>