<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br>From: <strong class="gmail_sendername" dir="auto">Hau, Emily</strong> <span dir="auto"><<a href="mailto:emily.hau@yale.edu">emily.hau@yale.edu</a>></span><br>Date: Tue, Oct 26, 2021 at 10:20 AM<br>Subject: [YINS] 10/27: Yasaman Bahri (Google Brain): "Dynamics and scaling laws in deep learning"<br>To: <a href="mailto:yins@mailman.yale.edu">yins@mailman.yale.edu</a> <<a href="mailto:yins@mailman.yale.edu">yins@mailman.yale.edu</a>><br></div><br><br>
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="m_4223931644022304491WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt"><img width="600" height="200" style="width:6.25in;height:2.0833in" id="m_4223931644022304491Picture_x0020_5" src="cid:17cbd3c1c784cff311" alt="cid:image001.png@01D7BF65.7BBD3AC0"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><u></u> <u></u></span></p>
<table border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse">
<tbody>
<tr>
<td width="623" valign="top" style="width:467.5pt;padding:0in 5.4pt 0in 5.4pt">
<h2 style="line-height:20.45pt;vertical-align:baseline"><strong><span style="font-size:16.0pt;font-family:inherit;color:#003d75;letter-spacing:.1pt;border:none windowtext 1.0pt;padding:0in">YINS Seminar:
</span></strong><strong><i><span style="font-size:16.0pt;font-family:inherit;color:#003d75;letter-spacing:.1pt;border:none windowtext 1.0pt;padding:0in">“Dynamics and scaling laws in deep learning”</span></i></strong><span style="font-size:16.0pt;font-family:"Museo Slab 300";color:#4d4d4d;letter-spacing:.1pt;font-weight:normal"><u></u><u></u></span></h2>
<p style="line-height:15.1pt;vertical-align:baseline;margin:0.75rem;outline:0px;font-variant-caps:normal;font-stretch:inherit;text-align:start;word-spacing:0px">
<strong><span style="font-family:inherit;color:#003d75;letter-spacing:.1pt;border:none windowtext 1.0pt;padding:0in">Speaker: Yasaman Bahri</span></strong><span style="font-family:"Open Sans";color:#4d4d4d;letter-spacing:.1pt"><br>
</span><em><span style="font-family:inherit;color:#4d4d4d;letter-spacing:.1pt;border:none windowtext 1.0pt;padding:0in">Research Scientist at Google Research, Brain Team<u></u><u></u></span></em></p>
<p style="line-height:15.1pt;vertical-align:baseline"><b><span style="font-family:"Open Sans";color:#002060;letter-spacing:.1pt">Wednesday, October 27, 2021 at 12:00pm<u></u><u></u></span></b></p>
<p style="line-height:15.1pt;vertical-align:baseline;margin:0.75rem;outline:0px;font-variant-caps:normal;font-stretch:inherit;text-align:start;word-spacing:0px">
<strong><span style="font-family:inherit;color:#003d75;letter-spacing:.1pt;border:none windowtext 1.0pt;padding:0in">Talk summary:</span></strong><span class="m_4223931644022304491apple-converted-space"><span style="font-family:"Open Sans";color:#4d4d4d;letter-spacing:.1pt"> </span></span><span style="font-family:"Open Sans";color:#4d4d4d;letter-spacing:.1pt">This
talk will discuss some topics in supervised deep learning from a scientist’s lens, describing efforts that seek to bridge theory and experiment. First, I’ll briefly review some of what is known about the learning dynamics of deep neural networks, including
highlights from the large-width limit of neural networks. I’ll then focus on efforts to understand empirical “scaling laws” for the performance of neural networks. Recent empirical work has found that the test loss often follows smooth power laws as a function
of basic variables such as model size and dataset size. I will discuss our work seeking to connect and understand some of these scaling laws. We introduce “variance-limited” and “resolution-limited” scaling regimes to distinguish the origin of the power-law
behavior. As an illustration, we investigate the case of teacher-student random feature models where we can study the problem exactly. I’ll close with a few empirical observations about task properties and scaling exponents. <u></u><u></u></span></p>
<p style="line-height:15.1pt;vertical-align:baseline;margin:0.75rem;outline:0px;font-variant-caps:normal;font-stretch:inherit;text-align:start;word-spacing:0px">
<strong><span style="font-family:inherit;color:#003d75;letter-spacing:.1pt;border:none windowtext 1.0pt;padding:0in">To participate:</span></strong><span style="font-family:"Open Sans";color:#4d4d4d;letter-spacing:.1pt"><br>
Join from PC, Mac, Linux, iOS or Android:<span class="m_4223931644022304491apple-converted-space"> </span><a href="https://yale.zoom.us/j/98819118892" target="_blank"><span style="font-family:inherit;color:#003d75;border:none windowtext 1.0pt;padding:0in">https://yale.zoom.us/j/98819118892</span><span class="m_4223931644022304491element-invisible"><span style="font-family:inherit;color:#003d75;border:none windowtext 1.0pt;padding:0in;text-decoration:none">(link
is external)</span></span></a><br>
Or Telephone</span><span style="font-family:"MS Gothic";color:#4d4d4d;letter-spacing:.1pt">：</span><span style="font-family:"Open Sans";color:#4d4d4d;letter-spacing:.1pt">203-432-9666 (2-ZOOM if on-campus) or 646 568 7788<br>
Meeting ID: 988 1911 8892<br>
International numbers available:<span class="m_4223931644022304491apple-converted-space"> </span><a href="https://yale.zoom.us/u/aeF0HRmJcA" target="_blank"><span style="font-family:inherit;color:#003d75;border:none windowtext 1.0pt;padding:0in">https://yale.zoom.us/u/aeF0HRmJcA</span><span class="m_4223931644022304491element-invisible"><span style="font-family:inherit;color:#003d75;border:none windowtext 1.0pt;padding:0in;text-decoration:none">(link
is external)</span></span></a> <u></u><u></u></span></p>
<p style="line-height:15.1pt;vertical-align:baseline;margin:0.75rem;outline:0px;font-variant-caps:normal;font-stretch:inherit;text-align:start;word-spacing:0px">
<strong><span style="font-family:inherit;color:#003d75;letter-spacing:.1pt;border:none windowtext 1.0pt;padding:0in">Speaker bio:</span></strong><span style="font-family:"Open Sans";color:#4d4d4d;letter-spacing:.1pt"> Yasaman Bahri is a Research Scientist at
Google Research on the Brain team. In the past several years, she has been working at the boundary of machine learning and the physical sciences, with a particular focus on bridging theoretical and empirical understanding in deep learning. Her doctoral work
is in the area of theoretical condensed matter physics, specifically on quantum many-body systems. She received her Ph.D. in Physics from UC Berkeley (2017) as well as earlier B.A. degrees from UC Berkeley in Physics and Math. She is a recipient of the 2020
Rising Stars Award in EECS and the NSF Graduate Fellowship and has co-organized two ICML workshops. <u></u><u></u></span></p>
</td>
</tr>
<tr>
<td width="623" valign="top" style="width:467.5pt;padding:0in 5.4pt 0in 5.4pt">
<p class="MsoNormal"><b><span style="font-size:11.0pt;color:#003d75;letter-spacing:.1pt;border:none windowtext 1.0pt;padding:0in">Upcoming YINS Seminars:<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#4d4d4d;letter-spacing:.1pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#4d4d4d;letter-spacing:.1pt"><a href="https://yins.yale.edu/event/yins-seminar-marinka-zitnik-harvard" target="_blank">11/3/21 Marinka Zitnik (Harvard)</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#4d4d4d;letter-spacing:.1pt"><a href="https://yins.yale.edu/event/yins-seminar-karan-singh-princeton" target="_blank">11/10/21 Karen Singh (Princeton)</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#4d4d4d;letter-spacing:.1pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#4d4d4d;letter-spacing:.1pt"><a href="https://mailman.yale.edu/mailman/listinfo/yins" target="_blank">Click here to subscribe to YINS emails.</a><u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-size:11.0pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif">Emily E. H. Hau | Director, Programs and Partnerships</span><span style="font-size:11.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><i><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#4472c4">Yale Institute for Network Science</span></i></b><span style="font-size:11.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif">Yale University</span></b><span style="font-size:11.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><u><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#954f72">17 Hillhouse Avenue | Room 341 | New Haven, CT 06511</span></u><span style="font-size:11.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><span lang="PT-BR" style="font-size:9.0pt;font-family:"Arial",sans-serif">c: (203) 273-7886 </span><span style="font-size:11.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><u><span lang="PT-BR" style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0563c1"><a href="mailto:emily.hau@yale.edu" target="_blank">emily.hau@yale.edu</a></span></u><span style="font-size:11.0pt"><u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
_______________________________________________<br>
YINS mailing list<br>
<a href="mailto:YINS@mailman.yale.edu" target="_blank">YINS@mailman.yale.edu</a><br>
<a href="https://mailman.yale.edu/mailman/listinfo/yins" rel="noreferrer" target="_blank">https://mailman.yale.edu/mailman/listinfo/yins</a><br>
</div></div>