[Sds-seminars] [Sds-announce] Fwd: [YINS] 10/27: Yasaman Bahri (Google Brain): "Dynamics and scaling laws in deep learning"

[Dan Spielman]

---------- Forwarded message ---------
From: Hau, Emily <emily.hau at yale.edu>
Date: Tue, Oct 26, 2021 at 10:20 AM
Subject: [YINS] 10/27: Yasaman Bahri (Google Brain): "Dynamics and scaling
laws in deep learning"
To: yins at mailman.yale.edu <yins at mailman.yale.edu>


[image: cid:image001.png at 01D7BF65.7BBD3AC0]


*YINS Seminar: **“Dynamics and scaling laws in deep learning”*

*Speaker: Yasaman Bahri*
*Research Scientist at Google Research, Brain Team*

*Wednesday, October 27, 2021 at 12:00pm*

*Talk summary:* This talk will discuss some topics in supervised deep
learning from a scientist’s lens, describing efforts that seek to bridge
theory and experiment. First, I’ll briefly review some of what is known
about the learning dynamics of deep neural networks, including highlights
from the large-width limit of neural networks. I’ll then focus on efforts
to understand empirical “scaling laws” for the performance of neural
networks. Recent empirical work has found that the test loss often follows
smooth power laws as a function of basic variables such as model size and
dataset size. I will discuss our work seeking to connect and understand
some of these scaling laws. We introduce “variance-limited” and
“resolution-limited” scaling regimes to distinguish the origin of the
power-law behavior. As an illustration, we investigate the case of
teacher-student random feature models where we can study the problem
exactly. I’ll close with a few empirical observations about task properties
and scaling exponents.

*To participate:*
Join from PC, Mac, Linux, iOS or Android:
https://yale.zoom.us/j/98819118892(link
is external) <https://yale.zoom.us/j/98819118892>
      Or Telephone：203-432-9666 (2-ZOOM if on-campus) or 646 568 7788
      Meeting ID: 988 1911 8892
      International numbers available: https://yale.zoom.us/u/aeF0HRmJcA(link
is external) <https://yale.zoom.us/u/aeF0HRmJcA>

*Speaker bio:* Yasaman Bahri is a Research Scientist at Google Research on
the Brain team. In the past several years, she has been working at the
boundary of machine learning and the physical sciences, with a particular
focus on bridging theoretical and empirical understanding in deep learning.
Her doctoral work is in the area of theoretical condensed matter physics,
specifically on quantum many-body systems. She received her Ph.D. in
Physics from UC Berkeley (2017) as well as earlier B.A. degrees from UC
Berkeley in Physics and Math. She is a recipient of the 2020 Rising Stars
Award in EECS and the NSF Graduate Fellowship and has co-organized two ICML
workshops.

*Upcoming YINS Seminars:*



11/3/21 Marinka Zitnik (Harvard)
<https://yins.yale.edu/event/yins-seminar-marinka-zitnik-harvard>

11/10/21 Karen Singh (Princeton)
<https://yins.yale.edu/event/yins-seminar-karan-singh-princeton>



Click here to subscribe to YINS emails.
<https://mailman.yale.edu/mailman/listinfo/yins>





Emily E. H. Hau | Director, Programs and Partnerships

*Yale Institute for Network Science*

*Yale University*

*17 Hillhouse Avenue | Room 341 | New Haven, CT 06511*

c: (203) 273-7886

*emily.hau at yale.edu <emily.hau at yale.edu>*


_______________________________________________
YINS mailing list
YINS at mailman.yale.edu
https://mailman.yale.edu/mailman/listinfo/yins
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20211026/bad60786/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 89589 bytes
Desc: not available
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20211026/bad60786/attachment.png>
-------------- next part --------------
-- 
Sds-announce mailing list
Sds-announce at mailman.yale.edu
https://mailman.yale.edu/mailman/listinfo/sds-announce