[Sds-seminars] [Sds-announce] Lunch and talk now

[Dan Spielman]

Reminding everyone that we have lunch and a talk now on the 13th floor

Sent from mobile phone


---------- Forwarded message ---------
From: Torres, Elizavette <elizavette.torres at yale.edu>
Date: Tue, Mar 25, 2025 at 12:39 PM
Subject: [Sds-faculty] [Sds-announce] S&DS Seminar: Blake Bordelon,
03/28/25, 12pm-1pm, KT 13th Floor, Rm. 1327, "Scaling Limits and Scaling
Laws of Deep Learning"
To: sds-announce at mailman.yale.edu <sds-announce at mailman.yale.edu>


[image: Department of Statistics and Data Science]
<https://statistics.yale.edu/>   *Department of Statistics and Data Science
<https://statistics.yale.edu/>*


*Blake Bordelon**, Harvard University*
Date: Friday, March 28, 2025
Time: 12:00PM to 1:00PM
<https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g>
Location In-Person: Kline Tower, 13th Floor, Rm. 1327 *See map
<http://maps.google.com/?q=219+Prospect+Street%2C+New+Haven%2C+CT%2C+06511%2C+us>*

219 Prospect Street
<https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g>
New Haven,
<https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g>
 CT
<https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g>
 06511
<https://www.google.com/maps/search/219+Prospect+Street+%0D%0A+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g>
Webcast Option:
https://yale.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=3eba2c4e-d770-410c-bb8a-b233012bcef8
Title: Scaling Limits and Scaling Laws of Deep Learning
Information and Abstract:
Scaling up the size and training horizon of deep learning models has
enabled breakthroughs in computer vision and natural language processing.
Empirical evidence suggests that these neural network models are described
by regular scaling laws where performance of finite parameter models
improves as model size increases, eventually approaching a limit described
by the performance of an infinite parameter model. In this talk, we will
first examine certain infinite parameter limits of deep neural networks
which preserve representation learning and then describe how quickly finite
models converge to these limits. Using dynamical mean field theory methods,
we provide an asymptotic description of the learning dynamics of randomly
initialized infinite width and depth networks. Next, we will empirically
investigate how close the training dynamics of finite networks are to these
idealized limits. Lastly, we will provide a theoretical model of neural
scaling laws which describes how generalization depends on three
computational resources: training time, model size and data quantity. This
theory allows analysis of compute optimal scaling strategies and predicts
how model size and training time should be scaled together in terms of
spectral properties of the limiting kernel. The theory also predicts how
representation learning can improve neural scaling laws in certain regimes.
For very hard tasks, the theory predicts that representation learning can
approximately double the training-time exponent compared to the static
kernel limit.

*Lunch at 11:30am in room 1307 Talk at 12:00-1:00pm in room 1327A*
For more details and upcoming events visit our website at
https://statistics.yale.edu/calendar.



Department of Statistics and Data Science

Yale University
Kline Tower

219 Prospect Street
<https://www.google.com/maps/search/219+Prospect+Street+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g>
New Haven, CT 06511
<https://www.google.com/maps/search/219+Prospect+Street+%0D%0ANew+Haven,+CT+06511?entry=gmail&source=g>

https://statistics.yale.edu/


-- 
Sds-announce mailing list
Sds-announce at mailman.yale.edu
https://mailman.yale.edu/mailman/listinfo/sds-announce
-- 
Sds-faculty mailing list
Sds-faculty at mailman.yale.edu
https://mailman.yale.edu/mailman/listinfo/sds-faculty
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20250328/fecbf0fc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Outlook-Department
Type: image/jpeg
Size: 2925 bytes
Desc: not available
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20250328/fecbf0fc/attachment.jpe>
-------------- next part --------------
-- 
Sds-announce mailing list
Sds-announce at mailman.yale.edu
https://mailman.yale.edu/mailman/listinfo/sds-announce