[Sds-seminars] [Sds-announce] FDS Colloquium: Eran Malach, 1/15/25, KT 13th Floor, Rm. 1327, 12pm-1pm, "Learning Hard Problems with Neural Networks and Language Models"

[elizavette.torres at yale.edu]

FDS Colloquium

 

Eran Malach, Harvard University



Date: Wednesday, January 15, 2025

Time: 12:00PM to 1:00PM

Location: Kline Tower, 13th Floor, Rm. 1327 See map
<http://maps.google.com/?q=219+Prospect+Street%2C+New+Haven%2C+CT%2C+06511%2
C+us>  

219 Prospect Street

New Haven, CT 06511

Website <https://www.eranmalach.com/> 

 

Webcast:
https://yale.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=6f3e7e6c-c1ef-4
066-b4f2-b2590138c39

 

Learning Hard Problems with Neural Networks and Language Models

 

Information and Abstract:  Modern machine learning models, and in particular
large language models, can now solve surprisingly complex mathematical
reasoning problems. In this talk I will explore how neural networks and
autoregressive language models can learn to solve computationally hard
reasoning tasks. I will begin by discussing the sparse parity problem, a
theoretical proxy for studying the challenges of learning complex functions
with Stochastic Gradient Descent (SGD). I will show that the computational
resources required for learning sparse parities with SGD scale exponentially
with the "sparsity" of the problem, making it computationally hard to learn.
Next, I will demonstrate how introducing step-by-step supervision through
auto-regressive language models overcomes these barriers, enabling simple
models trained on next-token prediction to efficiently learn any
Turing-computable function. These results serve as a basis for studying
machine learning with language models, with implications on data structure,
architecture design and training paradigms.

 

Bio: Eran Malach is a postdoc Research Fellow in the Kempner Institute at
Harvard University. Previously, he did his PhD at the School of Computer
Science and Engineering in the Hebrew University of Jerusalem, advised by
Prof. Shai Shalev-Shwartz. His research focus is Machine Learning and
Theoretical Foundations of Deep Learning and Language Models. He is mainly
interested in computational aspects of learning and optimization. He also
worked in Mobileye, where he developed machine learning and computer vision
algorithms for driver-assistance systems and self-driving cars. His research
is supported by the Rothschild Fellowship, the William F. Milton Fund and
the OpenAI Superalignment Fast Grant.

 

Lunch at 11:30am in room 1307
Talk at 12:00-1:00pm in room 1327A

 

For more details and upcoming events visit our website at
<https://statistics.yale.edu/calendar> https://statistics.yale.edu/calendar.


 

Department of Statistics and Data Science

Yale University
Kline Tower

219 Prospect Street
New Haven, CT 06511

 <https://statistics.yale.edu/> https://statistics.yale.edu/

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20250113/a781c987/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 32188 bytes
Desc: not available
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20250113/a781c987/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 4853 bytes
Desc: not available
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20250113/a781c987/attachment-0001.jpg>
-------------- next part --------------
-- 
Sds-announce mailing list
Sds-announce at mailman.yale.edu
https://mailman.yale.edu/mailman/listinfo/sds-announce