[Sds-seminars] S&DS Seminar, Tim G. J. Rudner, 10/30/23, KT 13th Floor, Rm. 1327, 4pm-5pm, "Regularization in Neural Networks: A Probabilistic Perspective"

[elizavette.torres at yale.edu]

 <https://statistics.yale.edu/>     <https://statistics.yale.edu/>
Department of Statistics and Data Science  

 

 <https://statistics.yale.edu/seminars/tim-g-j-rudner> Tim G. J. Rudner, New
York University



Date: Monday, October 30, 2023

Time: 4:00PM to 5:00PM

Location: Kline Tower
<http://maps.google.com/?q=219+Prospect+Street%2C+13+floor%2C+Rm+1327%2C+New
+Haven%2C+CT%2C+06511%2C+us> see map 

219 Prospect Street, 13 floor, Rm 1327

New Haven, CT 06511

Zoom Link: https://yale.zoom.us/j/94223816617 Meeting ID: 942 2381 6617

 <https://timrudner.com/> Website

 

Title: Regularization in Neural Networks: A Probabilistic Perspective

 

Information and Abstract: 

Conventional regularization techniques for neural networks, such as L2 or L1
regularization, explicitly penalize divergence of the model parameters from
specific parameter values. However, in most neural network models, specific
parameter configurations bear little to no physical meaning, and it is
difficult to incorporate domain knowledge or other relevant information into
neural network training using conventional regularization techniques. 

 In this talk, I will show that we can address this shortcoming by using
Bayesian principles to effectively incorporate domain knowledge or beliefs
about desirable model properties into neural network training. To do so, I
will approach regularization in neural networks from a probabilistic
perspective and define a family of data-driven prior distributions that
allows us to encode useful auxiliary information into the model. I will then
show how to perform approximate inference in neural networks with such
priors and derive a simple variational optimization objective with a
regularizer that reflects the constraints implicitly encoded in the prior.
This regularizer is mathematically simple, easy to implement, and can be
used as a drop-in replacement for existing regularizers when performing
supervised learning in neural networks of any size. 

 I will conclude the talk with an overview of applications of data-driven
priors, including distribution shift detection, drug discovery, and medical
diagnosis.

This is joint work with Sanyam Kapoor, Shikai Qiu, Xiang Pan, Lily Yucen Li,
Ya Shi Zhang, Ravid Shwartz-Ziv, Julia Kempe, and Andrew Gordon Wilson.

 Bio: Tim G. J. Rudner is an Assistant Professor and Faculty Fellow at New
York University's Center for Data Science and an AI Fellow at Georgetown
University's Center for Security and Emerging Technology. He conducted PhD
research on probabilistic machine learning in the Department of Computer
Science at the University of Oxford, where he was advised by Yee Whye Teh
and Yarin Gal. The goal of his research is to develop methods and
theoretical insights that enable the safe deployment of machine learning
systems in safety-critical settings. Tim holds a master's degree in
statistics from the University of Oxford and an undergraduate degree in
applied mathematics and economics from Yale University. He is also a Rhodes
Scholar and a Qualcomm Innovation Fellow.

3:30pm - Pre-talk meet and greet teatime - 219 Prospect Street, 13 floor,
there will be light snacks and beverages in the kitchen area.

 

For more details and upcoming events visit our website at
<http://statistics.yale.edu/> http://statistics.yale.edu/

 

Department of Statistics and Data Science

Yale University

Kline Tower
219 Prospect Street
New Haven, CT 06511

t 203.432.0666
f 203.432.0633

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20231023/09a510be/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2925 bytes
Desc: not available
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20231023/09a510be/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 260981 bytes
Desc: not available
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20231023/09a510be/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 50569 bytes
Desc: not available
URL: <http://mailman.yale.edu/pipermail/sds-seminars/attachments/20231023/09a510be/attachment-0001.png>