[WTI-trainee] FW: Paul Smolensky CompCog Talk This Tuesday 10AM, RM 1167

[Guerrero-Medina, Giovanna]

See below!

From: Psych.others <psych.others-bounces at mailman.yale.edu> on behalf of Brooke-Wilson, Tyler <tyler.brooke.wilson at yale.edu>
Date: Saturday, March 29, 2025 at 7:59 AM
To: psych.all at mailman.yale.edu <psych.all at mailman.yale.edu>
Subject: Paul Smolensky CompCog Talk This Tuesday 10AM, RM 1167
Forwarding a talk on Language Model’s In-Context learning that may be of interest –

Hi All,

Computational linguist Paul Smolensky<https://scholar.google.com/citations?user=PRtkZzYAAAAJ&hl=en> will be presenting current work on In-Context Learning in Transformers this Tuesday, 10-11:30AM. Title and abstract below, plus link to sign up for 1-1 meetings with Paul.

Time: Tuesday 4/1, 10-11:30AM
Location: 100 College St., RM 1167

Title: Mechanisms of Symbol Processing for In-Context Learning in Transformer Networks

Authors: Paul Smolensky, Roland Fernandez, Herbert Zhou, Mattia Opper, Jianfeng Gao

Abstract: Large Language Models (LLMs) have demonstrated impressive abilities in symbol processing through in-context learning (ICL). This success flies in the face of decades of predictions that artificial neural networks cannot master abstract symbol manipulation. We seek to understand the mechanisms that can enable robust symbol processing in transformer networks, illuminating both the unanticipated success, and the significant limitations, of transformers in symbol processing. Borrowing insights from symbolic AI on the power of Production System architectures, we develop a high-level language, PSL, that allows us to write symbolic programs to do complex, abstract symbol processing, and create compilers that precisely implement PSL programs in transformer networks which are, by construction, 100% mechanistically interpretable. We demonstrate that PSL is Turing Universal, so the work can inform the understanding of transformer ICL in general. The type of transformer architecture that we compile from PSL programs suggests a number of paths for enhancing transformers’ capabilities at symbol processing.

Suggested paper to pre-read: Section 1 of https://arxiv.org/abs/2410.17498

Note: There is a sign up sheet to meet with Paul here<https://docs.google.com/spreadsheets/d/1cUpdHD5bQNfuPlNKqmFP7Om15hR3r3Ke2hKja1wO8Ug/edit?usp=sharing>. Meetings will be at 100 College St. (Check the sign up sheet for exact location day of). Please feel free to send this to any students with relevant interests.

Best,
Tyler
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.yale.edu/pipermail/wti-trainee/attachments/20250331/83d35561/attachment.html>