Tom Sterkenburg (MCMP, LMU Munich) will join us at HLRS for a master class on the role of Occam’s Razor in Machine Learning. In the master class you will have the opportunity to discuss with Tom’s his two most recent papers on the topic.
Tom discusses the role simplicity plays in statistical learning and the type of justification we can expect from theory for preferring simple models in practice in Statistical learning theory and Occam’s razor: The core argument. This work suggest that in the end the most we can expect is a pragmatic justification for simplicity: we ought to prefer simpler model (-classes), if we are interested in better predictive accuracy. Tom also explores the many caveats that apply when making such a statement.
The second work we will discuss is Tom’s recent draft (under review) Statistical learning theory and Occam’s razor: Regularization. Phenomena such as double descent and the success of overparametrized deep learning models have cast some doubt on the simplicity preference expressed by Occam’s razor and the justification Tom has developed in the core argument. Here he defends this argument and points out how, in the light of these seemingly contradictory observations, a preference for simplicity can still be justified by using the meta-inductive framework of structural risk minimization.
The master class will take place at HLRS, Room Baden, July 21st. 11:30-13:00
This event will be in-person. We welcome external participants, but you have to register before July 15th by sending a mail to nico.formanek@hlrs.de.
Please also note Tom’s colloquium talk on July 20th.