This seminar simulates a machine learning conference, where the students take on the role of authors and reviewers. It consists of multiple phases.
Attend the mandatory first meeting either in person or remotely (details on TUWEL).
You select two topics/papers (i.e., two bullet points) from one of the topics below. You will work with the material mentioned in the overview and the topic-specific resources.
You choose your own topic to work on. This can be some existing machine learning paper/work or an own creative idea in the context of machine learning. We strongly encourage you to start from existing papers from the following venues: NeurIPS, ICML, ICLR, COLT, AISTATS, UAI, JMLR, MLJ. Importantly, your idea has to be specific and worked out well. Nevertheless, choose one of our suggestions as well.
Independent of the option you chose, understand the fundamentals of your topic and try to answer the following questions:
Select topics and write a short description of them together with the answers to the questions (~3 sentences should be sufficient) in TUWEL.
We can only accept your own proposals if you can answer the mentioned questions and have a well worked out topic.
You will also act as reviewers and bid on the topics of your peers you want to review. Based on the biddings, we (in the role as chairs of the conference) will select one of each student’s proposals as the actual topic you will work on for the rest of this semester. You do not need to work on the other topic, anymore. Additionally, we will also assign two different topics from other students to you, which you will have to review later in the semester.
Now the actual work starts. Gather deep understanding of your topic, write a first draft of your report and give a 5-minute presentation. We recommend to go beyond the given material.
You will again act as a reviewer for the conference by writing two reviews, one for each draft report assigned to you.
Based on the reviews from your peers (and our feedback) you will further work on your topic.
Give a final presentation and submit your report.
You should have access to the literature and papers through Google scholar, DBLP, the provided links, or the TU library.
Motivation: The data streaming model deals with processing large data sets that arrive continuously over time, where storing all data is impractical. Algorithms in this model make decisions using limited memory, typically by summarizing or sampling data on-the-fly, making it ideal for real-time applications such as network traffic analysis or sensor data processing.
Overview:
In the context of big data and real-time processing, it is essential to efficiently compute statistics on data streams. One important problem is estimating the number of distinct elements in a data stream, known as the Distinct Elements Problem. This problem has applications in fields such as network traffic monitoring, database management, and large-scale data analysis. This project aims to explore recent contributions in this area and their applications in various real-world scenarios. The goal of this project is to understand and explore space-efficient algorithms for dealing with streaming data, focusing on recent advancements, particularly the contributions of Sourav Chakraborty, Kuldeep Meel, and N.V. Vinodchandran.
Papers and Topics:
This project will focus on recent advances in algorithms for distinct elements in data streams and streaming model optimization. Key papers include:
Motivation: Ability to learn logically definable concepts from labelled data is a theoretical model of Machine Learning which is explainable by design, and integrates ideas from both logic (especially finite model theory) and PAC Learning.
Overview:
Papers and topics:
Motivation: In active learning, the learning algorithm is allowed to select the data points it wants to see labelled, for example, where it is most uncertain. The goal is to reduce the labelling effort. This is useful in applications where unlabelled data is abundant, yet labels are scarce, such as node classification in social networks, drug discovery, and autonomous driving.
Overview:
Papers and topics:
Motivation: Many modern datasets live on non-grid structured domains (graphs/meshes/manifolds) and/or come with continuous symmetries (e.g., E(3) in 3D geometry). Geometric deep learning seeks neural architectures that respect these structures, in particular equivariance to transformations (rotations, translations, permutations) and the ability to exploit continuous symmetry groups. Recent work shows that building these inductive biases into graph neural networks leads to models that are both more accurate and more interpretable when handling geometric data.
Papers:
Motivation: Learning theory studies computational and algorithmic aspects of machine learning algorithms to prove guarantees such as sample complexity bounds. This important to understand and devise novel learning algorithms. In recent years, many long-standing open questions in learning theory have been answered.
Overview:
Papers and topics:
Motivation:
Graphs can be used to model many areas of interest: e.g. molecules, geographical maps, spread of diseases, or communication networks. They can be used to model physical systems and solve partial differential equations. Even images and text can be seen as a special case of graphs. Learning on graphs nowadays is mostly done using message passing graph neural networks (MPNNs). MPNNs are connected to the classical Weisfeiler Leman graph isomorphism algorithm (WL) by the fact that they will always compute identical representations for two graphs which cannot be distinguished by WL. Current theoretical and practical work focuses hence on more expressive graph neural networks which can distinguish more graph pairs.
However, learning (particularly with neural networks) is based on similarities between representations. We will delve into recent methods which combine Weisfeiler Leman based graph representations with ideas from optimal transport. Ideally, we will investigate options to transfer some of the ideas to analyze similarities that MPNNs compute.
Papers of Interest:
Note: For very long papers we do not expect you to read the entire appendix.