Bachelor Seminar Wissenschaftliches Arbeiten
This seminar simulates a machine learning conference, where the students take on the role of authors and reviewers. It consists of multiple phases.
1. Proposal phase
Attend the mandatory first meeting on 22.03.2022, 15:00 (either in person in Seminarraum 127 in Gußhausstr. 27-29, room CD 03 24, or remotely at https://tuwien.zoom.us/my/patrickindri).
Option 1: our suggestions
You select two projects/papers (i.e. two bullet points) from one of the topics below. You will work with the material mentioned in the overview and the project-specific resources.
Option 2: your own projects
You choose two different own project ideas to work on. This can be some existing machine learning paper/work or an own creative idea in the context of machine learning. Importantly, it has to be specific and worked out well.
We can only accept your own proposals if you can answer the mentioned questions and have a well worked out project idea.
Independent of the option you chose, understand the fundamentals of your projects and try to answer the following questions:
- What is the problem?
- Why is it an interesting problem?
- How do you plan to approach the problem? /
How have the authors of your project approached the problem?
Select projects and write a short description of them together with the answers to the questions (~3 sentences should be sufficient) in TUWEL.
We can only accept your own proposals if you can answer the mentioned questions and have a well worked out project idea.
2. Bidding and assignment phase
You and your fellow students will act as reviewers and bid on the projects of your peers you want to review. Based on the biddings, we (in the role as chairs of the conference) will select one of each student’s proposals as the actual project you will work on for the rest of this semester. You do not need to work on the other project, anymore. Additionally, we will also assign two different projects from other students to you, which you will have to review later in the semester.
3. Working phase
Now the actual work starts. Gather deep understanding of your project, write a first draft of your report and give a 5-minute presentation. Feel free to go beyond the given material.
You will schedule two meetings with your supervisor to discuss your progress, but do not hesitate to contact him/her if you have any questions.
4. Reviewing phase
You will again act as a reviewer for the conference by writing two reviews, one for each draft report assigned to you.
5. Writing phase
Based on the reviews from your peers (and our feedback) you will further work on your project.
6. Submission phase
Give a final presentation and submit your report.
General resources (freely available books)
- Understanding machine learning: from theory to algorithms. Shai Shalev-Shwartz and Shai Ben-David (pdf)
- Foundations of machine learning. Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar (pdf)
- Foundations of data science. Avrim Blum, John Hopcroft, and Ravindran Kannan (pdf)
- Mathematics for machine learning. Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong (pdf)
- Mining of massive datasets. Jure Leskovec, Anand Rajaraman, and Jeffrey D. Ullman (pdf)
- Reinforcement learning: an introduction. Richard Sutton and Andrew Barto (pdf)
- Deep learning and neural networks. Ian Goodfellow and Yoshua Bengio and Aaron Courville (pdf)
Topics (Tentative)
You should have access to the literature and papers through Google scholar, DBLP, the provided links, or the TU library. Feel free to watch the linked talks to get an overview on the topics.
Kernels (click to expand)
Overview:
- preface and introduction up to section 1.5 of "Learning with kernels" by Bernhard Schölkopf and Alex Smola, 2002 (pdf).
- introduction to kernels: Bernhard Schölkopf - MLSS 2013 (youtube-link)
Papers and projects:
- support vector machines (Bennett and Campbell. "Support vector machines: hype or hallelujah?." ACM SIGKDD 2000)
- one class support vector machine (Khan and Madden. "A survey of recent trends in one class classification." Irish conference on artificial intelligence and cognitive science 2009)
- string kernels (Lodhi, et al. "Text classification using string kernels." Journal of machine learning research 2002)
- kernels for distances (Schölkopf. "The kernel trick for distances." NIPS 2001)
Online learning (click to expand)
Overview:
- chapter 1 of "A modern introduction to online learning" by Francesco Orabona, 2020.
- introduction to online learning (iterative learning / streaming settings): Nicolò Cesa-Bianchi - Mediterranean Machine Learning school 2021 (youtube-link)
Papers and projects:
- online (sub-)gradient descent (chapter 2 of "A modern introduction to online learning", Francesco Orabona, 2020)
- stochastic bandits (introduction and chapter 1 of "Introduction to multi-armed bandits", Aleksandrs Slivkins, 2019)
- online learning with expert advice (introduction and chapter 5 of "Introduction to multi-armed bandits", Aleksandrs Slivkins, 2019)
- adversarial bandits (introduction and chapter 6 of "Introduction to multi-armed bandits", Aleksandrs Slivkins, 2019)
- learning directed cuts (Gärtner and Garriga. "The cost of learning directed cuts." ECML 2007)
- predicting dynamic difficulty (Missura and Gärtner. "Predicting dynamic difficulty." NIPS 2011)
Dimensionality reduction (click to expand)
Overview:
- chapter 1 and 2 of "Dimension reduction: a guided tour" by Christopher Burges, 2010.
- introduction and overview on probabilistic dimensionality reduction: Neil Lawrence - MLSS 2012 (youtube-link)
Papers and projects:
- principal component analysis (PCA) and singular value decomposition (SVD) (chapter 3 of Foundations of Data Science book)
- random projections (chapter 23.2 of "Understanding machine learning" and Dasgupta. "Experiments with random projection." UAI 2000)
Equivariant neural networks (click to expand)
Overview:
- chapter 8 "equivariant neural networks" of "Deep learning for molecules and materials" by Andrew D. White, 2021. (pdf).
- introduction to equivariance: Taco Cohen and Risi Kondor - Neurips 2020 Tutorial (first half) (slideslive-link)
Papers and projects:
- deep sets (Zaheer, et al. "Deep sets." NIPS 2017)
- meta-learning symmetries (Zhou, et al. "Meta-learning symmetries by reparameterization." ICLR 2021)
- learning unitary opeartors (Hyland and Rätsch. "Learning unitary operators with help from u (n)." AAAI 2017)
Graph neural networks (click to expand)
Overview:
- introduction and overview on graph neural networks: Petar Veličković - Tensorflow Tech Talks 2021 (youtube-link)
- part II "graph representation learning" by William L. Hamilton (pdf)
Papers and projects:
- performance of graph neural networks (Dwivedi et al. "Benchmarking graph neural networks." 2020)
- counting substructures (Chen, Zhengdao, et al. "Can graph neural networks count substructures?" NeurIPS 2020)
- randomization (Abboud, Ralph, et al. "The Surprising Power of Graph Neural Networks with Random Node Initialization." IJCAI 2021)
Graph kernels (click to expand)
Overview:
- first 23 pages of "A survey on graph kernels" (Applied Network Science 2019) by Nils M. Kriege, et al.
- practical motivation for graph kernels in computational biology: Karsten Borgwardt - MLSS 2013 (the first 35 minutes) (youtube-link)
Papers and projects:
- hardness and walk-based kernels (Gärtner, et al. "On graph kernels: hardness results and efficient alternatives." Learning theory and kernel machines 2003)
- cyclic pattern kernel (Horváth, et al. "Cyclic pattern kernels for predictive graph mining." ACM SIGKDD 2004)
- Weisfeiler-Lehman kernel (Shervashidze, et al. "Weisfeiler-lehman graph kernels." Journal of machine learning research 2011)
Knowledge graph embedding (click to expand)
Overview:
- "graph representation learning" by William L. Hamilton (pdf)
- Knowledge Graph Embeddings Tutorial: From Theory to Practice, 2020 (https://kge-tutorial-ecai2020.github.io/)
Papers and projects:
- Knowledge Graph Embeddings (focus on deep learning approaches)
- Q. Wang, Z. Mao, B. Wang, L. Guo. "Knowledge Graph Embedding: A Survey of Approaches and Applications", 2017
- Y. Dai, S. Wang, N. Xiong, W. Guo. "A Survey on Knowledge Graph Embedding: Approaches, Applications and Benchmarks", 2020
- M. Wang, L. Qiu, X. Wang. "A Survey on Knowledge Graph Embeddings for Link Prediction", 2021
Semi-supervised learning (click to expand)
Overview:
- chapter 1/introduction of "Semi-supervised learning" by Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien, 2006 (pdf).
- introduction to semi-supervised learning: Tom Mitchell - Carnegie Mellon University 2011 (youtube-link)
Papers and projects:
- transductive support vector machines (chapter 6 in the "Semi-supervised learning" book mentioned above)
- label propagation (chapter 11 in the "Semi-supervised learning" book mentioned above)
- randomized min-cuts (Blum, Avrim, et al., "Semi-supervised learning using randomized mincuts.", ICML 2004)
Active learning (click to expand)
Overview:
- chapter 1 "Automating inquiry" of Burr Settles' "Active learning" (AL) book, 2012.
- introduction to active learning: Sanjoy Dasgupta - Microsoft 2016 (youtube-link)
Papers and projects:
- Bayesian active learning on graphs (Ma, Yifet, et al., "σ-optimality for active learning on gaussian random fields." NIPS 2013)
- active search on graphs (Wang, Xuezhi, et al., "Active search on graphs" KDD 2013)
- shortest-path-based active learning (Dasarathy, et al. "S2: an efficient graph based active learning algorithm with application to nonparametric classification." COLT 2015)
Explainable AI (click to expand)
Overview:
- Došilović, Filip Karlo, Mario Brčić, and Nikica Hlupić. "Explainable artificial intelligence: A survey." MIPRO 2018
- Samek, Wojciech, and Klaus-Robert Müller. "Towards explainable artificial intelligence." Explainable AI: interpreting, explaining and visualizing deep learning." Springer, Cham, 2019
Papers and projects:
- interpreting model predictions
- Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. ""Why should i trust you?" Explaining the predictions of any classifier." ACM SIGKDD 2016
- Lundberg, Scott M., and Su-In Lee. "A unified approach to interpreting model predictions." NIPS 2017
- reliability of saliency methods
- Adebayo, Julius, et al. "Sanity checks for saliency maps." NIPS 2018
- Kindermans, Pieter-Jan, et al. "The (un) reliability of saliency methods." Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer, Cham, 2019