course: Fundamentals of Data Science
- teaching methods:
- lecture with tutorials
- computer based presentation
- responsible person:
- Prof. Dr.-Ing. Aydin Sezgin
- Prof. Dr.-Ing. Aydin Sezgin (ETIT), M. Sc. Mohammadhossein Attar (ETIT)
- offered in:
- winter term
dates in winter term
- start: Monday the 07.10.2019
- lecture Mondays: from 12:15 to 13.45 o'clock in ID 04/401
- tutorial Fridays: from 14:15 to 15.45 o'clock in ID 2/103
All statements pertaining to examination modalities (for the summer/winter term of 2020) are given with reservations. Changes due to new requirements from the university will be announced as soon as possible.
Date according to prior agreement with lecturer.
|Form of exam:||oral|
|Registration for exam:||FlexNow|
The students understand the concepts of pattern recognition, machine learning, and information theory and are able to apply it to data analysis. Equipped with tools and methods acquired during the lectures, problems arising regularly in engineering disciples can be investigated.
The view taken in the course is based on the ideas that data science is fundamentally rooted in information theory, as information theory is the pillar of most machine learning algorithms. Naturally, stochastic processes will also play a role, as sequences of events can be modeled nicely. The course has also a focus on Bayesian statistics and includes new developments in neural networks and deep learning.
The table of contents is as follows:
Review: Linear Algebra
Review: Probability Theory, Random variables and, Markov Chains, processes (Gaussian, Markov Decision)
Least Mean Square Estimation
- Information theoretic learning
- Kullback-Leibler Divergence
- ICA, Dictionary Learning,
- k-SVD, Rate distortion theory,
- entropy maximization, information bottleneck
Neural networks and deep learning
As part of the exercise sessions, the students will implement various algorithms in Matlab:
- LMS, Kalman, Stochastic Gradient Descent,
- k-Means, KNN,
- Expectation Maximization, Backpropagation etc.
The focus of the course is on
- Discovery of regularities in data via Pattern recognition
- Development of algorithms via Machine learning (Classification, Clustering, Reinforcement Learning)
- Performance criteria via Information theory
- Hands-on experience
The main references for the course are:
- Sergios Theodoridis, Machine Learning- A Bayesian and optimization perspective.
- Simon Haykin, Neural Networks and Learning Machines
- Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning
- Math I-IV
- System theory I-III
- C. M., Bishop "Pattern Recognition and Machine Learning", Springer Verlag, 2006
The overall exam has two parts:
Part A: Oral Exam 45min Part B: Scientific Paper Review: Presenting the content of a paper (provided by us) orally and in written form (2 page, 2 column format, IEEE style in TeX)
Part A: 36 % Part B: 64 %