course: Fundamentals of Data Science

number:
141213
teaching methods:
lecture with tutorials
media:
computer based presentation
responsible person:
Prof. Dr.-Ing. Aydin Sezgin
lecturers:
Prof. Dr.-Ing. Aydin Sezgin (ETIT), M. Sc. Aya Ahmed (ETIT), M. Sc. Sampath Thanthrige (ETIT)
language:
english
HWS:
4
CP:
5
offered in:
winter term

dates in winter term

  • start: Monday the 08.10.2018
  • lecture Mondays: from 12:15 to 13.45 o'clock in ID 04/401
  • tutorial Fridays: from 14:15 to 15.45 o'clock in ID 04/401

Exam

Date according to prior agreement with lecturer.

Form of exam:oral
Registration for exam:FlexNow
Date:None
Duration:30min

goals

The stu­dents un­der­stand the con­cepts of pattern recognition, machine learning, and information theory and are able to apply it to data analysis. Equip­ped with tools and me­thods ac­qui­red du­ring the lec­tu­res, problems arising regularly in engineering disciples can be in­ves­ti­ga­ted.

content

The view taken in the course is based on the ideas that data science is fundamentally rooted in information theory, as information theory is the pillar of most machine learning algorithms. Naturally, stochastic processes will also play a role, as sequences of events can be modeled nicely. The course has also a focus on Bayesian statistics and includes new developments in neural networks and deep learning.

The table of contents is as follows:

  • Introduction

  • Review: Linear Algebra

  • Review: Probability Theory, Random variables and, Markov Chains, processes (Gaussian, Markov Decision)

  • Least Mean Square Estimation

  • Classification

  • Bayesian Learning

  • Information theoretic learning
    • Kullback-Leibler Divergence
    • ICA, Dictionary Learning,
    • k-SVD, Rate distortion theory,
    • entropy maximization, information bottleneck
  • Neural networks and deep learning

As part of the exercise sessions, the students will implement various algorithms in Matlab:

  • LMS, Kalman, Stochastic Gradient Descent,
  • k-Means, KNN,
  • Expectation Maximization, Backpropagation etc.

The focus of the course is on

  • Discovery of regularities in data via Pattern recognition
  • Development of algorithms via Machine learning (Classification, Clustering, Reinforcement Learning)
  • Performance criteria via Information theory
  • Hands-on experience

The main references for the course are:

  • Sergios Theodoridis, Machine Learning- A Bayesian and optimization perspective.
  • Simon Haykin, Neural Networks and Learning Machines
  • Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning

requirements

none

recommended knowledge

  • Math I-IV
  • System theory I-III
  • Optimization

literature

  1. C. M., Bishop "Pattern Recognition and Machine Learning", Springer Verlag, 2006