Information Theory, Coding and Cryptography, Spring 2011

Lecturers: Serge Fehr, Ronald Cramer     Teaching Assistant: Niek Bouman

Material & References

Serge's part:

• Lecture Notes (essentially all the material from Serge's part that you are supposed to know for the exam is covered in the lecture notes)

Ronald's part:

• (to be announced)

If you want to learn more about the subject, have a look at the following paper/books:

• A Mathematical Theory of Communication by Claude Shannon. This is Shannon's original paper that started the field of information theory.
• the 'standard' book: "Elements of Information Theory," Thomas Cover and Joy Thomas, Wiley
• another book, freely available online: "Information Theory, Inference, and Learning Algorithms," by David MacKay, Cambridge University Press.

The paper is a classic. The book by Cover & Thomas is concise and formal, MacKay's book is more lively written and practical, and also covers some modern coding theory, links to machine learning, and contains lots of examples from practice.

Time Table

The course schedule can be found in the roster.

Note: No class on Monday, February 28!

Examination

The mark for the course is determined via exercise sets (weight: 1/3) and a 20-minute oral exam (weight: 2/3). You need to pass for both parts.

Exercises

The exercises should be sent to bouman ((at)) cwi nl. They should either be typeset on a computer, or written by hand (but readable :-), scanned and converted into a compact format (preferably PDF).

Everyone has to hand in his/her own individual solution. Solutions with multiple names and copied solutions will not be accepted. Also, late hand-in's will not be accepted.

• Exercise Set 1
• Exercise Set 2
• Exercise Set 3

Links to Interesting Video Lectures about Information Theory

• Robert Gallager (M.I.T.) (lectures 1-4)
• David MacKay (Cambridge University)
• David Forney (M.I.T.) (lectures 5-12: block codes, finite fields, Reed-Solomon codes)