CS 380D: Distributed Systems

Welcome to Distributed Systems! This course will introduce you to the principles behind building robust distributed systems, and get practical experience building a small distributed system. We will also discuss a number of widely deployed distributed systems such as Amazon's Dynamo and Google's Spanner.

This course will be a discussion and research-oriented course. We will be reading and discussing a series of papers over the course of the semester. There will be two midterms, and two projects. The first project will allow you to utilize the principles learned in class to build a distributed key-value store. The second project is more open-ended, and will allow you to explore doing research in distributed systems.

Background

The expected background for this course is an undergraduate operating systems course, in which you learned the basics of an OS. Probably you learned it out of some book like Silberschatz or Tanenbaum or the free OS book from Wisconsin. If you have taken Alison Norman's excellent OS course (CS 439), you will be in a good position to take this course. If you took my Virtualization course and enjoyed it, you will like this course.

You should be willing to engage in discussion if you take this course. Each class, we will be discussing the assigned papers, talking about why the authors did something a certain way, and what other designs they could have considered. If everyone participates, this will lead to a lively discussion that will benefit everyone.

You should also be willing to do network development. Almost all the systems we build and discuss will involve processes communicating over the network. Thus, it is essential you know network programming, sockets, how to get processes to communicate over the network, etc.

Note to undergrads: this course will be more design-based and open-ended than your typical undergraduate courses. For example, the midterms will have you designing new systems for scenarios I make up. I know that not all undergrads enjoy this open-ended nature, so you have been forewarned. If you liked the discussion-based nature and the first design-based midterm in Virtualization, you will like this course.

Heavy Reading-Load Warning: you will be reading two research papers each class (a total of four per week!). This way, we get to cover more of the field than we would if we discussed just one paper. But it does mean you have to allocate time properly to read these papers. One of the skills you pick up in grad school is efficiently reading papers, so this will be good practice!

Resources

There is no required textbook for this class. You will complete reading assignments every week based on what we discussed in class. I will make notes available for each class.

If you want to purchase a book for you to keep as a reference, there are several to choose from:

• A.S. Tanenbaum and M. van Steen. Distributed Systems. Principles and Paradigms. (2nd Ed.) Prentice Hall, 2007.
• S. Mullender (editor) Distributed Systems, Second Edition, ACM Press, Addison-Wesley, MA, 1994.
• H. Attiya and J. Welch. Distributed Computing: Fundamentals, Simulations, and Advanced Topics (2nd Ed.) . Wiley 2004.
• N. Lynch Distributed Algorithms Morgan Kauffman, 1996.
• P.A. Bernstein, V. Hadzilakos and N. Goodman. Concurrency Control and Recovery in Database Systems Addison-Wesley, MA, 1987.

Course Policies

Students with disabilities may request appropriate academic accommodations from the Division of Diversity and Community Engagement, Services for Students with Disabilities, 512-471-6259, http://www.utexas.edu/diversity/ddce/ssd/.

Religious Holy Days: A student who is absent from an examination or cannot meet an assignment deadline due to the observance of a religious holy day may take the exam on an alternate day or submit the assignment up to 24 hours late without penalty, if proper notice of the planned absence has been given. Notice must be given at least 14 days prior to the classes which will be missed. For religious holy days that fall within the first 2 weeks of the semester, notice should be given on the first day of the semester. Notice must be personally delivered to the instructor and signed and dated by the instructor, or sent certified mail. Email notification will be accepted if received, but a student submitting email notification must receive email confirmation from the instructor.

Collaboration

1. The students are encouraged to do the projects in groups of two or three.
2. All exams are done individually, with absolutely no collaboration.
3. I strongly encourage you to discuss the projects and assignments with anyone you can. That's the way good science happens. But all work and write-up for the assignment must be your own, and only your own.
4. As a professional, you should acknowledge significant contributions or collaborations in your written or spoken presentations.
5. The student code of conduct is here. Intellectual dishonesty can end your career, and it is your responsibility to stay on the right side of the line. If you are not sure about something, ask.
6. The penalty for cheating on an exam, project or assignment in this course is an F in the course and a referral to the Dean of Students office.
7. You cross over from collaboration to cheating when you look at another person/team's source code. Discussing ideas is okay, sharing code is not.
8. You also may not look at any course project material relating to any project similar to or the same as this course's class projects. For example, you may not look at the work done by a student in past years' courses, and you may not look at similar course projects at other universities.
9. All submitted work must be new and original.

Exams

1. You will be allowed a A4 page of notes during exams. I will likely make the exams open book, but I reserve the right not to do so.
2. Laptops, tablets, and ereaders are BANNED from exams. You should not need them in an exam, and they are far too flexible as communication devices to make enforcement of non-communication policies enforceable. Any use of a communication device for any reason in the exam room will earn an automatic zero on the exam.

Late Policy

1. All projects/assignments must be submitted in class the day they are due.
2. For each day a project/assignment is late, you lose 10% of the points for that project. So if you submit two days after the deadline, your maximum points on that project will be 80%.
3. In this class, it is always better to do the work (even late) than not do it at all.
4. If you become ill: contact the instructor. A medical note is required to miss an exam.

Schedule

Office Hours: Tu-Th 3:30-4:30 PM
Where: GDC 6.436 (my office)
You can meet with me at other times by appointment.
Check my availability here.
Email: vijay@cs.utexas.edu

TA: Samantha Miller
Office Hours: MW 3-4 PM
Where: GDC First Floor TA Desk 1
Email: samantha.miller123@utexas.edu

Detailed Class Schedule

Guest Lectures

We will have guest lectures from senior engineers in industry so that we can learn about the current practices and problems in the field.

The first guest lecture will be from Prince Mahajan and Indrajit Roy, Staff Engineers at Google.

Our second guest lecture will be from Goranka Bjedov, Capacity Engineer at Facebook.

Canvas and Piazza

We will be using Canvas for course administration, grading, and submitting assignments and project. Check out the course page here. You will need your UT EID to log in.

We will be holding our class discussions on Piazza. Please sign up here.

Grading

Extra Credit

There are a number of opportunities to earn extra credit. One of the goals of this class is to get you started in research in distributed systems. As such, I offer extra credit for working on research and getting to a feasible submission.

• If you have a feasible workshop submission (5 pages) by Mar 15, you receive 0.5% extra credit. If you are attempting this, you need to get started on your research project early: come talk to me about this.
• By the end of the term, if you produce a full-length conference paper (12 pages) that is a feasible submission to OSDI, NSDI, SIGMETRICS, or FAST, you receive 1% extra credit. Each of these conferences has a different focus. The bar for a feasible submission at these conferences is quite high, so make sure you talk to me about it and put in the work!
• Class surveys. You will receive 0.25% (of the total grade) extra credit for completing an unofficial survey (before the first mid-term), and 0.25% for completing an official survey near the end of class.

Exams

There will be two mid-terms. Each mid-term will count for 20% of the grade. The exams will be conducted in-class, in place of our regular lecture.

Midterm 1: Thu Mar 8th (tentatively)
Midterm 2: Thu May 3rd (tentatively)

Deadlines

Every week on Friday (except when we have midterms), you will be required a submit an assignment answering questions on the papers we discussed that week.

Mar 2. Implementation Project Code and Report due.

Mar 8. Midterm 1.

Mar 15. Extra credit workshop paper due.

May 3. Midterm 2.

May 3 - May 10. Schedule out-of-class presentation for Research Project.

May 10. Research Project Report due.

Projects

There will be two big projects in the course. Students will work in groups of two or three for both projects.

Implementation Project. You will be building a distributed key-value store. The project document can be found here.

Research Project. You will be conducting original research in distributed systems. You will produce a 12 page research paper at the end of the semester. For extra credit, the strength and rigor of the research should be such that it is a viable submission at OSDI, NSDI, SIGMETRICS, or FAST. Alternatively, you could write a workshop paper based on your research; but this has to be done by Mar 15. More details will be added soon. This project will be due May 10th.

Acknowledgments

Copyright

Copyright Notice: These course materials, including, but not limited to, lecture notes, homeworks, and projects are copyright protected. You must ask me permission to use these materials.

I do not grant to you the right to publish these materials for profit in any form. Any unauthorized copying of the class materials is a violation of federal law and may result in disciplinary actions being taken against the student or other legal action against an outside entity. Additionally, the sharing of class materials without the specific, express approval of the instructor may be a violation of the University's Student Honor Code and an act of academic dishonesty, which could result in further disciplinary action. This includes, among other things, uploading class materials to websites for the purpose of sharing those materials with other current or future students.