Giter Club home page Giter Club logo

coursebook's Introduction

Coursebook

Build Status Current Pdf Link Current Wiki Link Current HTML Link Current Epub Link

Welcome to the systems programming coursebook! This repository houses a high-quality, open-source introductory systems programming textbook used by the CS 341: System Programming course at the University of Illinois at Urbana-Champaign The book assumes that you have taken a programming language course and are familiar with assembly instructions. All of the code and instruction will be in C, as it is the de-facto language of the Linux Kernel.

The Coursebook is an aim to standardize and build upon Angrave's original wikibook experiment.

The most recent version of the coursebook as a single pdf is here: main.pdf.

Overview

Goals

  • Improve the quality and rigour of the original wikibook while maintaining openness.
  • Improve the factualness by including citations, footnotes, extended reading, and a glossary.
  • Have exports in the form of pdf, markdown, and html.
  • Automagically build so writers can focus on writing.

Contributing, Building, and Structure

Read CONTRIBUTING.md

Credit Where Credit is Due

coursebook's People

Contributors

abhinavpappu avatar adiprerepa avatar aneeshdurg avatar angrave avatar ankitgohel1996 avatar ayushr2 avatar benjaminpollak avatar bhuvy2 avatar caoeri avatar daruuro avatar ezhang887 avatar hanxu12 avatar hungwoeineoh avatar hx507 avatar jtschuler avatar matthewscholefield avatar mayaanayak avatar nd-0r avatar pengyuejiang avatar pradyumnashome avatar rafilong avatar ram-g avatar ridhwaandev avatar rmenezes782 avatar smstong avatar st-arry avatar tudorsiminic avatar vivekmallampati avatar williamsentosa95 avatar wweil avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

coursebook's Issues

Example of MMAP Across Processes

Is your feature request related to a problem? Please describe.
We need to include an example of two processes using MMAP to communicate

Describe the solution you'd like
The example above

Describe alternatives you've considered
N/A

Additional context
N/A

More Why Doesn't This Work Examples

A huge part of systems programming is that we include why doesn't this work examples. Off the top of my head

  • Deadlock
  • pthreads
  • Concurrency
  • Networking
  • Signals

Could use more examples

Integrate with Pages Website

Ideally, we should figure out a way to integrate the website and the wikibook. We should have something like recursive build triggers like in the assignment-docs repository and will be easy because credentials are all set up.

Most of the leg work should be done in the website repository so that this project is as agnostic from the website as possible.

Worked Through Filesystems Example Question

We should have a worked through example of pretty much every calculation that you can think of for filesystems meaning.

  • Max/Min Size of a file
  • Size on disk vs Size of file

Pictures would be nice.

Author Resolve

From what Prof. Angrave said, we need a better way of keeping track of who contributed to the wikibook. To paraphrase, we should keep a automagically generated file from the previous wikibook plus the contributions from the new one. This is a little hard because of github doesn't do contributions that well (pretty much the same reason why Linux isn't on github for contribution reasons), but we can make it work by asking people for their name or if they wish to keep FERPA then submit changes anonymously.

We should also have a recognition and contributors page as well so people's names can forever live in the wikibook.

Figure Out Citations

Currently, citations are not being rendered correctly on the wiki. There is almost no way to integrate them with traditional build tools because pdflatex and latexmk use configuration files that have to be run ~3 times for the citation to go through not to mention the sheer amount of support code needed for that.

I more reasonable approach could be to try and manually resolve citation blocks and put a link to the bib file that contains it which would be a more reasonable amount, or just put it in plaintext and require a few clicks to get to the citation.

Detail Errno

We should define errno as well as let people know about posix error handling and what the general format is for problems and return codes in C (i.e. exception oriented programming languages with c++ and java and return oriented languages c and go).

Filesystems Read Write Model Demystified

We should walk through in the last section of the filesystems chapter of how to perform reads or writes in the filesystem as well as a more thorough explanation of the filesystem

Question Driven and Quick Reviews at the end of sections

So the wikibook at the current moment works really well for people (myself included) who like the straight read-through book aspect. I think we should marry the old book's approach of question driven answers with a new structure

  • Have each section maybe have a few driving questions. This helps people guide what they are looking for
  • At the end of each section there are a few do-you-know the answer, so that people can quickly check instead of just going to the review questions.

High Level Scheduler Overview

We need more background talking about the pros and cons of each of the scheduling algorithms at a high level i.e. how well they do on average, what their use cases. We should ask questions for a workload like X what would be the better algorithm?

Pandoc Magic Failures

Sometimes pandoc just refuses to convert tags even if there is an appropriate definition of them in a particular tex file on the system or even if they are named something different. The shim helps but it seems to be a bigger issue -- i.e. the proof environments aren't converted and the asides are not clearly marked.

Finding Filesystems

We need a high level overview of how to structure a filesystem, what the pertinent parts are and then dive into examples so someone should be able to implement a file-file system.

Change to Deploy Keys

Currently we are using secure tokens in the build process. Ideally we switch to public/private keys and deploy that way.

Backport Honors Lectures

Right now the honors section has a lot of good content that we should include in the wikibook asides or nice-to-knows so that people can have a fuller understanding of various topics not limited to.

  • Other Models of Parallelism
  • Intro to distributed
  • Intro to cryptography
  • Intro to lock-free
  • Intro to Security
  • Introduction to Kernel and Linking
  • Various mini-lectures
  • TCP and discussion of the two generals problem. Maybe include something alluding the byzantine tolerance but don't go too in depth into that.

Epoll Weirdness

Epoll is weird because it actually points to the file description not the file descriptor. We should list some of the pertinent quirks in the epoll chapter.

Dependencies /how to contribute?

Currently, for new contributors who may not have used latex before, it may be useful to document what steps should be taken to set up a development environment. We should determine what packages we used and what needs to be installed to contribute.

As a temporary measure, we should at least point people to installing texlive-full.

Furthermore, we may want to consider adding a quick latex tutorial or linking to one, and also consider adding a style guide if some sort.

Rename this project

Current name 'wikibook' is confusing and incorrect and makes it difficult to refer to the 'old' or 'new' resource.
It's not editable as a wiki.

"coursebook" would be better project name

Extra: Futexes

A lot of people think that mutexes are implemented as a spinlock but that isn't really the case when you have linux. We should talk about what a futex is, and how we could use one to implement an actual mutex as an extra part.

Expand on Lightweight Processes

Right now we don't talk about lightweight processes a lot. We should really detail the fact that a thread is a process with a few flags removed

Convert all Foreign Pictures to Our Own

Right now, we aren't claiming any copyright because of contributions and the fact that a lot of the images in the wikibook are created by someone else. It would be nice if we had completely original images for every chapter so we finally own all of those images.

IPC Implementation and Reasoning

Right now a lot of the IPC/Files chapter revolves around how to use, but not how it is implemented or when should you use a certain solution. We should fix that with preferably real world examples or contrived examples of pros and cons for each

Accessibility

So the website needs to be accessible to those with screen readers. Latex and PDFs are not the best format to do this, but HTML is arguably leagues better. Since Latex is just a markup language, we should be able to convert between this one and other ones. We can use pandoc to convert the latex into flat HTML and then transform it into accessible html that we then push to the website.

https://developer.mozilla.org/en-US/docs/Learn/Accessibility/HTML

Introduction Chapter/Teaching Someone to Fish

Although it is on the website in ways and forms now, I'd like to have a chapter strictly dedicated to tips and tricks for doing well in the course and programming career things like

  • How to search for errors
  • How to spend less time debugging
  • How to use editors/ssh better
  • How to use asserts better
  • How to plan out angle of attack before writing code
  • How to set up dev environments

233 Compatability Shim

Although you can, you need a bit of background information before you can really understand all the topics in C and some of that happens to be some 233 Topics namely

  • Caching
  • Interrupts
  • Assembly
  • Atomic Operations

Additional Topics we could cover

  • Hyperthreading

IPC Calculations with Diagrams

Ideally we should have

  • A worked through how much memory does this process take up if it's heap is x, stack is y, and all other sections take up n pages total
  • We should have pictures for how much total space a total level page table takes up
  • We should also have more worked through calculations on offset sizes and the process by which to dereference an address (an explain like I'm five).

Provide Travis Build Template

  • The travis build template should build the pdf and the html version
  • The html version should be put in a directory and trigger a website build
  • The pdf should live somewhere on the releases branch

Make HTML Build

Provide a way that the files can be molded into one website.

Pointer arithmetic rapid fire

We should have a bunch (~50, ~100) of pointer arithmetic rapid fire checks where one can go through a list of pointer arithmetic operations and tell you how many bytes are moved over

char **x;
x + 2 // how many bytes
(int)x + sizeof(x) // 64 bit system
*x + 3

etc etc

Extra: Queueing Theory

It would be neat to mention where all the scheduling algorithms come from with queueing theory just so people know where to look if they are interested.

As an added bonus we could link this back to packets going through a router to give a concrete example of why do I need to know this.

Epoll Example

It'd be really nice to just heavily explore epoll and the other IO mechanisms for event driven code.

More background info

A Common trend in the wikibook is that things make sense if you already know them. Meaning that if you are already familiar with the term packet them you understand them better. I'd like to try and do a survey of what we inherently assume is "background knowledge" and what we

I have done this for a few chapters but not all. We should do a more thorough survey and try to be a little more verbose

  • introc/introc
  • processes/processes
  • malloc/malloc
  • threads/threads
  • synchronization/synchronization
  • deadlock/deadlock (maybe cut down on this)
  • scheduling/scheduling
  • ipc/ipc
  • networking/networking (almost done)
  • filesystems/filesystems
  • signals/signals
  • review/review
  • appendix/appendix

Spell Checking

Grammar is hard, Spelling is (supposedly) easy. We'll have to define a dictionary full of nomenclature, but we should have some quick Travis checks that makes sure that all misspellings are tackled.

Code quality enforcement

In order to maintain a consistent sense of style/formatting in our tex source, I propose that we use a tool like texpretty or the online latex formatter (if we can find an API for it).

I tried playing around with texpretty and it seems promising, but it doesn't really like some of our macros/embedded code snippets.

Markdown Branch

We should have a branch that generates a markdown version of the wikibook so people can view it online. This should be really easy with pandoc + travis builds.

Concurrency Needs More Analysis Snippets

A problem a lot of students faced in the previous quiz is there wasn't enough analyze these code pieces. They understood the definitions correctly, but would like more examples of "what value is this" or "what does this deadlock".

Building chapters individually

Talked to @Ram-G and he felt that chapters should be built individually to reduce compilation time for small edits and I think it's also useful as that would open the door for custom build scripts for individual chapters.
The proposal is to generate a Pdf per chapter and have the global makefile make each chapter then stitch the pdfs together.

Mention `AF_UNSPEC`

We should mention in networking more portable ways of serving sockets like AF_UNSPEC for Inode Types

System Programming Case Study

I would really love a chapter that goes through famous examples of bugs and breaks them down. The reason that programmers get paid so much isn't to code, to implement documentation, or to include tests, it is to interpret the requirements and anticipate bugs that would cost millions of dollars. This would provide for a nice why? in terms of each topic in systems.

https://github.com/danluu/post-mortems

https://en.wikipedia.org/wiki/List_of_software_bugs

Off the top of my head

  • Heartbleed
  • Shellshock
  • Dirty COW
  • Meltdown
  • Spectre
  • Portsmash

Extra: Scatterlists

We should touch on scatterlists in the IPC section because it is a good peering point for user space programmers to know about the virtual memory abstraction and it is used in the linux crypto library

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.