Discussion forum for David Beazley

Python Projects Week


On June 18-22, 2018 I’ll be running my first ever “Python Projects Week” course (http://www.dabeaz.com/pythonprojects.html). This is a new course so I thought I’d say a few things about it.

For the past few years, I’ve been thinking about the idea of a more “Project Focused” Python course that could complement the material I teach in my more traditional “Practical Python” and “Advanced Python Mastery” courses. My Compilers course is certainly project based, but it’s also narrowly focused on a specific topic and involves a rather large project. I was thinking about something that could be a little more general that would feature a series of smaller, more self-contained projects–each project being something that could probably be completed in a single day. The projects also had to be targeted at more intermediate level programmers. These are not for total beginners nor are they freakishly advanced. However, they’re challenging in a way that allows you learn various elements of advanced Python programming as well as some interesting software architecture.

The real challenge is in coming up with suitable projects. I don’t really want to do projects that are too heavily focused on installing packages and looking up API functions on Google. There should be much more about coding practice, design, tradeoffs, and computational thinking. The projects should spark a lot of discussion and allow for exploration of different facets of Python. I’d also like to do some things that involve other interesting aspects of computing so that each project becomes so much more than simply learning a narrow bit of Python code.

With about two-and-half weeks to go, I’m pretty excited about how the course has been shaping up. Right now, it’s built around four core projects.

Project 1: Digital Circuit Simulator. We’re going to build a Python version of a digital circuit simulator. “Wait, I’m not designing circuits”, you say. This project actually involves a lot of design discussion concerning objects, relationships between objects, state, as well as time. Surprisingly, it intersects with a lot of the issues faced by people using async frameworks (callback hell, scheduling, coroutines, etc.). We’ll explore some of that. We might even dive into some unusual async/await usage. It also presents some interesting challenges with respect to testing (we’ll give pytest and hypothesis a whirl). Plus, it’s just kind of cool.

Project 2: Dataflow Library. A major use of Python is as a tool for command, control, and coordination. A lot of this involves the concept of streams, pipelines, and workflows. We’ll explore this idea in some depth. First, by various writing command line tools and looking at ways to hook them together. This involves a lot of system interfaces, files, subprocesses, and so forth. We’ll then look at iterators and generators. Finally, we’ll make a more general purpose data flow library. Thread programming and other elements of concurrency might make a guest appearance.

Project 3: Ray Tracer. We’ll work through Peter Shirley’s “Ray Tracing in One Weekend” project. There’s only a few challenges with doing that though. First, the whole project is presented in C++–so in order to do it, you’re going to have to figure out how to translate that code to something more Pythonic. The second problem is that Python sucks for ray-tracing–it’s crazily slow. So, we’re going to have to spend some time talking about profiling, performance optimization, and design choices for making things go faster. This could jump into a broad spectrum of topics ranging from numpy, pypy, C extensions, Cython, and JIT compilation tools such as Numba, Last, you might be able go beyond simple optimization–perhaps you could make it run in parallel using multiple CPUs. It’s hard to say where this project will end up, but at least you’ll have some cool pictures by the end of it.

Project 4: Twitter. Well, maybe not all of Twitter, but we’ll build a mini-version of some kind of “social network” and use it to learn about a variety of topics surrounding network programming, concurrency, distributed computing, databases, and related things. Note: this is not a “web programming” project where we install and try to figure out a bunch of frameworks. It’s more fundamental and focused on concepts related to system architecture. It should be useful–even if you are making a web app.

Overall, I’m pretty geeked out about all of these projects. There are still a few spots available for the inaugural week. You should sign up!