Teaching humans machine learning

One of the courses I audited for fun during grad school was Yaser Abu-Mostafa’s Learning Systems, an introductory course on machine learning. I didn’t bother to keep a notebook for it, but I did take a sheet of paper with me to class, and scribbled perhaps a couple of words/formulas per lecture—sometimes to help me think, mostly to take note of some important idea. I ended up using the same sheet for the whole semester, and by the end it was completely covered with short notes and equations. I was quite proud of that mess of a page, and the fact that I could recall the main ideas of almost all the lectures by looking at it.

Yaser Abu-Mostafa

Yaser’s lectures were often delightful; he taught with great clarity, insight, and enthusiasm—no wonder his was one of the few courses I was able to follow from beginning to end without any formal obligation to do so. I remember one lecture in particular, where he talked about Occam’s razor and the problem of overfitting. The buildup and the punchline were delivered so skillfully (and joyfully!) that I actually felt an urge to stand up and applaud (but did not).

Some time ago I came across a profile of him, written shortly after he received a prestigious teaching award. After reading a few snippets from interviews with him and his students, it became clear that Yaser was making a conscious effort to make each lecture into a little story, a focused presentation with a clear goal, boiling things down to the essentials, getting rid of clutter and inessential side issues. My being able to summarize each lecture with a few words was no accident; it was  his great skill and specific vision as a teacher that made this possible.

Gian-Carlo Rota

Interestingly, I had read an article by the mathematician/philosopher Gian-Carlo Rota that specifically stressed the benefits of such an approach, but somehow his advice started to register/make sense only after I read about Yaser’s teaching, and recalled my own experience with his lectures. Here is what Rota, a celebrated teacher himself, had to say:

a. Every lecture should make only one main point.

The German philosopher G. W. F. Hegel wrote that any philosopher who uses the word “and” too often cannot be a good philosopher. I think he was right, at least insofar as lecturing goes. Every lecture should state one main point and repeat it over and over, like a theme with variations. An audience is like a herd of cows, moving slowly in the direction they are being driven towards. If we make one point, we have a good chance that the audience will take the right direction; if we make several points, then the cows will scatter all over the field. The audience will lose interest and everyone will go back to the thoughts they interrupted in order to come to our lecture.

–Gian-Carlo Rota, Ten Lessons I Wish I Had Been Taught

I think it is worth noting that lectures with such laser-sharp focus may not be the best choice for all subjects and purposes. There may, for instance, be cases where one would prefer to encourage the students to brainstorm, to bounce ideas around, and develop their personal knowledge and thoughts in a more organic way. But if there is a set of clear, well-defined ideas that one wants to get across, there is much to gain from a focused, uncluttered presentation.

Yaser with Claude Shannon

As fate would have it, at a time when my formal background in the field consisted solely of my having (informally) attended Yaser’s lectures, I applied to an internship in machine learning and—thanks in part to Yaser’s kind letter of recommendation—had a chance to actually use what I learned in his class for fun (and to breathe the air of Silicon Valley, which I probably wouldn’t have otherwise—but that is another story).

The chance to audit courses unrelated to one’s own field was one of the great perks of grad school. Nowadays, of course, you don’t even need to be near a school to have such opportunity—any curious soul can sign up for whatever online course catches their fancy. Although the selection of available courses is still somewhat limited, there are true gems out there, one of which happens to be the online version of Yaser’s  course. If you are curious about his teaching, you can easily sign up or just , and see for yourself.

—

Yaser’s profile is here. Gian-Carlo Rota’s Ten Lessons is here. The latter piece is reproduced in Rota’s book Indiscrete Thoughts, from whose foreword we learn that Rota loved to simultaneously entertain and make one uncomfortable, a hint of which, perhaps, is the zoological metaphor above.