Programming By Example (Introduction)

When I first started to learn about programming, many more years ago than I care to think about, my idea of how it should work was that it should be like teaching someone how to perform a task. After all, isn’t the goal of programming to get the computer to learn and then actually perform some new behavior? And what better way to teach than by example?

So I imagined what you would do would be to show the computer an example of what you wanted it to do, go through it step by step, have the computer remember all the steps, and then have it try to apply what you had showed it in some new example. I guessed that you’d have to learn some special instructions that would tell it what would change from example to example and what would stay the same. But basically, I imagined it would work by remembering examples you showed it and replaying the remembered procedures.

Imagine my shock when I found out how most computer programmers worked. There were these things called "programming languages" that didn’t have much to do with what you were actually working on. You had to write out all the instructions for the program in advance, without being able to see what any of them did. How could you know whether they did what you wanted? If you didn’t get the syntax exactly right (and who could?) nothing would work. Even after you had the program, tried it out, and something went wrong, you couldn’t see what was going on in the program. How could you tell which part it was wrong? Wait a second, I thought, this approach to programming couldn’t possibly work.

I’m still trying to fix it.

Over the years, a small but dedicated group of researchers came to feel the same way I did, ultimately developing a radically different approach to programming, called "programming by example" (PBE). It is sometimes also called "programming by demonstration," because the user demonstrates examples of the desired behavior to the computer. A software agent records the interactions between the user and a conventional "direct-manipulation" interface and writes a program corresponding to the user’s actions. The agent can then generalize the program so it works in other situations similar to, but not necessarily exactly the same as, the examples on which it was taught.

This ability makes PBE like macros on steroids. Conventional macros are limited to playing back exactly the steps recorded, making them brittle, because if even the slightest detail of the context changes, the macro ceases to work. Generalization is also PBE’s central problem, the solution of which should enable PBE to replace practically all conventional programming.

Children might represent the first real commercial market for PBE systems. They are not spoiled by conventional ideas of programming; for them, usability and immediacy are paramount. That’s why it’s with children in mind that this special section explores two notable PBE systems recently brought to market to enthusiastic receptions from their initial users, many of whom are children. David Canfield Smith and Allen Cypher’s Stagecast Creator (described in Smith et al.’s article), which evolved from Apple Computer’s Cocoa and KidSim, brings rule-based PBE to a graphical grid world. And Ken Kahn’s ToonTalk (described in Kahn’s article), a programming system that is simultaneously a video game, uses a radically different programming model, as well as a radically different user interface. ToonTalk solves the problem of generalizing examples in a simple, almost obvious way—by removing detail. The program is less specialized and therefore more applicable in a wider range of situations.

PBE is one of the few technologies with the potential for breaking down the Berlin Wall that has always separated programmers from users.

We also analyze PBE’s user requirements, examples of functioning PBE systems, and directions for the future of PBE that hopefully all demonstrate the power and potential of this innovative technology.

One way PBE departs from conventional software is how it applies new techniques from AI and machine learning. Incorporating these techniques represents a tremendous opportunity for PBE but incurs the risk that the system will make unwanted generalizations. Myers et al. treat the thorny issue of how much intelligence a PBE system should have from their wide experience building intelligence into PBE systems.

We can’t convince people about PBE’s innate value unless we offer at least some good examples of how PBE is being used in specific application areas. For example, Bauer et al. unite PBE and the Web—everybody’s favorite application area today. The Web is a great focus for PBE because of its accessibility to a wealth of knowledge, along with the pressing need for helping users organize, retrieve, and browse it all. Recent developments in intelligent agents can help—but only if users are able to communicate their requirements to and control the behavior of their agents. PBE is ideal. PBE can also be used to automate many other common but mundane tasks that under conventional circumstances consume a frustratingly large fraction of programmers’ and users’ time.

So, you may ask, if PBE is so great, how come everybody isn’t using it? PBE represents a radical departure from what we now know as programming; it can’t help but take a while before it becomes widespread, despite the existence of many systems demonstrating its feasibility and value in improving applications in a variety of domains. The conservatism of the programming community is the biggest obstacle to widespread PBE use.

Repenning and Perrone show how to make PBE more like human learning by using analogy—an important intuitive cognitive mechanism. We often explain new examples by way of analogy with things we already know, allowing us to transfer and reuse old knowledge. They show how we can use analogy mechanisms to edit PBE programs, as well as to create such programs from scratch.

Finally, St. Amant et al. explore what at first might seem a crazy approach. We (I’m one of the authors) have the computer simulate the user’s visual system in interpreting images on the screen, rather than accessing the underlying data. Though it may seem inefficient, this approach neatly sidesteps one of PBE’s thorniest problems—coexistence with conventional applications. It enables what we call "visual generalization," or generalizing applications on how things appear to users on the screen, as well as on the properties of the data.

PBE is one of the few technologies with the potential for breaking down the Berlin Wall that has always separated programmers from users. It allows users to exploit the procedural generality of programming while remaining in the familiar user interface. Users today are generally at the mercy of software providers delivering shrink-wrapped, one-size-fits-all, unmodifiable applications. With PBE, they could create personalized solutions to one-of-a-kind problems, modifying existing programs and creating new ones, without going through the arcane voodoo characterizing conventional programming.

Back to Top