What To Do With Those Idle Cores?

Moore’s Law keeps on pushing more and more cores in our machines.  Quad core is common these days.  Eight core server chips will arrive later this year.  Many hundreds of cores are expected for processors in the next decade.

On a desktop machine, sometimes many of these cores will not be able to be used productively by the main task on the machine due to insufficient parallelism.  Sometimes, many cores will be idle. 

What should we do with all those idle cores?  One option might be to leave them idle.  Save power.

But, we have bought the hardware at this point.  It is a sunk cost.  In fact, the only cost we have is the marginal cost of the electricity between an idle and busy core.  If we can find any productive work that does not interfere with the main task that is greater than that small marginal cost, we should do that work.

So, given such a low bar, what work might be worth doing?

The most obvious category might be maintenance work such as indexing for search, compression, reorganizing, and pattern recognition.  Generally preparing the machine to move faster in the future.

Another might be defensive computer security, analyzing past and current patterns of activity on the machine, communicating with other machines, and working to prevent malicious activity.

More broadly, perhaps we should look to intelligent software agents, programs that act on your behalf autonomously, pulling down and compiling information for you for the future.  Examples of this might include monitoring agents that mine the Web to find interesting news or good deals for you.

Perhaps we could do batch processing work for others.  Folding@home might be a good example, using the idle processing power of many computers around the world to advance science.

Another option might be to focus on speculative execution of likely paths out from the main task, doing work that has a chance, perhaps only a very small chance, of being needed by the user in the next second or so.  Speculative execution on processors is already a form of this, but now rather than executing a few nanoseconds ahead we might now be executing seconds ahead.  We could do speculative execution of entire threads, prefetching and catching data, and implicit partial execution of actions in user interfaces.

What do you think?  What else could we do with these idle cores we are likely to have?