The problem with using past performance to predict future results

So I've been thinking lately about how common it is in job interviews (for software developer roles) for companies to make judgements (both good and bad) about a candidate's suitability for the job based on references or experiences from years in the past. And how problematic that is.

I'm sure you've improved as a programmer in the last few years. I definitely have. And if you haven't, there is something seriously wrong.

And it's not just programming skill, but also in dealing with people, managing your time, being able to follow and suggest industry best practices, etc. All of these things are important to how well you are able to do your job.

It is so hard to even define what makes a good developer, let alone trying to predict how good someone is based on a subjective recollection of the work someone did 5 years ago; probably in entirely different circumstances.

So how far back should a company look? And if the answer is 'not very far', where does that leave companies who want to hire good software developers? What can they look at to get a decent indication of whether they should hire a candidate or not?

Recently at RailsCamp (Melbourne 2013) I heard someone say that when you're interviewing someone, you need to find out 3 things:

1. Is the candidate capable of doing the job?
2. Does the candidate want to do the job?
3. Do I like this person enough to work with them?

2 and 3 seem pretty straightforward to determine. Capability on the other hand, seems much harder. I would suggest that you should put the least weight on point number 1. If the candidate really wants to do the job, and you like them, they are probably capable of doing it.