They do have a narrative connection, though. They're all incarnations of the Doctor. Yes, it's flimsy, but there's lots of stuff that's flimsy that we do. For instance, does Captain John Hart have a narrative connection to James I, by your definition? No. But they do under my definition: Both were monarchs of England at one point. And look: They both have the Monarch template.