The characters that the player chooses only affect gameplay, not the storyline. Even if they aren't aren't on your team, the Eleventh Doctor will still talk to Amy, Rory, the First Doctor, the Third Doctor in level 69 of Chapter 3. It is stated in multiple chapter 4 levels that the events that occur within the game are experienced by all of the characters.
There are even some levels where the average player will have a team that does not match up with the story being told, such as level 28 of chapter 4 which narratively only happens to the Twelfth Doctor and Clara. This shows that the gameplay is separated from the story. The story will affect the gameplay, but not vice versa. All the characters do is give you abilities and change your attack, defence, and health.