What if the TARDIS wasn’t actually picking up on Clara’s timeline. What if there was some residual DNA transfer from Danny on her hands. The Doctor asking who her date was with was more than just an attempt at conversation. He was actually hinting at the fact that MAYBE there was another reason for their destinations being Rupert’s childhood and future great-grandson. Possibly, they weren’t actually following Clara.
If Danny’s DNA plus her thinking about him when the phone rang formed a psychic link between Danny and the TARDIS. It could have sent them off on a tangent. We saw that later when Clara was distracted while doing ‘a thing’ and flying the TARDIS. Distracted by the Doctor, she flew the TARDIS into his childhood by mistake.
Nothing directly says that Clara and Danny end up together. Orson’s great-grandparent was not specified as being a woman. Basically, you could plug in any time traveler.
What if it was just Danny? If Danny joins the Doctor and Clara later as a companion, we do not NEED Clara to make this story work. Danny himself would qualify as the time traveler and familial link.
As we know, the TARDIS has a problem with past - present - future. The TARDIS could have simply anticipated Danny joining the group, and stretched a bit further than expected.
So I am seeing a bit of a running “thing” with the Doctor and his chalk.
Keeping my eye on the chalk.