Good point! i never thought about it like that!
and if someone has pointed this out then sorry but what about in the ep Outside Man(I think) when he said to the guy don't you want to stick around to find out who killed the mother of your children?
that could be about his past too i guess