Avrohom looked so much closer to Yishmael. In fact Yishmael did Teshuva during his father’s lifetime, while Aisav seemed so evil. Under the surface however, Yitzchok’s connection to Aisav is limitless (he was a Jew, only a sinner) while Yishmael’s relationship to Avrohom was on the outside only. This reflected the difference between who Avrohom and Yitzchok were themselves.