I think that Signpost Corner is a fair spot to watch. A fast approach. a decent 90degree bend, and you can watch them disappear round Bedstead.
I once walked it from the Sefton Hotel on the promenade, and it took about an hour and 10minutes.
You walk into Martin Bullock's garden, make a contribution to the Helicopter Fund, and stand on the bank in the garden right on the bend.
You've got toilet facilities, tea and sandwiches, plus a decent radio commentary.
Definitely where I would go, even for just one race.
You could cut out most of the walk by catching a bus to the Manx Arms in Onchan.