Actually, you hit the crux of the problem right on the head, without intending to. While they don't happen at a high rate, they do happen, "obvious missed/bad calls", some of which currently, there is no way to correct them.
Plays that are seen over and over after they occur and everyone is screaming "that can't be, the refs messed up! This can't happen again, fix it!"
Really, these unchallengeable mistakes are not much different than why instant replay was enacted in the first place. Replay was put into place to correct missed calls on catches, fumbles, touchdowns, etc. So why not include other potentially important outcomes? Why is a blatant pass interference excluded? Why can't a late hit call or no call on a QB be looked at again?
I don't buy the "you can't challenge a judgement call" excuse. Because all of us have seen more than enough plays to admit, sometimes a refs "judgement" is just way off, possibly through no fault of his own.