If for no other reason than it helped shape the direction of two characters, this one is worth watching, even if it's not too great otherwise.
The Doctor and Seven were two characters the show never really got tired exploring, so it was only natural that they would develop some kind of relationship. Given that Seven basically replaced Kes, it's also appropriate that this happened, since The Doctor tended to try and live vicariously through the more innocent members of the crew, and you don't get more innocent than Kes and Seven. In his quest to push his own limitations, however, The Doctor could sometimes make huge mistakes, and he didn't seem to mind dragging others with him.
Hence what happens in "Retrospect." Actually, that's pretty much all there is. It doesn't really do much for Seven, which is weird, because she's supposed to be the hot new thing on the show, and one of her own episodes is co-opted by an existing character's story. The fact that The Doctor's lesson is a little redundant here is why I don't find it to be one of his stronger entries, but it's worth a look for its historical significance.
Franchise * Series * Essential * Character
Memory Alpha summary.