In that case, it doesn't sound like an audio interface buffer issue.
The vocal harmony *can* easily get distorted; try checking the vu levels by going to the master compressor (from mixer). I find you have to dial the mic gain *just* right, too little and vocal harmony is barely there (and VH itself will have trouble tracking the note you've sung, in which case instead of a normal chordal VH, you should change to a *vocoder*, which doesn't need to track your original sung note) too much gain, and while it tracks your singing easily, you'll have constant crackling from distortion (even though it *might* not appear too loud internally, it's still too much gain for the VH send)
It's like when you setup a compander (compressor /expander/ limiter); the output might appear to stay in the green, but the signal going in may have already exceeded the red-line)
Conversely, you can also leave the gain high, but don't sing close to the mic
Mark