Latency in gaming means you would pull the trigger and have a delay before your gun actually shoots.
By that logic, although changing channels might take longer, once its playing, you shouldn't get any buffering.
If you had lots of high spikes then maybe...but that screenshot only has a couple....so yeah not too sure.
Have you asked virgin to do a line check? You may have too much noise on it which can be fixed at their end.