Referee Phil Dowd awarded Villa a penalty, that James Milner converted, after Vidic had hauled down Villa striker Gabriel Agbonlahor inside four minutes.
Friedel said: “We started the game really well but we have seen the penalty incident now and it really should have been a sending-off.
“You feel really hard done by after you have viewed it afterwards because you would have liked to be playing against 10 men.
“We feel a bit hard done by that we didn’t play with a man advantage after that, especially on a pitch as heavy as that and cutting up like that.
“With 10 men, they would have really tired towards the end but it wasn’t to be.”