When I flip someone off, it's an expression of anger. I'm wholly intending to express displeasure, frustration, and even fury with the recipient. It's a very in-the-moment gesture.
But if I give someone a thumbs-down? That's not indicative of anger at all. I'm sitting in judgment.