I think it's both the mating sense, but also and perhaps more the inherent physicality and vulnerability in the M+F pairing. Yes, some women are bigger than men blah blah blah but on average, most heterosexual liasons see the woman put in a position of physical vulnerability due to their size and strength. So from that, a female cheating on a male could be viewed as not only a betrayal but a usurping of that "safe/vulnerable" space dynamic the female and S.O. first had; even more so if the affair-guy is seen as a risk for aggression or violence, which is often the case and probably part of the appeal of themes in games like this.
To me the whole trust part is overstated though; I often hear that as a progressive platitute; "oh, its not the physical act, it's the betrayal of trust in the emotional relationship".
Pardon my french, but that's fuckn bullshit.
If a guy's girlfriend tells him she's going out to eat at McDonalds, then comes back home and he asks, "How was McDonalds?", and she replies, "Hahaha I lied to you, I actually went to Taco Bell", he doesn't feel some deep emotional cut at the betrayal, no, he'd probably say; "Uhhhh well did you get me any gorditas?"
He feels devastated at sexual cheating because another guy had his dick in an area she normally only shows to her doctor and significant other; the physical act itself is extremely relevant in the betrayal.