This is a question to which I have had different answers at different times, involving many of the points that have been mentioned on this thread and elsewhere on this forum. But I have come to the conclusion that Ennis was gay. In explaining why I think this, I will start by defining terms, as Welshwitch suggested. By "gay" in this context, I mean having an ongoing physical/sexual attraction for people of the same sex/gender, which is stronger than any such attraction for people of "the opposite" sex. It also involves identity of one's self, though in this case Ennis fights against it throughout the entire film. It was interesting for me to read that Ang Lee himself had no qualms about Ennis being gay, and said he thougt it was obvious as to what/who attracted Ennis. Obvious from Ang Lee's perspective translates quite differently for me and obviously for many others here. For us it wasn't obvious at all.
For me, it boils down to the conversation Ennis and Jack have at their next-to-last (in the film) camping trip, where Ennis asks Jack if things between Jack and Lureen are "normal", and does she-Lureen-ever "suspect". Then he gets down to brass tacks and asks the question that has been eating at him for years, does he-Jack-ever get the feeling that other people "know". He gives that word "know" special emphasis. It implies others are aware of something that he has been trying to hide. In this context, I don't think he is merely referring to his ongoing affair with, or love for, Jack Twist, the details of which may have become public-via Alma- after his divorce. The question arises from a fear that is more central to his being and sense of who he is. It is so central and so frightening for him that he cannot bring himself to utter just exactly what it is others know about him, but he assumes-correctly-that Jack will understand his meaning. He interestingly does not phrase it as other people thinking he's queer,-to use his word- but knowing, thus, perhaps inadvertantly, certifying its truth. This is Ennis's great secret that he has been hiding, not least from himself, all his life.
Jack's response, that maybe Ennis ought to get out of there and move somewhere else, suggests that this is exactly how he understood the question, and being intimately knowlegeable about Ennis's fear, he offers the first suggestion that comes to his mind. That's our Jack, remember? "Ready, shoot, aim." He doesn't seem to quite grasp the signficance of what Ennis is asking/telling him. As one who struggled mightilyfor more than two decades with trying to keep exactly this secret, I cannot really see any equally plausible interpretation of just what Ennis is so afraid of other people knowing about him.
As for Ennis wanting a family, it is here that I think Ennis displays one of his most striking character flaws. He may indeed want a family, but he is unwilling to commit himself to making the sacrifices necessary to provide for them adequately, and spends all of his free vacation time-according to the story-on fishing trips with Jack, never with Alma or his daughters. He compromises his 'career' in doing so. This doesn't speak well of a sincere desire to be a good husband and father, at least in the ways which would count in his rural Wyoming culture. On the other hand, having a wife and kids is the best insurance he can provide himself and the outside world that he is not, in fact, gay/queer. They inadvertantly help him prop up his denial system, to their detriment. I'm not saying he doesn't love his wife and daughters, in his way, but he fails-or is unable-to give them priorty in his life. So, that's my two cents worth.