I'm certain that absolute democracy will under all circumstances lead to absolute idiocracy, given enough time.
I firmly believe that all voting must be limited to certain topics, but not certain people. If we allowed any sort of interference about the question who gets to vote, the "loud" ideas will inevitably overrule the "quiet" ideas, and history shows that the loudest ideas have almost always been the worst. Taking away the ability to vote from certain groups would in turn lead to this group being ignored, since their voice stops to count, and thus it also stops to matter. You can freely take rights, choices and property from those people without ever getting bad results, on the contrary, you will get better results from the people who dislike the non-existant group.
For the same reason, some things must be exempt from voting, call it basic human rights or civil rights or whatever you will. In fact, i think that it's much better for the continued existence of a society when you don't allow people to vote on laws at all, but only on representatives for some form of parliament every once in a while (3-5 years, imo), who are in turn personally accountable for the decisions they make while in power. The representative towards the outside should be elected by the parliament itself, as primus inter pares.
This would allow people to lead in a meaningful way, and give the general population the option to actually influence the way they live.
Funny part is, all of us always exempt a very large part of the relevant population from all decision by sticking to the notion of borders. Small wonder no country actually cares for 3rd world development or immigration rights, after all, those people can't cast a vote.
So, in my opinion, all votes need to include the whole known population to be effective in making everyone's lives better, in all other cases it's always about making the life of some group better, usually at the expense of another group.
TL,DR: I disagree with the question in itself, because i'm quite sure that there is no vote that should exempt anybody at all. If the voting is narrowed down so much that it only applies to a certain group, the vote is inherently bad.