Do you think fans should get their responsibility as all-star voters taken away?

This whole Rory Fitzpatrick thing awhile back and Buffalo bandwagoners stuffing the ballot boxes so a guy like Ryan Miller starts over Brodeur in the all-star game. I know the game is pretty meaningless but some guys miss out because of irrational fans. Thoughts?