Fantasy football league quality is generally dependent on the resolve of the person designated as commissioner. Disputes must be resolved fairly and objectively, team managers must be involved and want to succeed, and any money collected must be distributed according to the pre-determined structure. First, I will discuss the various tools available to the commissioner, and then go into the qualities that make a good commissioner.
The commissioner determinates some of the most important aspects of the league before the draft. First, the commissioner is generally responsible for recruiting enough players to have a competitive league. General leagues are between 8 to 12 players, but 6 or 14 in extreme scenarios can be used. Generally, four teams are frowned upon because every team will be an all-star team and skill will be relegated to luck. Sixteen teams is in my opinion way too much, because you will be scrapping the bottom of the barrel of players to fill out your roster.
If you are doing a cash league, recruit players that can and will pay before or at the draft. I have had several leagues with that one bad apple that always has a sob story about how they do not have the money, forgot the money, or will pay you in the future. Avoid these people if possible, because they are very hard to collect from. As the commissioner, it is your responsibility to promote fairness and collect funds, so if you can not collect then you might be responsible for their share, which of course is not really fair.
Finally, try and make sure that your players are knowledgeable about sports and competitive. Chances are that most passive fans will give up as soon as the going gets rough, so try to find people with fantasy experience, football knowledge, or a competitive streak. Most leagues have one or two people that might give up halfway through, but a league where half of the managers give up is just no fun for anyone. Try to control the types of people that enter your league.
After you have a general idea of who will be in your league, you will need to make some decisions about your league. Scoring is pretty important, and most league hosting sites have default values already installed for their leagues. If you are really unfamiliar with fantasy football, you might want to leave the values as they are. Depending on how you want to weigh things like defense vs. defense, position importance, and special team scores, you might want to adjust some of the default scores, however. Also, some leagues get really technical and use decimal points to represent yardage or assisted tackles, and some use negative points. Decide how realistic and serious you want your league to be and adjust your scoring options accordingly.
As commissioner, you also need to decide which roster positions are best suited for your league. Once again, the sites generally have default roster positions set, but you will need to decide on a couple of issues relating to positions. Generally, basic leagues include team defenses, but more complex leagues can include individual defensive positions in addition. It is not uncommon to have one or two positions for each level of defense, from the defensive line to the linebackers to the secondary. Keep in mind that defensive line players will be hard to get points for because they do not register too many tackles, so anything in excess of two will really be a stretch to roster formidable linemen.
Another simple decision that a commissioner must make is whether to have a live draft or an online draft. If at all possible, invest in a live draft. It gets the competitive juices flowing, allows everyone to meet each other, and makes it easier to collect league dues. Live drafts are not always practical, however, because people have different schedules and in many cases live too far away from each other to make it happen. In these cases, there's nothing wrong with doing an online draft. Just make sure that everyone can make the draft, because not having people there can make them lose interest or be happy with the procedure taken to choose their teams. Overall, maybe one or two people might miss the draft and it's not the worst thing that could happen, but plan for a way for no one to miss, and that should take care of most of the risk.
There are several other pre-draft decisions that a commissioner must make, but these are pretty simple decisions. The Commissioners must decide whether to organize their league in divisions, how much money to charge (if any), whether or not to set a limit for number of acquisitions, and whether or not to make the league a keeper league (some players are kept for more than one season) or an auction league (waiver acquisitions are determined by a bidding format.
Once the draft has begun, the commissioner for the most part is just another manager, but he or she may occasionally have to settle disputes. In these cases, generally try to include the opinions of as many league members as you can, but it is your responsibility to make firm and sound decisions. Use managers as you would advise, realizing that they may make decisions based on their own interest as well because of stands or friendships. In the end, you need to make judgments fairly and objectively.
Finally, pretty much any one can be a good commissioner, but there are several qualities that should help. First, managers should be good leaders. People need to believe in the decisions that a commissioner makes, and the commissioner must be confident enough to enforce rulings. The Commissioners should be responsible, as they will need to invest a good deal of time communicating with managers and collecting dues. They need to remain invested in the league even if others decide to quit, because the entire league depends on them. Finally, commissioners should be fair, because decisions will need to be made that for the integrity of the league will need to be made objectively.