You can tell that a new spell is too powerful, says the Dungeons and Dragons 3.5E DM Guide, if every caster wants to use it. I think the same philosophy applies to other new game elements, including magic items and prestige classes. If a new Rod of Meteor Showers is so good that most everyone will get one if they can afford it, then it is too potent. If pretty much every fighter would take the Supreme Warrior prestige class instead of sticking with fighter, then something is too bogus with the PrC.
I would even go further: In general, a prestige class should not be more powerful than a standard option. For example, a fighter-like prestige class could start with +1 BAB per level, 2+Int skill ranks per level, one decent save, and a broad choice of feats every other level.
And I also subscribe to the idea that we should strive to make new rules that are like core rules. Yet the core rules for PrCs are often more powerful than core classes. How can we reconcile existing powerful DMG prestige classes with the desire to make them no more powerful (or attractive to players) than core classes? That is, if PrCs should be no more powerful than core classes, why did the authors of the DMG make their official PrCs more powerful than core classes? On an apples-to-apples basis, what gives?
In fact, the DMG PrCs ARE about the same in power as core classes, because they feature drawbacks -- limitations, high costs, and so on -- that make it work out. There are four ways to make sense of it: Rejiggerings, Requirements, Restrictions, and Relationships.
Rejiggerings are fixes to core rules that are actually broken. For example, spellcaster multiclassing simply does not work in the core PHB rules. A wizard 10 / cleric 10 is not as good a choice as a wizard 20 or a cleric 20. A sorcerer 5 / rogue 5 is not as powerful as a sorcerer 10 or a rogue 10. To solve this problem, prestige classes were added to the 3.5 rules edition -- PrCs designed to boost caster multiclasses. The three main examples are the Eldritch Knight (for warrior/mages), the Mystic Theurge (for priest/mages), and the Arcane Trickster (for mage/rogues). In these cases, the fix is in the prestige class. Similar new prestige classes might be designed for druid/bards, or for cleric/barbarians, and so on.
Requirements can be high indeed, and a steep cost can make a prestige class less appealing and more balanced. For example, the Shadowdancer requires three feats and three skills. Your feat progession is very limited if you want to be a shadowdancer. Therefore, high requirements make a balanced PrC by taking away the freedom to choose feats and skills as you might like.
Moreover, if the requirements force sub-optimal choices on the character, then a high-power PrC may be balanced -- read, “paid for” -- by requiring things that are not useful (depending on how useless the requirement turns out to be). The Shadowdancer’s high requirements are mitigated by their utility. But the Horizon Walker’s required ranks in Knowledge (Geography) and required Endurance Feat are of limited utility. Arguably, the Horizon Walker’s requirements are steeper than the Shadowdancer’s, since the Shadowdancer will get more mileage out of the required ranks and feats.
Another factor has to be considered in any debate over utility. A feat like Weapon Focus is inarguably more useful for a fighter-oriented PrC, while a skill like Concentration is more useful for a caster. The question is one of synergies. Stacking combat powers is a prime way to increase a PC’s effectiveness. Both Blackguard and Assassin require Hide skill, but for a Blackguard playing the traditional anti-paladin role (fighter or ex-paladin background, wearing armor), those hide ranks are a genuine pain in the neck to gain -- you have to multiclass or buy them cross-class. But the Assassin is very likely to max out his hide ranks anyway: for her the requirement is really no requirement at all.
Restrictions, which you might think of as disadvantages, are strict, relevant limitations on the use of an otherwise game-breaking power. The idea here is straightforward enough. If you throw too many powers into a prestige class, you have to balance it out with compensatory disadvantages. Here are a few examples. The Dwarven Defender gains incredible combat ability -- but he cannot move. The Duelist is similarly potent, but she cannot wear armor and must keep one hand free. These are clear and dramatic disadvantages -- no movement; no armor. Restrictions can be a little more subtle. The Thaumaturgist requires seven levels of cleric casting, but its base attacks and saves are worse than a cleric’s. Poor saves and attacks is a disadvantage.
I have to note that for a restriction to be an effective game balancer, it has to be something that actually affects the character. Restricting weapons and armor is no restriction for a PrC that advances arcane casting, for example. Both the Dwarven Defender and the Duelist are excellent models because their restrictions dramatically affect character options.
Relationships between a prestige class' powers and those of other classes and prestige classes are key -- especially to how a power-gamer can abuse a PrC. Power gamers excel at finding synergies among classes, that is, an ability with one class that serves as a force-multiplier when combined with an ability from another class.
For example, a power-gamer creating a sorcerer/rogue/arcane trickster would never cast magic missile. There's no attack roll for magic missile, so you cannot use it to sneak attack an enemy. On the other hand, Acid Arrow is a sweet spell for that combo, because if you have denied your target his dexterity bonus (maybe you cast Grease under him last round, so he is off balance, or maybe you are invisible), then if you hit and are within 30 feet, you can add sneak attack damage to your Acid Arrow damage.
Or for another example, a fighter/mage might specialize in spells with no somatic component, and take the Still Spell feat, so as to cast spells while wearing the heaviest of armors.
The relevance is that a balanced PrC does not offer those kinds of synergistic relationships, or keeps them to a minimum. A balanced PrC offers class powers that do not stack with the powers of other classes.
If you are creating a prestige class for fighters, adding a spell list and a casting table, like the Blackguard's, is less potent than giving extra martial feats or advancing spell casting in an existing class. A balanced prestige class can have fewer abilities that offer a multiplying relationship with base class skills, or it can offer more abilities that offer no real synergy. Combat PrCs that help attacks and caster PrCs that boost spell casting offer great synergies, so their balance must be achieved by restrictions and/or requirements. But a PrC that does not offer those synergies does not need those balancers.
Take the Horizon Walker. It gives a fighter’s base attack advancement, 4+Int skill ranks per level, and a class ability more powerful than a feat each and every level. For a ranger or ranger multiclass, the cost is almost negligible. There are no restrictions. By this enumeration, the Horizon Walker is way overpowered. So how can the Horizon Walker be a balanced option? In brief, the class abilities offer few synergies with core abilities. A barbarian might take a level dip for desert mastery’s immunity to fatigue. But the Horizon Walker adds abilities and powers that generally do not stack with those of other classes. No character seeking combat prowess over all else will pick this PrC.
Similarly, the Dragon Disciple (ignoring the abomination of the powers granted at level 10) is balanced because it grants combat bonuses and extra spells, without actually benefitting combat or casting to any great degree. The natural attacks are generally not better than weapon damage. The spells do not boost caster level or grant higher level spells. It is hard to imagine a powerful application of this PrC that leverages all or most of the class’s options synergistically.
So to sum up, a new prestige class should ideally be just about as powerful as a core class. If it is more powerful, it must have some combination of high-cost requirements and real restrictions to pay for that power -- or its class abilities need to be non-synergistic, so that they do not act as power multipliers.
