Game Design

Balance in CA should always be approached hollistically, in respect to its design. It is a complex network of niches, counter mechanics and roles. These can be unit-unit counters, or situational dynamics such as terrain. When designing the game, what a unit is meant to be good at, what it is meant to be bad at, what it is meant to beat what is meant to beat it, are the most important thing. Balance should respect this.

<[FlagHax]CarRepairer> it's simple really. unit cost = (pwn - fail) * baw

However, when dealing with a unit you think needs to be changed for balance reasons, there are three steps you should take.

Ingame experience

This is the most important. If a unit cannot be used effectively in game, then it isnt OP (yet) and if everyone uses it then it isnt UP (yet). However, beware the culture of balance. Often a unit will go for weeks without anyone using it, and then someone discovers and exploits it. Soon everyone is using it, and it goes from being 'useless' to being OP. Be careful not to repeatedly buff an unpopular unit, it can take a long time before people come back to an unpopular unit and give it another shot. Dont be too hasty to nerf a popular unit- perhaps this is a 'fad', perhaps someone will find a counter. If you do nerf it, be conservative- dont try and over-nerf it just to ensure people stop using it (Then its useless and will get buffed again).

Balance based on what good players do. This does not mean that you should balance the game entirely around the high-micro, hard to use 'pro' units or balance so that the game is only viable at a high level. However, good or inventive players can totally change the face of the way the game is played by effectively exploiting an underused unit. There are many units which an average player will not understand the use for until they have seen it used well.

Dont balance based on one game. Consult other players on their experiences too. You should test exhaustively, on several maps, with varying skill levels between the players. It is useful if testing a strategy to try it out vs someone, then get them to try it out vs you, both trying to counter.

1v1 is the most pure testing ground for a strategy. Team games often have different requirements but the various factors in play in team games make it very hard to get an impartial judgement, there are too many stray variables.

A good way to get impartial data on the use of units is through  real game statistics. Statistical abnormalities are excellent candidates for trying to exploit ingame. Remember however that what causes the most cost in damage in a game isnt necessarily what decided the outcome- It might just be the mop-up crew, or the victory may be economic.

Maths and Stats

Look at the actual stats of a unit. Sometimes identical units in both the factions will have widely different use rates and sucess. Sometimes one unit will appear to be OP when another one is actually far better. This is an incredibly useful tool in determining the problem in a unit might lay, as often the problem will stand out immediately. Other times, it is something very subtle. Remember there are a huge range of values besides hp/dps. Range, speed, manoeuvrability, and even variables in the script like turret turnrate.

Think about everything in terms of cost. Knowing how much damage or HP something has is an abstract figure, what is important is its cost/damage cost/hp.

The use of a unit is not always determined by good stats, but often by the right combination and values. There is a huge difference between just outranging an LLT, and having the same range as an LLT. Try to think more in terms of unit relationships than pure numbers. When balancing a unit, try and think of other areas that can be changed to make it useful, rather than just its hp/dps. What specifically is holding it back/making it too strong ingame? Also remember knock-on effects. If you nerf something, another unit which it has until now effectively countered may suddenly be OP. Remember to ensure that changes respect the role of the unit.

Also, see MathematicalBalancing.

Closed testing

Do for-cost tests of units. /spectator /godmode is your friend here! Micro both sides in various circumstances. This can help to nullify a whole range of factors that may be skewing your assessment, like economic differences and terrain. Note however that those factors are often precisely why those units are op, and that this is useful merely as a guide to get an idea of where problems might be.

All these options should generally be explored when attempting to balance a unit.

Links

 Issac's essay on balance design Saktoth: I would call the 'Game Balance' simply 'Design'. Prevents ambiguities.