In order to evaluate trading strategies, it’s very handy to have one number that represents how good a given strategy is – bigger is better.  I’ve suggested a couple of ways of doing that –  win rate and expectation.  Both are uselful, but both also have substantial limitations that render them problematic in the real world.  What I want to do here is briefly describe those limitations, and then suggest an alternate mathematical construct called “profit factor” you can use for evaluating systems.

Underlying this whole question is the concept of a probability mass function (PMF).  You can think of this as simply a list of the possible outcomes of a trade or bet, with each outcome annotated with the probability that you will get that outcome.  So if you have trade with two possible outcomes, winning $100 and losing $100, and each is 0.5 likely, that’s a simple PMF.  In the real world, we don’t necissarily know the outcome PMF for the trades we are making – frequently the best we can do is observe the same trades historically, and use what happened in the past as our expected PMF for the future.  This approach is basically sound, although there are lots of caveats I’ll get into in other posts.

A large part of evaluating trading strategies is simply “scoring” these PMFs – evaluating how attractive a given trading system is and thus deciding whether to assign trading capital to it.  What we’re trying to do here is reduce all the information in the PMF into one value, one figure of merit.

Win rate is a first attempt at this.  And in cases where there are only two possible outcomes – one good, one bad, it can be sufficient.  If the two outcomes are also of the same size (as in the +$100/-$100 example above) then it works very well – a win rate of 50% is breaking even and anything higher is profit.  Even if they’re of different sizes, the break-even win rate is simply different – it was 52% in a previous example because of brokerage fees.

Win rate becomes uninformative when there are multiple levels of profit or loss that can occur.  A common example of this is trend following systems where losses are generally bounded by a stop loss order, but profits are allowed to run as far as the trend continues.  So you would end up with a large number of small losses, some small wins, and a few much larger wins.  In a system like that, win rate is uninformative – a 30% win rate could be highly profitable if the wins were many times the size of the losers.  Some other metric is needed.

A common newbie mistake is to focus too much on a trading system’s win rate without considering the size of the wins and losses.  New traders frequently develop strategies with a large number of small wins and the occasional disastrously large loss.  This is probably psychologically driven as much as anything – winning feels good and seems like confirmation of your intelligence regardless of the amount won.  These sorts of systems can also be very deceptive if the event that causes the huge loss is rare – it’s easy to have that rare event never appear in your system’s back and forward testing by simple luck or subconscious avoidance.

Computing expectation is a means of dealing with some of these problems.  To compute expectation, you simply take each result in the PMF, multiply it by its probability, and sum those.  This is the same as computing the mean or center of mass of the PMF.  Expectation cleans handles the issue of large numbers of different outcomes.  You can read all the tweaky mathematical details here.  Expectation is much more useful than win rate – if the result is greater than zero then you will, on average, make money taking that bet.  This is a handy filter – while not all positive expectation bets are worth taking, negative expectation bets are always worth avoiding (exception: hedges that reduce the volatility of another bet).

There is however a more subtle and serious problem with expectation: it loses a lot of information about the shape of the PMF which radically effects the desirability of a bet.  Consider these two bets with the same expectation:

Bet 1: for every dollar wagered, you either:

• Lose $1, probability 0.495
• Win $1, probability 0.505

Bet 2: for every dollar wagered you

• Win $0.01, probability 1.0

Both bets have an expectation of +$0.01 – for every dollar wagered, you will on average make a 1 cent profit.  But they are not equally desirable.  You would be happy to wager everything you had on Bet 2 assuming the time period was short.  In fact, if you could borrow money at less than 1% cost over the time period of this bet, you’d be willing to bet however much money you could borrow on it without any concern.  The same is not true of bet 1 – you would not want to wager everything you had on it, let alone borrow money.  Bankruptcy would be the result of the course of action 49.5% of the time – presumably an unacceptable level of risk.  So clearly there’s something going on which expectation doesn’t capture.

This issue is the spread, or in finance terms volatility, of the results.  Bet 2 has no volatility – you always get the same results.  It is in finance terms “risk free” (which is a loaded term).  Bet 1 has risk.  The real problem with expectation as a figure of merit for trading strategies is that it factors out volatility and pretends it doesn’t matter, when it reality it matters a lot.  This is closely related to another problem: you can double the amount of the expected return of a bet by doubling the size of the bet.  Because expectation averages out risk, this appears attractive.

Enter profit factor (PF), which tries to deal with these issues.  It’s computed as follows:

PF = SUM(winning results * P(result) / SUM(losing results * P(result))

note: for this calculation, losses are positive numbers, not negative.  So the PF is always positive between zero (no wins) and infinity (no losses).

In plain English you sum up all the wins, times their probabilities, and divide (or really, normalize) by the losses times their probabilities.  A break-even strategy, where wins exactly equal losses on average, will always have a PF of 1.0.  Values higher than 1.0 are potentially attractive.  Anything less than 1.0 is unprofitable..  As a general rule, PFs between 1.0 and 1.2 are not worth trading despite being profitable – the risk is too high for the return.  There are some exceptions for systems that generate a huge number of trades.  Values between 1.2 and 2.0 are the types of systems that represent the bread and butter of most systems traders.  Anything over 2.0 is fantastic – but I would carefully check the data that produced it, because something is likely to be off.

If you do the math, Bet 1 above has a PF of only 1.02.  And if you’ve never worked with PFs before, I suggest you actually do the math as practice to make sure you get the same result I did.  That low profit factor is a numeric representation of what we already knew intuitively – you can’t assign much of your net worth to Bet 1 out of concern for the risk. Say you assigned only 2% of your net worth to the bet (to tamp down the risk to a reasonable level), then your expected return on net worth would be only 0.02% – basically not worth the time to mess with.  This is what I mean about profit factors less than about 1.2 not being worth trading.  Whereas with Bet 2 you could assign your whole net worth and get a 1% return on net worth.  Note that Bet 2 has effectively an infinite PF – you end up dividing by zero because there are no losses.

I’ll be making heavy use of profit factors in this blog to evaluate trading systems – they’re not the only thing that matters by any means, but they do a nice job of capturing both risk and reward in a fair way.