You might be wondering what, exactly, it is that a trader does all day.  Or in my case where I have a 9-5 job, what I do with the 3 or so hours a day I spend working on trading.  If you look at the example trades I’ve posted on this blog, they don’t take very long – the Tickle Me Elmo trades were only open for a few minutes each.  This morning I waited an hour and a half to take a single trade that lasted 7 minutes (a slow one by my standards).  Even taking several such trades a day, there’s no way the actual act of trading is going to add up to more than about an hour of time.  And since my trade entry and management is largely computer automated, most of that time when I have a trade on is really spent just sitting there making sure the computer doesn’t crash.  In an average day I only spend about 30 seconds to a minute actively entering or canceling orders.

Point being, actually trading makes up a tiny part of what a trader does.  Some of the rest of the time is spent watching for trading opportunities to develop, although that task is arguably better off-loaded to a computer.  But the vast majority of time is spent preparing to trade.  This is somewhat akin to training and practice in athletics – good players spend tens of hours preparing for every hour of game clock time spent on the field or court.  Of course, the analogy can only be stretched so far – sports preparation is largely physical whereas trading preparation is academic in nature.  But the same sort of ratio holds – massive amounts of preparation are required in order to achieve good results for a very short period of actual performance.

There’s a problem though.  New traders have no clue what to do with all that prep time.  It’s easy to see what the end result should be: placing trades where you have a high confidence they will make money overall, and doing so without making errors.  But how do you prepare to get to that point?  The answer is trading system development.  A trading system is a set of rules that describe a trade in full – what instrument is being traded, under what conditions you’re going to put on a trade, how big a position you’re going to take, how you’re going to exit the trade etc.  It’s a formal way of encoding a method of making money in the market.

Not all trading systems are equally good.  In fact, the vast majority you could dream up will lose money on average.  Of those that make money, many will have other bad properties that make them unusable – they involve unacceptable risks, or are psychologically impossible to execute (don’t underestimate this), or require technology and execution capabilities you don’t have.  But a few systems will have almost entirely positive properties: they make money, are easy to execute, don’t involve outsized risk, etc.  What you want to do as a trader is find one or more of those good systems, and put your money to work trading them.  So the process is really one of search: you’re looking through a large space of systems to find one of the few systems that have the properties you’re looking for.

This search task is also a matter of expertise.  For the experienced trader who has developed many successful trading systems in the past, creating another one is not hard.  All the technical tools and experience required are already in place.  It’s still not a fast process, but it’s not that difficult.  For the novice, it’s incredibly difficult – each step is fraught with potential errors that will cost you real money.  Now, one way to become an expert is to dive in, make the mistakes, lose that money, and learn.  That’s what I did, and I don’t recommend it.  I’d be much richer today if, when I started trading, someone had laid out the process for system development to me and explained what it was I needed to do in order to find a working trading system.  And all the information is out there, but it’s scattered and mixed in with all sorts of junk and took a long time to find.

The trading system development process is the meat and potatoes of this blog.  If you understand it in depth and follow it you can make money as a speculator.  The process is long and complicated, so I’m not going to be able to explain it all in this post – it’s a textbook sized chunk of material and will probably take me about a year to write out.  What I’m going to give you today is an overview – the basic steps of the process without any of the details in the form of a flowchart and some brief descriptions.  This should be enough to get you thinking in the right direction.  Then I’ll come back and do the deep dive in future posts.

Here’s the process:

• Idea Generation: Before you can make a trading system, you have to have an idea.  You have to spot a situation in which the market behaves in a somewhat predictable, non-random way that you can exploit.  This is the most free-form part of the process, and the place where the best traders distinguish themselves from the merely good.  For a new trader the most effective means of generating system ideas is to study known trading systems from the past, understand how the work (or no longer work) and experiment with variations on them.
• Trading Trial: once you have an idea that you think might make money, the next step is to try trading it on paper or in a demo account where you can’t lose real money.  At this point, you don’t have many concrete notions of how the idea is going to be implemented.  You’re just experimenting to see how you go about constructing trades that exploit the market behavior you’ve identified. You may also want to start developing technical tools – indicator software and the like to help you recognize the situation you’re trying to exploit.  Frequently, during this step, your conception of how the trade will work will change as you discover the situation is different once you’re “inside” it trying to trade than it was looking from the outside.  Eventually via experimentation you should arrive at a method of trading your idea that seems to make money most of the time.
• Formal Specification: if your trading trial seems promising then the next step is to formally specify your trading system: under what circumstances do you enter a trade?  When and how do you exit?  What market(s) do you place the trades in.  What time of day will you trade?  These questions should be answered in sufficient detail such that your new system could be given to another person, and they would generate the same trades implementing it that you would.
• Backwards Testing: once you have a specified system, you need to do rigorous statistical testing to determine whether it works.  This is akin to the type of confidence testing you may have done in a statistics class, but with some additional complexity.  The backward testing phase is done on historical data – the question you’re trying to answer is whether your method would have made money in the past, and if so how profitable it was.  If the backwards testing gives favorable results, you move on to forwards testing.  If it shows the system didn’t work acceptably, you move to re-work.
• Forward Testing: forward testing is simulated trading on paper or against a demo account.  The purpose of forward testing is to ensure that your method continues to work going forward.  This serves both as a statistical confirmation of the back-test (the reasons confirmation is needed are complicated) and as a trial run for actually trading the method.  The goal is to make this step as much like real money trading as possible without actually risking money.  You want to expose execution issues that may not have been visible in the back test – for example lack of liquidity and slippage on market orders, limit orders that don’t get filled because of FIFO order execution, and slippage on any stops you’re using.  You need to asses whether these problems are sufficiently severe that the method no longer works.  If the forward test gives good results, you move on to Risk Assesment & Tailoring.  Otherwise you go to Re-Work.
• Risk Assessment & Tailoring: when you have a method that appears to work, you then need to do some further analysis on the risks of trading it.  The analysis should highlight two types of risks – infrequent risks that might cost you a large amount of money in one go, and simple risk of failure where the method produces a string of losing trades.  The investigation into infrequent risks needs to consider scenarios which did not appear in your back or forward tests – market crashes, broker crashes and bankruptcies, etc.  The losing trade analysis is mostly concerned with determining how big a position you should trade such that losses akin to what you saw in your tests won’t cause you financial distress.  This process frequently benefits from Monte Carlo analysis.  The output of this step is an understanding of how your method will behave in various unlikely scenarios and an understanding of what constitutes reasonable position sizing.  If the method behaves unacceptably badly in infrequent cases, it may have to be modified to reduce risk (go to Re-Work).  Otherwise, continue to Real Money Trading.
• Real Money Trading: This is where you finally get to make some money if you’ve done everything right.  You can switch over from demo trading to a real money account.  If you’ve been rigorous up to this point, real money trading should be no different from your forward test.
• Continued Performance Evaluation: as real money trading proceeds, you need to monitor your strategy to make sure it’s still working.  This is a statistical process more or less like the forward test.  If you become convinced your strategy has somehow stopped working, it’s Re-Work time.  Otherwise keep trading it.
• Re-Work: for some reason your system doesn’t work, or has stopped working.  You need to do analysis to understand what has happened, and figure out if the system can be adjusted to get it working again.  This is in essence a repeat of the specification process.  If adjustments can be made, you go to Backwards Testing.  If the problems cannot be fixed, the strategy should be abandoned.  You need to be particularly careful about backwards testing after re-work to ensure that you don’t test on the same historic data that motivated the re-work in the first place.
• Diversification: when a strategy works well, you want to get as much out of it as possible.  This can consist of just increasing your trading size as your account equity increases, but it can also be accomplished by applying the same or a similar strategy in different venues – other instruments, other time frames etc.  In other words, a working system is an excellent source of ideas to generate yet more systems and perpetuate the process.
• End Of Life: sometimes trading systems just don’t work out, or what worked for a while eventually dies.  When this happens, and Re-Work proves futile, the best policy is generally just to give up on the system and go find a new idea.

As you can see, there’s a lot to the process.  If this were a textbook, each bullet point would be a chapter – this post in in essence the “index” to this blog.  I’ll link relevant information in as I write it.

This also explains why it doesn’t work to just go out and buy a canned trading method.  Even if the person selling the method has your best interests at heart (and they rarely if ever do), they’re not giving you the whole process.  Usually they just provide a method specification (often incomplete) and maybe some back testing information.  As you can see, that’s a tiny portion of the entire process and the parts they failed to give you can easily sink the whole enterprise.  Of course, if you were sufficiently experienced you could fill in the missing parts.  But if you were that experienced, you wouldn’t need to buy the method in the first place and could have just done the specification work yourself.  So don’t buy trading methods.