html menu by Css3Menu.com
Optimization versus Monte Carlo analysis
What does the Monte Carlo analysis in TradeSim do ?
When trading a system using a portfolio of securities the outcome is not unique due to the number of different permutations and combinations of securities that can be traded. This non-unique outcome gives rise to the need for proper statistical analysis in order to thoroughly evaluate a trading system. For example on a given day there maybe 7 entries of which you can only take 3 due to capital limitations or other constraints. Because of this there are many different combinations of trade entries that can be taken.
The Enterprise Edition of TradeSim incorporates proper statistical analysis through exhaustive testing using a Monte Carlo analysis approach aiming to evaluate every possible trading 'route' and then analyzing and correlating the resulting data. This is done by repeatedly simulating a trading system where each simulation uses a random selection of trades, which satisfy the trading rules. Although it is possible to manually run multiple trade simulations on TradeSim and analyze the resulting data the Enterprise edition of TradeSim automates the process for you and presents the results in easy to read reports and graphical formats.
The resulting in-depth trade analysis is like a 'footprint' or 'signature' of the trading system. It gives you a multi dimensional view of your trading systems performance. Rather than display a rats nest of equity curves overlaid on top of each other it makes more sense to do a proper statistical analysis and display the results in terms of frequency distributions of net profit, winning and losing trades as well as trade expectation if it is applicable to the position size model used.
What is Optimization ?
An optimization procedure attempts to step through some trading system parameter(s) and run a backtest for each step. Usually one performance metric such as profit or drawdown is plotted for each step and the best result is assumed to result from the optimum set of system parameter(s). When you try to optimise a system you always run into the risk of curve fitting your system even with a walk forward strategy. Also an optimization procedure is really only designed to work with a single market and not a portfolio or multiple market.
For a portfolio a different approach is needed where you run a Monte Carlo analysis for each trade parameter step and plot all of the resulting histograms on a common chart. It is quite possible that the distributions of results may not obey a simple normal or gaussian distribution having a single mean and standard deviations so therefore it is not possible to rely on a single statistical metric such as average or standard deviation etc. Also it is most likely that even if you use ranking and force a single outcome for a portfolio simulation this may not produce the most optimum result for that set of simulations so optimisation based on this single outcome can be quite misleading.
What is a Parametric Sweep ?
In the latest version of TradeSim we have added Parametric sweep function to the Enterprise Edition in the current. http://www.compuvision.com.au/ParmetricSweep.htm
If you observe, when we step through a trade parameter we run a Monte Carlo analysis for each step and then plot the resulting histograms for several metrics on a common chart. You will find that a conventional optimisation strategy does not do this nor is capable of doing this because it is to slow. Competing products usually get around this by using a ranking criteria to force a single outcome for each simulation. This makes their optimisation procedure even more dubious because you are only seeing one snapshot for each optimisation step and this most likely will not be the most optimum value had an in depth Monte Carlo analysis procedure been carried out. In the end this single optimisation performance metric becomes statistically insignificant and misleading. The nice 3-D surface chart produced makes for a nice poster on the wall but that's about all.
To give you some idea of the difference between Optimization and Parametric Sweep an optimisation procedure may run 1,000 simulations corresponding to 1000 steps. With our new Parametric sweep we might just run 20 steps of "risk per trade" from 1% to 5% in 0.2% steps and then run 5,000 simulations per step. That is 100,000 simulations for only 20 steps ! As you can see there is much more information being shown in just 20 parameter steps than there is for a 1,000 steps optimization procedure which only displays information for 1,000 simulations versus 100,000 simulations for the parametric sweep. Also, as can be seen it is not just a simple matter of looking for a single metric per step like it is for a convention optimisation procedure. You will note that some histograms can be quite complex with double humps etc. You can't possibly gauge this information by just running one simulation per step.
You might ask what happened to all of the missing information from the portfolio optimisation procedure that now exists in the parametric sweep ? Well because of trade ranking which is usually based on some arbitrary and untested ranking criteria all of this extra information is missing !
