Inconsistent Results

[ajwalker]

David,
I have set up a Metastock/TradeSim simulation on ASX stocks and it basically seems to be running OK.

It has an entry trigger of
Cross(Stoch(5,3),Mov(Stoch(5,3),3,S))
together with a number of other qualifiers, and a TradeSim Trailing Stop (6*ATR(10)) exit trigger. I have set the number of records to 32,000 and do a 10,000 simulation Monte Carlo analysis.

When I run the same full (Metastock and TradeSim) simulation it gives different results each time it is run. On 4 different occasions it gave average profits of $347K, $373K, $407K and $399K respectively. The drawdown results likewise widely diverge. (On each of these occasions none of the programming or parameters in either Metastock or TradeSim were changed. The share portfolio is unchanged and I have tried deleting the "trb" file in between simulations without any improvement).

If I simply rerun TradeSim on the same Metastock file the results are near enough to being the same, although they do vary slightly.

This effectively means that if I want to do a slight change to the Metastock software and then compare the results I can have little confidence that any changes observed are significant.

I am obviously doing something wrong and have scoured the manuals and the TradeSim forum looking for answers with no luck.

I have submitted this to Jose and he has suggested that it may have something to do with the settings in TradeSim. Any assistance you can give would be much appreciated.

Regards
John Walker :?:

[ajwalker]

David,
I should add to the above that I have discovered that the "Total Number of Trades found in Database" entry in the TradeSim message log varies for the identical explorations. For example 1854, 1996 and 2133 were three of the results when the explorations were run as per my previous message. Higher numbers do not necessarily mean higher profit.

Regards
John Walker :?:

[David Samborsky]

Yes it will give different results for two different reasons. Every time you run a Monte Carlo analysis it may use some simulations which did not occur in another run thus altering the statistics slightly. If this is the case then you should try 50000 runs for example. The higher the better but not to high because then you would be wasting CPU time running identical simulations. This of course depends on your trading system and size of the portfolio as well as the number of trades.

Also every time you generate a new trade database for a given record count some trades at the begininng of the security data drop off because the first bar that was there on a previous day is no longer used in todays exploration because your record window advances by one bar every day so you get newer trades at the end of the security data and early trades disappear.

You can avoid this aberration by using more runs in the Monte Carlo analysis thus smoothing out the statistics.

[ajwalker]

David,
Thankyou for your response.

I think I may solved the problem. Wanting to exclude those securities with more than 3 digits in the code, I included in the main exploration code (Column A) as part of the Entry conditions the TradeSim function:
ExtFml("TradeSim.GetSymbolLength")<=3.

This apparently is something that works on filtering securities not assessing the suitability of trades. I therefore tried placing it in the Filter section of the Exploration and checked the "Use Filter" box.

Now, every time I re-run the Exploration/Simulation it gives virtually identical results (within the limits of the Monte Carlo statistical variations).

With best regards
John Walker :idea: