tradesim vs metastock testing should be similar?

[gawdib]

I understand that Metastock tests multiple securities by just taking each one. one by one, so that is not reflective of how one can trade, since, to duplicate metastock a tradesim coder may have to put in to have 50 open positions and enter 30 trades in one day , etc. Therefore, tradesim simulations will differ often in overall statistics,
even tho the dates of entry and exit were identical , comparing the 2 systems, for each trade for each stock, and prices were quite similar.

However, as a way of checking out tradesim,
I would think that the results of tradesim would be nearly identical to metastock
IF
one programs tradesim to allow such a large number of trade (200) open positions that tradesim does not reject any trades because of max open positions. then one sees the amount of available capital that can be used
( either % of capital
or the same fixed dollar amount per trade), so that the capital used was no more than ONE'S AVAILABLE CAPITAL , so that
NO TRADES ARE REJECTED by the tradesim simulation.
Would not this force tradesim to aact like metastock system testing?

Then, if one puts that fixed investment/trade (that worked out in tradesim, so no trades are rejected)into a metastock simulation then the results with both should be the same?


But when I do make these unnatural changes,

I find occassionally the results in terms of overall profit are very similar.. other times the differences can be 50 times less in tradesim.

even tho tradesim is programed to take every trade, just like metastock has done.

(and I am not referring to the times that frequently tradesim will not work because it does not register exit prices accurately with certain exit triggers)

I usually hope to do tentative system development in Metastock, then do a final testing with Tradesim, but the difference in results make me feel unsure as how to use the two programs to help with system development ,
and distrustful of both programs now.

Any guidance/comment you could give would be appreciated as to why the results would not be the same after I do these manipulations. What Are the other ways that the 2 programs work differently?
thank you, regards, geoffrey

[David Samborsky]

There are as many a variety ways of testing a system just as there are many ways of trading a system. The results obtained from testing should however mimic as closely as possible the results from trading.


I usually hope to do tentative system development in Metastock, then do a final testing with Tradesim, but the difference in results make me feel unsure as how to use the two programs to help with system development ,
and distrustful of both programs now.

Why use both?

If in doubt TradeSim produces a comprehensive Trade Log which can be used to validate the results.

I can't speak for Metastock and the algorithms used but the algorithms used by TradeSim are designed to mimic real world trading conditions as practically as possible.

[gawdib]

Why I use both is because anyone can do extensive overnight optimization for testing of which formulae work on Metastock,but not with Tradesim. this is invaluable. Tradesim does not allow this, so it would take years to set each one up manually to see which varients work on most stocks

[sectorbets]

I think I'm finally starting to understand--maybe--your long series of postings over the last several months. This most recent post suggests that what you are doing is curve fitting your systems to past results by running optimizations against a long list of stocks.

This is a major error that many if not most folks make early in their expereince with developing trading systems. It isn't all that difficult to back test any set of variables to arrive at the perfect system for any number of stocks, but since you are likely curve fitting by over optimizing, you simply cannot achieve similar results in the real word, hence David's comments about TradeSim trying to act as closely as possible to real world trading situations.

Read about the quest for the holy grail again and again and again until you get it--curve fitting won't work--period! Keep it simple--the code you have posted on and off is anything but simple--and don't over optimize and then you will see if your system will work during different market conditions, using different stocks, and when it does, you are on your way to trading for a living.

[Jose]

Geoffrey, even MetaStock's developers don't seem to understand how their own System Tester works. This module begun life as a basic optimizing tool, and has become more unreliable and buggier with each new version of MetaStock.

My advice, for all that it's worth, is:
don't waste time on the System Tester.

And you will also save yourself a lot of heartache if you take Sectorbets' good advice (above): stay clear of curve-fitting.

jose '-)

[gawdib]

thnaks, Jose:
I have been consistently very impressed by your work and comments. I have found that , when using a system for 100 or more stocks, that Metastock system tester has helped me find varients that , later,Tradesim proves to be improvements. That is why I am reluctant to give it up, because I could never have enough time on Tradesim to type in each varient each time, so that I can have some hint as to the worthwhileness of an idea. best regards, Geoffrey

[David Samborsky]

TradeSim can do what Metastock can do but it does it better! :wink:

Set the simulation type to "Basket Test" and run a simulation. You can then inspect the trade statistics (ie Trade Statistics, Security Profit Distribution, Trade Profit Distribution etc) in the Trade Report in both tabular and graphical forms.

The beauty of the basket test is that it tests each trade in isolation according to your trade parameters and combines the results of trades with the same symbol to produce the security statistics. :D