Welcome guest, is this your first visit? Click the "Create Account" button now to join.
Results 1 to 3 of 3
  1. Default Why "Walk Forward Analysis" is still unreliable and useless! [DISCUSS]

    Hello, I am Darwin and today I want to talk about the limitations that walk forward analysis suffers from. This is my 3rd article, so if you do not know how WFA works, please read the other 2 (you can find them here on the forum).

    The target audience of this article is everybody that deals with ExpertAdvisors and Backtesting / Walk Forward Analysis

    Some of you might already have seen a few posts of me where I talk about some research I do in the fields of Trading System Analysis (in the course of writing a meta-algorithm that can build, analyse and trade strategies on its own). The goal is to write an algortithm that is so powerful that it can take every EA and, due to in-depth analysis, tell you how and when to trade it in order to make profit, no matter how good or bad the underlying EA is

    So here is a new article in which I would like to lay down some insights that I could get in the process of writing this algorithm (DATFRA - Darwins Algorithmic Trading Framework)

    Well, lets begin. My first concern is that the design of Walk Forward Analysis is, in its nature, unrewarding and not the kind of analysis a trader wants.

    Also, I claim that the results of a WFA are more or less random, and if a system works well after a successful WFA, then not because the test was successful, but because the trader designing the system did a good job.

    In this article I do not yet want to show how this problems can be solved, I just want to demonstrate that they exist. In my next article I will explain how I think this all can be solved in an elegant way.

  2. Default

    The problem is, due to limitations of WFA algorithms, optimisation has to be done in every single Walk Forward Window.

    Let's say we do a WFA on 10 years of data and our Forward Trading Timespan is 2 weeks: That makes 240 Walk Forward Windows. That means 500.000 tested parameter combinations per window would need 120.000.000 single simulations.

    And then, remember that WFA relies on a trial&error principle, so you will most likely have to do this a few times.

    You see? Evaluating the "real" picture would take very, very long, and therefore most WFA implementations are forced to only evaluate an very much cropped fraction of the actual parameterspace because it is not possible to evaluate the whole parameterspace (or a meaningfull sample of it) in a reasonably small timespan, because optimisation has to be done in every single WF-Window.

    This means, WFA most likely does not evaluate 500.000 parameter combinations per window but only 10.000 or 50.000 or something like that. So eventually we already lose like 90% of all data in this step.

    This is a problem that could be solved if the trader has lots of time for his/her analysis (which is not likely, especially based on the trial&error method), or with a more efficient design of these algorithms. Nevertheless, in praxis, this problem is ever-present.

    For comparison: DATFRA, which is my private research project, only has to do one single simulation per parameter-combination, no matter how many WF-Windows it analyses. In the above example, that would already decrease the computing time by the factor 240.

  3. Default

    Ok, now lets look at the second step of WFA, and the second problem: Missed data because of _wrong_ algorithm design and computing time concerns.

    Now remember, a meaningfull sample of our trading system's parameterspace would be 500.000, and we have 240 WF-Windows. That would make a total of 120.000.000 optimisation-candidates. And out of this huge amount, a WFA algorithm takes the very best per window, 240 in this example.

    That is 0,0002% of the total amount of all datapoints that we could use to describe/analyse this system and it's ability to produce good forward trading results, based on good optimisation results.

    And then WFA takes these few datapoints and claims it gives a somehow realistic view on a trading system's performance / robustness.

    Thats nonsense! You also would not judge a picture's colour by looking at 1 pixel, would you?



Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts