in Search

Formula too complex (temp array overflow)

Last post 05-29-2009, 19:11 by wabbit. 13 replies.
Sort Posts: Previous Next
  •  05-25-2009, 16:13 29782

    Formula too complex (temp array overflow)

    Im writing a custom formula within the Indicator Builder. But for some reason MS is telling me that the "Formula too complex (temp array overflow)". Is there go around this somehow, break it up into serperate formulas then reference them etc..... Any insight would be helpful?


    ExtFml("OpenCSVFile","C:\DataSet\","A") OR ExtFml("OpenCSVUpdateFile","C:\DataSet\","A") OR
    ExtFml("A.GetData","A",Fml("1")) OR
    ExtFml("A.GetData","B",Fml("2")) OR
    ExtFml("A.GetData","C",Fml("3")) OR
    ExtFml("A.GetData","D",Fml("4")) OR
    ExtFml("A.GetData","E",FmlVar("5","X")) OR
    ExtFml("A.GetData","F",FmlVar("6","X")) OR
    ExtFml("A.GetData","G",FmlVar("7","X")) OR
    ExtFml("A.GetData","H",Fml("8")) OR
    ExtFml("A.GetData","I",Fml("9")) OR
    ExtFml("A.GetData","J",Fml("10")) OR
    ExtFml("A.GetData","K",Fml("11")OR
    ExtFml("A.GetData","L",Fml("12")OR
    ExtFml("A.GetData","M",Fml("13")) OR
    ExtFml("A.GetData","N",Fml("14")) OR
    ExtFml("A.GetData","O",Fml("15")) OR
    ExtFml("A.GetData","P",Fml("16")) OR
    ExtFml("A.GetData","Q",Fml("17")) OR
    ExtFml("A.GetData","R",Fml("18")) OR
    ExtFml("A.GetData","S",Fml("19")) OR
    ExtFml("A.GetData","T",Fml("20")) OR
    ExtFml("A.GetData","U",Fml("21")) OR
    ExtFml("A.GetData","V",Fml("22")) OR
    ExtFml("A.GetData","W",Fml("23")) OR
    ExtFml("A.GetData","X",Fml("24")) OR
    ExtFml("A.GetData","Y",Fml("25")) OR
    ExtFml("A.GetData","Z",Fml("26")) OR
    ExtFml("A.GetData","A1",Fml("27")) OR
    ExtFml("A.GetData","B1",Fml("28")) OR
    ExtFml("A.GetData","C1",Fml("29")) OR
    ExtFml("A.GetData","E1",Fml("30") OR
    ExtFml("SMG.WriteData") OR
    ExtFml("SMG.CloseCSVFile") OR
    ExtFml("SMG.CloseCSVUpdateFile")

     

  •  05-25-2009, 18:24 29784 in reply to 29782

    Re: Formula too complex (temp array overflow)

    Being a fully evaluated scripting language, MS is going to perform every operation in this formula.  You might consider doing more of the "hard work" in the external function which will ease MS script limitations, be easier to debug and be more efficient.


    Hope this helps.

    wabbit Big Smile [:D]


    "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."
    Edsgar W. Dijkstra

     

    MS: 6.52 EOD, 7.x EOD, 8.0 PRO, 9.2 PRO w/QC, 10 PRO w/QC C, 11 PRO w/QC & MDK
    For custom MetaStock programming : http://www.wabbit.com.au
    My SkyPE status :
    My SkyPE account : wabbit.com.au

  •  05-25-2009, 22:30 29786 in reply to 29784

    Re: Formula too complex (temp array overflow)

    You mean convert the MS language indicators 1-30 and convert them into the dll? If thats what your refering to that will take way to long to program Im talking hundreds of man hrs.

    The Indicator is having trouble with the last 4 functions. Is there a way to seperate the indicator into two parts and have 1 indicator reference the other, such as a latch or something?...

  •  05-25-2009, 22:43 29787 in reply to 29786

    Re: Formula too complex (temp array overflow)

    With the code you have, MS will attempt to create a data array for every component of the code, so in your case this is 30 array; from memory, the limit is 26 (one for each of the 20 allowed variables, and one each for OHLCVOI)

    Again, if you don't provide specific details about what you are trying to achieve, you are only going to get "average" answers.  We have no idea what each of you functions is doing as you've never provided the codes for them, and have only provided some vague concept of the goal.  Is there any similarity between each of the functions meaning that all 30 functions don't have to be rewritten, or, are these simple functions that can be passed as arguments to the external function?  The more information you provide, the better answers you get.

    If you don't want to post your code in the public forum, contact someone who can help you by PM and ask them to assist.  The problem then is you are relying on only one person to help, when in the public forum, there are many members who can answer the question.


    wabbit Big Smile [:D]


    "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."
    Edsgar W. Dijkstra

     

    MS: 6.52 EOD, 7.x EOD, 8.0 PRO, 9.2 PRO w/QC, 10 PRO w/QC C, 11 PRO w/QC & MDK
    For custom MetaStock programming : http://www.wabbit.com.au
    My SkyPE status :
    My SkyPE account : wabbit.com.au

  •  05-26-2009, 18:43 29790 in reply to 29787

    Re: Formula too complex (temp array overflow)

    You mean post the code for all 30 MS indicators? thats a ton of code to post. So I have added my MS Indicator which uses simple functions and also simple custom functions I didnt add names for the custom formulas I just name then A, B,C etc. Does that help?

     

    ExtFml("SMG.OpenCSVFile","C:\DataSet\","A") OR ExtFml("SMG.OpenCSVUpdateFile","C:\SMGDataSet\","A") OR
    ExtFml("SMG.GetData","A",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","B",Fml("OBV True Range Short-Term")) OR
    ExtFml("SMG.GetData","C",Fml("OBV True Range Intermediate-Term")) OR
    ExtFml("SMG.GetData","D",Fml("OBV True Range Long-Term")) OR
    ExtFml("SMG.GetData","E",FmlVar("NYSE (Adv-Dec) Vol Difference","X")) OR
    ExtFml("SMG.GetData","F",FmlVar("Nasdq (Adv-Dec) Vol Difference","X")) OR
    ExtFml("SMG.GetData","G",FmlVar("Amex (Adv-Dec) Vol Difference","X")) OR
    ExtFml("SMG.GetData","H",Fml("Inverse Fisher Reform NYSE Adv/Dec Issues")) OR
    ExtFml("SMG.GetData","I",Fml("Q Indicator")) OR
    ExtFml("SMG.GetData","J",Fml("NYSE Adv-Dec Vol Difference Y Value")) OR
    ExtFml("SMG.GetData","K",Fml("B Indicator")) OR
    ExtFml("SMG.GetData","L",Fml("TSI Fast")) OR
    ExtFml("SMG.GetData","M",Fml("TSI Fast Ma")) OR
    ExtFml("SMG.GetData","N",Fml("ADX")) OR
    ExtFml("SMG.GetData","O",Fml("Instantaneous Trend Line")) OR
    ExtFml("SMG.GetData","P",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","Q",Fml("ADX 1min MA")) OR
    ExtFml("SMG.GetData","R",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","S",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","T",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","U",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","V",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","W",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","X",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","Y",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","Z",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","A1",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","B1",Fml("Custom Formula")) OR
    ExtFml("SMG.GetData","C1",Fml("ADX 5min MA")) OR
    ExtFml("SMG.GetData","E1",Fml("ADX (metastock internal)")) OR
    ExtFml("SMG.WriteData") OR
    ExtFml("SMG.CloseCSVFile") OR
    ExtFml("SMG.CloseCSVUpdateFile")

  •  05-26-2009, 22:18 29791 in reply to 29790

    Re: Formula too complex (temp array overflow)

    What is the purpose of these external functions? i.e. what do they do?  And what are the parameters for?

    ExtFml("SMG.OpenCSVFile","C:\DataSet\","A")

    ExtFml("SMG.OpenCSVUpdateFile","C:\SMGDataSet\","A")

    ExtFml("SMG.GetData","A",Fml("Custom Formula"))

    ExtFml("SMG.CloseCSVFile")

    ExtFml("SMG.CloseCSVUpdateFile")



    MS uses a fully evaluated scripting language, which mean every operation in your expression is going to be evaluated and a data array value is going to be returned.


    wabbit Big Smile [:D]


    "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."
    Edsgar W. Dijkstra

     

    MS: 6.52 EOD, 7.x EOD, 8.0 PRO, 9.2 PRO w/QC, 10 PRO w/QC C, 11 PRO w/QC & MDK
    For custom MetaStock programming : http://www.wabbit.com.au
    My SkyPE status :
    My SkyPE account : wabbit.com.au

  •  05-27-2009, 15:44 29798 in reply to 29791

    Re: Formula too complex (temp array overflow)

    ExtFml("SMG.OpenCSVFile","C:\DataSet\","A") OR   ----------------> create csv file Entire Only 
    ExtFml("SMG.OpenCSVUpdateFile","C:\DataSet\","A") OR -------> create csv Update Only (NEW function)
    ExtFml( "SMG.GetData","A", Mov(C,1,S)) OR --------------------------------> get data from Metastock with your own indicator (e.g. mov(c,1,s))
    ExtFml("SMG.WriteData") OR -----------------------------------------------------> write data to csv file (both Entire and Update)
    ExtFml("SMG.CloseCSVFile") OR -------------------------------------------------> close csv file Entire Only
    ExtFml("SMG.CloseCSVUpdateFile") --------------------------------------------> close csv file Update Only (NEW function)
  •  05-27-2009, 16:12 29799 in reply to 29798

    Re: Formula too complex (temp array overflow)

    As the entire expression is going to be evaluated; what happens if you remove the ORs?

    Another idea is to pass more than one array to the function, reducing the number of calls made; you can use up to nine arguments per call.


    wabbit Big Smile [:D]

    "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."
    Edsgar W. Dijkstra

     

    MS: 6.52 EOD, 7.x EOD, 8.0 PRO, 9.2 PRO w/QC, 10 PRO w/QC C, 11 PRO w/QC & MDK
    For custom MetaStock programming : http://www.wabbit.com.au
    My SkyPE status :
    My SkyPE account : wabbit.com.au

  •  05-27-2009, 17:26 29801 in reply to 29799

    Re: Formula too complex (temp array overflow)

    Wabbit - k I will try both.

    Can you give me an example with "pass more than one array to the function, reducing the number of calls made; you can use up to nine arguments per call".

     

    Thanks for your help really appreciate it

  •  05-27-2009, 17:36 29802 in reply to 29801

    Re: Formula too complex (temp array overflow)

    e.g. sending a swag of MAs

    extfml("myLibrary.myFunction",
    mov(c,3,s),
    mov(c,5,s),
    mov(c,8,s),
    mov(c,13,s),
    mov(c,21,s),
    mov(c,34,s),
    mov(c,55,s),
    mov(c,89,s),
    mov(c,144,s)
    );



    wabbit Big Smile [:D]


    "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."
    Edsgar W. Dijkstra

     

    MS: 6.52 EOD, 7.x EOD, 8.0 PRO, 9.2 PRO w/QC, 10 PRO w/QC C, 11 PRO w/QC & MDK
    For custom MetaStock programming : http://www.wabbit.com.au
    My SkyPE status :
    My SkyPE account : wabbit.com.au

  •  05-27-2009, 22:55 29805 in reply to 29802

    Re: Formula too complex (temp array overflow)

    Thanks for your help,

    K played around with things and my brian is a little off right now, but now MS is giving me this error?

    "Internal compiler error. Insufficient memory for binary" what the heck does that mean? Is there a way to correct it?

  •  05-28-2009, 18:41 29814 in reply to 29805

    Re: Formula too complex (temp array overflow)

    ICE could be caused by a few things :-

    If its your IDE generating the alert through incomplete/erroneous compilation of the DLL, then shutdown the IDE, restart and recompile the DLL

    If it is just MS internal functions, then you sometimes see this error if your expression is too complex (without generating the complex error alert!) such as too many nested If() statements or something like that.


    The causes of ICE are hard to pinpoint... they are hard to reproduce.  Change one small component of your code and see what the effect is, or, rewrite the functionality by starting simply and adding to it until you error out.



    wabbit Big Smile [:D]


    "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."
    Edsgar W. Dijkstra

     

    MS: 6.52 EOD, 7.x EOD, 8.0 PRO, 9.2 PRO w/QC, 10 PRO w/QC C, 11 PRO w/QC & MDK
    For custom MetaStock programming : http://www.wabbit.com.au
    My SkyPE status :
    My SkyPE account : wabbit.com.au

  •  05-28-2009, 18:47 29815 in reply to 29805

    Re: Formula too complex (temp array overflow)

    Ok question. Is there a way to break the main MS Indicator (which contains 30 Outputs) into two separate MS Indicators which will contain 15 Outputs and then reference both those Indicators within one new Indicator?

     

    So it would look something like this (see below)? I Know I have really butched the code but I was thinking more about the idea. The reason is if I break it apart I think it might compensate for the following MS errors I keep getting.

     

    "Formula too complex (temp array overflow)" matrix overflow issue or an “Internal Compiler error. Insufficient memory for binary”.

     

     

    Original Indicator:

     

    ExtFml("A.OpenCSVFile","C\:Dataset\","A");

    ExtFml("A.OpenCSVUpdateFile","C\:Dataset\","A");

    ExtFml("A.GetData","A",Fml("1"));

    ExtFml("A.GetData","B",Fml("2"));

    ExtFml("A.GetData","C",Fml("3"));

    ExtFml("A.GetData","D",Fml("4"));

    ExtFml("A.GetData","E",FmlVar("5","X"));

    ExtFml("A.GetData","F",FmlVar("6","X"));

    ExtFml("A.GetData","G",FmlVar("7","X"));

    ExtFml("A.GetData","H",Fml("8"));

    ExtFml("A.GetData","I",Fml("9"));

    ExtFml("A.GetData","J",Fml("10"));

    ExtFml("A.GetData","K",Fml("11"));

    ExtFml("A.GetData","L",Fml("12"));

    ExtFml("A.GetData","M",Fml("13"));

    ExtFml("A.GetData","N",Fml("14"));

    ExtFml("A.GetData","O",Fml("15"));

    ExtFml("A.GetData","P",Fml("16"));

    ExtFml("A.GetData","Q",Fml("17"));

    ExtFml("A.GetData","R",Fml("18"));

    ExtFml("A.GetData","S",Fml("19"));

    ExtFml("A.GetData","T",Fml("20"));

    ExtFml("A.GetData","U",Fml("21"));

    ExtFml("A.GetData","V",Fml("22"));

    ExtFml("A.GetData","W",Fml("23"));

    ExtFml("A.GetData","X",Fml("24"));

    ExtFml("A.GetData","Y",Fml("25"));

    ExtFml("A.GetData","Z",Fml("26"));

    ExtFml("A.GetData","A1",Fml("27"));

    ExtFml("A.GetData","B1",Fml("28"));

    ExtFml("A.GetData","C1",Fml("29"));

    ExtFml("A.GetData","E1",Fml("30"));

    ExtFml("A.WriteData");

    ExtFml("A.CloseCSVFile");

    ExtFml("A.CloseCSVUpdateFile")

     

     

    New Indicator 1

     

    ExtFml("A.OpenCSVFile","C\:Dataset\","A");

    ExtFml("A.OpenCSVUpdateFile","C\:Dataset\","A");

    ExtFml("A.GetData","A",Fml("1"));

    ExtFml("A.GetData","B",Fml("2"));

    ExtFml("A.GetData","C",Fml("3"));

    ExtFml("A.GetData","D",Fml("4"));

    ExtFml("A.GetData","E",FmlVar("5","X"));

    ExtFml("A.GetData","F",FmlVar("6","X"));

    ExtFml("A.GetData","G",FmlVar("7","X"));

    ExtFml("A.GetData","H",Fml("8"));

    ExtFml("A.GetData","I",Fml("9"));

    ExtFml("A.GetData","J",Fml("10"));

    ExtFml("A.GetData","K",Fml("11"));

    ExtFml("A.GetData","L",Fml("12"));

    ExtFml("A.GetData","M",Fml("13"));

    ExtFml("A.GetData","N",Fml("14"));

    ExtFml("A.GetData","O",Fml("15"));

    ExtFml("A.WriteData");

    ExtFml("A.CloseCSVFile");

    ExtFml("A.CloseCSVUpdateFile")

     

     

     

    New Indicator 2

     

    ExtFml("A.OpenCSVFile","C\:Dataset\","A");

    ExtFml("A.OpenCSVUpdateFile","C\:Dataset\","A");

    ExtFml("A.GetData","P",Fml("16"));

    ExtFml("A.GetData","Q",Fml("17"));

    ExtFml("A.GetData","R",Fml("18"));

    ExtFml("A.GetData","S",Fml("19"));

    ExtFml("A.GetData","T",Fml("20"));

    ExtFml("A.GetData","U",Fml("21"));

    ExtFml("A.GetData","V",Fml("22"));

    ExtFml("A.GetData","W",Fml("23"));

    ExtFml("A.GetData","X",Fml("24"));

    ExtFml("A.GetData","Y",Fml("25"));

    ExtFml("A.GetData","Z",Fml("26"));

    ExtFml("A.GetData","A1",Fml("27"));

    ExtFml("A.GetData","B1",Fml("28"));

    ExtFml("A.GetData","C1",Fml("29"));

    ExtFml("A.GetData","E1",Fml("30"));

    ExtFml("A.WriteData");

    ExtFml("A.CloseCSVFile");

    ExtFml("A.CloseCSVUpdateFile")

     

     

    Combined New Indicator 1 and 2

     

    ExtFml("A.OpenCSVFile","C\:Dataset\","A");

    ExtFml("A.OpenCSVUpdateFile","C\:Dataset\","A");

    Ref (“New Indicator 1”);

    Ref(“New Indicator 2”);

    ExtFml("A.WriteData");

    ExtFml("A.CloseCSVFile");

    ExtFml("A.CloseCSVUpdateFile")

     

  •  05-29-2009, 19:11 29825 in reply to 29815

    Re: Formula too complex (temp array overflow)

    I don't know the specifics about how your external functions work, but it seems to me that you have a problem with what you are trying to do with respect to the MS code.

    Your combined indicator opens the files (and I will assume, leaves them open for writing?), then you call the function "New Indicator 1" which also opens the data file for writing?  Is this the same file as is already open?  Then you try to do it again with "New Indicator 2".  At the end of "New Indicator 1" you are closing the files?  So it is possible NI2 needs to reopen the files... all too confusing.

    Without thinking too deeply about this; if I was writing this functionality, I would have one external function which might take a number of arguments:

    destination file/folder
    write all data / write only last value
    append / overwrite data
    data array

    Each time the function is called, the file is checked to exist or created if it doesn't exist, then opened for writing.  Depending on the required format of the data file, if appending the data, seek the EOF and append data there, or, for each exisiting line of data in the file, append the data to the end of that line (you'll need to synchronise the date/time to ensure everything lines up in time).  Once the data is written, close the file.

    (I might have to do some fine tuning, but I think you're doing something the hard way when the solution should be quite simple.)

    Using global handles to point to the open file, then creating dynamic sized arrays to store data until written is an efficient way to go about the problem, but I don't think you have accomplished what you think you need.

    Have a look at your pseudo-code/flowchart again and ask yourself it makes sense.  Think about the interfacing, file accesses and the memory usage.  I am sure you will find a better way to achieve the goal.


    Hope this helps.

    wabbit Big Smile [:D]


    "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."
    Edsgar W. Dijkstra

     

    MS: 6.52 EOD, 7.x EOD, 8.0 PRO, 9.2 PRO w/QC, 10 PRO w/QC C, 11 PRO w/QC & MDK
    For custom MetaStock programming : http://www.wabbit.com.au
    My SkyPE status :
    My SkyPE account : wabbit.com.au

View as RSS news feed in XML