Oh, well then I've completely misunderstood where the problem is.

Look you posted this:

f1[a_] := a[[1]] (1 - a[[2]]) + a[[2]] - a[[2]]^2;
f2[a_] := a[[1]] - 0.5 a[[1]]^2 + 2 (1 - a[[1]]) a[[2]];
av0 = {1., 1.};
varex = Table[av[i][t], {i, 2}];
eqnlist = {Derivative[1][av[1]][t] == f1[varex], 
   Derivative[1][av[2]][t] == f2[varex], av[1][0] == av0[[1]], 
   av[2][0] == av0[[2]]};
solDEsuper = NDSolve[eqnlist, varex, {t, 0, 1.4}]

and said that it worked.

I posted this:

avInit = {1, 1};
avFns = Array[av, 2];
avFnsApplied = Through[avFns[t]]; 

avProtoList[a_List] := 
  0.5  (1 - a[[2]])  a[[1]]^2 + (1 - a[[1]])  a[[2]]^2 + 
   a[[1]]   a[[2]];
avProto[args___] := avProtoList[{args}];

g1 = Derivative[1, 0][avProto];
g2 = Derivative[0, 1][avProto];

eqnlist = {Derivative[1][av[1]][t] == g1 @@ avFnsApplied, 
   Derivative[1][av[2]][t] == g2 @@ avFnsApplied, 
   av[1][0] == avInit[[1]], av[2][0] == avInit[[2]]};
solDEsuper = NDSolve[eqnlist, avFnsApplied, {t, 0, 1.4}]

and you said that it didn't work.

When I look at eqnlist for each of them, I can't tell the difference. I thought if we used a simpler function, I could see the difference by inspection. But you don't want to provide a simpler function. Fine, just tell me what's different about these two. I'm sorry that I'm just not picking up on it, but that's where we are.

But then later, I forced you to tell me whether that expression was correct or not, and you said that it wasn't. So, I thought the problem was in generating eqnlist, and that maybe you could indicate the difference by giving me a reference. But now you seem to be saying that you don't know what eqnlist should even look like, you just know it's wrong. Okay, that's fair, but now we're talking about a mathematics problem, not a programming problem. So, you'll have to lead me through the mathematics before I can provide the code.

Now, you probably think I'm being stupid, and you'll say "no, no, that's not what I said/meant", and that's totally fine. Communication is hard. But I, as of right now, have no idea what/where the problem is. I know that for you, the problem is something about whether arguments are passed as a sequence or as a list. To me, that's a trivial problem, and I'm extremely confident that I can solve it. But I've been failing to produce a good result even though it seems to match the good results you've previously given me. It sounds like you're getting "good answers" with sequences, so I'm just trying to pin down what a good answer is.

So, at the end of the day, it sounds to me like we'd like to feed some inputs into NDSolve. We've got to generate those inputs somehow. You seem to be able to recognize when those inputs are good or not. I don't seem to be able to figure that part out. Also, I'm not really sure where we're starting here. It sounds like there are things "before" we start defining f, f1, etc. I know in your real world problem, those things are big and complicated, but in our toy problem, I would think you could give them to me explicitly. So, my brain is saying, "if he gives me the uber inputs, A, and then the inputs to NDSolve, B, then I can figure out how to transform A into B".

If you can tell me where I'm going wrong in any of the above thinking, that would help greatly. If you can give me any concrete examples that I can compare to at any point in the computation, that would help greatly. If you can try different words to explain your intent, that might help.

I think I solved it! So flattening the lists before passing to ffoutsuper, and reconstructing them should have worked, because that is what I had before, no lists. But I also had the arguments passed to ffoutsuper be _Real. So once I did that, it works again. I don't really know why. Somehow the conditionals work now. It would have been nice not to have to flatten the lists and reconstruct them. Something for the developers to figure out.

I tried your ArrayReshape. It makes no difference either to the timing for Derivative (way too slow) or the bug with the Re'[1].

I'm a bit surprised that it didn't make a difference on timing, but I didn't intend it to fix anything. It's just more wolfram-idiomatic style.

There is something very wrong with Derivative, and I want you to admit it, instead of putting it on me to come up with something simpler (are you working for Wolfram?).

I'm not a Wolfram employee. I don't have to admit anything. Finding the simplest demonstration of something is a tried-and-true debugging technique, but if you don't want to try it, that's fine by me. I'm not invested in any of this--it's your baby.

If I understood what you're trying to achieve mathematically, maybe I could give you suggestions, but I don't, so I can't. But as a Mathematica programming problem, you might be beating your head against a wall. I'm not sure that we can expect Mathematica to figure out the derivative that you want from the form that you're providing to it.

Not that it really matters to the underlying problem, but you can avoid using temporary variables:

ffoutsuper[args__] := Max[Re[Eigenvalues[Dot @@ ArrayReshape[Drop[{args}, 10], {2, 4, 4}]]]]

I really don't think I can help you if you can't provide simpler examples. I mean, just forget your actual domain completely, and just show an example of dynamically generating a deferred derivative (or whatever we're trying to do here). Show an example where you know the right answer and show me the right answer. The stuff you give me is too far along, and I don't know where to start looking, and so I end up spending a lot of time reverse engineering your code. I'm not willing to do that any longer.

Some general advice: start small and take baby steps. Don't get this far into your coding before you try to fix it. Start with the absolute simplest representation of your problem (or a minimal alternative to your problem) and just do one computation/transformation and make sure it works. Then build on that foundation with one more dead simple computation/transformation. You have so much crazy code here that I don't know if you've made a straight up semantic error or if you're just mis-using a Mathematica function or if we just need to change the order of evaluation. There may be a simple fix, or this whole thing might actually be impossible. I have no idea where we are on that spectrum.

Getting close, but still no cigar. I'm now able to give ffoutsuper a sequence, and resonstruct lists inside it (I only did it for 2 of the 2dimensional lists to keep it simple). But now I'm still getting superslow evaluation, and the Re'[number] bug. Any ideas?

So, back to trying without lists in the args to ffoutsuper, since we Derivative has problems with lists in my real case, either being too slow, or trying to do nonsense like differentiaing the real part of a variable. This should not be complicated, but I don't know how to do it. I want to reconstruct the lists inside ffoutsuper. ffoutsuper gets a flat sequence, so the information of the list stuctures is lost to it, but not in my head. For each simulation I will have a constant list structure, though these will vary from simulation to simulation, and for now I don't mind rewriting code for different simulations.

So I know where in the flat structure to start and where to end, for reconstructing all the lists, no matter what shape they have. I just need a way to refer to the nth argument of the flat sequence in the args of ffoutsuper. Can you show me how to do that? Do I need to define a pure function and use Slot? Here is a failed attempt to reconstruct one of the lists for a simple example Wolfram Notebook

Sorry, I am not familiar with associations

Sorry for the distraction then. Associations aren't inherent to the problem you're trying to solve. I was just trying to demonstrate how to transform between representations.

I'll try to refactor your code later. I only have a couple of minutes right now. So for now, I'll just try to explain why it failed.

Here's your definition for fflat1 (I'm removing the comments for clarity)

fflat1[Sequence @@ Flatten[{flag_, LRc_, cL_}]] := cL[[1]]

If you look at just the argument list you created, it looks like this.

Sequence @@ Flatten[{flag_, LRc_, cL_}]
(* Sequence[flag_, LRc_, cL_] *)

So, your function expects exactly three arguments. But when you tried to apply it:

fflat1[Sequence @@ Flatten[{1, 0.1, cout0L}]]

you actually provided 6 arguments:

Sequence @@ Flatten[{1, 0.1, cout0L}]
(* Sequence[1, 0.1, 4., 4., 4., 4.] *)

When you applied it the other way:

fflat1[1, 0.1, cout0L]

you did provide exactly three arguments:

Sequence[1, 0.1, cout0L]
(* Sequence[1, 0.1, {4., 4., 4., 4.}] *)

but the last argument was itself a list. So, with the argument constraint satisfied, the function was evaluated to be the first item in the third argument, which is 4..

No need to pay.

Ooooooooohhhhhhh. Good grief. I finally understand. It's just so much easier to look at something concrete. Not that it matters now, but maybe you can appreciate my confusion if I explain what I thought you meant. You said

I need to tell flist that the first argument is a scalar called flag, the second also a scalar called RLc,

Now, since you already had arguments named flag and RLc, I thought your problem was telling flist that they were scalars. And so I gave you several ways to apply the constraint that an argument is a scalar. Then you said

the third through 6th correspond to a 1D list of dimension 4 called cL, etc.

which sounded to me like each of these arguments was a list of length 4, not that you wanted to re-interpret the 3rd through 6th arguments as members of a new list. Given your language of "tell flist that..." I wasn't understanding that you were talking about the implementation details instead of the interface. Telling a function about its arguments is what it means to give a function a signature. I don't think I ever, ever would have understood this to mean "flist's implementation should construct a list from arguments 3-6".

Okay, with that out of the way, I'll send some suggestions.

I need to tell flist that the first argument is a scalar called flag, the second also a scalar called RLc, the third through 6th correspond to a 1D list of dimension 4 called cL, etc.

If you mean you need to set these conditions as constraints on the arguments, then you could try any of these (I'm going to simplify your example, we don't need eleven arguments to demonstrate this):

myFuncTakesScalars[arg1_Real, arg2_Integer, arg3_?NumericQ, arg4_?AtomQ] := "constraints satisfied";
myFuncTakesScalars[1., 2, Pi, "a"]
(* "constraints satisfied" *)

myFuncTakesLists[arg1 : {_, _, _, _}, arg2_?ArrayQ, arg3_?MatrixQ] := "constraints satisfied";
myFuncTakesLists[{1, 2, 3, 4}, RandomReal[{0, 1}, 100], RandomInteger[{1, 10}, {20, 30}]]
(* "constraints satisfied" *)

myFuncTakesListsWithFurtherConstraints[arg1 : {_, _, _, _}, arg2_?ArrayQ, arg3_?MatrixQ] := "constraints satisfied" /; 4 == Length[arg1] && 2 == ArrayDepth[arg3];
myFuncTakesListsWithFurtherConstraints[{1, 2, 3, 4}, RandomReal[{0, 1}, 100], RandomInteger[{1, 10}, {20, 30}]]
(* "constraints satisfied" *)

I need to tell flist that the first argument is a scalar called flag, the second also a scalar called RLc, the third through 6th correspond to a 1D list of dimension 4 called cL, etc.

If the argument list always has this same structure, you can just refer to the pattern names you've already used:

fList[{flag_, LRc_, cL_, cR_, alphaL_, alphaR_, LRC_, CL_, CR_, AlphaL_, AlphaR_}] := 
 StringForm["``-``-``-``-``-``-``-``-``-``-``", LRc, CR, AlphaL, flag, cL, alphaL, cR, alphaR, CL, AlphaR, LRC];

fList[Range[11]] // ToString
(* "2-9-10-1-3-5-4-6-8-11-7" *)

But if you really want to access by index, then you'll need a pattern name for the whole list:

fList[args : {flag_, LRc_, cL_, cR_, alphaL_, alphaR_, LRC_, CL_, CR_, AlphaL_, AlphaR_}] := 
 StringForm["``-``-``-``-``-``-``-``-``-``-``:::``:::(``)", LRc, CR, AlphaL, flag, cL, alphaL, cR, alphaR, CL, AlphaR, LRC, args, args[[5]]];

fList[Range[11]] // ToString
(* "2-9-10-1-3-5-4-6-8-11-7:::{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}:::(5)" *)

And if you only want to access by index, you don't need all those other pattern names:

fList[args : {_, _, _, _, _, _, _, _, _, _, _}] := args[[RandomSample[Range[11]]]];    

fList[Range[11]]
(* {3, 1, 4, 6, 8, 11, 2, 7, 10, 5, 9} *)

And if you don't actually want to restrict it to 11 arguments, but any arbitrary number:

fList[args : {___}] := RandomSample[args];

fList[Range[5]]
(* {1, 5, 4, 2, 3} *)

And also, you can index into any one particular argument:

fList[{flag_, LRc_, cL_, cR_, alphaL_, alphaR_, LRC_, CL_, CR_, 
   AlphaL_, AlphaR_}] := cL[[2]];

fList[{1, 2, {100, 101, 102, 103}, 4, 5, 6, 7, 8, 9, 10, 11}]
(* 101 *)

Yep, pretty sure we can take lists and transform them to sequences for Derivative.

OK, should be able to work on it tonight and tomorrow. I will let you know if it works, and if you're amenable I still want to give you that financial reward. Thanks!

Maybe! Just to make sure I understand your suggestion, here is the test file I implemented it in. Do I need to combine this with your previous suggestions on the 2-way conversions of lists to sequences, so I can pass varex (there will be several of these) to other functions from f? I fear there will be scoping problems, as varex is global and not scoped within f (I could be wrong about this). All the lists need to be updated as NDSolve chugs along.

Also, I need to figure out how to automate taking derivatives before forming eqnlist. I was using such constructs as this, when I was hoping I can pass f lists directly, to differentiate f for forming eqnlist:

nderaR[i1_, i2_, m_, 
   f_] := (Derivative @@ {0, 0, ConstantArray[0, m], 
      ConstantArray[0, m], ConstantArray[0, {m, m}], 
      Normal[SparseArray[{{i1, i2} -> 1}, {m, m}]], 0, 
      ConstantArray[0, m], ConstantArray[0, m], 
      ConstantArray[0, {m, m}], ConstantArray[0, {m, m}]})[f];

one for each list

Okay, I think I see what you're saying now. I'll try to take a stab at it later. But in the meantime, here's the simplest thing I can think of to satisfy your suggestion to

take a list, turn it to a sequence before passing to f, then reconstruct it inside f, so that I can pass lists to other functions that I call from within f.

In fact, I'll show both "directions":

functionList[list_List] := dependencySeq @@ list;
functionList[{1, 2, 3}]
(* dependencySeq[1, 2, 3] *)

functionSeq[args___] := dependencyList[{args}];
functionSeq[1, 2, 3]
(* dependencyList[{1, 2, 3}] *)

inputList = {a, b, c};
functionSeq @@ inputList
(* dependencyList[{a, b, c}] *)

My intuition is that this is too simplistic to work, but without going back to look at eqnlist or ffoutsuper right now, that's the simplest thing that satisfies my best attempt to interpret your words here.

Eric, I'm thinking you might have missed one of my posts, please go back through them all (I should too, to make sure they all posted successfully). I've done my utmost best to simplify the problem and tell you why your solution doesn't work, and tell you exactly what is needed. Your comment sounded sarcastic because if I knew how to "manually change the thing that doesn't work to the thing that does work and show me that" then the problem would be solved.

I'm very serious. I opened the notebooks, saw how large they were, and closed them. I'm not spending my day reverse-engineering all of that code.

You don't need to find a simpler function (although, I don't know why that's such a big deal), you just need to tell me what the "answer" is. Something I can use as a reference to check against. I'm not going to try to figure out the mathematics and I'd rather not reverse engineer a ton of code. I'm pretty sure there is a simple code issue here, but I'm tired of guessing. So, just show me exactly what a correct result would look like so that we don't just keep wasting time.

Look, you're the one with a problem to solve. I'm just trying to help out. My only investment here is just being curious about solving the programming problem. If it's not worth it to you to give me something explicit I can use as a test case, then it's not worth it. I can just move on to the rest of my life.

so it it NOT what is needed.

Excellent! That's progress. So, what is needed?

Clearly we're having a communication issue. You think you've provided me with enough info, but I don't understand. Why can't you literally just manually change the thing that doesn't work to the thing that does work and show me that? If you do that, I can almost guarantee that I can show you how to get there.

You said

The first shows that as long as there are no lists in the arguments to f, NDSolve is happy and so am I.

and

You can see from these 2 files that f has no hope of being evaluated symbolically (maybe there is a clever way to do it using certain symmetries, but I will have cases in the future where these symmetries are not present that might make f be evaluatable symbolically)

I know you think you're being helpful, but this is just a distraction. I need to know the actual things you want to pass in to NDSolve. Just forget about the list versus non-list arguments for now. I just need to have a target to shoot for. These informal descriptions are just not helping me.

I'm like 95% sure this is doable in a very easy way, but so far I've failed to hit your target, and I don't know what my errors are. Don't tell me in words like "evaluate f symbolically". Just give me a concrete target that I can shoot for.

I am willing to pay you $200 to solve this problem. OK, so we need two things: 1. Arguments to f have to be a bunch of lists (and some non-lists too) 2. eqnlist that is given to NDSolve should not attempt to evaluate f (or its non-time derivatives) but should be in symbolic, unevaluated Derivatives form. The reason is that f (and hence its derivatives) is not possible to evaluate symbolically, only numerically.

I don't know how to come up with a simple f that can't be evaluated symbolically (like it is in all the examples we have produced so far). I can send you two files that have the real f (called ffoutsuper in there, you will have to scroll down to see its definition which involves eigenvalues of a high rank matrix and calls to another ODE solver, and then scroll down even further to see where NDSolve is called): The first shows that as long as there are no lists in the arguments to f, NDSolve is happy and so am I.

The second doesn't work because there are lists in the args to f. You can see from these 2 files that f has no hope of being evaluated symbolically (maybe there is a clever way to do it using certain symmetries, but I will have cases in the future where these symmetries are not present that might make f be evaluatable symbolically)

Attachments:

nestedlevels4.3.nb

nestedlevels4.5.nb

So close! It works, but unfortunately eqnlist has evaluated the derivative of f symbolically. Recall that in the actual case this cannot be done, since f can only be evaluated numerically. If I recall, what happens if I don't use the "Real" hack (which gives me a "non-numerical derivative at t=0 when I have lists, but works without lists), which is also what will happen if I use your solution in the actual case, is that it keeps trying to evaluate Eqnlist symbolically, never even getting to evaluate NDSolve and just doesn't give me output.

Ugh. Got wrapped around the axle in my translation. What about this?

eqnlist = {
  Derivative[1][av[1]][t] == g1 @@ avFnsApplied,
  Derivative[1][av[2]][t] == g2 @@ avFnsApplied,
  av[1][0] == avInit[[1]],
  av[2][0] == avInit[[2]]}

The only reason I want at least one of the elements of the list of arguments passed to f to be real, is a hack. I want to pass NDSolve a list of symbolic equations. Since f in my actual problem can't be symbolically evaluated, the Real qualifier prevents evaluation when the derivatives are taken (the ones with respect to the arguments of f, not the ones with respect to t), to produce acceptable (to NDSolve) symbolic equations. Once NDSolve gets going, it replaces the arguments by real numbers, and then f can be evaluated. This can be seen to work in the fourth cell, which does not have a list of arguments passed to f. But I need to have a bunch of lists passed to f in my real (meaning actual, not as in real vs complex vs symbolic) example.

Perhaps this could be done with Hold and Release as well (instead of the Real hack) but I have been so far unsuccessful in achieving that.

In your third cell, why don't you just do the following?

f[a : {_Real, _}] := 0.5 (1 - a[[2]]) a[[1]]^2 + (1 - a[[1]]) a[[2]]^2 + a[[1]] a[[2]]

I'm not at all sure what you're trying to do, but here are some thoughts to get you started:

f1[args__] := {args}[[1]] + {args}[[2]];
f1[1, 2]
(* 3 *)

f2 = #1 + #2 &;
f2[1, 2]
(* 3 *)

Or, you can just overload your function:

f3[list : {_, _}] := list[[1]] + list[[2]];
f3[a1_, a2_] := f3[{a1, a2}];
f3[1, 2]
(* 3 *)

Or you can change the way you apply your function:

f4[a1_, a2_] := a1 + a2;
f4 @@ {1, 2}
(* 3 *)

But I don't understand why you think using Table and Array is creating some sort of impedance, so I don't have confidence in any of these approaches.