Group Abstract Group Abstract

Message Boards Message Boards

Speed up numerical solution of differential equation using BSplines?

Posted 9 years ago

Dear Wolfarm community, I am very new with Mathematica. I've written a code which works pretty well, but the big problem is that it is so slow that I cannot use it. I usually use Matlab for this type of coding. The equivalent Matlab version is thousand times faster. Here in the following I report a dummy (very much simplified) version of code that contains the main operations I need to perform. Maybe I make some basic mistakes in programming with Mathematica that dramatically slow down the execution. When the parameters n and m increase, the code becomes unusable. Note also that I tried to replace all the functions defined in the code with compiled functions with command Compile. I expected to see big improvements but unfortunately it was not so. Is there a more efficient way to write this code? Thank you very much! Enzo

Clear["Global`*"]

(*Differential equation*)

Eq = D1w[1, 1] + D2w[1, 1, 2]

D1w[1, 1] + D2w[1, 1, 2]

(*Define BSpline parameters*)

p = 2;
q = 2;
n = 5;
m = 5;
r = n + p + 1;
s = m + q + 1;
rint = r + 1 - 2 (p + 1);
sint = s + 1 - 2 (p + 1);
Uint = Range[0, 1, 1/(rint + 1)]; 
Vint = Range[0, 1, 1/(sint + 1)];
U = N[Join[ConstantArray[0, p], Uint , ConstantArray[1, p]]];
V = N[Join[ConstantArray[0, q], Vint , ConstantArray[1, q]]];
Uc = Table[(Sum[U[[h + k]], {k, 1, p}])/p, {h, 1, n + 1}];
Vc = Table[(Sum[V[[h + k]], {k, 1, q}])/q, {h, 1, n + 1}];

(*Define BSpline approximations (Discretization) *)

D1w11[ic_, jc_, i_, j_] := 
  D[BSplineBasis[{p, U}, i - 1, u], {u, 1}] BSplineBasis[{q, V}, j - 1, v] W1[
     i, j] //. {u -> Uc[[ic]], v -> Vc[[jc]]};
D2w112[ic_, jc_, i_, j_] := 
  D[BSplineBasis[{p, U}, i - 1, u], {u, 1}] D[
     BSplineBasis[{q, V}, j - 1, v], {v, 1}] W1[i, j] //. {u -> Uc[[ic]], 
    v -> Vc[[jc]]};

(*Substitute the above approximate derivatives into the original differential equation*)

subw11[ic_, jc_, i_, j_] := {D1w[1, 1] :> D1w11[ic, jc, i, j]};
subw112[ic_, jc_, i_, j_] := {D2w[1, 1, 2] :> D2w112[ic, jc, i, j]};
sub[ic_, jc_, i_, j_] := Join[subw11[ic, jc, i, j], subw112[ic, jc, i, j]];
EqDisc[ic_, jc_, i_, j_] := Eq //. sub[ic, jc, i, j];

(*Now collects terms in w1*)

KEq[ic_, jc_, i_, j_] := Coefficient[EqDisc [ic, jc, i, j], {W1[i, j]}];
KEqW1[ic_, jc_, i_, j_] := KEq[ic, jc, i, j][[1]];

(*Transform double -index formulation into single - index formulation*)

iMat = Table[Range[1, n + 1, 1], m + 1];
ivec = Flatten[ArrayReshape[iMat, {1, (n + 1) (m + 1)}]];
jMat = Transpose[Table[Range[1, m + 1, 1], n + 1]];
jvec = Flatten[ArrayReshape[jMat, {1, (n + 1) (m + 1)}]];
KKEqW1[r_, s_] := 
  KEqW1 @@ List[ivec [[r]], jvec [[r]], ivec [[s]], jvec [[s]]];

(*Finally, create a matrix*)

K = Table[KKEqW1[r, s], {r, 1, (n + 1) (m + 1)}, {s, 1, (n + 1) (m + 1)}];
POSTED BY: Enzo Marino
14 Replies
Posted 9 years ago

Hello Neil, Thank you for your reply. I agree with you, there is a bug in the calculation and evaluation of BSplineBasis derivatives. Actually, it is correct that the derivatives of BSplineBasis functions involve denominators which are 0. However, this indeterminate result 1/0 is normally replaced by 0. It is a pity that Mathematica does not do it when one uses the following syntax

D[BSplineBasis[{p, U}, i - 1, x], {x, 1}] /. {i -> 1, x -> 0.2}

The workaround you suggest is fine, but it still requires the definitions of derivatives as functions implying definig other functions calling them.... and this slowdown the code. I reformulated the entire code with the overloading operator and in the end the new version is approximately 10 times faster than the initial one. However, it is still much slower than the equivalent matlab version and I am afraid I cannot use it for further developments.

I think I have tried all I could. I don't think that I have other chances I am afraid I will return to Matlab. I will possibly use Mathematica only for small n and m to make comparisons and facilitate the debugging process.

Thank you for all your help!

Enzo

POSTED BY: Enzo Marino

Enzo,

I think you found a bug in Mathematica's BSplineBasis. It should be putting a conditional on the terms of the derivative. I would report this at Wolfram Tech Support or better yet, you should call them and see if they can suggest a good workaround until it is fixed. I added a few lines to your file showing the problematic expression. The derivative has two terms and the first term will go to zero in the denominator when n->0. The second term has the answer that agrees with your answer when you substitute in zero for n and then take the derivative. Note that the derivative expression also goes to infinity when n > 5. In this case the second term of the derivative has zero in its denominator.

As for a workaround:

D[BSplineBasis[{d, U}, n, x], {x,1}] appears to work properly if the n value is numeric before you take the derivative. I suggest you write your code so that you make sure it happens that way. One possibility is to use operator overloading to compute all the derivative expressions as functions of x and then evaluate them.

If you define:

bSplineDeriv[n_] := 
 bSplineDeriv[n] = D[BSplineBasis[{p, U}, n, x], {x, 1}]

As you iterate through your values of n, the symbolic expressions for the derivative will be computed. The nice part of this definition is that it will only compute the derivative for each n value once. See the enclosed nb file.

I hope this helps. Maybe someone else has another workaround suggestion...

Attachments:
POSTED BY: Neil Singer
Posted 9 years ago

Thank you Neil, I think now I am very close to solve the problem.

Yes, as you siuggested, I do not define any function. I always use = instead of :=. Only in the end, when I need to iterate, I evaluate the symbolic expressions with numbers.

There is only one (I hope minor!) thing that is still not working... very basic issue I guess: can you tell me why I get the error 1/0 indeterminate when I try to evaluate the first derivative of a BsplineBasis? Please see the following short code:

(* Input data *)
p = 2;
n = 5;
r = n + p + 1;
rint = r + 1 - 2 (p + 1);
Uint = Range[0, 1, 1/(rint + 1)]; 
U = N[Join[ConstantArray[0, p], Uint , ConstantArray[1, p]]];

what I do not understand is why the BSplineBasis can be evaluated easily in the following way:

BSplineBasis[{p, U}, i - 1, x] /. {i -> 1, x -> 0.2}

whereas its first derivative CANNOT:

D[BSplineBasis[{p, U}, i - 1, x], {x, 1}] /. {i -> 1, x -> 0.2}

In the example I have chosen to compute the first derivative of the 0th Bspline basis and evaluate it at x=0.2. If you try you'll get "Infinite expression 1/0. encountered. But, as shown in the plot (see file attached), this derivative must DOES exist!

Plot[Evaluate[D[BSplineBasis[{p, U}, 0, x], {x, 1}]], {x, 0, 1}]

Could you please help me on this? After that I think I have done! Many thanks! Enzo

Attachments:
POSTED BY: Enzo Marino

Enzo,

Next, try to do the procedure exactly as you would do in Matlab but using Mathematica to handle the equations. Enter the PDE equations, fit splines, take derivatives and construct large matrices -- Construct functions that you use for each step but don't defer the computation. For example, Don't create a formula to construct spines and use it later. Define the formula for splines and create a matrix of splines (I may be showing my lack of understanding of your algorithm here -- but you get the idea). After you have an example that works, then you can create functions that do the steps in that order without delaying the calculations until the end (which causes repetition). If you have done this in Matlab it should be relatively straightforward in Mathematica. If you post some Matlab code I might be able to suggest more details but essentially you should let Mathematica handle the large expressions and expand everything out into symbolic or numerical values early -- at the same point in the procedure that you would do so in Matlab. It is better to consume memory than to repeat your calculations so it is better to iterate and create a large matrix of expressions and then take derivatives of the matrix and later turn it into numbers than to defer the calculation and do the derivatives and expression handling while you are creating the large numerical matrix. (the later may seem more "elegant" but it will be slower).

I hope that helps.

Regards

POSTED BY: Neil Singer

Enzo,

At first glance, I think the Code that Sander posted shows methods to address the issue that I described. Note that he cleverly defines:

bspU[i_]:=bspU[i]=BSplineBasis[{p,U},i-1,u]

This is relying on operator overloading -- for example, when you call the function bspU[3] the calculation is done once so future calls of bspU[3] just return that answer instantly and do no further calculations. If you do this in my simple (contrived) posted example:

fun1[i_, var_] := 
 fun1[i, var] = Coefficient[computefun[eq, var], var, i]

The first call to define mat would take the full 2 seconds but a subsequent call would take less than a millisecond.

Referring to Sander's code:

D1w11V2[{i_,j_}]:=D[bspU[i],u]bspV[j]
D2w112V2[{i_,j_}]:=D[bspU[i],u] D[bspV[j],v]

You should also see if you can define enough of your problem to compute the derivatives once (or do the same function overloading trick). As you iterate over i, the code takes the derivative of bspU[i] repeatedly.

Lastly, Did you try the built in functions for numerically solving PDE's? NSolve and NSolveValue should be helpful since you can express the problem symbolically in Mathematica and have it generate the numerical solution.

Regards

POSTED BY: Neil Singer
Posted 9 years ago

Thank you Neil for your comments! Yes, I see the definition `bspU[i_]:=bspU[i] helps! And I agree with you: I think that a lot of time can be saved by avoiding Mathematica to compute the derivative everytime with command D. The derivative of a bspline can be defined recursively form the bspline itself. Unfortunately, I haven't found it between the Mathematica built-in functions so I will try to define it by myself hopig that it will be faster than computing it via D.

As regards Sander's version of my code, it is faster. The only thing is that I need to let Mathematica to collect terms since in the real applications the original differential equations is rather lengthy and qiute complex.

Many thanks!

Enzo

POSTED BY: Updating Name

Enzo,

I did not extensively study your approach but I think I see your problem.

Easy question first. To make a number a float just add the decimal point or a prime (above the tab key) -- instead of 1,2,3 you use 1.,2.,3. or 1.0,2.0,3.0 (because it looks better!), or 1`, 2`, 3` (The prime ties into the Precision)

When you switch from Matlab to Mathematica you need to change the way you think about solving a problem. In general, using Matlab, matrices and coefficients are computed as you proceed because it is a matrix-based language. Mathematica is primarily symbolic. The way you handled these calculations is you created functions. By creating Mathematica functions you delay the calculation of everything until you call it. Therefore many of your calculations are duplicated MANY times. You need to rework your approach a bit to make this faster. For a simple example (notebook attached):

Create a large equation

In[1]:= exp = 1000;

In[2]:= eq = (x + 3)^exp

Out[2]= (3 + x)^1000

Compute the derivative with the large equation (just to have something to compute)

In[3]:= eq2 = D[eq, x]/exp

Out[3]= (3 + x)^999

Create a function that takes the derivative -- and slow it down some more by Expanding first

In[4]:= computefun[f_, var_] := D[Expand[f], var]

Create two functions -- one that calls the compute function and one that takes the computed result and make a function

In[5]:= fun1[i_, var_] := Coefficient[computefun[eq, var], var, i]

In[6]:= fun2[i_] := Coefficient[eq2, x, i]

In[7]:= 

Slow -- it recomputes the result

In[8]:= Timing[mat = Table[fun1[i, x], {i, exp}];]

Out[8]= {2.20229, Null}

Faster -- it calculates it once using table

In[9]:= Timing[mat = Table[fun2[i], {i, exp}];]

Out[9]= {0.008603, Null}

This would even be faster -- create a matrix using CoeeficientList

In[10]:= Timing[mat = CoefficientList[eq, x];]

Out[10]= {0.002729, Null}

The upshot is that by delaying your computations until the very end, you redo some computations many times over. The lure of using Mathematica functions is that the representation is very compact but for efficiency you sometimes must evaluate the expressions earlier and handle the numbers as matrices. Your profile says you are calling Coefficient too many times so that is a place to start -- create the expression once and use CoeeficientList on the result.

Regards

Neil

Attachments:
POSTED BY: Neil Singer
Posted 9 years ago

Hello Neil,

Thank you very much for your helpful and instructive reply to my post. The example that you kindly provided helps a lot to understand the issue.
Actually, I also had the feeling that something was wrong since all the functions remain unevaluated until the very end and they are called (unnecessarily) many times, but I did not know how to face it. I will try to reformulate my code taking inspiration from the approach you suggest in your example. I will let you know.

Many thanks!

Enzo

POSTED BY: Enzo Marino
Posted 9 years ago

Hello Sander,

thank you very much for your kind reply. I will try all your suggestions (/. intead of //. , SubDivide, etc). However, to answer your main question about “what I want to calculate”, all I need to calculate is reported in the sample code that I posted. My final goal is to build the matrix K (see last line of the code). The only difference in real applications of the full code is that I need to compute many matrices K (say 25) each of them is very big (not like in the posted example). For example, if you change the parameters p, q, n and m by setting them as follows: p = q = 8 and n = m = 100 you will see the code takes ages. Making it UNUSABLE :(

I have only 6 variables which are integers: i, j, ic, jc, r, s. All the rest are real numbers. For me it would be great to treat all variables as reals. How can I tell this to Mathematica? My final result (K) must be made of reals.

Finally, I run the code in debug mode with the command // RuntimeToolsProfile at the end. See profile report in attachment. In this case I set p = q = 2 and n = m = 20. I see that it takes a lot of time in making the substitution Eq //. sub[ic, jc, i, j]. Moreover, I do not understand why it keeps calling many times functions like Coefficient[EqDisc[ic, jc, i, j], {W1[i, j]}] which should be called once for all, instead.

Many thanks,

Enzo

Attachments:
POSTED BY: Enzo Marino

Again, tell us what K is. What does it represent? I know K is a matrix, but what do the elements represent? Without knowing any background it is SUPER HARD to come up with a faster solution. Your code is also very hard to read because of all the functions that nearly don't do anything, very convoluted. I cleaned up your code a lot, which took me a lot of time to figure out what you are doing!

p=5;
q=5;
n=8;
m=8;
r=n+p+1;
s=m+q+1;
rint=r+1-2 (p+1);
sint=s+1-2 (p+1);
Uint=N@Subdivide[rint+1];
Vint=N@Subdivide[sint+1];
U=Join[ConstantArray[0.0,p],Uint,ConstantArray[1.0,p]];
V=Join[ConstantArray[0.0,q],Vint,ConstantArray[1.0,q]];
Uc=MovingAverage[U[[2;;-2]],p];
Vc=MovingAverage[V[[2;;-2]],p];

ClearAll[DoIt,D1w11V2,D2w112V2,bspU,bspV]

bspU[i_]:=bspU[i]=BSplineBasis[{p,U},i-1,u]
bspV[j_]:=bspV[j]=BSplineBasis[{q,V},j-1,v]

D1w11V2[{i_,j_}]:=D[bspU[i],u]bspV[j]
D2w112V2[{i_,j_}]:=D[bspU[i],u] D[bspV[j],v]

DoIt[rr_List,ij:{i_,j_}]:=Module[{eq},
    eq=D1w11V2[ij]+D2w112V2[ij];
    Table[eq/.{u->Uc[[r[[1]]]],v->Vc[[r[[2]]]]},{r,rr}]
]

ivec=Flatten[ConstantArray[Range[n+1],m+1]];
jvec=Flatten[Transpose[ConstantArray[Range[m+1],n+1]]];
ijvec={ivec,jvec}\[Transpose];

K=Transpose[Table[DoIt[ijvec,s],{s,ijvec}]];

Total[Abs[Chop[(K - Korig), 10^-10]], \[Infinity]] (* check if K and original K are the same within numerical error *)

This runs already a bit faster, but without knowing what you're doing, mathematically, or even in spoken words, I'm afraid it will be very tough to speed it up. What does a row represent in 'k'? what does a column represent? Also posting your Matlab code would be helpful maybe... Attach it to your post.

POSTED BY: Sander Huisman

BTW, if you have n = m =100, then your output will be of order 10^8 elements, is that correct? that needs quite a bit of memory...

POSTED BY: Sander Huisman
Posted 9 years ago
POSTED BY: Enzo Marino
Posted 9 years ago

Hello Sander,

I went through your reformulation of my example code. It is faster, however I see that you have defined eq = D1w11V2[ij] + D2w112V2[ij]. This works fine but only in this specifc case. I wanted Mathematica to define it because in the general applications of the method the equation can be very long and complex. That's why I moved to Mathematica... to avoid doing it by hand. I hope that after my explanation of the whole meaning of the code (see my yestrday post) you have a clearer picture of my final goal.

Many thaks,

Enzo

POSTED BY: Enzo Marino
POSTED BY: Sander Huisman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard