Message Boards Message Boards

Speed up numerical solution of differential equation using BSplines?

Posted 8 years ago

Dear Wolfarm community, I am very new with Mathematica. I've written a code which works pretty well, but the big problem is that it is so slow that I cannot use it. I usually use Matlab for this type of coding. The equivalent Matlab version is thousand times faster. Here in the following I report a dummy (very much simplified) version of code that contains the main operations I need to perform. Maybe I make some basic mistakes in programming with Mathematica that dramatically slow down the execution. When the parameters n and m increase, the code becomes unusable. Note also that I tried to replace all the functions defined in the code with compiled functions with command Compile. I expected to see big improvements but unfortunately it was not so. Is there a more efficient way to write this code? Thank you very much! Enzo

Clear["Global`*"]

(*Differential equation*)

Eq = D1w[1, 1] + D2w[1, 1, 2]

D1w[1, 1] + D2w[1, 1, 2]

(*Define BSpline parameters*)

p = 2;
q = 2;
n = 5;
m = 5;
r = n + p + 1;
s = m + q + 1;
rint = r + 1 - 2 (p + 1);
sint = s + 1 - 2 (p + 1);
Uint = Range[0, 1, 1/(rint + 1)]; 
Vint = Range[0, 1, 1/(sint + 1)];
U = N[Join[ConstantArray[0, p], Uint , ConstantArray[1, p]]];
V = N[Join[ConstantArray[0, q], Vint , ConstantArray[1, q]]];
Uc = Table[(Sum[U[[h + k]], {k, 1, p}])/p, {h, 1, n + 1}];
Vc = Table[(Sum[V[[h + k]], {k, 1, q}])/q, {h, 1, n + 1}];

(*Define BSpline approximations (Discretization) *)

D1w11[ic_, jc_, i_, j_] := 
  D[BSplineBasis[{p, U}, i - 1, u], {u, 1}] BSplineBasis[{q, V}, j - 1, v] W1[
     i, j] //. {u -> Uc[[ic]], v -> Vc[[jc]]};
D2w112[ic_, jc_, i_, j_] := 
  D[BSplineBasis[{p, U}, i - 1, u], {u, 1}] D[
     BSplineBasis[{q, V}, j - 1, v], {v, 1}] W1[i, j] //. {u -> Uc[[ic]], 
    v -> Vc[[jc]]};

(*Substitute the above approximate derivatives into the original differential equation*)

subw11[ic_, jc_, i_, j_] := {D1w[1, 1] :> D1w11[ic, jc, i, j]};
subw112[ic_, jc_, i_, j_] := {D2w[1, 1, 2] :> D2w112[ic, jc, i, j]};
sub[ic_, jc_, i_, j_] := Join[subw11[ic, jc, i, j], subw112[ic, jc, i, j]];
EqDisc[ic_, jc_, i_, j_] := Eq //. sub[ic, jc, i, j];

(*Now collects terms in w1*)

KEq[ic_, jc_, i_, j_] := Coefficient[EqDisc [ic, jc, i, j], {W1[i, j]}];
KEqW1[ic_, jc_, i_, j_] := KEq[ic, jc, i, j][[1]];

(*Transform double -index formulation into single - index formulation*)

iMat = Table[Range[1, n + 1, 1], m + 1];
ivec = Flatten[ArrayReshape[iMat, {1, (n + 1) (m + 1)}]];
jMat = Transpose[Table[Range[1, m + 1, 1], n + 1]];
jvec = Flatten[ArrayReshape[jMat, {1, (n + 1) (m + 1)}]];
KKEqW1[r_, s_] := 
  KEqW1 @@ List[ivec [[r]], jvec [[r]], ivec [[s]], jvec [[s]]];

(*Finally, create a matrix*)

K = Table[KKEqW1[r, s], {r, 1, (n + 1) (m + 1)}, {s, 1, (n + 1) (m + 1)}];
POSTED BY: Enzo Marino
14 Replies
Posted 8 years ago

Hello Neil, Thank you for your reply. I agree with you, there is a bug in the calculation and evaluation of BSplineBasis derivatives. Actually, it is correct that the derivatives of BSplineBasis functions involve denominators which are 0. However, this indeterminate result 1/0 is normally replaced by 0. It is a pity that Mathematica does not do it when one uses the following syntax

D[BSplineBasis[{p, U}, i - 1, x], {x, 1}] /. {i -> 1, x -> 0.2}

The workaround you suggest is fine, but it still requires the definitions of derivatives as functions implying definig other functions calling them.... and this slowdown the code. I reformulated the entire code with the overloading operator and in the end the new version is approximately 10 times faster than the initial one. However, it is still much slower than the equivalent matlab version and I am afraid I cannot use it for further developments.

I think I have tried all I could. I don't think that I have other chances I am afraid I will return to Matlab. I will possibly use Mathematica only for small n and m to make comparisons and facilitate the debugging process.

Thank you for all your help!

Enzo

POSTED BY: Enzo Marino

Enzo,

I think you found a bug in Mathematica's BSplineBasis. It should be putting a conditional on the terms of the derivative. I would report this at Wolfram Tech Support or better yet, you should call them and see if they can suggest a good workaround until it is fixed. I added a few lines to your file showing the problematic expression. The derivative has two terms and the first term will go to zero in the denominator when n->0. The second term has the answer that agrees with your answer when you substitute in zero for n and then take the derivative. Note that the derivative expression also goes to infinity when n > 5. In this case the second term of the derivative has zero in its denominator.

As for a workaround:

D[BSplineBasis[{d, U}, n, x], {x,1}] appears to work properly if the n value is numeric before you take the derivative. I suggest you write your code so that you make sure it happens that way. One possibility is to use operator overloading to compute all the derivative expressions as functions of x and then evaluate them.

If you define:

bSplineDeriv[n_] := 
 bSplineDeriv[n] = D[BSplineBasis[{p, U}, n, x], {x, 1}]

As you iterate through your values of n, the symbolic expressions for the derivative will be computed. The nice part of this definition is that it will only compute the derivative for each n value once. See the enclosed nb file.

I hope this helps. Maybe someone else has another workaround suggestion...

Attachments:
POSTED BY: Neil Singer
Posted 8 years ago

Thank you Neil, I think now I am very close to solve the problem.

Yes, as you siuggested, I do not define any function. I always use = instead of :=. Only in the end, when I need to iterate, I evaluate the symbolic expressions with numbers.

There is only one (I hope minor!) thing that is still not working... very basic issue I guess: can you tell me why I get the error 1/0 indeterminate when I try to evaluate the first derivative of a BsplineBasis? Please see the following short code:

(* Input data *)
p = 2;
n = 5;
r = n + p + 1;
rint = r + 1 - 2 (p + 1);
Uint = Range[0, 1, 1/(rint + 1)]; 
U = N[Join[ConstantArray[0, p], Uint , ConstantArray[1, p]]];

what I do not understand is why the BSplineBasis can be evaluated easily in the following way:

BSplineBasis[{p, U}, i - 1, x] /. {i -> 1, x -> 0.2}

whereas its first derivative CANNOT:

D[BSplineBasis[{p, U}, i - 1, x], {x, 1}] /. {i -> 1, x -> 0.2}

In the example I have chosen to compute the first derivative of the 0th Bspline basis and evaluate it at x=0.2. If you try you'll get "Infinite expression 1/0. encountered. But, as shown in the plot (see file attached), this derivative must DOES exist!

Plot[Evaluate[D[BSplineBasis[{p, U}, 0, x], {x, 1}]], {x, 0, 1}]

Could you please help me on this? After that I think I have done! Many thanks! Enzo

Attachments:
POSTED BY: Enzo Marino

Enzo,

Next, try to do the procedure exactly as you would do in Matlab but using Mathematica to handle the equations. Enter the PDE equations, fit splines, take derivatives and construct large matrices -- Construct functions that you use for each step but don't defer the computation. For example, Don't create a formula to construct spines and use it later. Define the formula for splines and create a matrix of splines (I may be showing my lack of understanding of your algorithm here -- but you get the idea). After you have an example that works, then you can create functions that do the steps in that order without delaying the calculations until the end (which causes repetition). If you have done this in Matlab it should be relatively straightforward in Mathematica. If you post some Matlab code I might be able to suggest more details but essentially you should let Mathematica handle the large expressions and expand everything out into symbolic or numerical values early -- at the same point in the procedure that you would do so in Matlab. It is better to consume memory than to repeat your calculations so it is better to iterate and create a large matrix of expressions and then take derivatives of the matrix and later turn it into numbers than to defer the calculation and do the derivatives and expression handling while you are creating the large numerical matrix. (the later may seem more "elegant" but it will be slower).

I hope that helps.

Regards

POSTED BY: Neil Singer

Enzo,

At first glance, I think the Code that Sander posted shows methods to address the issue that I described. Note that he cleverly defines:

bspU[i_]:=bspU[i]=BSplineBasis[{p,U},i-1,u]

This is relying on operator overloading -- for example, when you call the function bspU[3] the calculation is done once so future calls of bspU[3] just return that answer instantly and do no further calculations. If you do this in my simple (contrived) posted example:

fun1[i_, var_] := 
 fun1[i, var] = Coefficient[computefun[eq, var], var, i]

The first call to define mat would take the full 2 seconds but a subsequent call would take less than a millisecond.

Referring to Sander's code:

D1w11V2[{i_,j_}]:=D[bspU[i],u]bspV[j]
D2w112V2[{i_,j_}]:=D[bspU[i],u] D[bspV[j],v]

You should also see if you can define enough of your problem to compute the derivatives once (or do the same function overloading trick). As you iterate over i, the code takes the derivative of bspU[i] repeatedly.

Lastly, Did you try the built in functions for numerically solving PDE's? NSolve and NSolveValue should be helpful since you can express the problem symbolically in Mathematica and have it generate the numerical solution.

Regards

POSTED BY: Neil Singer
Posted 8 years ago

Thank you Neil for your comments! Yes, I see the definition `bspU[i_]:=bspU[i] helps! And I agree with you: I think that a lot of time can be saved by avoiding Mathematica to compute the derivative everytime with command D. The derivative of a bspline can be defined recursively form the bspline itself. Unfortunately, I haven't found it between the Mathematica built-in functions so I will try to define it by myself hopig that it will be faster than computing it via D.

As regards Sander's version of my code, it is faster. The only thing is that I need to let Mathematica to collect terms since in the real applications the original differential equations is rather lengthy and qiute complex.

Many thanks!

Enzo

POSTED BY: Updating Name

Enzo,

I did not extensively study your approach but I think I see your problem.

Easy question first. To make a number a float just add the decimal point or a prime (above the tab key) -- instead of 1,2,3 you use 1.,2.,3. or 1.0,2.0,3.0 (because it looks better!), or 1`, 2`, 3` (The prime ties into the Precision)

When you switch from Matlab to Mathematica you need to change the way you think about solving a problem. In general, using Matlab, matrices and coefficients are computed as you proceed because it is a matrix-based language. Mathematica is primarily symbolic. The way you handled these calculations is you created functions. By creating Mathematica functions you delay the calculation of everything until you call it. Therefore many of your calculations are duplicated MANY times. You need to rework your approach a bit to make this faster. For a simple example (notebook attached):

Create a large equation

In[1]:= exp = 1000;

In[2]:= eq = (x + 3)^exp

Out[2]= (3 + x)^1000

Compute the derivative with the large equation (just to have something to compute)

In[3]:= eq2 = D[eq, x]/exp

Out[3]= (3 + x)^999

Create a function that takes the derivative -- and slow it down some more by Expanding first

In[4]:= computefun[f_, var_] := D[Expand[f], var]

Create two functions -- one that calls the compute function and one that takes the computed result and make a function

In[5]:= fun1[i_, var_] := Coefficient[computefun[eq, var], var, i]

In[6]:= fun2[i_] := Coefficient[eq2, x, i]

In[7]:= 

Slow -- it recomputes the result

In[8]:= Timing[mat = Table[fun1[i, x], {i, exp}];]

Out[8]= {2.20229, Null}

Faster -- it calculates it once using table

In[9]:= Timing[mat = Table[fun2[i], {i, exp}];]

Out[9]= {0.008603, Null}

This would even be faster -- create a matrix using CoeeficientList

In[10]:= Timing[mat = CoefficientList[eq, x];]

Out[10]= {0.002729, Null}

The upshot is that by delaying your computations until the very end, you redo some computations many times over. The lure of using Mathematica functions is that the representation is very compact but for efficiency you sometimes must evaluate the expressions earlier and handle the numbers as matrices. Your profile says you are calling Coefficient too many times so that is a place to start -- create the expression once and use CoeeficientList on the result.

Regards

Neil

Attachments:
POSTED BY: Neil Singer
Posted 8 years ago

Hello Neil,

Thank you very much for your helpful and instructive reply to my post. The example that you kindly provided helps a lot to understand the issue.
Actually, I also had the feeling that something was wrong since all the functions remain unevaluated until the very end and they are called (unnecessarily) many times, but I did not know how to face it. I will try to reformulate my code taking inspiration from the approach you suggest in your example. I will let you know.

Many thanks!

Enzo

POSTED BY: Enzo Marino
Posted 8 years ago

Hello Sander,

thank you very much for your kind reply. I will try all your suggestions (/. intead of //. , SubDivide, etc). However, to answer your main question about “what I want to calculate”, all I need to calculate is reported in the sample code that I posted. My final goal is to build the matrix K (see last line of the code). The only difference in real applications of the full code is that I need to compute many matrices K (say 25) each of them is very big (not like in the posted example). For example, if you change the parameters p, q, n and m by setting them as follows: p = q = 8 and n = m = 100 you will see the code takes ages. Making it UNUSABLE :(

I have only 6 variables which are integers: i, j, ic, jc, r, s. All the rest are real numbers. For me it would be great to treat all variables as reals. How can I tell this to Mathematica? My final result (K) must be made of reals.

Finally, I run the code in debug mode with the command // RuntimeToolsProfile at the end. See profile report in attachment. In this case I set p = q = 2 and n = m = 20. I see that it takes a lot of time in making the substitution Eq //. sub[ic, jc, i, j]. Moreover, I do not understand why it keeps calling many times functions like Coefficient[EqDisc[ic, jc, i, j], {W1[i, j]}] which should be called once for all, instead.

Many thanks,

Enzo

Attachments:
POSTED BY: Enzo Marino

Again, tell us what K is. What does it represent? I know K is a matrix, but what do the elements represent? Without knowing any background it is SUPER HARD to come up with a faster solution. Your code is also very hard to read because of all the functions that nearly don't do anything, very convoluted. I cleaned up your code a lot, which took me a lot of time to figure out what you are doing!

p=5;
q=5;
n=8;
m=8;
r=n+p+1;
s=m+q+1;
rint=r+1-2 (p+1);
sint=s+1-2 (p+1);
Uint=N@Subdivide[rint+1];
Vint=N@Subdivide[sint+1];
U=Join[ConstantArray[0.0,p],Uint,ConstantArray[1.0,p]];
V=Join[ConstantArray[0.0,q],Vint,ConstantArray[1.0,q]];
Uc=MovingAverage[U[[2;;-2]],p];
Vc=MovingAverage[V[[2;;-2]],p];

ClearAll[DoIt,D1w11V2,D2w112V2,bspU,bspV]

bspU[i_]:=bspU[i]=BSplineBasis[{p,U},i-1,u]
bspV[j_]:=bspV[j]=BSplineBasis[{q,V},j-1,v]

D1w11V2[{i_,j_}]:=D[bspU[i],u]bspV[j]
D2w112V2[{i_,j_}]:=D[bspU[i],u] D[bspV[j],v]

DoIt[rr_List,ij:{i_,j_}]:=Module[{eq},
    eq=D1w11V2[ij]+D2w112V2[ij];
    Table[eq/.{u->Uc[[r[[1]]]],v->Vc[[r[[2]]]]},{r,rr}]
]

ivec=Flatten[ConstantArray[Range[n+1],m+1]];
jvec=Flatten[Transpose[ConstantArray[Range[m+1],n+1]]];
ijvec={ivec,jvec}\[Transpose];

K=Transpose[Table[DoIt[ijvec,s],{s,ijvec}]];

Total[Abs[Chop[(K - Korig), 10^-10]], \[Infinity]] (* check if K and original K are the same within numerical error *)

This runs already a bit faster, but without knowing what you're doing, mathematically, or even in spoken words, I'm afraid it will be very tough to speed it up. What does a row represent in 'k'? what does a column represent? Also posting your Matlab code would be helpful maybe... Attach it to your post.

POSTED BY: Sander Huisman

BTW, if you have n = m =100, then your output will be of order 10^8 elements, is that correct? that needs quite a bit of memory...

POSTED BY: Sander Huisman
Posted 8 years ago

Hello Sander,

Thank you so much for your reply and for the time you spent in cleaning up my code.

Here the mathematical meaning of what I need to do:

I want to solve a differential equation (d.e.) through a numerical method which basically transforms the d.e. into an algebraic system in the standard form Kx = b. Once I find K, then I will just need to invert it and get the unknown vector x = K^-1 b. For the sake of simplicity, I have posted a very dummy example of differential equation. (Please, note that it is a meaningless example since neither the term b nor the boundary conditions are present). The d.e. of the dummy example reads as follows

w1,1 + w1,12 = 0 (1)

where w1(u,v) is a function defined from a square domain [0,1] x [0,1] to R. In the code that I posted D1w[1, 1] is w1,1, namely the partial derivative of w1 with respect to u, while D2w[1, 1, 2] is w1,12, namely the second derivative of w1 with respect to u and v. There are several ways to approximate w1. I have chosen to approximate it by means of BSpline basis functions. So I have

w1(u,v) = \Sum{i=0,n} \Sum{j=0,m} Ni(u)*Nj(v) W1(i,j) (2)

where W1(i,j) represent a (finite) set of (n+1)(m+1) control variables, and Ni and Nj are the i-th and j-th BSpline basis functions. Now, it is easy to take the derivatives of the approximate form of w1 as given in Eq. (2) and replace them into the original d.e. Eq (1). If one makes the substitution, which is what I try to do with `EqDisc[ic, jc, i, j] := Eq //. sub[ic, jc, i, j]; (see my code) one gets a discretized version of Eq (1), which represents one (approximated) equation in (n+1)(m+1) unknowns.

If now one evaluates this equation in (n+1)(m+1) points belonging to the domain [0,1]x[0,1] (let’s say a grid of points) one gets (n+1)(m+1) equations. A linear system is obtained and, after collecting terms (see command Coefficient), the matrix K of the system is found. So, each row of K represents the discretized equation evaluated in a specific point of the domain, while each element on a row represents the term appearing in the summations over i and j. Please note that a conversion from two-index to one-index is needed since the unknowns W1(i,j) are in the final system arranged in a vector and not in a matrix.

Sorry for the very informal explanation, but I hope this is enough to give you a general idea of what my final goal is.

This is a known technique which works well when I implement it in Matlab environment where I use a more standard programming approach. Now I would like to develop the same technique in Mathematica.

Many thanks, Enzo

POSTED BY: Enzo Marino
Posted 8 years ago

Hello Sander,

I went through your reformulation of my example code. It is faster, however I see that you have defined eq = D1w11V2[ij] + D2w112V2[ij]. This works fine but only in this specifc case. I wanted Mathematica to define it because in the general applications of the method the equation can be very long and complex. That's why I moved to Mathematica... to avoid doing it by hand. I hope that after my explanation of the whole meaning of the code (see my yestrday post) you have a clearer picture of my final goal.

Many thaks,

Enzo

POSTED BY: Enzo Marino

Hi Enzo,

the codes takes 225 milliseconds on my machine, 1000x faster? 225 microseconds? It would be very nice to explain WHAT you are doing/what you want to accomplish. A literal code-translation is generally not a good way of doing things efficiently in Mathematica.

I was also wondering why you have:

 Flatten[ArrayReshape[ ... ]]

The ArrayReshape doesn't do anything, right? Also using Table to make copies is slower than when you use ConstantArray. In some occasions you use //. while a single replacement /. is probably enough right?

Uint = Range[0, 1, 1/(rint + 1)]; 
Vint = Range[0, 1, 1/(sint + 1)];

I think SubDivide is the function you want to use here, faster and more legible. Lastly, I see that the end result is part integers, part real numbers. Try to 'plug in' real values as soon as you can, this can speed up code by a very large amount, also having only reals (not mixed types) speeds things up a lot.

Again, tell us what you (want to) calculate, now it is difficult to guess because of all the functions that call each other...

POSTED BY: Sander Huisman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract