Message Boards Message Boards

4
|
6687 Views
|
4 Replies
|
6 Total Likes
View groups...
Share
Share this post:

Negative effects possible from "too many" LibraryFunctions?

Szabolcs Horvát
Posted 8 years ago

Hello,

I hope some of the LibraryLink experts will be able to comment on this.

Could there be any negative effects from loading "too many" LibraryFunctions? My package has 156 at the moments and the number is steadily increasing. Package loading is already unusually slow because of this: 2 seconds on OS X and 40 seconds or so on a Raspberry Pi. Of these functions, 21 use LinkObject passing, which means that there is a loopback link created for each of them (as shown by Links[]).

Now I didn't observe any negative effects, but I am not sure if anyone has pushed LibraryLink so far before.

So, in principle, can there be any negative effects from loading lots of LibraryFunctions, or from creating too many loopback links? Could it affect general performance? Should I take steps now to reduce the number of functions in the package? Or is the system designed to cope with arbitrarily many loaded functions?

Just to be clear, I have not observed any problems so far apart from the mildly annoying loading performance. But it would give me some peace of mind to know that such use cases were anticipated for LibraryLink.
POSTED BY: Szabolcs Horvát
4 Replies
Sort By:

Thank you, this is very useful information! I can use lazy loading, so that's no problem.
POSTED BY: Szabolcs Horvát

Another way would be to generate C code from the compiled functions and find a way to join the source files into a single one, and then compile it into a single dll containing all functions, similarly to what you would get when compiling manually a LibraryLink dll in Visual C++ for example.
POSTED BY: Faysal Aberkane

Actually the original question was about a single library (single DLL), containing many (hundreds) of manually written C functions. But it does not seem to be a problem.
POSTED BY: Szabolcs Horvát

I will just pass the other's word, but don't ask me for details. I just hope it will give you a hint.

I don't think there is any intrinsic problem with many LibraryLink functions other than the time and resources they take to load. But once they are loaded they should not interfere or slow down due to large numbers of functions. We know cases of loading about 600 functions and this didn't stop it working, just made it slow to load. A way around that is to just load the library functions lazily. Another case has ~800 functions and loads everything lazily, which means you only pay for the functions you use.
POSTED BY: Vitaliy Kaurov
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract