Message Boards Message Boards

GROUPS:

MongoDB and Mathematica

Posted 5 years ago
7049 Views
|
17 Replies
|
19 Total Likes
|
Hi everyone,

I'm looking for a database link between Mathematica and MongoDB. Is it possible? I see nothing about this in the DatabaseLink reference. Any ideas?

Steeve
17 Replies
Hi Steeve,

I am not aware of a direct link between mathematica and MongoDB but could I recommend using JLink and then writting your mongo interface through Java?

Even though it is not a complete solution I know MongoDB does have a REST interface that you could interface through... however it has no support for insert/update/remove operations... But you could read your data at least and you can use Mathematica's ability to communicate through HTTP requests to read the data.

If you aren't completely locked into Mongo you could also look at couchdb (http://couchdb.apache.org/) which does have a fully formed REST API. It also takes an optimistic approach to data and instead of locking an entire local instance of the database to write data it versions data which also allows for ACID compliance as opposed to Mongo's partial ACID compliance. All of this being said CouchDB does seem to only allow you to query through map/reduce (and often you just have to write the map function. I am also still not clear on how their indexing works, but it appears to index through the map functions that are written.. I really need to do something with this data store to understand how it works better).

All in all I have used mongo before and love it. I have not used couchdb yet but it could be an alternative if you aren't willing/can't create a java interface for mongodb.

Adam Martinek
Posted 3 years ago

Hello! Old thread, but I recently made my MongoDB Mathematica driver open-source here: https://github.com/zbjornson/MongoDBLink.

Here is a related SE post: mathematica.stackexchange.com/questions/40731

There is a good presentation with practical code examples about Mathematica with MongoDB here:

MongoDB with WL- .NB Presentation

Wolfram Language 11.3 now has support for interfacing with MongoDB: http://reference.wolfram.com/language/MongoLink/guide/MongoLinkOperations.html

Hello Sebastian,

I've put some simple collection with several documents with MongoCollectionInsert. When i try to make a simple query like find the people with 30 years old it works: MongoCollectionFind[coll, <|"Age" -> 30|>] But when i try to know the people with more than 30 years old i cant find the right sintaxe for it.

I've tried the mongoDB standard "Age" -> {$gt}->30 sintaxe in all combinations of every other type of brackets and it doesn't work. The examples in the F1 Help only use the equal never the other ones (<,>,<=, >=, !=)

From the MongoLink Tutorial, applying the format suggested in the section "Modifying Documents" suggests this is the correct syntax:

MongoCollectionFind[coll, <|"Age" -> <|"$gt" -> 30|>|>

Well, On youtube i found the answer: MongoCollectionFind[coll, <|"Age" -> |<"$gt"->30>||>] Simple, right? But now i have this problem: - if the "Age" Values are strings (previously "exported" as such by a lazy exporter) the comparison does not work.I've tried using ToExpression applied to the Key "Age" or to Values["Age"] but it doesn't accepted it. This works with mathematica objects (arrays, lists) and with Map for instance but i can't figure out how to use it in cursor objects.

This works with mathematica objects (arrays, lists) and with Map for instance but i can't figure out how to use it in cursor objects.

If you are doing a MongoDB query, then Mathematica functions (like ToExpression) definitely not supported. The issue is then a MongoDB issue, and there are various options you have (eg see this https://stackoverflow.com/questions/18039358/mongodb-gt-lt-operators-with-prices-stored-as-strings)

Thank you all,

@Robert Ferguson - You were really quick and concise in you reply

@Sebastian Bodenstein

The issue is then a MongoDB issue...

Indeed but couldn't it be different in future versions of WM? - If the solution to this trivial problem is to convert the source of the data (defeating a little bit the idea of NoSQL way of thinking) or to get hole information to some variable (defeating a bigger bit of the idea of using the cursors) doesn't this situation weakens the all point of using Mongolink on WM?

Hope this gets resolved in the version 12.n and in the help menus someone address this issue with examples and some kind of warning, because i'm really enjoying working with WM which was unknown to me just a few weeks ago.

If the solution to this trivial problem is to convert the source of the data (defeating a little bit the idea of NoSQL way of thinking)

That is the correct solution if you intend to represent this data in any database and perform database-native queries on the data. NoSQL does not imply 'no types for fields', just no fixed schema.

Indeed but couldn't it be different in future versions of WM?

How so? How can MongoLink change the behaviour of the MongoDB $gt operator?

Again, Thank you for your reply. I know MongoLink has to work with the behaviors inherit by MongoDB, but why not create a new function (e.g, named MongoCollectionFindType(coll, {query, typeexpect} )) in which the user say what kind of data it expects to find in the query and if some documents have different types of the expected the function does that conversion in order to make the comparison...and of course this new function does not return a cursor of the documents matching the query, it has to return a list of some sort (maybe not all "columns" but only some chosen by the user), but with the advantage of already having converted to the "typeexpected". Other "problem" is the fact that you, after getting a cursor of the documents return by a MongoFindCollection query, cannot use it as a first argument of a new MongoFindCollection.....refining the query and not handling the data just yet. I'm sorry if i'm not making sense.

I know MongoLink has to work with the behaviors inherit by MongoDB, but why not create a new function (e.g, named MongoCollectionFindType(coll, {query, typeexpect} )) in which the user say what kind of data it expects to find in the query and if some documents have different types of the expected the function does that conversion in order to make the comparison

Because its a deeply non-performant hack to the correct solution: storing the data in the correct format. And if users really insist on storing data with incorrect types, they are free to use MongoCollectionAggregate with the $convert op (https://docs.mongodb.com/manual/reference/operator/aggregation/convert/) if they have MongoDB +4.0 and are willing to pay the appropriate performance penalties.

My questions were mostly to confirm what i thought....and if there was something basic that i was missing.

Again, thank you... specially for having the patience to explain the why's and the if's.

In my case, as the model of the data to be exported to MongoDB is also being defined, is important not to miss this issues before the data gets stored and starts to grow. Some of my data comes from machines that "don't care" about this sort of inconsistencies so we have to "pass the comb" before we can work on the data.

is important not to miss this issues before the data gets stored and starts to grow.

Use MongoDB schema validation in this case to prevent invalid data from being inserted into the database: https://docs.mongodb.com/manual/core/schema-validation/

Hi, Using some testing collection i've put MongoDB (using MongoCollectionInsert[]) I've been exploring the MongoLink function MongoCollectionUpdateMany by using the equivalent sintaxe from MongoDB for create/update ( $set), for increment ($inc), etc for field and arrays but one problem has come when the field (or sub-fields) are arrays: The MongoCollectionUpdateMany[collection, filter, update] has 3 parameters and the equivalent (i assume) on MongodDB is db.collection.updateMany(filter, update, arraysFilters), although appears to also have 3 parameters in reality it has 4 because you define the coolection "outside" the brackets.....and so it became a problem because that arraysFilters has no place on the MongoLink function. db.students.insert([ { "_id" : 1, "grades" : [ 95, 92, 90 ] }, { "_id" : 2, "grades" : [ 98, 100, 102 ] }, { "_id" : 3, "grades" : [ 95, 110, 100 ] } ]) db.students.updateMany( { grades: { $gte: 100 } }, { $set: { "grades.$[element]" : 100 } }, { arrayFilters: [ { "element": { $gte: 100 } } ] } ) (because grades is a array its no enought to filter it (first parameter), you have to filter by element of the arrays ans the MongoLink don't have this forth parameter) I may give you some examples but i was hopping that someone has dealt with this issue and give me some help.

That's cool! Thanks for sharing this new feature.

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract