Message Boards Message Boards

[WSG20] Daily Study Group: Biodiversity Explorations with Machine Learning

enter image description here

In this study group you'll practice applying machine learning techniques using the Wolfram Language and data from the natural world. Topics include functions to access biodiversity data, examples of classification, text analysis in social media, audio processing of bird sounds and deploying a trained neural network image classifier to your mobile phone. Sessions run daily, Monday through Friday. Join any session 15 minutes early for help getting started. I will guide each session by sharing lessons, polling the group to review key concepts, introducing practice problems and answering questions. A certificate of program completion will be awarded to participants who attend online sessions and pass a quiz. Sign up: https://register.gotowebinar.com/register/5988979773822061838
POSTED BY: Jofre Espigule-Pons
30 Replies
Sort By:

In response to the query about using neural networks for medical image analysis, here is a list of past posts from community on medical related machine learning projects that might be useful:

https://community.wolfram.com/groups/-/m/t/1729656

https://community.wolfram.com/groups/-/m/t/1732071

https://community.wolfram.com/groups/-/m/t/1731911

https://community.wolfram.com/groups/-/m/t/1120757

https://community.wolfram.com/groups/-/m/t/1379717
POSTED BY: Jofre Espigule-Pons

I noticed that the anole downloads from GBIF are labeled as *.csv files, but the don't open properly in Excel (which is easier for me to use for filtering the data). They appear to be either space or tab delimited files. I tried to import them in Excel several different ways, but nothing worked, The files do import fine in Mathematica, however, using SemanticImport.

I assume the process Jofre used to load the iOS mushroom app was for his iPhone only, and did not place the app in the Apple app store - Correct?

In the quiz, none of the possible answers for Question 5 will work because of a syntax error. I'm pretty sure I've seen this same question on previous quizzes with the same error.

UPDATE: I just discovered that my earlier post about the syntax error in Question 5 of the quiz is wrong. Some operators (those that contain multiple characters) need to be entered using key combinations between Escape ... Escape. Apologies for any confusion.
POSTED BY: James Kralik
Rohit Namjoshi
Rohit Namjoshi
Posted 3 years ago

Hi James,

The problem is caused by GBIF giving the file a .csv extension. It is not comma separated, it is tab separated. Changed the file extension from .csv to .tsv and Excel will open it correctly.
POSTED BY: Rohit Namjoshi
C Ellis
C Ellis
Posted 3 years ago

Jofre - Great presentation yesterday on audio processing. Couldn't help but reflect upon how the results were somewhat inconclusive (ie end of slide 3) despite both the overall process and coding appearing quite sound. With that said, is it possible the bird you recorded wasn't actually an owl?

Mourning doves can sound like owls ( https://www.audubon.org/news/no-thats-not-owl-outside-your-window ) and would be much more common habitants in the forest of lamp poles populating the UICI stadium parking lots. The initial WL entity search did return "pigeon, dove" as its AudioIdentify result.

A search of inaturalist showed the Rock Dove could serve as another likely suspect for A/B test use.

Just a thought. Again, excellent presentation. Thank you.
POSTED BY: C Ellis

Oh that would be embarrassing! I'm not an owl expert and you are right that mourning dove can sound like owls. Thanks for pointing out the audubon article. If you want to check my observation and audio recording on iNaturalist, you can do it here: https://www.inaturalist.org/observations/35127698

Since there were other iNaturalists identifying the species as great horned owl, I did not perform an exhaustive search. I don't remember if I really saw the animal or just heard the sound. :)
POSTED BY: Jofre Espigule-Pons

Today (Day-5), we are going to train the "Wolfram ImageIdentify Net V1"- Neural Network with additional images from “INaturalistSearch”.

I would like to do the same with "YOLO V2 Trained on MS-COCO Data" and or "YOLO V3 Trained on Open Images Data". The input shall be an image and the outputs are Label, Probability, and Bounding Box.

How to train such a network? I have not found any documentation. The final objective is to identify all mushrooms on the following image. enter image description here
POSTED BY: Jürgen Kanz

Hi @Juergen Kanz , there was presentation on this particular topic at the Wolfram Tech Conference 2020, that you might want to watch. https://www.wolfram.com/broadcast/video.php?c=488&p=3&v=3252

It isn't an easy task yet. Probably, the most laborious task will be to manually tag the labels and add bounding boxes on your training dataset. Hopefully in the near future there will be more tools for automating this task.
POSTED BY: Jofre Espigule-Pons

Hi Jofre,

Yes, this presentation is going into the right direction. Is it possible to get access to the notebook?

Thanks in advance.
POSTED BY: Jürgen Kanz

I was finally able to create animations similar to that in Jofre's stink bug example for the brown and green anole. Both animations are based on GBIF data. When the distributions are compared for a given year, it appears both populations are increasing and that green anoles are apparently not being displaced by the brown anoles. What's being depicted, however, is probably the increasing frequency of observations being reported, rather than any change in the true populations of these two species year-by-year. I can report, however, that I have not seen any green anoles around my house or on my walks this year. Perhaps because I'm not looking in the right places; e.g., the brown anoles have taken over the ground and the green anoles have moved into other niches, such as tree canopies or less urbanized areas like parks.

Attachment

Attachment

Attachments:
BrownAnole_Houst...gif
GreenAnole_Houst...gif
POSTED BY: James Kralik

Nice animations! You are right, for a comparative analysis one would need to take into account the rise in number of observers over the last years. Or maybe you could plot their relative abundance over time for these two species. In this article they also point out the change in habitat for the green anole as you mentioned: https://www.anoleannals.org/2015/06/10/are-brown-anoles-in-florida-really-driving-green-anoles-to-extinction/
POSTED BY: Jofre Espigule-Pons

@James Kralik , I recommend you to use GBIFImport instead, specially when the number of observations is very large. You can download the occurrences data directly on the GBIF site: https://www.gbif.org/occurrence/search?taxon_key=4287413

Once you have downloaded the dataset for this species, you can then import the local CSV/Text file containing the occurrences using ResourceFunction["GBIFImport"]
POSTED BY: Jofre Espigule-Pons

I'm trying to expand on the anole example, but apparently iNaturalistSearch is limited to a maximum of 200 observations, even though 3046 observations for the brown anole were counted for Houston. I'm unable to use the Page and MaxItems options to automate the extraction (although I can brute force it by manually extracting 200 observations at a time into separate variables and then using Join. I also tried to extract 3 different species using the Or ("|") operator, but that did not work. My goal is to visually show the displacement of the green anole by the brown anole over the last 20 years in Houston.

In Jofre's stink bug example, he used a SemanticImport of a *.csv file, but I'm not sure of its source.

In my meandering around the web, I became aware of another invasive lizard that's beginning to get noticed in Texas, the Argentine tegus - they're the size of a dog and grow to about 4 feet long. Oh, and BTW, they're voracious omnivores!
POSTED BY: James Kralik
POSTED BY: Andrew Skipor

Note: At the "Predict" section from today's session there was an issue for those running 12.1 version or earlier. You will need to change a bit the code. Use to the following:

pm = PredictorMeasurements[p, testing -> "Rings"]

Instead of using the 12.2 version code:

pm = PredictorMeasurements[p, testing]

Thank you, Neil for catching this issue.
POSTED BY: Jofre Espigule-Pons

POSTED BY: Jofre Espigule-Pons
Updating Name
Updating Name
Posted 3 years ago

At the end of the Day 2 challenge example Jofre provided, the syntax trained["TrainedNet"][image] would not execute, but if I used c[image] it works as expected. Is there some code missing (i.e., I didn't see trained[] defined)?

Also, is there a shortcut for ResourceFunction["iNaturalistSearch"] that will create the iconized form on the fly like Ctrl + Enter for entities?
POSTED BY: Updating Name

Thanks for pointing to this syntax error. I edited the code. Concerning the shortcut for ResourceFunction, I'm not aware of any trick. It would be nice to have one though.
POSTED BY: Jofre Espigule-Pons

Thanks very much, Andrew
POSTED BY: Andrew Skipor

Hello , Can you please post the Notebook, you showed in regards to the "Tick" example. I want to try to correlate its population growth with weather such as rainfall... I am interested in the central WI area..

Thanks Andrew
POSTED BY: Andrew Skipor

The following sections are the ones related to tick species. If you are interested in getting tick observations from central Wisconsin only. I recommend you using ResourceFunction["INaturalistSearch"] (see the next last example)
POSTED BY: Jofre Espigule-Pons

You showed the histogram data, but during the class you increased the resolution for the month of May. I don' recall how you did that? Did you change "Month" to "Day" in the Histogram plot?

Thanks Andrew
POSTED BY: Andrew Skipor

Yes, if you change "Month" to "Day" you will get daily observations on DateHistogram.
POSTED BY: Jofre Espigule-Pons

Probably you are already aware of WeatherData function. You can use it to get rainfall info:

DateListPlot[
 WeatherData[
  Entity["AdministrativeDivision", {"Wisconsin", "UnitedStates"}][
   "Coordinates"], 
  "TotalPrecipitation", {{2018, 1, 1}, {2022, 12, 1}, "Month"}], 
 Joined -> True]
POSTED BY: Jofre Espigule-Pons

Thank you very much, this is very helpful!! Andrew Skipor
POSTED BY: Andrew Skipor

POSTED BY: James Kralik

Other Biodiversity posts from Joffre:
POSTED BY: Abrita Chakravarty

POSTED BY: Jofre Espigule-Pons

Looking forward to it, @Jofre Espigule-Pons!
POSTED BY: Jamie Peterson

Hi Jamie, I wasn't able to attend today. Will you be posting the recording soon? Sincerely, Jay Morreale Member p-brane LLC
POSTED BY: Jay Morreale

Hi @Jay Morreale, the recording link was included in today's email reminder. The link is: https://www.gotostage.com/channel/daily-study-group
POSTED BY: Jamie Peterson
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract