Message Boards Message Boards

Incremental Machine Learning with feeding data every hour?

GROUPS:

I ran couple of simple ML examples available from the Documentation. A small data set with less than 100 records would take a whole 10 sec to be trained in my laptop. If I have a new set of sample data of 100 records available every hour, do I need to add those records to the population and train the entire Classifier all over again every hour? Is Classifier capable of retaining of its previous trained knowledge, while we keep feeding in newly available set of data to the Classifier every now & then?

POSTED BY: Peter Lim
Answer
2 months ago

On the site: https://medium.com/wolfram-events/multiparadigm-data-science-5-things-you-need-to-know-14ab75a15210 John McLoone explains the ML framework. In part 7 of the lecture at 3:25 minutes he shows that you can add data in a already trained network. https://youtu.be/JRW-BICsUG4?list=PLxn-kpJHbPx12mbdcVVWCelvcIL_R0X

POSTED BY: l van Veen
Answer
2 months ago

Hi I van Veen,

Thanks for your info. The given link seems broken. Been trying to click on it for the past few days. However, after watching John McLoone's video, it seems like by adding one additional labelled data, he had to retrain the entire classification. My concern is the efficiency of retraining the classification. For instance, I have a sample model that takes about 3-4 hours to train. If I just want to add a few more sample data to the previously trained population, does it take equally the same amount of time to retrain if using this method of John Mcloone's?

POSTED BY: Peter Lim
Answer
2 months ago

Thanks I Van Veen, I just tried using the same approach of John's. It works well with a much shorter time in retraining. Wonderful!

POSTED BY: Peter Lim
Answer
2 months ago

Group Abstract Group Abstract