SVM incremental learning (partial_fit)

[jvirtuel]

Hi,

i am trying to implement incremental learning using SVM/ SVC classifier using river library. When I am trying to use the learn_one command, I get an error stating that - "X is expecting 17 vectors, getting 2". I am guessing this is because when I am trying to pass documents one by one through the algo in a loop, it creates a matrix of features based on the first pass, and sets that as the length. When I go over the second pass of the loop, it is trying to find those words in the corpora to increment the frequencies.
Can you please tell me if i am thinking in the correct direction here? if not, please suggest how can I train a model incrementally using SVM.
Thank you in advance for your help !

[MaxHalford]

Hello! Could you please provide a minimal, reproducible example? We can't help you with only this description. Thanks in advance.

[jvirtuel]

Hello, thanks for your answer, here it is :

am trying to incrementally train a model using SGDClassifier using the River library. I am providing the data and the code to train it below:
data = [("tried the program","software"),
("my unit test failed","software"),
("i need a new power supply","hardware"),
("the drive has a 2TB capacity","hardware"),
("unit-tests","software"),
("program","software"),
("power supply","hardware"),
("drive","hardware"),
("it needs more memory","hardware"),
("check the API","software"),
("design the API","software"),
("they need more CPU","hardware"),
("code","software"),
("i found some bugs in the code","software"),
("i swapped the memory","hardware"),
("i tested the code","software")]

test_data = [('he writes code daily','software'),
('the disk is faulty','hardware'),
("refactor the code","software"),
('no empty space on the drive','hardware')]

Create a Pipeline

from river.compose import Pipeline
from river import preprocessing
from river import compat
from sklearn import linear_model
model = preprocessing.StandardScaler()
model |= compat.convert_sklearn_to_river(estimator=linear_model.SGDClassifier(loss='log',eta0=0.01,learning_rate='constant'),classes=['software', 'hardware'])
pipe_nb = Pipeline(('vectorizer',TFIDF),('nb',model))

learning one at a time

for text,label in data:
try:
pipe_nb = pipe_nb.learn_one(text,label)
print("yes")
except Exception as e:
print(e)

When I am trying to run the code, I am getting the following error message:

ERROR MESSAGE :

yes
X has 4 features, but SGDClassifier is expecting 3 features as input.
X has 4 features, but SGDClassifier is expecting 3 features as input.
X has 5 features, but SGDClassifier is expecting 3 features as input.
X has 2 features, but SGDClassifier is expecting 3 features as input.
X has 1 features, but SGDClassifier is expecting 3 features as input.
X has 2 features, but SGDClassifier is expecting 3 features as input.
X has 1 features, but SGDClassifier is expecting 3 features as input.
X has 4 features, but SGDClassifier is expecting 3 features as input.
yes
yes
X has 4 features, but SGDClassifier is expecting 3 features as input.
X has 1 features, but SGDClassifier is expecting 3 features as input.
X has 6 features, but SGDClassifier is expecting 3 features as input.
yes
yes

From my understanding, this is happening because the number of features (words/tokens) in each sentence are different, and it tries to find the same features in the second iteration that it found on the first time to create the matrix. Can you explain why is this error coming up, and how can I incrementally train on this data?

Thanks again for your help!

[MaxHalford]

scikit-learn SGDClassifier does indeed require all the vocab to be specified in the first sample. May I ask why you're using it, and not River's LogisticRegression? The latter has the exact same performance as its scikit-learn equivalent, but doesn't have this sample shape consistency issue.

Meta: you can highlight code with ``` when using GitHub, it makes it easier on the eyes

[jvirtuel]

Hi,

Thanks for the quick response. I could have used Logistic Regression but my actual problem is not binary in terms of classes. Logistic Regression would work only in the cases where there are two classes. My actual dataset consist of multiple labels (8). Can I do a multinomial regression using River and a partial fit?
Thanks

[jvirtuel]

Actually, I just realised that there is a Softmax Regression library on River. I guess I could use it for multinomial logistic regression.
Thanks

[MaxHalford]

You can also use the OneVsRestClassifier from River. It's actually what scikit-learn's SGDClassifier does under the hood.