Shouldn't KNNClassifier expect inconsistent number of features

[gr33ndata]

I've noticed that KNNClassifier expects consistent number of features, i.e. training instances should stick to same the features as seen in the first instance, or documents should stick to the same vocabulary as seen by the first one.

I understand this is by design:

Raises
    ------
    ValueError
        If at any moment, a sample with a different number of attributes than
         those already observed is passed.

But doesn't this contradict with the online learning nature of the library, especially when using KNN to classify text documents, then it is very hard to know the vocabulary beforehand. Happy to research possible solutions if you are up for that

[MaxHalford]

I'll let @smastelini answer.

I'm quite sure this can be handled, and is just a limitation of the current implementation.

[smastelini]

You're right @gr33ndata. This is currently a limitation of our implementation, as internally we rely on arrays for performance reasons.

I should undergo a research project in the next few months focused on overcoming this limitation. Quite some while ago I did some experiments using LSH, but the amount and sensibility of the hyper-parameters are a bit overwhelming. I intend to explore multiple alternatives to provide efficient, yet easy-to-use k-NN implementations.

Also expect a baseline k-NN implementation based on brute-force search to become available too.

[smastelini]

Frankly, I don't have high hopes for LSH in its current shape. Maybe other LSH projection families could be explored, aiming at reducing the setup complexity. I'm not very familiar with LSH projection function families (but certainly want to become familiarized in the following months), but this sounds like a promising way to explore.

By the time I wrote the code, I was able to get nice results concerning the accuracy, memory, and running time, but the hyper-parameters had to be carefully adjusted.