Sequential model doesn't have outputs

[stsievert]

Shouldn't this code work?

from sklearn.datasets import make_classification
from scikeras.wrappers import KerasClassifier
import tensorflow as tf

def model():
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Dense(8))
    model.add(tf.keras.layers.Dense(1))
    return model

X, y = make_classification(n_features=8)
est = KerasClassifier(model=model, loss="sparse_categorical_crossentropy")
est.fit(X, y=y)

This throws a ValueError: object of type NoneType [self.model_.outputs] has no len().

Full traceback

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/Downloads/_junk2.py in <module>
     11 X, y = make_classification(n_features=8)
     12 est = KerasClassifier(model=model, loss="sparse_categorical_crossentropy")
---> 13 est.fit(X, y=y)

~/anaconda3/envs/scikeras/lib/python3.7/site-packages/scikeras/wrappers.py in fit(self, X, y, sample_weight, **kwargs)
   1375             sample_weight = 1 if sample_weight is None else sample_weight
   1376             sample_weight *= compute_sample_weight(class_weight=self.class_weight, y=y)
-> 1377         super().fit(X=X, y=y, sample_weight=sample_weight, **kwargs)
   1378         return self
   1379

~/anaconda3/envs/scikeras/lib/python3.7/site-packages/scikeras/wrappers.py in fit(self, X, y, sample_weight, **kwargs)
    739             epochs=getattr(self, "fit__epochs", self.epochs),
    740             initial_epoch=0,
--> 741             **kwargs,
    742         )
    743

~/anaconda3/envs/scikeras/lib/python3.7/site-packages/scikeras/wrappers.py in _fit(self, X, y, sample_weight, warm_start, epochs, initial_epoch, **kwargs)
    855         X = self.feature_encoder_.transform(X)
    856
--> 857         self._check_model_compatibility(y)
    858
    859         self._fit_keras_model(

~/anaconda3/envs/scikeras/lib/python3.7/site-packages/scikeras/wrappers.py in _check_model_compatibility(self, y)
    541             # we recognize the attribute but do not force it to be
    542             # generated
--> 543             if self.n_outputs_expected_ != len(self.model_.outputs):
    544                 raise ValueError(
    545                     "Detected a Keras model input of size"

TypeError: object of type 'NoneType' has no len()

[adriangb]

Hmm, let me take a look.

[adriangb]

I think you need to add an Input layer:

def model():
    model = tf.keras.Sequential()
    modeo.add(tf.keras.layers.Input(shape=(8,)))
    model.add(tf.keras.layers.Dense(8))  # this does not have to be 8
    ...

Also, related to #206 , it seems like "sparse_categorical_crossentropy" requires the output to have 2 units. "binary_crossentropy" works with 1 output unit.

[adriangb]

Did #207 (comment) solve your problem @stsievert ? Happy to dig deeper otherwise.

[stsievert]

Yes, this code works:

from sklearn.datasets import make_classification
from scikeras.wrappers import KerasClassifier
import tensorflow as tf

def model():
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Input(shape=(8,)))
    model.add(tf.keras.layers.Dense(8))
    model.add(tf.keras.layers.Dense(1))
    return model

X, y = make_classification(n_features=8)
est = KerasClassifier(model=model, loss="binary_crossentropy")
est.fit(X, y=y)

I think this error should be caught.

[adriangb]

We can leave this open to track it. Ideally, we can come up with a larger category of "Keras Model does not have inputs or outputs".

By the way, I believe that the reason Keras allows this is because Model is just a layer, and thus can be chained, like I did in #200

[stsievert]

This should be easy to catch:

With model.add(tf.keras.layers.Input(shape=(8,))): est.model_.input raises an AttributeError, as does est.model_.output ("'Sequential' object has no attribute _nested_outputs")

Without model.add(tf.keras.layers.Input(shape=(8,))): est.model_.input) is a KerasTensor.

[adriangb]

I agree, it shouldn't be too hard to catch. I'll cook up an implementation once we resolve some of the outstanding PRs, especially #143 . Unless you think this should be incorporated into / superseed #143?

[stsievert]

Eh, I like specific and narrow PRs. Let's keep them separate.

[invexed]

Hi, I'm encountering a similar issue when trying to wrap a TensorFlow Decision Forests model:

from sklearn.datasets import make_classification
import tensorflow_decision_forests as tfdf
from scikeras.wrappers import KerasClassifier

X, y = make_classification(1000, 20, n_informative=10, random_state=0)
model = tfdf.keras.CartModel()
model.compile()
clf = KerasClassifier(model=model)
clf.fit(X, y)

Traceback (most recent call last):
  File "/home/username/dev/tfdf-sklearn/main.py", line 10, in <module>
    clf.fit(X, y)
  File "/home/username/.local/lib/python3.9/site-packages/scikeras/wrappers.py", line 1453, in fit
    super().fit(X=X, y=y, sample_weight=sample_weight, **kwargs)
  File "/home/username/.local/lib/python3.9/site-packages/scikeras/wrappers.py", line 725, in fit
    self._fit(
  File "/home/username/.local/lib/python3.9/site-packages/scikeras/wrappers.py", line 888, in _fit
    self._check_model_compatibility(y)
  File "/home/username/.local/lib/python3.9/site-packages/scikeras/wrappers.py", line 528, in _check_model_compatibility
    if self.n_outputs_expected_ != len(self.model_.outputs):
TypeError: object of type 'NoneType' has no len()

Is there a way to get this to work?
Thanks.

[adriangb]

Does that model have an input layer? Can you post the results of model.summary()?

[invexed]

I trained a tfdf.keras.CartModel using a different dataset and the start of model.summary() showed

Model: "cart_model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
=================================================================
Total params: 1
Trainable params: 0
Non-trainable params: 1
_________________________________________________________________
Type: "RANDOM_FOREST"
Task: CLASSIFICATION
Label: "__LABEL"

[invexed]

So I suppose the answer to your question is no, the model does not have an input layer. Are such models compatible with SciKeras?

[adriangb]

Correct. But Keras allows you to compose models, so you can use your pretrained model as an layer in a new model (kinda) and then just add an input layer before it (and maybe an output layer, depending on your model). Take a look at this SO question: https://datascience.stackexchange.com/a/21620.

Let me know if that works or if it gives you trouble, I can try to write an example closer to your use case if needed.

[invexed]

inputs = Input(shape=(20,))
initial_model = tfdf.keras.CartModel()
outputs = Flatten()(initial_model(inputs))
model = Model(inputs=inputs, outputs=outputs)
model.compile(loss='mse')

The above seems to work, although I get a warning printed:

Warning:  The model was called directly (i.e. using `model(data)` instead of using `model.predict(data)`) before being trained. The model will only return zeros until trained. The output shape might change after training Tensor("inputs:0", shape=(None, 20), dtype=float32)

Have I done something wrong? Do you have to pre-train the initial_model before you can add it as a layer?

[adriangb]

A couple things:

1. I'm not sure if you need the Flatten() and such there, did you do that just because you copied that from the example or is that actually something your model needs?

2.I would troubleshoot this by using Keras directly: instead of giving your model building function to SciKeras just do model = model_build_fn(...). See if it trains, predicts, accuracy, etc. Once you're confident your Keras model works as expected, then you can wrap it in SciKeras, put it in a Scikit-Learn pipeline, grid search, etc. Trying to figure out how to make your Keras model work while also doing grid search or something is going to be painful: there's too many variables and unknowns.

[invexed]

The original model definitely works as expected - it's a default model from TensorFlow Decision Forests and does indeed train and predict correctly. It looks like the warning is something unique to TFDF (I think Keras' functional API is confusing it for whatever reason), however the wrapped model seems to be working correctly. And you were right about the Flatten being superfluous.

Thanks a lot for your help. It's much appreciated.