The error you're encountering stems from an issue with using masking in the Embedding layer (mask_zero=True) and passing that through subsequent layers like LSTM and TimeDistributed. Here's how you can address the issue:

Key Points to Fix: Mask Propagation: Ensure that layers following the Embedding layer correctly handle the mask. The LSTM layer natively supports masking, but there may be issues when passing the mask to TimeDistributed. Loss Function: The sparse_categorical_crossentropy expects integer labels, and masking should be handled properly in the output to avoid issues. Input Shapes: Check the compatibility of shapes being passed through the layers. Eager Execution: If you're using TensorFlow 2.x, ensure eager execution is enabled (which is the default mode in TF 2.x). Here's the corrected and modified version of your code:

python Copy Edit import tensorflow as tf from tensorflow import keras

Define the model architecture

model = keras.Sequential([ keras.Input(shape=(200,)), # Input shape matches the padded sequence length keras.layers.Embedding(input_dim=vocab_len, output_dim=50, weights=[embedding_matrix], mask_zero=True), # Enable masking for padding tokens keras.layers.Bidirectional(keras.layers.LSTM(units=100, return_sequences=True)), # LSTM supports masking keras.layers.Bidirectional(keras.layers.LSTM(units=100, return_sequences=True)), keras.layers.TimeDistributed(keras.layers.Dense(units=tags_len, activation="softmax")) # TimeDistributed for sequence output ])

Compile the model

model.compile( optimizer="adam", loss="sparse_categorical_crossentropy", # Suitable for integer labels metrics=["accuracy"] )

Display model summary

model.summary()

Training

model.fit(X_train, Y_train, epochs=10) Changes and Fixes: Masking Compatibility: Layers like LSTM and Bidirectional handle the mask propagated from the Embedding layer. There's no need for manual intervention. Output Layer: The TimeDistributed layer is designed to handle the sequence output of the LSTM layer. Loss Function: Ensure Y_train is formatted correctly for sparse_categorical_crossentropy, with integer labels and no one-hot encoding. Debugging: If the issue persists, wrap the model.fit call in tf.function: python Copy Edit @tf.function def train(): model.fit(X_train, Y_train, epochs=10) train() Additional Tips: Masking with TimeDistributed: If the problem persists in the TimeDistributed layer, consider explicitly handling the mask: python Copy Edit keras.layers.TimeDistributed(keras.layers.Dense(units=tags_len, activation="softmax"), input_shape=(None, 200, 100)) Debugging with Eager Execution: Ensure eager execution is enabled by adding: python Copy Edit tf.config.run_functions_eagerly(True) Data Validation: Confirm that X_train and Y_train are correctly padded to the same sequence length (200) and formatted as NumPy arrays.

Comment From: mehtamansi29

Hi @ARforyou -

Thanks for reporting this issue and adding key points to fix. Your key points are correct as error is related to graph error. So need to run the code via enable in eager mode.

Debugging with Eager Execution: Ensure eager execution is enabled by adding: python Copy Edit tf.config.run_functions_eagerly(True

Addition to your point, adding run_eagerly=True also enable eager execution. Attached gist here for the reference. More details you can find here in this issue #20754

Comment From: github-actions[bot]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

Comment From: Surya2k1

Hi @ARforyou ,

Can you please test with latest master now. The PR #20765 might have addressed this.