ValueError: Dimension size must be evenly divisible by 50 but is 1 for '{{node Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32](

[MaitriSavla2003]

I am trying to integrate Differential Privacy in my code, but the code is working fine only when I am using microbatch size as 1, other than that it is throwing the error. I even tried keeping it equal to the batch size. But the error is still not resolved.

EXACT ERROR:
ValueError: Dimension size must be evenly divisible by 50 but is 1 for '{{node Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32](sparse_categorical_crossentropy/weighted_loss/value, Reshape/shape)' with input shapes: [], [2] and with input tensors computed as partial shapes: input[1] = [50,?].

[sws943]

I get the same error (tfp 0.9, tf 2.15) with the DPKerasAdamOptimizer in a colab env (colab release 2024-09-23)

More precisely:

first installing ft privacy via

!pip install tensorflow_privacy

and running the code below will result in

ImportError                               Traceback (most recent call last)
[<ipython-input-4-24f585dbd19e>](https://localhost:8080/#) in <cell line: 2>()
      1 import tensorflow as tf
----> 2 from tensorflow_privacy.privacy.optimizers.dp_optimizer_keras import DPKerasAdamOptimizer
      3 from tensorflow_privacy.privacy.analysis import compute_dp_sgd_privacy
      4 import numpy as np
      5 

8 frames
[/usr/local/lib/python3.10/dist-packages/tensorflow_estimator/python/estimator/estimator.py](https://localhost:8080/#) in <module>
     32 from tensorflow.python.checkpoint import checkpoint_management
     33 from tensorflow.python.checkpoint import graph_view
---> 34 from tensorflow.python.distribute import estimator_training as distribute_coordinator_training
     35 from tensorflow.python.eager import context
     36 from tensorflow.python.eager import monitoring

ImportError: cannot import name 'estimator_training' from 'tensorflow.python.distribute' (/usr/local/lib/python3.10/dist-packages/tensorflow/python/distribute/__init__.py)

Explicitly switching to an older tf version with:

!pip install tensorflow==2.15.* tf-keras==2.15.* tensorflow_privacy==0.9.*

and runnig the code below again I get the same error as @MaitriSavla2003 .

Epoch 1/10
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-2-24f585dbd19e>](https://localhost:8080/#) in <cell line: 30>()
     28 model.compile(optimizer=optimizer, loss='mse', metrics=['mae'])
     29 
---> 30 model.fit(X, y, epochs=10, batch_size=32)

2 frames
[/usr/local/lib/python3.10/dist-packages/tensorflow_privacy/privacy/optimizers/dp_optimizer_keras.py](https://localhost:8080/#) in _compute_gradients(self, loss, var_list, grad_loss, tape)
    275           num_microbatches = self._num_microbatches
    276         microbatch_losses = tf.reduce_mean(
--> 277             tf.reshape(loss, [num_microbatches, -1]), axis=1)
    278 
    279         if callable(var_list):

ValueError: in user code:

    File "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py", line 1401, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py", line 1384, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py", line 1373, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py", line 1154, in train_step
        self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
    File "/usr/local/lib/python3.10/dist-packages/keras/src/optimizers/legacy/optimizer_v2.py", line 598, in minimize
        grads_and_vars = self._compute_gradients(
    File "/usr/local/lib/python3.10/dist-packages/tensorflow_privacy/privacy/optimizers/dp_optimizer_keras.py", line 277, in _compute_gradients
        tf.reshape(loss, [num_microbatches, -1]), axis=1)

    ValueError: Dimension size must be evenly divisible by 32 but is 1 for '{{node Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32](mean_squared_error/weighted_loss/value, Reshape/shape)' with input shapes: [], [2] and with input tensors computed as partial shapes: input[1] = [32,?].

import tensorflow as tf
from tensorflow_privacy.privacy.optimizers.dp_optimizer_keras import DPKerasAdamOptimizer
from tensorflow_privacy.privacy.analysis import compute_dp_sgd_privacy
import numpy as np

# Create a synthetic dataset
np.random.seed(0)
X = np.random.rand(1000, 1)
y = 3 * X + np.random.normal(0, 0.1, (1000, 1))

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(1,))
])


l2_norm_clip = 1.0
noise_multiplier = 0.5
num_microbatches = 32

optimizer = DPKerasAdamOptimizer(
    l2_norm_clip=l2_norm_clip,
    noise_multiplier=noise_multiplier,
    num_microbatches=num_microbatches,
    learning_rate=0.01
)

# Compile the model
model.compile(optimizer=optimizer, loss='mse', metrics=['mae'])

model.fit(X, y, epochs=10, batch_size=32)