Logics and Labels must be the same size error using SoftmaxCrossEntropyWithLogits
I am completely new to Tensorflow. I tried to tweak Deep MNIST to predict movie ratings in the MovieLens dataset. I've simplified the model a bit so that instead of using a 5-point scale, it's a simple binary Y / N rating (similar to the most recent rating system on Netflix). I am trying to use only partial scores to predict preferences for new items. When training the model, I get the following error in the stack trace:
Traceback (most recent call last):
File "/Users/Eric/dev/Coding Academy >Tutorials/tf_impl/deep_tf_group_rec_SO.py", line 223, in <module>
train_step.run(feed_dict={x: batch_xs, y_: batch_ys, keep_prob: 0.5})
File "/Library/Python/2.7/site->packages/tensorflow/python/framework/ops.py", line 1550, in run
_run_using_default_session(self, feed_dict, self.graph, session)
File "/Library/Python/2.7/site->packages/tensorflow/python/framework/ops.py", line 3764, in >_run_using_default_session
session.run(operation, feed_dict)
File "/Library/Python/2.7/site->packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/Library/Python/2.7/site->packages/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/Library/Python/2.7/site->packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/Library/Python/2.7/site->packages/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: logits and >labels must be same size: logits_size=[1,2] labels_size=[50,2]
[[Node: SoftmaxCrossEntropyWithLogits = >SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, >_device="/job:localhost/replica:0/task:0/cpu:0"](Reshape_2, Reshape_3)]]
Caused by op u'SoftmaxCrossEntropyWithLogits', defined at:
File "/Users/Eric/dev/Coding Academy >Tutorials/tf_impl/deep_tf_group_rec_SO.py", line 209, in <module>
cross_entropy = >tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y_, >logits=y_conv))
File "/Library/Python/2.7/site->packages/tensorflow/python/ops/nn_ops.py", line 1617, in >softmax_cross_entropy_with_logits
precise_logits, labels, name=name)
File "/Library/Python/2.7/site->packages/tensorflow/python/ops/gen_nn_ops.py", line 2265, in >_softmax_cross_entropy_with_logits
features=features, labels=labels, name=name)
File "/Library/Python/2.7/site->packages/tensorflow/python/framework/op_def_library.py", line 763, in >apply_op
op_def=op_def)
File "/Library/Python/2.7/site->packages/tensorflow/python/framework/ops.py", line 2327, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/Library/Python/2.7/site->packages/tensorflow/python/framework/ops.py", line 1226, in __init__
self._traceback = _extract_stack()
InvalidArgumentError (see above for traceback): logits and labels must >be same size: logits_size=[1,2] labels_size=[50,2]
[[Node: SoftmaxCrossEntropyWithLogits = >SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, >_device="/job:localhost/replica:0/task:0/cpu:0"](Reshape_2, Reshape_3)]]
The code causing the error can be viewed here
Dimensions of variables used in the model:
-
x (?, 1682)
-
y_ (?, 2)
- x_history (?, 290, 290, 1)
- h_pool1 (?, 145, 145, 32)
- h_pool2 (?, 73, 73, 64)
- h_pool3 (?, 37, 37, 128)
- h_pool4 (?, 19, 19, 256)
- h_pool5 (?, 10, 10, 512)
- h_fc1 (?, 1024)
- h_fc1_drop (?, 1024)
- y_conv (?, 2)
source to share
The problem is when you change your input batch (form 50 training instances x 1682 functions) to [-1, 290, 290, 1] on this line:
x_history = tf.reshape(x, [-1, 290, 290, 1])
You can see that the x_history form ends up running:
x_history.eval(feed_dict={x:batch[0], y_:batch[1], keep_prob:1.0}).shape
=> (1, 290, 290, 1)
This effectively uses all the functionality for your batch of 50 instances and puts them in one instance (first dimension is 1, where it should be 50) and then runs through the rest of the network. So your cross_entropy score is failing as it cannot put out a batch of 50 target shortcuts with a single outlet on your network.
You need to select the shapes of the layers so that the batch size ("in your plates") is saved over the network. One way to do this would be to cross out your pre-1764 functions and change to [-1,42,42,1], since 42 * 42 = 1764.
It's worth noting that 2d convolution is most commonly used with image data that is naturally two-dimensional. Given that your functions are not 2D, might you be better off starting with a simple fully connected layer network?
source to share
To overcome this problem, I reduced the batch size from 50 to one.
batch = create_batches(train_data_pairs, 1)
and dramatically increasing the number of training iterations. Also, for accuracy, I test the model against a large number of small chunks of test data, then average that estimate.
This is more of a hack than a solution to the problem, but I allowed myself to move on to exploring other aspects of TensorFlow and reproduce the data in different ways.
source to share