Hi,

First thank you so much for providing us this excellent code!

I am wondering whether Movie-len 100k is a suitable dataset for this task? Since the Movie-len 100k does not consider any long-term behavior for the user (user only rated for each movie individually).

Therefore, I am concerned about if it is not good enough for the list-wise recommendation task which focus on the user long-term behavior.

Looking forwards to your reply!

Runsheng

hello. I'm student of South Korea.
I'm studying this paper and i want to run your code
but i can't find train data (train.csv) T^T
where is it..?
Is this secret?
if it is not secret, please tell me how can i get the data please.
I will be waiting for your answer. thank you ^^

Episode 1/100 Reward=543 Time=6s No replay
Episode 2/100 Reward=543 Time=57s Loss=2958.6747
Episode 3/100 Reward=543 Time=74s Loss=203.2957
Episode 4/100 Reward=543 Time=74s Loss=80.1686
Episode 5/100 Reward=543 Time=74s Loss=42.6469
Episode 6/100 Reward=543 Time=73s Loss=37.9031
Episode 7/100 Reward=543 Time=73s Loss=29.8142
Episode 8/100 Reward=543 Time=73s Loss=26.4902
Episode 9/100 Reward=543 Time=73s Loss=19.2726
Episode 10/100 Reward=543 Time=73s Loss=16.2351
Episode 11/100 Reward=543 Time=73s Loss=14.5593
Episode 12/100 Reward=543 Time=73s Loss=13.3819
Episode 13/100 Reward=543 Time=73s Loss=12.0543
Episode 14/100 Reward=543 Time=73s Loss=9.9012
Episode 15/100 Reward=543 Time=73s Loss=10.0397
Episode 16/100 Reward=543 Time=73s Loss=9.2727
Episode 17/100 Reward=543 Time=72s Loss=7.4542
Episode 18/100 Reward=543 Time=72s Loss=7.2437
Episode 19/100 Reward=543 Time=73s Loss=7.3171
Episode 20/100 Reward=543 Time=73s Loss=6.2746
Episode 21/100 Reward=543 Time=73s Loss=6.2563
Episode 22/100 Reward=543 Time=72s Loss=6.2598
Episode 23/100 Reward=543 Time=72s Loss=5.5634
Episode 24/100 Reward=543 Time=73s Loss=6.2925
Episode 25/100 Reward=543 Time=73s Loss=5.2713
Episode 26/100 Reward=543 Time=73s Loss=4.6282
Episode 27/100 Reward=543 Time=72s Loss=4.6075
Episode 28/100 Reward=543 Time=72s Loss=4.7174
Episode 29/100 Reward=543 Time=72s Loss=4.4310
Episode 30/100 Reward=543 Time=73s Loss=4.8185
Episode 31/100 Reward=544 Time=72s Loss=4.1476
Episode 32/100 Reward=543 Time=72s Loss=3.5478
Episode 33/100 Reward=543 Time=73s Loss=3.7355
Episode 34/100 Reward=543 Time=72s Loss=3.5698
Episode 35/100 Reward=543 Time=73s Loss=3.8600
Episode 36/100 Reward=543 Time=72s Loss=3.3381
Episode 37/100 Reward=543 Time=73s Loss=3.1626
Episode 38/100 Reward=543 Time=72s Loss=3.3410

I got an error of ValueError: Input tensors to a Functional must come from tf.keras.Input. Received: 0 (missing previous layer metadata) and i cant find the cause

this is my error trace and my code

    ---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-15-8058f3a2fd50> in <module>()
      6   test_loss, test_accuracy = eg.test(dg.user_test)
      7   print('Test set: Loss=%.4f ; Accuracy=%.1f%%' % (test_loss, test_accuracy * 100))
----> 8   eg.save_embeddings('embeddings.csv')

7 frames
<ipython-input-5-54ff9897b1c3> in save_embeddings(self, file_name)
     66     inp = self.m.input                                           # input placeholder
     67     outputs = [layer.output for layer in self.m.layers]          # all layer outputs
---> 68     functor = K.function([inp, K.learning_phase()], outputs )   # evaluation function
     69 
     70     #append embeddings to vectors

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py in function(inputs, outputs, updates, name, **kwargs)
   3934     from tensorflow.python.keras import models  # pylint: disable=g-import-not-at-top
   3935     from tensorflow.python.keras.utils import tf_utils  # pylint: disable=g-import-not-at-top
-> 3936     model = models.Model(inputs=inputs, outputs=outputs)
   3937 
   3938     wrap_outputs = isinstance(outputs, list) and len(outputs) == 1

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in __new__(cls, *args, **kwargs)
    240       # Functional model
    241       from tensorflow.python.keras.engine import functional  # pylint: disable=g-import-not-at-top
--> 242       return functional.Functional(*args, **kwargs)
    243     else:
    244       return super(Model, cls).__new__(cls, *args, **kwargs)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py in __init__(self, inputs, outputs, name, trainable)
    113     #     'arguments during initialization. Got an unexpected argument:')
    114     super(Functional, self).__init__(name=name, trainable=trainable)
--> 115     self._init_graph_network(inputs, outputs)
    116 
    117   @trackable.no_automatic_dependency_tracking

/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py in _init_graph_network(self, inputs, outputs)
    142       base_layer_utils.create_keras_history(self._nested_outputs)
    143 
--> 144     self._validate_graph_inputs_and_outputs()
    145 
    146     # A Network does not create weights of its own, thus it is already

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py in _validate_graph_inputs_and_outputs(self)
    637                          'must come from `tf.keras.Input`. '
    638                          'Received: ' + str(x) +
--> 639                          ' (missing previous layer metadata).')
    640       # Check that x is an input tensor.
    641       # pylint: disable=protected-access

ValueError: Input tensors to a Functional must come from `tf.keras.Input`. Received: 0 (missing previous layer metadata).

please help

I would like to know once the model is live how would we capture user's(negative or positive) ratings given as recommendation is based on cosine similarity of the embeddings vector , which is derived from the overall rating for the content from the historical data.

Hello, thank you for sharing Nice notebook!

I try to run this notebook in my pc. Can I get requirements.txt? thanks!

Hello
I use this code, but in part `environment = Environment(data, embeddings, alpha, gamma, fixed_length)

tf.compat.v1.reset_default_graph() # For multiple consecutive executions

sess = tf.compat.v1.Session()

'1: Initialize actor network f_θ^π and critic network Q(s, a|θ^µ) with random weights'

actor = Actor(sess, state_space_size, action_space_size, batch_size, ra_length, history_length, embeddings.size(), tau, actor_lr)
critic = Critic(sess, state_space_size, action_space_size, history_length, embeddings.size(), tau, critic_lr)

train(sess, environment, actor, critic, embeddings, history_length, ra_length, buffer_size, batch_size, discount_factor, nb_episodes, filename_summary), I encounter problems such as getting errors from thisline:tf.reset_default_graph() and this line:sess = tf.Session()`
I have tried the following methods but they all did not work.

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

I even manually created the commands that were in the actor class, etc., but I still get the following error:
:50: FutureWarning: In a future version of pandas, a length 1 tuple will be returned when iterating over a groupby with a grouper equal to a list of length 1. Don't supply a list with a single grouper to avoid this warning.
for rewards, group in self.embedded_data.groupby(['reward']):

RuntimeError Traceback (most recent call last)
in <cell line: 7>()
5 sess = tf.compat.v1.Session()
6 # '1: Initialize actor network f_θ^π and critic network Q(s, a|θ^µ) with random weights'
----> 7 actor = Actor(sess, state_space_size, action_space_size, batch_size, ra_length, history_length, embeddings.size(), tau, actor_lr)
8 critic = Critic(sess, state_space_size, action_space_size, history_length, embeddings.size(), tau, critic_lr)
9

2 frames
/usr/local/lib/python3.10/dist-packages/tensorflow/python/ops/array_ops.py in placeholder(dtype, shape, name)
3356 """
3357 if context.executing_eagerly():
-> 3358 raise RuntimeError("tf.placeholder() is not compatible with "
3359 "eager execution.")
3360

RuntimeError: tf.placeholder() is not compatible with eager execution.
, what should I do?

The source of error is : if True: # Generate embeddings?
save_embeddings('embeddings.csv')