deshpandenu / time-series-forecasting-of-amazon-stock-prices-using-neural-networks-lstm-and-gan- Goto Github PK

Great work here Nupur. Just not sure why in notebook 5 you create an estimator but you don't use it to predict.

Please note that there is the possibility of data leakage. The way data are standardize is very dangerous because you shift back in time future information. Just try to delete (.shift(-num_historical_days)) in your scaling method and you will see how results will get worser.

Hello @deshpandenu This is really a cool project. I am trying to use the model for prediction but unable to output it.

class Predict:
  def __init__(self, num_historical_days=20, days=10, pct_change=0, gan_model=f'{googlepath}/deployed_models/gan', cnn_modle=f'{googlepath}/deployed_models/cnn', xgb_model=f'{googlepath}/deployed_models/xgb'):
    self.data = []
    self.num_historical_days = num_historical_days
    self.gan_model = gan_model
    self.cnn_modle = cnn_modle
    self.xgb_model = xgb_model
    
    files = [f"{googlepath}stock_data/{f}" for f in os.listdir(f"{googlepath}stock_data")] 
    for file in files:
      
      print(file)
      df = pd.read_csv(file, index_col='timestamp', parse_dates=True)
      df = df[['open','high','low','close','volume']]
            # data for new column labels that will use the pct_change of the closing data.
            # pct_change measure change between current and prior element. Map these into a 1x2
            # array to show if the pct_change > (our desired threshold) or less than.
            
      df = ((df -
            df.rolling(num_historical_days).mean().shift(-num_historical_days))
            /(df.rolling(num_historical_days).max().shift(-num_historical_days)
            -df.rolling(num_historical_days).min().shift(-num_historical_days)))
      df = df.dropna()
      self.data.append((file.split('/')[-1], df.iloc[0], df[200:200+num_historical_days].values))
      #split the df into arrays of length num_historical_days and append
      # to data, i.e. array of df[curr - num_days : curr] -> a batch of values
      # appending if price went up or down in curr day of "i" we are lookin
      # at
      
      
  def gan_predict(self):
    tf.reset_default_graph()
    gan = GAN(num_features=5, num_historical_days=self.num_historical_days, generator_input_size=200, is_train=False)
    with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      saver = tf.train.Saver()
      saver.restore(sess, self.gan_model)
      clf = joblib.load(self.xgb_model)
      for sym, date, data in self.data:
        features = sess.run(gan.features, feed_dict={gan.X:[data]})
        features = xgb.DMatrix(features)
        print('{} {} {}'.format(str(date).split(' ')[0], sym, clf.predict(features)[0][1] > 0.5))
        #predictions = np.array([x for x in gan_estimator.predict(p.gan_predict())])
        #print(predictions)

p = Predict()
p.gan_predict()

Questions:

1. How should I interpret the output? For eg: Is there a plot or time-series or forecasting that shows the prediction?
   After running the prediction, how to interpret the output?

2. The prediction is using gan to get features and use xgboost to predict. How to use the cnn model for stacking?

Amazing project. Thanks for the contribution.

The code fails during the execution of TrainXGBoost due to the outdated library. XGBoost no longer allows a list of NumPy arrays. It always has to have the input as a 2D array.

dmlc/xgboost#3970

Answer to the XGBoost list input to DMatrix:
@markli123 Hi, I actually found the cause of the weird behavior. See dmlc/xgboost#3970 8. The list gets silently converted into scipy.sparse.csr_matrix, which causes unexpected behavior causing wrong prediction. This is why we later disallowed converting list into a DMatrix directly.

hey,
i have a couple of qouestions:

1. i would like to know what is the input (shape, content) of the generator, discriminator?
2. do have special matrix to calculte the loss of the GAN?
3. what is the architecter of the generator and the discriminator?
   thank you

Hey Nupur,
I'm following your great notebooks :-) and in notebook five I came across two problems:

1. at the gan_estimator you have a variable that is - tfgan = tf.contrib.gan it works only in TensorFlow version below 1.15, do you have a solution for TensorFlow 2.3 because I could not find one...
2. in the class predict in def gan_predict there is a line - tf.reset_default_graph() that throws an error about nestef graphs but if i remove that line i get an error at this line - predictions = np.array([x for x in gan_estimator.predict(p.gan_predict())])
   that there two varibles with the same name
   thank you

Hello
all regards to your work in the relation of GAN
I am being stuck in error "tape is required when a Tensor loss is passed." in GAN_class
any urgent help

Can anyone please share the sample data of this project?

Where are the sources for the data mentioned in the sentimental analysis component?Can you please give a brief on that?

Congratulations. Its a fabulous piece of work.