kumarlabjax / mousetracking Goto Github PK

Dear KumarLab,

As you mention in the Docs part: "This software was designed for 480x480 monochromatic images and is untested for different image size. Due to the pooling and upsampling layers, this exact structure will only work with images in multiples of 96 pixels without adjusting the network layers.
Functionally, the input images must be square and be the same shape as all the other images tested."

I would like to use the code on video that is not square (960x540) so I was wondering if you are planning to add the use of videos with different dimensions.

In any case, if you could share some advice on how to modify the network to make it work on other video formats it would be really helpful.

Thanks!

I've been trying to use this code for inference of top view mice videos and I've been using the Full Model pretrained data set with 415000 steps. I've also tried using different pretrained models to see if I can get better results, but the Full Model has given me the best results. I am looking for some advice on how to get better results, like, is it better to train the model more than 415000 steps or is it more worthwhile to use the provided tools to create a dataset from our own data? Our data has a few objects in the environment with the mouse and this program is really promising because it has worked on data that includes mice and extra objects. Currently it predicts the mouse's position about 30% of the time but am unable to figure out how to do better. I was curious if there was a way we could discuss over zoom or some other medium? I would really appreciate any help/advice!

Hi,

I downloaded this code a couple weeks ago as I am working on a very similar mouse position tracking project. I set up a conda virtual environment and installed python (2.7.18 because I could not find 2.7.12) along with the packages in the versions you specified. I downloaded and unzipped the KOMP_Dataset.tar off of your Zenodo site and ran the line "find Ell -name '*.txt' | sed -e 's/Ell///' -e 's/.txt//' | shuf | split -l 1000" where I edited the size from the original 16000 (as in the README) to 1000 to match the dataset. This generated a xaa and xab file which I renamed to Train_Split.txt and Valid_Split.txt respectively (attached) so that they would match the given input line. Then, in terminal I input "python main.py --net_type segellreg --batch_size 50 Train --model construct_segellreg_v8 --log_dir Training_Segellreg --num_steps 100000 --learn_function gen_train_op_adam --train_list Train_Split.txt --valid_list Valid_Split.txt --dataset_folder Ell --start_learn_rate 1e-5" and recieved the following error:

Constructing model...
Adding loss function...
WARNING:tensorflow:From /pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/training.py:10: mean_squared_error (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-12-30.
Instructions for updating:
Use tf.losses.mean_squared_error instead.
WARNING:tensorflow:From /pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/contrib/losses/python/losses/loss_ops.py:530: compute_weighted_loss (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-12-30.
Instructions for updating:
Use tf.losses.compute_weighted_loss instead.
WARNING:tensorflow:From /pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/contrib/losses/python/losses/loss_ops.py:151: add_loss (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-12-30.
Instructions for updating:
Use tf.losses.add_loss instead.
Populating input queues...
Starting input threads...
Initializing optimizer...
Generating summaries and savers...
Initializing model...
Beginning training...
Traceback (most recent call last):
File "main.py", line 101, in
main(sys.argv[1:])
File "main.py", line 92, in main
trainNet.trainNetwork(arg_dict)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/TrainNetwork.py", line 277, in trainNetwork
trainSegEllfitNetwork(arg_dict)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/TrainNetwork.py", line 130, in trainSegEllfitNetwork
img_batch, seg_label_batch, ellfit_label_batch = sess.run([image_train_batch, seg_train_batch, label_train_batch])
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_3_Input_Decoding/shuffle_batch_join_1/random_shuffle_queue' is closed and has insufficient elements (requested 50, current size 0)
[[Node: Input_Decoding/shuffle_batch_join_1 = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](Input_Decoding/shuffle_batch_join_1/random_shuffle_queue, Input_Decoding/shuffle_batch_join_1/n)]]

Caused by op u'Input_Decoding/shuffle_batch_join_1', defined at:
File "main.py", line 101, in
main(sys.argv[1:])
File "main.py", line 92, in main
trainNet.trainNetwork(arg_dict)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/TrainNetwork.py", line 277, in trainNetwork
trainSegEllfitNetwork(arg_dict)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/TrainNetwork.py", line 104, in trainSegEllfitNetwork
image_train_batch, seg_train_batch, label_train_batch = get_train_batch_segellreg(arg_dict['dataset'], arg_dict['n_reader_threads'], arg_dict['batch_size'], arg_dict['input_size'], max_trans = arg_dict['aug_trans_max'], max_rot = arg_dict['aug_rot_max'])
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/readers.py", line 235, in get_train_batch_segellreg
image_batch, seg_batch, ellfit_batch = tf.train.shuffle_batch_join(example_list, batch_size=batch_size, shapes=shapes, capacity=capacity, min_after_dequeue=min_after_dequeue)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1305, in shuffle_batch_join
name=name)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 776, in _shuffle_batch_join
dequeued = queue.dequeue_many(batch_size, name=name)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 458, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 1310, in _queue_dequeue_many_v2
timeout_ms=timeout_ms, name=name)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1226, in init
self._traceback = _extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_3_Input_Decoding/shuffle_batch_join_1/random_shuffle_queue' is closed and has insufficient elements (requested 50, current size 0)
[[Node: Input_Decoding/shuffle_batch_join_1 = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](Input_Decoding/shuffle_batch_join_1/random_shuffle_queue, Input_Decoding/shuffle_batch_join_1/n)]]

After consulting the help file I modified the Train_Split.txt and Valid_Split.txt files to include the names of the training and validation data files and '.txt' within Train_Split.txt and Valid_Split.txt and ran the following line in terminal (to use as many defaults as possible): "python main.py --net_type segellreg Train --model construct_segellreg_v8 --log_dir Training_Segellreg --learn_function gen_train_op_adam --train_list Train_Split.txt --valid_list Valid_Split.txt --dataset_folder Ell". I encountered the following error:

Constructing model...
Adding loss function...
WARNING:tensorflow:From /pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/training.py:10: mean_squared_error (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-12-30.
Instructions for updating:
Use tf.losses.mean_squared_error instead.
WARNING:tensorflow:From /pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/contrib/losses/python/losses/loss_ops.py:530: compute_weighted_loss (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-12-30.
Instructions for updating:
Use tf.losses.compute_weighted_loss instead.
WARNING:tensorflow:From /pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/contrib/losses/python/losses/loss_ops.py:151: add_loss (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-12-30.
Instructions for updating:
Use tf.losses.add_loss instead.
Populating input queues...
Starting input threads...
Initializing optimizer...
Generating summaries and savers...
Initializing model...
Beginning training...
Traceback (most recent call last):
File "main.py", line 101, in
main(sys.argv[1:])
File "main.py", line 92, in main
trainNet.trainNetwork(arg_dict)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/TrainNetwork.py", line 277, in trainNetwork
trainSegEllfitNetwork(arg_dict)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/TrainNetwork.py", line 130, in trainSegEllfitNetwork
img_batch, seg_label_batch, ellfit_label_batch = sess.run([image_train_batch, seg_train_batch, label_train_batch])
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_3_Input_Decoding/shuffle_batch_join_1/random_shuffle_queue' is closed and has insufficient elements (requested 5, current size 0)
[[Node: Input_Decoding/shuffle_batch_join_1 = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](Input_Decoding/shuffle_batch_join_1/random_shuffle_queue, Input_Decoding/shuffle_batch_join_1/n)]]

Caused by op u'Input_Decoding/shuffle_batch_join_1', defined at:
File "main.py", line 101, in
main(sys.argv[1:])
File "main.py", line 92, in main
trainNet.trainNetwork(arg_dict)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/TrainNetwork.py", line 277, in trainNetwork
trainSegEllfitNetwork(arg_dict)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/TrainNetwork.py", line 104, in trainSegEllfitNetwork
image_train_batch, seg_train_batch, label_train_batch = get_train_batch_segellreg(arg_dict['dataset'], arg_dict['n_reader_threads'], arg_dict['batch_size'], arg_dict['input_size'], max_trans = arg_dict['aug_trans_max'], max_rot = arg_dict['aug_rot_max'])
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/GuetherOrig/MouseTracking-master/utils/readers.py", line 235, in get_train_batch_segellreg
image_batch, seg_batch, ellfit_batch = tf.train.shuffle_batch_join(example_list, batch_size=batch_size, shapes=shapes, capacity=capacity, min_after_dequeue=min_after_dequeue)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1305, in shuffle_batch_join
name=name)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 776, in _shuffle_batch_join
dequeued = queue.dequeue_many(batch_size, name=name)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 458, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 1310, in _queue_dequeue_many_v2
timeout_ms=timeout_ms, name=name)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/pfs/tc1/project/rongsongcodeoptim2022/demmons/virtual2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1226, in init
self._traceback = _extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_3_Input_Decoding/shuffle_batch_join_1/random_shuffle_queue' is closed and has insufficient elements (requested 5, current size 0)
[[Node: Input_Decoding/shuffle_batch_join_1 = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](Input_Decoding/shuffle_batch_join_1/random_shuffle_queue, Input_Decoding/shuffle_batch_join_1/n)]]

Is there an issue with the way I am calling the code or is it wise to switch datasets? Also, if there's any other information useful for debugging please let me know and I will provide it if I can
Train_Split.txt
Valid_Split.txt
.

Dear KumarLab,

First of all, thank you for sharing your valuable research materials. I found your research a month ago, thanks to your kind code description, I could follow your work more easily.

At this moment, I have one question about your description.
The question is how can I test some images based on the trained model.
For example, after I trained KOMP_Dataset based on your instruction(modified number of steps), I obtained three files denoted as model.ckpt-1000(index, meta, data respectively). Now I want to test some images or videos, but, I cannot find video data on your website except for your inference code.
Is it possible to test some images from OFA_Dataset? If it is possible, could you let me know how to do it?
If not, could you share your video data?

Thank you for your help.