jwyang / jule.torch Goto Github PK
View Code? Open in Web Editor NEWTorch code for our CVPR 2016 paper "Joint Unsupervised LEarning of Deep Representations and Image Clusters"
License: MIT License
Torch code for our CVPR 2016 paper "Joint Unsupervised LEarning of Deep Representations and Image Clusters"
License: MIT License
Hi,
Thank you for the code. I had a doubt regarding the Affinity function used in the reformulated loss(eq 11). From what I understand, after computing the triplets by organize_samples() function, they are fed to the criterion_triplet to get the loss.
In TripletEmbedding.lua are the Affinities( A(xi,xj) and A(xi,xk) in eq(11)) represented by delta_pos and delta_net ?
I am a bit confused because I could not see the actual Affinity function(calculated in the agg_clustering.c and as mentioned Graph Degree Linkage paper[68]) to be used while calculating the loss.
Thanks.
Hi,
I tried to install your codes however I have no idea how to install lua-knn. Specifically, how to set "CUDA_TOOLKIT_ROOT_DIR=/your/cuda-toolkit/dir" ? Can you please provide the more details about installing lua-knn ?
Thanks!
Hi,
I tried to implement the loss layer using python you mentioned in the paper (eq(11)), however I have no idea how the diff should look like, because usually there will be three diffs for anchors/positives/negatives. Can you please provide the sample using only feats and labels as inputs?
Thanks!
There is no GPU on my computer. How to use cpu to run the torch code. I have set gpu to -1, but still got error: module 'cunn' not found: No LuaRocks module found for "cunn".
If we want to use this package but not using GPU drivers, what's the best solution?
The lua-knn package is specifically for GPU, and the only other package I found is https://github.com/rlowrance/kernel-smoothers which I have not been able to figure out how to "install" -- the makefile just looks like a test suite, and it's not included in luarocks despite being referred to by the official docs.
Best option I can think of right now is to code the knn myself.
In the lines 65-68 (agg_clustering.c file), you first get the length of cluster i and cluster j, which are saved in Y_i_size, Y_j_size, respectively. But I am confused that you use THTensor_(get2d)(Y, j, n) - 1 to call the element Y[j,n], in which the maximum of n is Y_j_size, but the column size of Y is Y->size[1].
Hi, I was wondering if it is possible to cluster an arbitrary directory of images to their final cluster labels after training on a custom dataset? It would also suffice to be able to generate the embeddings of a set of images after training on a custom dataset (I think this is what #8 is requesting). I'd like to map the images in my custom dataset to their final cluster label.
Is this code directed against unlabeled data? Why the data in the example have label? Can I run this code If my data have no label?
The code in run_step_fast(), doesn't match the formula in annotation. This part aims at updating a_us_t[idx_c_a,:]
below is the annotation
update A_t(i->idx_c_a) = r_a * A_t(i->idx_c_a) + r_b * A_t(i->idx_c_b) (fast algorithm)
but code doesn't not contain r_a and r_b, which looks like:
A_us_t:indexAdd(1, torch.LongTensor{idx_c_a}, A_us_t:index(1, torch.LongTensor{idx_c_b}))
this makes me confused, should I change the code to annotation version(implementation in function merge_two_clusters())?
Thanks for your work!
I'd like to use this feature representation for further clustering.
Thanks in advance!
Hi,
I see that in the organize_samples function, you are using trainingData_labels. Should I pass arbitrary value if I don't have labels?
-Lalit
hello,
I have readed you CVPR paper “Joint Unsupervised Learning of Deep Representations and mage cluster”,but I don't know how to run the program given. I want you help ,thank you!
when I installed cunn, there's a error happened as shown below.
Error: Failed installing dependency: https://raw.githubusercontent.com/torch/rocks/master/cutorch-scm-1.rockspec - Build error: Failed building.
Do you have the dataset YTF? Could you privide it for me if you had?Thanks!
I use 80000 samples to train the jointed net. But when I finished the first CNN update, it is difficult to run the next step. This code seemly have a large amount of computation in computing the 'Affinity',
How can I solve this problem?
Is there anyway we can control the minimum number of cluster items?
E.g. Would like to avoid having a cluster with 2 items.
Thank you
Bruno
There is still probelm when I tried architecture for MNIST in models_def:
`online epoch # 0 [batchSize = 100] [learningRate = 0.01]
/home/ai/torch/install/bin/luajit: /home/ai/torch/install/share/lua/5.1/nn/Container.lua:67:
In 2 module of nn.Sequential:
In 1 module of nn.Sequential:
In 1 module of nn.Sequential:
/home/ai/torch/install/share/lua/5.1/nn/THNN.lua:110: Need input of dimension 4 and input.size[1] == 1 but got input to be of shape: [100 x 3 x 28 x 28] at /tmp/luarocks_cunn-scm-1-3260/cunn/lib/THCUNN/generic/SpatialConvolutionMM.cu:49
stack traceback:
[C]: in function 'v'
/home/ai/torch/install/share/lua/5.1/nn/THNN.lua:110: in function 'SpatialConvolutionMM_updateOutput'
...ai/torch/install/share/lua/5.1/nn/SpatialConvolution.lua:79: in function <...ai/torch/install/share/lua/5.1/nn/SpatialConvolution.lua:76>
[C]: in function 'xpcall'
/home/ai/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
/home/ai/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function </home/ai/torch/install/share/lua/5.1/nn/Sequential.lua:41>
[C]: in function 'xpcall'
/home/ai/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
/home/ai/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function </home/ai/torch/install/share/lua/5.1/nn/Sequential.lua:41>
[C]: in function 'xpcall'
/home/ai/torch/install/share/lua/5.1/nn/Container.lua:63: in function 'rethrowErrors'
/home/ai/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward'
train.lua:421: in function 'opfunc'
/home/ai/torch/install/share/lua/5.1/optim/sgd.lua:44: in function 'sgd'
train.lua:436: in function 'updateCNN'
train.lua:487: in main chunk
[C]: in function 'dofile'
...e/ai/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406670
WARNING: If you see a stack trace below, it doesn't point to the place where this error occurred. Please use only the one above.
stack traceback:
[C]: in function 'error'
/home/ai/torch/install/share/lua/5.1/nn/Container.lua:67: in function 'rethrowErrors'
/home/ai/torch/install/share/lua/5.1/nn/Sequential.lua:44: in function 'forward'
train.lua:421: in function 'opfunc'
/home/ai/torch/install/share/lua/5.1/optim/sgd.lua:44: in function 'sgd'
train.lua:436: in function 'updateCNN'
train.lua:487: in main chunk
[C]: in function 'dofile'
...e/ai/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406670`
I think that It may be my data is three-channel RGB data. I found FRGC is the three-channel RGB data. Then I resize my data to 32323. But there is still trouble when I tried architecture for FRGC:
==> online epoch # 0 [batchSize = 100] [learningRate = 0.01] loss: 0.037374177345863 /home/ai/torch/install/bin/luajit: bad argument #2 to '?' (out of range at /home/ai/torch/pkg/torch/generic/Tensor.c:913) stack traceback: [C]: at 0x7f453c6d9b30 [C]: in function '__index' train.lua:366: in function 'organize_samples' train.lua:422: in function 'opfunc' /home/ai/torch/install/share/lua/5.1/optim/sgd.lua:44: in function 'sgd' train.lua:436: in function 'updateCNN' train.lua:487: in main chunk [C]: in function 'dofile' ...e/ai/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x00406670
Do you know how to do? Or can you teach me how to build my network architecture. Especially, how to choose the parameters in the model_def? For example, how to choose the parameters of nInputPlanes, nOutputPlanes, nn.View, nn.Linear, nn.Normalize in the model_def?
How can I use your trained model to extract image(my dataset) features and then save them for further clustering?
Thank you in advance for the answer
Hi,
very thankful to your released code, it helps me a lot!
I have tried running on my own dataset.
If the image size is 3x32x32, all things go well.
However, when I set image size to 3x64x64, I got the error "/usr/local/bin/luajit: bad argument #2 to '?' (out of range at /root/torch/pkg/torch/generic/Tensor.c:913)"
Could you please help me with this problem?
I'm having trouble getting my custom data formatted into an appropriate hdf5 file, per your description. I manage to create the hdf5 file but I think I am having trouble with the 'labels' component of the hdf5 file:
stack traceback:
/home/ec2-user/distro/install/bin/luajit: train.lua:104: the class torch.LongTensor cannot be indexed
stack traceback:
[C]: in function '__newindex'
train.lua:104: in function 'cvt2TabelLabels'
train.lua:162: in main chunk
[C]: in function 'dofile'
...ser/distro/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x004064c0
In general I have a writer like this (where the labels are dummies for now for demonstration purposes):
import glob
import os.path
import time
from PIL import Image
import tempfile
import numpy as np
ims = [np.array(Image.open(f)) for f in glob.glob('*.jpg')]
def setup_hdf5_file(shape, name, **options):
f = h5py.File(name+'.h5', 'w')
f.create_dataset('data', shape, **options)
f.create_dataset('labels', (len(ims),1), **options)
return f
def save_images(f):
for i, im in enumerate(ims):
f['data'][i,...] = im.T
f['labels'][i,...] = int(1)
f.close()
def benchmark(name, **options):
tstart = time.time()
f = setup_hdf5_file((len(ims),3,256, 256), name, **options)
save_images(f)
tstop = time.time()
size = os.path.getsize(name+'.h5')
print("{0}: {1:.1f}s, {2}MB".format(name, tstop-tstart, size//1e6))
benchmark('data4torch')
it is not clear to me what format may be expected for the 'labels' (also are they even needed given the context of your use case? You don't mention this in the readme.
@jwyang hi, when i try to use this file deep_seminmf_nonlinear.m. I found it fail to run due to missing function deep_recon(). Could you help me with this problem? thx.
There is only class in my clustering result when I run my data(4 classes).
It is an urgent question. Please help me as soon as possible.
Thank you! I look forward to your reply as soon as possible!
Thank you!
Hi, I find this paper to be very interesting, Im part of a research program at the University of Barcelona and we want to try this network for Egocentric vision, we have our dataset in a folder structure were each folder represents a category, in our case daily activities.
I have no experience working with hd5f files, can you please help me with the code to create the h5 file?
Also do you think that JULE will work with egocentric vision?
Thank you for your time and good work
Where are the test datasets of other datasets (USPS, FRGC, YTF, etc..)
(apart from MNIST-test)?
hi,
I have run your code successfully! now, I want to know how to set the target number of clusters, the initial image labels and CNN parameters.
thank you!
Hi,
I'm trying to learn clustering on CIFAR-10 dataset. As a feature extraction model I'm using FRGC's model with minimal changes:
nInputLayers[0]
incrreased to 4 because I have RGBA images,local chunkSizes = {5, 5}
because I'm planning to change the size for each layer separately. So I'm using convolutional NN as following:module:add(backend.SpatialConvolution(nInputPlane, nOutputPlane, chunkSize, chunkSize, 1, 1, 0, 0))
But I'm catching the error:
[torch.LongStorage of size 2]
sigma: 0
/home/velkerr/PHd/libs/torch/install/bin/luajit: bad argument #2 to '?' (out of range at /home/velkerr/PHd/libs/torch/pkg/torch/generic/Tensor.c:704)
stack traceback:
[C]: at 0x7f0a0f314580
[C]: in function '__newindex'
./affinity/affinity.lua:39: in function 'compute'
train.lua:201: in function 'updateLabels'
train.lua:284: in function 'merge_label'
train.lua:482: in main chunk
[C]: in function 'dofile'
...libs/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405d00
As I understood, the error caused by too small dataset or too large batch size. I've experimented with different batchSizes (500, 100, 50) but I'm still seeing this error.
What I'm doing wrong? Thank you for help in advance.
Hello, I have two questions when I install the torch with ubuntu16.04.
Ubuntu 16.04 don't provide the libqt4-core & libqt4-gui in the origin package, so when I run the command
curl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-deps | bash
I failed and do not know whether it is affect the next steps.
When I skip the first step, and accomplish all the install steps. I run the first test code.
th train.lua -dataset USPS -eta 0.9
then the fault as follows:
/home/haichao/torch/distro-master/install/bin/luajit: ...torch/distro-master/install/share/lua/5.1/trepl/init.lua:3
89: module 'hdf5' not found:No LuaRocks module found for hdf5 no field package.preload['hdf5']
no file '/home/haichao/.luarocks/share/lua/5.1/hdf5.lua'
no file '/home/haichao/.luarocks/share/lua/5.1/hdf5/init.lua'
no file '/home/haichao/torch/distro-master/install/share/lua/5.1/hdf5.lua'
no file '/home/haichao/torch/distro-master/install/share/lua/5.1/hdf5/init.lua'
no file './hdf5.lua'
no file '/home/haichao/torch/distro-master/install/share/luajit-2.1.0-beta1/hdf5.lua'
no file '/usr/local/share/lua/5.1/hdf5.lua'
no file '/usr/local/share/lua/5.1/hdf5/init.lua'
no file '/home/haichao/.luarocks/lib/lua/5.1/hdf5.so'
no file '/home/haichao/torch/distro-master/install/lib/lua/5.1/hdf5.so'
no file '/home/haichao/torch/distro-master/install/lib/hdf5.so'
no file './hdf5.so'
no file '/usr/local/lib/lua/5.1/hdf5.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
I remember I have installed the hdf5 in other folder, and I have configured the .bashrc . So are there any steps I need to do ?
I tried to run this code in my data. But I am having trouble when run my data:
/home/ai/torch/install/bin/luajit: train.lua:382: cuda runtime error (59) : device-side assert triggered at /tmp/luarocks_cutorch-scm-1-3009/cutorch/lib/THC/generic/THCTensorCopy.c:18 stack traceback: [C]: in function 'indexCopy' train.lua:382: in function 'organize_samples' train.lua:422: in function 'opfunc' /home/ai/torch/install/share/lua/5.1/optim/sgd.lua:44: in function 'sgd' train.lua:436: in function 'updateCNN' train.lua:487: in main chunk [C]: in function 'dofile' ...e/ai/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x00406670
How to solve this problem?
I look forward to your response at your earliest convenience.
Thanks.
I have spent one day on configuring the environment for your torch code, when i go to the last step, i.e., install lua-knn, it failed. What's the problem? I'm so tired.
Many Thanks for your code.
And I still have a question that HOW TO GET THE CLUSTERED DATA AFTER TRAINING.
HI @jwyang
Great work,and I think this is what I am looking for !
But I found that it is a little difficult for me to train custom dataset .Could your please give some help?
My question is :
How to create training dataset?What 's the format of the datas?
Now i want train my custom images,I have no idea how to transform them to .h5 file
could you please give a step by step training instructions()?I think this will help other people too.
Thank you very much!!!!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.