Hi, my friend, thank you for your work, I try to debug and find a tiny bug:

in file - 0_gen_sampled_data.py:

unique_cate_id = np.concatenate(
    (ad['cate_id'].unique(), log['cate'].unique()))

lbe.fit(unique_cate_id)

in file - 2_gen_dsin_input.py:

data = pd.merge(sample_sub, user, how='left', on='userid', )
data = pd.merge(data, ad, how='left', on='adgroup_id')

here merge method lost some data(cate_id, and brand)

sparse_feature_list = [SingleFeat(feat, data[feat].nunique(
) + 1) for feat in sparse_features + ['cate_id', 'brand']]

so here data['brand'].nunique() is small than input data index.

and I log all unique input brand number, and update the fd, then code can run without error.

from dsin.

Hi, my friend, thank you for your work, I try to debug and find a tiny bug:

in file - 0_gen_sampled_data.py:

unique_cate_id = np.concatenate(
    (ad['cate_id'].unique(), log['cate'].unique()))

lbe.fit(unique_cate_id)

in file - 2_gen_dsin_input.py:

data = pd.merge(sample_sub, user, how='left', on='userid', )
data = pd.merge(data, ad, how='left', on='adgroup_id')

here merge method lost some data(cate_id, and brand)

sparse_feature_list = [SingleFeat(feat, data[feat].nunique(
) + 1) for feat in sparse_features + ['cate_id', 'brand']]

so here data['brand'].nunique() is small than input data index.

and I log all unique input brand number, and update the fd, then code can run without error.

Hi, I met the same problem, could you tell us how to fix the bug?

Sorry for the late reply, I am not sure whether it is ok or not.

1. log the dimension in file 0_gen_...:

   pd.to_pickle({
   'cate_id': SingleFeat('cate_id', len(np.unique(unique_cate_id)) + 1),
   'brand': SingleFeat('brand', len(np.unique(unique_brand)) + 1),
   },
   '../model_input/dsin_fd_cate_brand_' + str(FRAC) + '.pkl')

2. update input fd in train_dsin.py:

   cate_brand_fd = pd.read_pickle('../model_input/dsin_fd_cate_brand_' +
   str(FRAC) + '.pkl')

   fd['sparse'][13] = cate_brand_fd['cate_id']
   fd['sparse'][14] = cate_brand_fd['brand']

3. rerun the script.

from dsin.

please update your code to the latest and run them on the environment written on
https://github.com/shenweichen/DSIN#operating-environment

from dsin.

I run this code on tf-cpu1.4.0, cause my cuda is 10.0 and cannot run on gpu.
Do you know what does this error info mean?

from dsin.

have you run your code on python3.6?

from dsin.

right

from dsin.

check your code is up to date with the the latest commit

from dsin.

It is the latest commit with deepctr==0.4.1

from dsin.

yes i suggest you to clone the whole repo and re-run again

from dsin.

from dsin.

Hi, my friend, thank you for your work, I try to debug and find a tiny bug:

in file - 0_gen_sampled_data.py:

unique_cate_id = np.concatenate(
    (ad['cate_id'].unique(), log['cate'].unique()))

lbe.fit(unique_cate_id)

in file - 2_gen_dsin_input.py:

data = pd.merge(sample_sub, user, how='left', on='userid', )
data = pd.merge(data, ad, how='left', on='adgroup_id')

here merge method lost some data(cate_id, and brand)

sparse_feature_list = [SingleFeat(feat, data[feat].nunique(
) + 1) for feat in sparse_features + ['cate_id', 'brand']]

so here data['brand'].nunique() is small than input data index.

and I log all unique input brand number, and update the fd, then code can run without error.

I have also encountered this problem. Could you please tell me how to modify this bug in detail？

from dsin.

Hi, my friend, thank you for your work, I try to debug and find a tiny bug:

in file - 0_gen_sampled_data.py:

unique_cate_id = np.concatenate(
    (ad['cate_id'].unique(), log['cate'].unique()))

lbe.fit(unique_cate_id)

in file - 2_gen_dsin_input.py:

data = pd.merge(sample_sub, user, how='left', on='userid', )
data = pd.merge(data, ad, how='left', on='adgroup_id')

here merge method lost some data(cate_id, and brand)

sparse_feature_list = [SingleFeat(feat, data[feat].nunique(
) + 1) for feat in sparse_features + ['cate_id', 'brand']]

so here data['brand'].nunique() is small than input data index.

and I log all unique input brand number, and update the fd, then code can run without error.

Hi, I met the same problem, could you tell us how to fix the bug?

from dsin.

Hi, my friend, thank you for your work, I try to debug and find a tiny bug:

in file - 0_gen_sampled_data.py:

unique_cate_id = np.concatenate(
    (ad['cate_id'].unique(), log['cate'].unique()))

lbe.fit(unique_cate_id)

in file - 2_gen_dsin_input.py:

data = pd.merge(sample_sub, user, how='left', on='userid', )
data = pd.merge(data, ad, how='left', on='adgroup_id')

here merge method lost some data(cate_id, and brand)

sparse_feature_list = [SingleFeat(feat, data[feat].nunique(
) + 1) for feat in sparse_features + ['cate_id', 'brand']]

so here data['brand'].nunique() is small than input data index.

and I log all unique input brand number, and update the fd, then code can run without error.

Hi, I met the same problem, could you tell us how to fix the bug?

Sorry for the late reply, I am not sure whether it is ok or not.

1. log the dimension in file 0_gen_...:
   pd.to_pickle({
   'cate_id': SingleFeat('cate_id', len(np.unique(unique_cate_id)) + 1),
   'brand': SingleFeat('brand', len(np.unique(unique_brand)) + 1),
   },
   '../model_input/dsin_fd_cate_brand_' + str(FRAC) + '.pkl')
2. update input fd in train_dsin.py:
   cate_brand_fd = pd.read_pickle('../model_input/dsin_fd_cate_brand_' +
   str(FRAC) + '.pkl')
   fd['sparse'][13] = cate_brand_fd['cate_id']
   fd['sparse'][14] = cate_brand_fd['brand']
3. rerun the script.

thank you too much, please let me try

from dsin.

sorry for this mistake, we are planning to refactor our code in the future.
I think this error can be fixed by using

sparse_feature_list = [SingleFeat(feat, data[feat].max(
    ) + 1) for feat in sparse_features + ['cate_id', 'brand']]

instead of

from dsin.