The causaldiscoverytoolbox from fentechsolutions

GS/MMPC algorithm removed all non-related variable

Hi there,
This is my very first post on Github so parden me if this is a simple fix to my problem.

I am currently exploring the different graph inference algorithms on my own dataset. One thing I realized is that when implementing algorithm like GS/MMPC the uncorrelated variable is not shown(automatically removed) in the nx.adjacency_matrix(output).todense() command.

For example I fed 50 variables into the MMPC algorithm and out of which only 35 variables are correlated and 15 are not. The nx.adjacency_matrix(output).todense() will only spit out the matrix for 35 variables and I do not know the variables that are uncorrelated and removed.

While CDT does provide plotting option I do perfer to use package like Graphviz. Thus it will be helpful to acquire the matrix for all 50 input variables instead.

Is there a way for me to obtain such matrix?
Thank you in advance!

load_dataset in cdt.data is "random"

Hello. I am currently working on multiple Pairwise algorithms, and found the following problems.
Whenever i load the TCEP data as follows,
data, labels = load_dataset('tuebingen')

the apparent order of the pairs is different.
I had this problem because of the following: to test threshold-dependent algos (such as ANM, GNN) i had a single jupyter file in which i would have a single instance of data, labels loaded, and then threshold and compute metrics on the predictions of different pre-recorded scores.

But each time i would call data, labels = load_dataset('tuebingen') and then compute_metrics(preds,labels) , they would all change ?!
I was quite worried when the accuracy of cdt implementations of RCC,ANM and IGCI were as low as 40% on TCEP ...

Thank you for this wonderful work by the way!

Reproducing Jarfo results on Kaggle challenge data

Hi,
I've been trying to reproduce Jarfo results from the Kaggle challenge (2013) with its final dataset from http://www.causality.inf.ethz.ch/CEdata/, i.e., with CEfinal_train_text.zip and CEfinal_test_text.zip.

It seems to output very different results compared to the original code, while the learning parameters and the used features seems to be the same.

import numpy as np
import pandas as pd
from cdt.causality.pairwise.Jarfo import Jarfo
from cdt.utils.io import read_causal_pairs
from cdt import SETTINGS

SETTINGS.GPU = False
SETTINGS.NJOBS = 1

train_data = read_causal_pairs(".../CEfinal_train_pairs.csv")
train_target = pd.read_csv(".../CEfinal_train_target.csv").iloc[:,:2].set_index("SampleID")
test_data = read_causal_pairs(".../CEfinal_test_pairs.csv")
test_target = pd.read_csv(".../CEfinal_test_target.csv").iloc[:,:2].set_index("SampleID")

j = Jarfo()
j.fit(train_data, train_target)

jp = j.predict(test_data)

acc = np.mean(jp * test_target.values > 0)
print(acc)

0.25491827465325406

I've tested it with
python 3.7.3
cdt 0.5.14

Thank you in advance for any hint.
Best
Tom

RuntimeError: Expected object of backend CUDA but got backend CPU for argument #3 'index'

I get the following error while trying to fit the NCC model using a GPU

obj.fit(data, labels, epochs=1000, batch_size=32, learning_rate=0.01, device = cdt.SETTINGS.default_device)

using other data sets - pandas data frame

Hi @Diviyan-Kalainathan

How do we use this package with other data sets.
Do we just create a pandas data frame and use the load_data function?

Many thanks,
Best,
Andrew

All labels in TCEP are equal

Hi,
I was doing some more experiments using CGNN based algorithms on TCEP, and I couldn't replicate the results. I also got only 60% accuracy using the hyperparameters that got me 75%~ a few months ago. I couldn't find the cause of this, but when I printed the labels they were all 1. Is this a mistake?

Edit: I checked the commit warnings, apparently you did permute all pairs so that the labels is 1, and still is the right label. I'll compare with other instances of the dataset but I'm now reassured!

[enhancement] Causal Estimation Code?

Hey. Maybe this is a dumb question, but has any thought been put into performing causal estimation in a graph in this package? It's great to have a package like this that does causal discovery, but I'd also like to have the functionality that can generate the conditional probability estimates over the causal graph, as well as a method for generating answers to Pr(Y|do(X=x)) for x and y being continuous and discrete. I've found the ability to do this kind of general calculation to be absent from most python packages. I currently use the pomegranate package to build a causal bayes net for my discrete variables (and I discretize continuous variables when I have them), and then use another package to determine backdoor or frontdoor based variables I can condition on, and then use the adjustment formula. It just all seems to be glued together however because all of this doesn't exist in the same package, especially for graphs with all discrete variables, which I would think is a simpler case than the continuous case.

I know that 'DoWhy' exists, but it only allows for estimation of cause and effect for the direct effect case, where a treatment variable directly impacts an outcome variable.

Docker image usage

Hi Diviyan,

Thank you for your project!

I am not very familiar with Docker, but I think it's worth trying.
I did the following:

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
divkal/cdt-py3.6    0.4.0               2af9c5c51ac1        2 weeks ago         3.34GB
$ docker run -it --entrypoint /bin/bash 2af9c5c51ac1
root@35b4b45ecfc8:/#

But it seems there is no python or conda in the image:

root@35b4b45ecfc8:/# python
bash: python: command not found
root@35b4b45ecfc8:/# conda
bash: conda: command not found

I believe it must be my fault. Could you please give some pointers?

Best,
Abel

FSGNN

Hi,

When trying to run:

Fsgnn = FSGNN(train_epochs=1000, test_epochs=500, l1=0.1, batch_size=1000)

From the example notebook of the LUCAS data (with my own data, though) I get the following error:

TypeError: __init__() got an unexpected keyword argument 'batch_size'
I wonder whether there are some changes on the definition of the FSGNN subject. Second, I would like to know how does the package manage categorical data, as, from what I have noticed, the values are converted into floats at some point in the objects generation.

Thanks in advance,
Sergio

In cdt.causality.pairwise , the "RCC" example uses "Jarfo"

Hi,
minor problem, just a confusing example I found through experiments, see below

class RCC(PairwiseModel):
    """Randomized Causation Coefficient model. 2nd approach in the Fast
    Causation challenge.
    **Description:** The Randomized causation coefficient (RCC) relies on the
    projection of the empirical distributions into a RKHS using random cosine
    embeddings, then classfies the pairs using a random forest based on those
    features.
    **Data Type:** Continuous, Categorical, Mixed
    **Assumptions:** This method needs a substantial amount of labelled causal
    pairs to train itself. Its final performance depends on the training set
    used.
    Args:
        rand_coeff (int): number of randomized coefficients
        nb_estimators (int): number of estimators
        nb_min_leaves (int): number of min samples leaves of the estimator
        max_depth (): (optional) max depth of the model
        s (float): scaling
        njobs (int): number of jobs to be run on parallel (defaults to ``cdt.SETTINGS.NJOBS``)
        verbose (bool): verbosity (defaults to ``cdt.SETTINGS.verbose``)
    .. note::
       Ref : Lopez-Paz, David and Muandet, Krikamol and Schölkopf, Bernhard and Tolstikhin, Ilya O,
       "Towards a Learning Theory of Cause-Effect Inference", ICML 2015.
    Example:
        >>> from cdt.causality.pairwise import RCC
        >>> import networkx as nx
        >>> import matplotlib.pyplot as plt
        >>> from cdt.data import load_dataset
        >>> from sklearn.model_selection import train_test_split
        >>> data, labels = load_dataset('tuebingen')
        >>> X_tr, X_te, y_tr, y_te = train_test_split(data, labels, train_size=.5)
        >>>
        >>> obj = Jarfo()
        >>> obj.fit(X_tr, y_tr)
        >>> # This example uses the predict() method
        >>> output = obj.predict(X_te)
        >>>
        >>> # This example uses the orient_graph() method. The dataset used
        >>> # can be loaded using the cdt.data module
        >>> data, graph = load_dataset('sachs')
        >>> output = obj.orient_graph(data, nx.DiGraph(graph))
        >>>
        >>> # To view the directed graph run the following command
        >>> nx.draw_networkx(output, font_size=8)
        >>> plt.show()
    """

Example

Hey man in trying your example, I have run into an error. I am not sure what to make of it, as it might be device related. I will include the message here and the file link.

https://github.com/snowde/firmai.github.io/blob/master/Discovery_LUCAS.ipynb


# So the question is, if you only have the data can you find the
# structure of the graph
from cdt.independence.graph import FSGNN

Fsgnn = FSGNN()

start_time = time.time()
ugraph = Fsgnn.predict(data, train_epochs=2000, test_epochs=1000, threshold=5e-4, l1=0.01)
print("--- Execution time : %4.4s seconds ---" % (time.time() - start_time))
nx.draw_networkx(ugraph, font_size=8) # The plot function allows for quick visualization of the graph.
plt.show()
# List results
pd.DataFrame(list(ugraph.edges(data='weight')))

`---------------------------------------------------------------------------
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 350, in call
return self.func(*args, **kwargs)
File "/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/joblib/parallel.py", line 131, in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/joblib/parallel.py", line 131, in
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/cdt/independence/graph/model.py", line 43, in run_feature_selection
return self.predict_features(df_features, df_target, **kwargs)
File "/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/cdt/independence/graph/FSGNN.py", line 79, in predict_features
x = th.FloatTensor(scale(df_features.as_matrix())).to(device)
AttributeError: 'torch.FloatTensor' object has no attribute 'to'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/dereksnow/anaconda/envs/py36/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 359, in call
raise TransportableException(text, e_type)
joblib.my_exceptions.TransportableException: TransportableException

AttributeError Sat Jun 30 16:19:48 2018
PID: 92373 Python 3.6.3: /Users/dereksnow/anaconda/envs/py36/bin/python
...........................................................................
/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/joblib/parallel.py in call(self=<joblib.parallel.BatchedCalls object>)
126 def init(self, iterator_slice):
127 self.items = list(iterator_slice)
128 self.size = len(self.items)
129
130 def call(self):
--> 131 return [func(*args, **kwargs) for func, args, kwargs in self.items]
self.items = [(<bound method FeatureSelectionModel.run_feature...n of <cdt.independence.graph.FSGNN.FSGNN object>>, ( Allergy Anxiety Genetics Peer_Pressure...0.858699 -1.037579

[500 rows x 11 columns], 'Allergy', 0), {'l1': 0.01, 'test_epochs': 1000, 'train_epochs': 2000})]
132
133 def len(self):
134 return self._size
135

...........................................................................
/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/joblib/parallel.py in (.0=<list_iterator object>)
126 def init(self, iterator_slice):
127 self.items = list(iterator_slice)
128 self.size = len(self.items)
129
130 def call(self):
--> 131 return [func(*args, **kwargs) for func, args, kwargs in self.items]
func = <bound method FeatureSelectionModel.run_feature...n of <cdt.independence.graph.FSGNN.FSGNN object>>
args = ( Allergy Anxiety Genetics Peer_Pressure...0.858699 -1.037579

[500 rows x 11 columns], 'Allergy', 0)
kwargs = {'l1': 0.01, 'test_epochs': 1000, 'train_epochs': 2000}
132
133 def len(self):
134 return self._size
135

...........................................................................
/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/cdt/independence/graph/model.py in run_feature_selection(self=<cdt.independence.graph.FSGNN.FSGNN object>, df_data= Allergy Anxiety Genetics Peer_Pressure...0.858699 -1.037579

[500 rows x 11 columns], target='Allergy', idx=0, **kwargs={'l1': 0.01, 'test_epochs': 1000, 'train_epochs': 2000})
38 list_features = list(df_data.columns.values)
39 list_features.remove(target)
40 df_target = pd.DataFrame(df_data[target], columns=[target])
41 df_features = df_data[list_features]
42
---> 43 return self.predict_features(df_features, df_target, **kwargs)
self.predict_features = <bound method FSGNN.predict_features of <cdt.independence.graph.FSGNN.FSGNN object>>
df_features = Anxiety Genetics Peer_Pressure Attentio...0.858699 -1.037579

[500 rows x 10 columns]
df_target = Allergy
0 -0.266076
1 -0.579084
2 -0...8 -0.064685
499 -0.638704

[500 rows x 1 columns]
kwargs = {'l1': 0.01, 'test_epochs': 1000, 'train_epochs': 2000}
44
45 def predict(self, df_data, threshold=0.05, **kwargs):
46 """Get the skeleton of the graph from raw data.
47

...........................................................................
/Users/dereksnow/anaconda/envs/py36/lib/python3.6/site-packages/cdt/independence/graph/FSGNN.py in predict_features(self=<cdt.independence.graph.FSGNN.FSGNN object>, df_features= Anxiety Genetics Peer_Pressure Attentio...0.858699 -1.037579

[500 rows x 10 columns], df_target= Allergy
0 -0.266076
1 -0.579084
2 -0...8 -0.064685
499 -0.638704

[500 rows x 1 columns], nh=20, idx=0, dropout=0.0, activation_function=<class 'torch.nn.modules.activation.ReLU'>, lr=0.01, l1=0.01, train_epochs=2000, test_epochs=1000, device='cpu', verbose=False, nb_runs=3)
74 activation_function=th.nn.ReLU, lr=0.01, l1=0.1, # batch_size=-1,
75 train_epochs=1000, test_epochs=1000, device=None,
76 verbose=None, nb_runs=3):
77 """For one variable, predict its neighbours."""
78 device, verbose = SETTINGS.get_default(('device', device), ('verbose', verbose))
---> 79 x = th.FloatTensor(scale(df_features.as_matrix())).to(device)
x = undefined
df_features.as_matrix.to = undefined
device = 'cpu'
80 y = th.FloatTensor(scale(df_target.as_matrix())).to(device)
81 out = []
82 for i in range(nb_runs):
83 model = FSGNN_model([x.size()[1], nh, 1],

AttributeError: 'torch.FloatTensor' object has no attribute 'to'

"""

The above exception was the direct cause of the following exception:

TransportableException Traceback (most recent call last)
~/anaconda/envs/py36/lib/python3.6/site-packages/joblib/parallel.py in retrieve(self)
698 if getattr(self._backend, 'supports_timeout', False):
--> 699 self._output.extend(job.get(timeout=self.timeout))
700 else:

~/anaconda/envs/py36/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
643 else:
--> 644 raise self._value
645

TransportableException: TransportableException