Comments (8)
Hello, the results seem good, the accuracy should follow, could you join a sample of data_
?
I think the predictions are not in the expected shape..
Best,
from causaldiscoverytoolbox.
I have added the csv file that comes out after training the model.
Thanks for your help!
Best
from causaldiscoverytoolbox.
Here is also the way I loaded the data. I add this in cdt/data/loader.py
def load_ce_gauss(shuffle=False):
dirname = os.path.dirname(os.path.realpath(__file__))
data = read_causal_pairs('{}/resources/CE-Gauss_pairs.csv'.format(dirname), scale=False)
labels = pd.read_csv('{}/resources/CE-Gauss_targets.csv'.format(dirname)).set_index('SampleID')
if shuffle:
for i in range(len(data)):
if random.choice([True, False]):
labels.iloc[i, 0] = -1
buffer = data.iloc[i, 0]
data.iloc[i, 0] = data.iloc[i, 1]
data.iloc[i, 1] = buffer
return data, labels
def load_ce_multi(shuffle=False):
dirname = os.path.dirname(os.path.realpath(__file__))
data = read_causal_pairs('{}/resources/CE-Multi_pairs.csv'.format(dirname), scale=False)
labels = pd.read_csv('{}/resources/CE-Multi_targets.csv'.format(dirname)).set_index('SampleID')
if shuffle:
for i in range(len(data)):
if random.choice([True, False]):
labels.iloc[i, 0] = -1
buffer = data.iloc[i, 0]
data.iloc[i, 0] = data.iloc[i, 1]
data.iloc[i, 1] = buffer
return data, labels
def load_ce_net(shuffle=False):
dirname = os.path.dirname(os.path.realpath(__file__))
data = read_causal_pairs('{}/resources/CE-Net_pairs.csv'.format(dirname), scale=False)
labels = pd.read_csv('{}/resources/CE-Net_targets.csv'.format(dirname)).set_index('SampleID')
if shuffle:
for i in range(len(data)):
if random.choice([True, False]):
labels.iloc[i, 0] = -1
buffer = data.iloc[i, 0]
data.iloc[i, 0] = data.iloc[i, 1]
data.iloc[i, 1] = buffer
return data, labels
from causaldiscoverytoolbox.
Hello,
Whoops I forgot to ask if you had the labels as well ?
from causaldiscoverytoolbox.
oh yeah I have
which I downloaded from https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/3757KX
Thanks for the reply
Best
from causaldiscoverytoolbox.
Thanks for getting back to me quickly,
There seems to be an issue with your accuracy computation ; i got an accuracy of .72 on this dataset:
import pandas as pd
import numpy as np
from sklearn.metrics import average_precision_score, accuracy_score
preds = pd.read_csv('res2_gauss.csv')
labels = pd.read_csv('CE-Gauss_targets.csv')
print(labels.shape, preds.shape)
print(labels.columns, preds.columns)
# Returns :(300, 2) (300, 2)
# Returns : Index(['SampleID', 'Target'], dtype='object') Index(['SampleID', 'Predictions'], dtype='object')
average_precision_score(labels.Target, preds.Predictions) ## Equals to AUPR
# Returns :0.8027886920926466
preds.loc[preds.Predictions > 0, 'Predictions'] = 1
preds.loc[preds.Predictions < 0, 'Predictions'] = -1
accuracy_score(labels.Target,preds.Predictions)
# Returns : 0.7233333333333334
From my point of view, accuracy however might not be the best metric for evaluating causal algorithms: The confidence of an algorithm has to be taken into account, thus giving the possibility of not committing into a prediction if the prediction is not certain (Not answering is better that giving a wrong causal direction).
Best regards,
Diviyan
from causaldiscoverytoolbox.
Thanks a lot
Sorry, I was an idiot ...
I forgot to increment the idx variable
Thanks for your help
Sorry for the inconvenience
from causaldiscoverytoolbox.
No issues, glad I could help you!
I'll be closing this issue, have a good day !
from causaldiscoverytoolbox.
Related Issues (20)
- SID and SHD do not get the same results as the author of SID HOT 3
- SID error HOT 1
- Is it possible to insert prior knowledge before the causal graph creation? HOT 3
- [BUG] CGNN (Causal Graph Generation) + Usage of multiprocessing with pytorch HOT 1
- R Package (k)pcalg/RCIT is not available. RCIT has to be installed from https://github.com/Diviyan-Kalainathan/RCIT HOT 6
- [BUG] cdt.data.load_dataset('sachs') + one of the returned objects, 'target', is inconsistent with the paper(Sachs,etc 2005) HOT 1
- [fileNotFoundError: [Errno 2]] cdt.causality.graph.LiNGAM + No such file or directory: 'C:\\anaconda\\lib\\site-packages\\cdt\\utils\\R_templates\\test_import.R' HOT 1
- GIES targets and target.index parameter needs to be exposed HOT 2
- [BUG] orient_graph removes some of the edges
- [Question] What does the causal score in the pairwise model really indicate?
- ImportError: R Package (k)pcalg/RCIT is not available. HOT 3
- [BUG] CGNN run() Wrong way to calculate the score HOT 1
- FloatingPointError: The system is too ill-conditioned for this solver. The system is too ill-conditioned for this solver HOT 1
- Help! HOT 1
- Can PC algorithm be used for causal discovery under mixed types of data๏ผ
- ImportError: R Package pcalg is not available
- [BUG] autoset_settings() fails with MIG GPU
- CCDr algorithm execution error
- CCDr Algorithm + estimate.dag in R Script, Error in weights HOT 3
- CGNN running time is too long
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from causaldiscoverytoolbox.