DECI demo simulation error about project-azua HOT 8 CLOSED

@microsoftopensource @msftgits

from project-azua.

@ae-foster any help? I just can't run our deci demo script, you can refer to above errors.
Could you please give us data such that we can just load it from and move with other methods ?

from project-azua.

Hi @saichaitanyamolabanti , apologies that you are having difficulty with the CSuite data generation script. Unfortunately, we were not able to reproduce the problem in our internal repository. We aim to release the latest version of our internal code soon, at which point the problem should be resolved. In the meantime, we are exploring the possibility of releasing the raw data so you can bypass running simulate.py and get on with running some experiments. We will let you know as soon as we find out whether this is possible.

from project-azua.

Hey @ae-foster I did a couple of edits on the code and got the results, but don't know where I went wrong because of the observations I made, please look into below and let me know any suggestions:

scenario-1: to make simulate.py work
observed, there were problems with pairplots and assert statements atleast in my case, so removed them in the scripts to generate data from simulate.py script
pyro_utils.py file - Line 206:
"""
if discrete:
sns.pairplot(pd.concat(df_list), hue="intervention", plot_kws=dict(), grid_kws=dict(), kind="hist")
else:
sns.pairplot(pd.concat(df_list), hue="intervention", plot_kws=dict(alpha=0.05, size=0.7), grid_kws=dict())
"""
utils file - Line:69 #assert intervention_samples.shape == reference_samples.shape == original_samples.shape
able to generate data, but able to see interventions file in all folders as NaN values, you can see below.

Scenario-2: generating causal graph
even though, intervention dataset was almost Nan filled, I wanna give a try in generating causal graph.
load_data has some problems again with assert statements. so commented -
if is_counterfactual:
assert (conditioning_values.shape[0] == intervention_samples.shape[0]), "Counterfactual data expects the conditioning to be of equivalent shape as the interventional data.
assert (conditioning_mask == 1).all(), "Counterfactual data expects the conditioning to be full."
now, data got loaded and sample folder graphs got generated

Scenario-3: csuite_large_backdoor generated not a DAG
experimented in generating csuite_large_backdoor causal graph but able to see a error message "not a DAG generated"
commented #assert (np.trace(scipy.linalg.expm(adj_mat)) - self.num_nodes) == 0, "Generate non DAG graph" to generate forecfully, got below graph

could you please provide me suggestions on which way I can try better, because there's definitely something wrong. If it's about the dataset, could we please attach a valid dataset to just mitigate all these effects ?

from project-azua.

Hi @saichaitanyamolabanti - thanks for your patience. We have agreed as a team to release the data into the public domain, which should resolve the issues you're having. We need to go through an internal process to have final agreement on releasing the synthetic data. We will update you when it is released!

from project-azua.

thanks @ae-foster, i'll wait for you guys responding back on the synthetic data for atleast one use-case. btw, could you please tell the above causal graph is incorrect right? any suggestions on my previous comment ?

from project-azua.

For the causal DAG generated, it could be caused by incorrect data, or alternatively a problem with optimization. Augmented Lagrangian training can take a long time to converge. Did you run using the same hyperparameter settings that were recommended (including the auglag parameters)?

from project-azua.

Referring back to the original question, which was about access to the synthetic data, I am happy to let you know that you can now access our "CSuite" synthetic datasets here https://github.com/microsoft/csuite without having to run any data generation code! This repository contains all the documentation about these datasets, how to download and use them, etc.

from project-azua.