evaluate LiNLAM and compare to FCI

[Alex-Markham]

We need to complete this line in the fit_linlam.py script so that pag is an adjacency matrix of type np.ndarray

[smilesun]

the only line that GIN-LINLAM could add non-directed edge is
https://github.com/py-why/causal-learn/blob/08655cda0d065d0962e47366844001808b53f5ba/causallearn/search/HiddenCausal/GIN/GIN.py#L142

[smilesun]

py-why/causal-learn#209

[smilesun]

@Alex-Markham since GIN output causal order, can we use this to evaluate the performance?

[Alex-Markham]

Good idea. If we can find some code that does this, then I think it's worth using that. However, if we have to implement something ourselves, I don't think it's worth focusing on at the moment. I think evaluation metrics based on the causal order are a bit questionable (which is also true of SHD evaluation), and I'm also not sure how to handle the fact that a given DAG (and even worse, more general graphs) typically represents an equivalence class of causal orders.

[smilesun]

the solution is here:
https://github.com/marrlab/causalspyne/blob/main/causalspyne/ancestral_acc.py

what do you think?

[Alex-Markham]

Okay, yeah, I think this is good (especially for now). Nice work!

[Alex-Markham]

So now we can:

• 1. change the rule and script fit_linlam to output the general graph and the causal order
• 2. add a rule and script for ancestral accuracy
• 3. if it's not too hard, add a rule that extracts a causal order from a PAG, so we can also get ancestral acc from the FCI output

[smilesun]

I am currently working on the unit test for ancestral graph for causal order, when i am done, i will put the test link here.

[smilesun]

in case you have time, you can work in parallel for the step 2. I think we can transform the current unit test i am working on in to step 1 you posted.

[smilesun]

done here: marrlab/causalspyne#29,

example for step 1: https://github.com/marrlab/causalspyne/blob/95b065640895b337efbee4bdd2aee2d7ab8bce3e/examples/gin.py#L35

It might be faster if you start some draft for step 1 and step 2 which i can fill in. Otherwise I will look into that tomorrow.

I will go to sport now. @Alex-Markham

[Alex-Markham]

@smilesun should be able to do 1 and 2 pretty quickly. I'll have some time to work on it soon, and then I'll let you know when it's done

[Alex-Markham]

@smilesun it's running into some dimension mismatch and indexing errors. Do you think you could take a look? I don't really have more time to work on this today

[smilesun]

ok, I have less than 1 hour today. If I can not fix it, will continue on Monday. So i run uv run snakemake all?

[Alex-Markham]

yup, exactly

[smilesun]

do you have a quick anser to https://github.com/Alex-Markham/causalspyne-expt/pull/3/files#r1893933238

[Alex-Markham]

should be done c49a278

[smilesun]

debugging

MissingOutputException in rule generate_data in file /home/playtime/causa
lspyne-expt/workflow/Snakefile, line 12:                                 
Job 4  completed successfully, but some output files are missing. Missing
 files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
results/datasets/method=standard/graph_size=small/seed=0/samp_size=1000/node_names.pkl (missing locally, parent dir contents: ground_truth_dag.csv, ground_truth_dag.pkl, data_subdag_v0uv1_ind_1.csv, adj.csv, hidden_nodes.csv, data.csv)

[smilesun]

same error with --latency-wait=15

[Alex-Markham]

my last commit fixed it for me

[Alex-Markham]

there was another problem, but 74cbff5 fixes that

[smilesun]

cool, thanks! I will check the results and see if I can check if the causal order is oriented (low to high vs high to low ) correctly

[smilesun]

there was another problem, but 74cbff5 fixes that

should be fixed in your commit [74cbff5] for this error

TypeError: 'MatDAG' object is not subscriptable

[smilesun]

will see what went wrong

['X5', 'X0', 'X1', 'X4', 'X2', 'X6', 'X7']
Traceback (most recent call last):
  File "/home/playtime/causalspyne-expt/.venv/lib/python3.13/site-package
s/numpy/lib/_npyio_impl.py", line 1630, in savetxt
    v = format % tuple(row) + newline
        ~~~~~~~^~~~~~~~~~~~
TypeError: %d format: a real number is required, not numpy.str_

The above exception was the direct cause of the following exception:

[smilesun]

The above exception was the direct cause of the following exception:     
                                                                         
Traceback (most recent call last):                                       
  File "/home/playtime/causalspyne-expt/.snakemake/scripts/tmp3dg_irf_.fi
t_linlam.py", line 22, in <module>                                       
    np.savetxt(snakemake.output["causal_order"], obs_causal_order, delimi
ter=",", fmt="%d")                                                       
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^                                                       
  File "/home/playtime/causalspyne-expt/.venv/lib/python3.13/site-package
s/numpy/lib/_npyio_impl.py", line 1632, in savetxt
    raise TypeError("Mismatch between array dtype ('%s') and "
                    "format specifier ('%s')"
                    % (str(X.dtype), format)) from e
TypeError: Mismatch between array dtype ('<U2') and format specifier ('%

[Alex-Markham]

just need to figure out the appropriate dtype/formatting for L20, when the causal order is saved

[Alex-Markham]

I can try using json instead of np.save; that might be the quickest fix

[smilesun]

i tried something here: #4

[smilesun]

with error : id_ancestor = true_dag._dict_node_names2ind[ancestor]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: np.int64(0)

[smilesun]

debugging using this line
marrlab/causalspyne@9e61d61

will go to another meeting now

[Alex-Markham]

after bugfix marrlab/causalspyne@c4b7b86 I think I've gotten the rest of it working now