hemile / kenn-pytorch Goto Github PK

Also sigmoid transformation on clause weights for lukasiewicz

Hi,

I found an unexpected behavior when the KnowledgeEnhancer is instanced with the parameter save_training_data = True

The error can be seen by running this notebook

I also found the solution: at line 49 of KnowledgeEnhancer.py the instruction weights.append(enhancer.clause_weight.numpy())[0][0] has to be modified in this way: weights.append(enhancer.conorm_boost.clause_weight.numpy())

Tell me if this is a correct solution

Hi,

I found a strange behavior of the code. After solving the other issue I posted as explained inside the issue, I used the code to see the deltas returned by the KnowledgeEnhancer (as you can see in the notebook).

I found that if I manually sum the returned deltas with the original preactivation I obtain a different vector w.r.t. the one returned by the KnowledgeEnhancer. I used the debugger to inspect the behavior and I found that in line 98 of ClauseEnhancer.py the variables scattered_delta and delta contain the same values but reversed. I also checked that scattered_delta is the summed variable to obtain the enhanced prediction and delta is the returned variable when save_training_data = True.

Why these variables are reversed?

I think that the correct one is scattered_delta, is it correct?

Thanks

Hi there,

I experience OOM issues when I execute the code on a GPU. You have already mentioned this as a #TODO in the file Clause_Enhancer.py.
I solved this issue by using torch_scatter.scatter_add() instead of torch.stack(scatter_deltas_list).sum(dim=0) to build the sum over the changes by the clause enhancers. Is there any particular reason why you are not using a scatter operation? Did you experience it to be slower in terms of runtimes or does it not fulfil the same functionality as the solution based on torch.stack(...)?

Also, I got the problem that not all tensors were on the same device when executing on GPU. I solved this by registering the relevant tensors as buffers in ClaueEnhancer.py:

self.register_buffer('signs', signs)
self.register_buffer('gather_literal_indices', gather_literal_indices)
self.register_buffer('scatter_literal_indices', scatter_literal_indices)

Thank you,
Luisa