I've encountered a strange issue: abess() does not te

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

[Bug] No termination within reasonable time for Poisson regression in a specific case about abess HOT 6 OPEN

abess-team commented on July 20, 2024

[Bug] No termination within reasonable time for Poisson regression in a specific case

from abess.

Comments (6)

Mamba413 commented on July 20, 2024

Thanks. I can reproduce this on my laptop. It may be caused by the extremely large value of the deviance when setting support.size = 0:14.

> abess(x, y, tune.type = "gic", family = "poisson", support.size = 0:13)
Call:
abess.default(x = x, y = y, family = "poisson", tune.type = "gic",  support.size = 0:13)

  support.size           dev          GIC
1             0 -7.581848e+14 -1.51637e+15
2             1 -2.298525e+34 -4.59705e+34
3             2 -2.298525e+34 -4.59705e+34
4             3 -2.298525e+34 -4.59705e+34
5             4 -2.298525e+34 -4.59705e+34
6             5 -2.298525e+34 -4.59705e+34
7             6 -2.298525e+34 -4.59705e+34
8             7 -2.298525e+34 -4.59705e+34
9             8 -2.298525e+34 -4.59705e+34
10            9 -2.298525e+34 -4.59705e+34
11           10 -2.298525e+34 -4.59705e+34
12           11 -2.298525e+34 -4.59705e+34
13           12 -2.298525e+34 -4.59705e+34
14           13 -2.298525e+34 -4.59705e+34

from abess.

Mamba413 commented on July 20, 2024

@oooo26 , I have uploaded two files poisson_y.csv and poisson_x.csv that corresponds to y and x, respectively. Can you test whether this issue happens in python?
poisson_x.csv
poisson_y.csv

from abess.

oooo26 commented on July 20, 2024

Hi, sorry for the late response. I have checked in Python, but the problem seems to not happen.

ABESS version: latest, v0.4.6(PyPI)
Python version: 3.9.12

Here is the test code:

import numpy as np
import pandas as pd
import abess

X = pd.read_csv("poisson_x.csv")
y = pd.read_csv("poisson_y.csv").squeeze()
print(X.shape)
print(y.shape)

model = abess.PoissonRegression(
    support_size=range(15),     # 0:14
    cv=5                        # both CV and IC are working
)
model.fit(X, y)

print(f"Sparsity: {np.count_nonzero(model.coef_)}")
print(f"Non-zero: {np.nonzero(model.coef_)[0]}")
print(f"Train Loss: {model.train_loss_}")
print(f"Test Loss: {model.eval_loss_}")
######
# Sparsity: 4
# Non-zero: [122 352 573 769]
# Train Loss: -2360540438301305.5
# Test Loss: -729389503380903.0
######