Needed:

• assemble_regression_matrix
• fit_with_breaks
• fit_with_breaks_force_points
• predict
• fit_with_breaks_opt
• fit_force_points_opt
• standard_errors
• prediction_variance
• r_squared
• fit
• fitfast
• fit_guess
• use_custom_opt

Probably won't change:

• calc_slopes
• p_values

Add errors/warnings:
- [ ] small number of data
- [ ] square matrix
I added try except catches to the optimization functions that would result in either a failed Cholesky decomposition or non-invertible matrix.

Hi,

Is there a way to get the polynomial coefficients in a similar way it was provided in numpy.polyfit?

I did the following as a simple example to illustrate what I was thinking:

import numpy as np
import matplotlib.pyplot as plt
import pwlf

# create the data
x = np.arange(10)
y = np.zeros(10)
y[:5] = 0 + 2 * x[:5]
y[5:] = 20 - 3 * x[5:]

degree = 1
segments = 2

my_pwlf = pwlf.PiecewiseLinFit(x, y, degree=degree, disp_res=True)
res = my_pwlf.fit(segments)

# predict
xHat = np.linspace(min(x), max(x), num=10000)
yHat = my_pwlf.predict(xHat)

# get polynomial coefficients
def get_coeffs(pwlf) -> np.ndarray:
    coefficients = np.array(list(zip(pwlf.slopes, pwlf.intercepts)))
    return coefficients

coeffs = get_coeffs(my_pwlf)
p1 = np.poly1d(coeffs[0])
p2 = np.poly1d(coeffs[1])

breaks = my_pwlf.fit_breaks[1]

# test polynomial
ytest = np.zeros(len(x))
for i in range(len(x)):
    if x[i] <= breaks:
        ytest[i] = p1(x[i])
    else:
        ytest[i] = p2(x[i])

plt.figure()
plt.plot(x, y, 'o', label='data')
plt.plot(xHat, yHat, '-', label='predict')
plt.plot(x, ytest, '--', label='fit test')
plt.legend()
plt.show()

Thank you

Add an option to constraint linear coefficients to a range of values (e.g >0 ).

Hello,

How can I add a constraint in the outer optimization function: minimum length of each segment should be >= x?

Hey cjekel,

Thanks a lot for your work.
Is there a way I can retrieve the p-values of the coefficients of piecewise regression?
Thank you.

I found this document gives demos which predict on a one-dimension input array x. According to your comments, I find that this model seems only supports 1-d input array

        x : array_like
            The x or independent data point locations as list or 1 dimensional
            numpy array.

I just wondering if pwlf can predict on multidimensional input such as the following, and I provide something like a fit_breaks list, which tells pwlf to break on certain values of clock? Moreover, if I want to remove the clock feature afterwards, is it possible?

clock    feat1    feat2
0            1           2
100        3           4

You have a function to calculate slopes, but not offsets.

y_hat = self.predict(self.fit_breaks[0:-1])
self.offsets = y_hat - self.slopes*self.fit_breaks[0:-1]

I think it would be really helpful to allow a mixture of degrees for a model. This could be passed as a list or numpy array to the degree parameter. This has real-life applications: https://pdfs.semanticscholar.org/a6cf/634e8788586ba1d37c10c744db6a2f790109.pdf

In this scenario, a mixture of [0, 1] and [1, 0] degrees would be needed. However, it might be interesting to see if mixtures of higher degrees could be useful.

Hi,

First of all, thanks for a great software. I'm however struggling somewhat with the standard errors. I would like to plot both the data, a piece wise linear fit and a 95 % confidence interval for the fit, but I'm not quite sure how to actually do this. Could you for instance update the fitForSpecifiedNumberOfLineSegments.py example to also plot the 95% confidence interval?

What I have been trying is
se = pwlf_Fz_dy.standard_errors()

fz_soil_xHat_95 = pwlf_Fzdy_soil.predict(brkpoints_y) + se * 1.96
fz_soil_xHat_05 = pwlf_Fzdy_soil.predict(brkpoints_y) - se * 1.96
some plotting function here

but I'm not sure if this is the correct approach.

Best regards
Torstein Skår

I just read the paper for this project and it looks amazing and has given me great insight to piece-wise algorithms. I think that it would be amazing if you could include a section to your paper looking at the time complexity of your algorithm.

After running your program against my rather large data set It finds an answer relatively fast for a small number of breaks; but, takes a very long time when it tries to look for more than 5 break points.

What are your thoughts? I would love to help out in any way that I can.

Calling predict() with a float type argument raises IndexError (vs TypeError).

       if sorted_data is False:
           # sort the data from least x to max x
           order_arg = np.argsort(x)
           x = x[order_arg]  <<--- HERE

Consider detecting non-array arguments to predict?

This Library has grown significantly since it's inception. I've included statistical features that have not been formally written (standard errors and prediction variance).

Before I fall behind, I need to write a paper describing all of the features and use cases of this library. This will need to describe the underlining mathematics and statistics the library follows.

Hi,
Thanks for your work, this package is exactly what I've been looking for!

However, I have some issues. My data has a few different regions, however, I'm not sure how many: two or three. So, I tried both, and with three different fits your package works perfect, whereas when I try to fit my data with two straight lines, I get this error:

>>> res = myPWLF.fit(2)

Traceback (most recent call last):

File "<stdin>", line 1, in <module>


File "/home/user/.local/lib/python3.6/site-packages/pwlf/pwlf.py", line 308, in fit
  polish=True, init='latinhypercube', atol=1e-4)

File "/home/user/.local/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 213, in differential_evolution

  return solver.solve()

File "/home/user/.local/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 511, in solve

  self._calculate_population_energies()

File "/home/user/.local/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 590, in _calculate_population_energies
  *self.args)

File "/home/user/.local/lib/python3.6/site-packages/pwlf/pwlf.py", line 226, in fitWithBreaksOpt
  if np.isclose(var[0],var[1]) == True:
IndexError: index 1 is out of bounds for axis 0 with size 1```

Could you please clarify this moment?
Sincerely, V.

I should either calculated the R^2 value for each fit, or create a function that returns the R^2 value for a given fit.

The R^2 example is given here:
c25e5b3

I have a particular problem that requires a piece-wise approximation of similar datasets.
My current solution is to process the data one by one,
next I extract the slopes, find the average and the standard deviation.
However, I found someone having a similar issue and the offered solution is to "fit a multilevel model".
Is it possible to implement this feature somehow?

When I use this method https://github.com/cjekel/piecewise_linear_fit_py/blob/master/pwlf/pwlf.py#L1240 it provides the
"LinAlgError: Singular matrix" too often.
When debugging, I see that the Ad matrix calculated in here:
https://github.com/cjekel/piecewise_linear_fit_py/blob/master/pwlf/pwlf.py#L1304
has the last column is all 0, such as:
(Pdb) p Ad
array([[ 1., 2., 0.],
[ 1., 4., 0.],
[ 1., 6., 0.],
[ 1., 7., 0.],
[ 1., 8., 0.],
[ 1., 9., 0.],
[ 1., 11., 0.],
[ 1., 12., 0.],
[ 1., 16., 0.],
[ 1., 18., 0.],
[ 1., 19., 0.],
[ 1., 20., 0.],
[ 1., 21., 0.],
[ 1., 22., 0.],
[ 1., 23., 0.]])
so:
(Pdb) np.dot(Ad.T, Ad)
array([[ 15., 198., 0.],
[ 198., 3310., 0.],
[ 0., 0., 0.]])
and then https://github.com/cjekel/piecewise_linear_fit_py/blob/master/pwlf/pwlf.py#L1323
will throw the error.
Have you consider using pinv instead of inv here, which still provides a good pseudo inverse matrix to use later?
(Pdb) linalg.pinv(np.dot(Ad.T, Ad))
array([[ 0.3168677 , -0.01895462, 0. ],
[-0.01895462, 0.00143596, 0. ],
[ 0. , 0. , 0. ]])
I can make a PR if you see it helpful

Hi again,

in my application it would perform better if it has the ability to constraint the breaks into integers only.

To explain this situation, my application is a piecewise 3rd polynomial degree linearization for some electronic sensors, so I use the coefficients of the polynomial into values given by an ADC (analog-to-digital converter) of 10 bits, which gives-me unsigned integers values from 0 to 1023, so right now I am breaking the linear pieces flooring it (rounding down) and it potentially gives-me some error there.

Any suggestion on how to achieve that?

Hi Charles,
this is an awesome lib. But as it seems there is no possibility to dump the learned linear spline model to a file. This would make it quite more applicable to practical use cases....are there any plans to support this feature?

It's not good practice to invert matrices. I need to rework the prediction variance and standard errors calculation such that the matrix inversion is not necessary.

Is it possible to use a solver instead of matrix inversion?

This needs further investigation.

I have no idea what's going on with documentation and why it wasn't building...

The assembly of A is very slow in numpy. This is currently the reason why tensorflow is faster for larger data sets.

Hi,

How can I find the error of intercepts and slopes? is there such possibility in the code?

Thanks in Advance
Az

It looks like this library requires a user to specify the number of line segments before fitting a model. This means this library can't be used for automatic trend detection(this requires a program to figure out the right number of line segments itself)? It is not quite useful if it requires a user to visually inspect the chart to figure out how many line segments are.

Hi there,

When running the following code:
my_pwlf = pwlf.PiecewiseLinFit(x, y)
res = my_pwlf.fit(4)
xHat = np.linspace(min(x), max(x), num=10000)
yHat = my_pwlf.predict(xHat)

I get the error:
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pwlf/pwlf.py", line 55, in __init__ self.x_data = x[order_arg]
TypeError: only integer scalar arrays can be converted to a scalar index

I have managed to work out what was causing this error but I will post this anyway in case others run into the same issue. The x and y array fed into the PiecewiseLinFit function has to be an np.array type.

To fix, I just changed the code to:
my_pwlf = pwlf.PiecewiseLinFit(np.array(x), np.array(y))

I'm getting an error if I use the fit function for the standard example. I'm using Python3.6 and scipy version 1.1.0.

my_pwlf = pwlf.PiecewiseLinFit(x, y)
res = my_pwlf.fit(2)

The error message is:
`
File "/home/svimal/miniconda3/lib/python3.6/site-packages/scipy/optimize/lbfgsb.py", line 378, in _minimize_lbfgsb
hess_inv = LbfgsInvHessProduct(s[:n_corrs], y[:n_corrs])
File "/home/svimal/miniconda3/lib/python3.6/site-packages/scipy/sparse/linalg/interface.py", line 144, in new
obj = super(LinearOperator, cls).new(cls)

TypeError: super(type, obj): obj must be an instance or subtype of type`

Does this code support multivariate piecewise linear regression?
Or is there any plan to support this?

Hi again!

Sorry for disturbing, but I would like to ask you a few more things.
First, is it possible to add a key parameter which turns off all the output messages (except errors)?
I have to deal with massive numbers of files and I need to see my own print messages.

Second, can your algorithm in general use boundaries for resulting parameters? Say, I know in general, that the values of slopes a priori lie in range [0:2], thus, the result has to be in the range.

Sincerely,
V.

I've noticed that for small data sets, sometimes the predict function gives a result that appears to have the incorrect number of segments, even though .n_segments, .slopes, .fit_breaks, all seem to indicate the correct number of segments.

Here is an example which seems to give consistently the incorrect results

import numpy as np
import matplotlib.pyplot as plt
import pwlf

numSegRequest = 2

x = np.array([0.11445023, 0.49628489, 0.67287169, 0.83479312, 0.90068457,
       1.15980533, 1.34981648, 1.52888812, 1.70987057, 1.73051513])
y = np.array([ 0.0732206 ,  0.11161097,  0.01289929, -0.02448141, -0.17952348,
       -0.33457269, -0.48479134, -0.83837053, -1.41280894, -1.45807414])

my_pwlf = pwlf.PiecewiseLinFit(x,y)
res = my_pwlf.fit(numSegRequest)

plt.plot(x,y,'o')
plt.plot(x,my_pwlf.predict(x))
plt.title('number of segments: '+str(my_pwlf.n_segments))
plt.show()

print('breaks: ',my_pwlf.fit_breaks)

Which has the following results

breaks: [0.11445023 1.29188065 1.73051513]

I would be great if this package were to integrate in with scikit-learn, identify itself as a regressor, and implement the associated APIs. This would unlock scikit workflows as well as packages that build on scikit, like Yellowbrick. Overall the package is great, this would be greater!

https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/base.py#L372

I want fit the data (yellow ones),using piecewise horizontal and vertical lines but failed. Could pwlf make it possible?

Hi there,
The paper says, " the solution is expressed as",

It equals to:

Why use the equation 8?

Hi, I am trying to force a fit through a point using the pwlf library. If I force the fit through a certain data point with a degree=1 everything works perfectly, but as soon as I set the degree=2 the following error comes up:

ValueError: could not broadcast input array from shape (5,1) into shape (11,1)

This is the code I used:

import numpy as np
import pwlf


#generate sin wave data
x = np.linspace(0, 10, num=100)
y = np.sin(x * np.pi / 2)
# add noise to the data
y = np.random.normal(0, 0.15, 100) + y

#linear fit
my_pwlf_1 = pwlf.PiecewiseLinFit(x, y, degree=1)
my_pwlf_1.fit(n_segments=6, x_c = [0], y_c = [0])
y_predict_1 = my_pwlf_1.predict(x)

# quadratic fit
my_pwlf_2 = pwlf.PiecewiseLinFit(x, y, degree=2)
my_pwlf_2.fit(n_segments=5, x_c = [0], y_c = [0])
y_predict_2 = my_pwlf_2.predict(x)

I've made a number of changes since the 1.0.0 release that make the current library significantly different than the March 19, 2019 paper.

Currently the fit functions assume that a breakpoint occurs at the smallest x_data point and the largest x_data point. This results in a simplified optimization problem.

However it may not result in the best continuous piecewise linear model.

Hi,

Great lib !

I think I have spotted an issue with it however. It seems that when the breaks are spot on the xdata the break point belongs at the same time in two arrays of sepData. Thus it increases the size of yHat in fitWithBreaks and an error is raised when the function calculate the sum of the square of residuals
e = self.yData-yHat

My guess would be to change either the bTest or aTest in separateData to a strict inequality.

I am not sure I am being very clear, my english is a little bit rusty.

Have a nice day,

Hey @cjekel ,

thank you for this library! It's been a great help in prototyping some ideas this week.
I found a slight mismatch in the documentation:

PieceWiseLinFit.fitfast and PieceWiseLinFit.fit_guess mention that scipy.optimize.differential_evolution is used, but that's not true. It's only called for PieceWiseLinFit.fit.

PieceWiseLinFit.fitfast and PieceWiseLinFit.fit_guess both use scipy.optimize.fmin_l_bfgs_b instead.

It's correct further down in the documentation (near the examples) though.

See:

Possible objective functions:

• Elastic net
• test/train split
• k fold cross validation

related to #17

Hi, thanks for the package!

When trying to run the examples here https://jekel.me/piecewise_linear_fit_py/examples.html, I am consistently running into the error AttributeError: 'PiecewiseLinFit' object has no attribute 'beta'. For example, following the 'fit for specified number of line segments' example, I get:

    yHat = my_pwlf.predict(xHat)
  File "/Users/tom/phd/entity_extraction/piecewise_linear_fit_py/pwlf/pwlf.py", line 609, in predict
    y_hat = np.dot(A, self.beta)
AttributeError: 'PiecewiseLinFit' object has no attribute 'beta'

I am using Python 3.5.6. I tried both pip install pwlf and installing from source and got the same issue. Let me know if you need any more info. Thanks!

when use pwlf, I encounter errors: TypeError: must be real number, not NoneType
Below is my code:
x = np.random.rand(100,1)
y = np.random.rand(100,1)
myPWLF = pwlf.PiecewiseLinFit(x.reshape(-1), y.reshape(-1))
myPWLF.fit(6)
But when I changed another PC, this problem disappears. So I think it may be due to some depend library.

Traceback (most recent call last):
File "test.py", line 69, in
y_plf, x_plf = model_tm.plf(zkOrder_start, zkOrder_end, y_total, z_fit)
File "/media/disk2/sgcc_model_tm2/Model/model_tm.py", line 129, in plf
myPWLF.fit(z_fit)
File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/pwlf/pwlf.py", line 971, in fit
atol=1e-4)
File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 213, in differential_evolution
return solver.solve()
File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 511, in solve
self._calculate_population_energies()
File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/scipy/optimize/_differentialevolution.py", line 590, in _calculate_population_energies
*self.args)
File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/pwlf/pwlf.py", line 670, in fit_with_breaks_opt
beta, ssr, rank, s = np.linalg.lstsq(A, self.y_data, rcond=None)
File "/home/lukuan/.pyenv/versions/anaconda3-5.1.0/envs/lk3/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 1953, in lstsq
0, work, -1, iwork, 0)
TypeError: must be real number, not NoneType

Thank you!

If independent variable x and dependent variable y=f(x) is the same, in other words, y=f(x)=x.
Then the training process takes ages to train...

differential_evolution step 1: f(x)= 2.08676e-23
differential_evolution step 2: f(x)= 2.08676e-23
differential_evolution step 3: f(x)= 2.08676e-23
differential_evolution step 4: f(x)= 2.08103e-23
differential_evolution step 5: f(x)= 2.08103e-23
.....
.....
.....
differential_evolution step 990: f(x)= 2.02263e-25
differential_evolution step 991: f(x)= 2.02263e-25
differential_evolution step 992: f(x)= 2.02263e-25
differential_evolution step 993: f(x)= 2.02263e-25
differential_evolution step 994: f(x)= 2.02263e-25
differential_evolution step 995: f(x)= 2.02263e-25
differential_evolution step 996: f(x)= 2.02263e-25
differential_evolution step 997: f(x)= 2.02263e-25
differential_evolution step 998: f(x)= 2.02263e-25

So such circumstances should be avoided?

Hi,
I got some problems to run your fitForSpecifiedNumberOfLineSegments.py. I get the following error message:

Can you help me, please?
My numpy version is 1.13.3 and scipy version is 1.0.0.

Sincerely,
s.

Hi

I am using pwlf within https://github.com/pyinstaller/pyinstaller (it compiles python to exe on windows).

However, pwlf is causing some troubles, because version handling is too simple - the file "https://github.com/cjekel/piecewise_linear_fit_py/blob/master/pwlf/VERSION" is not included automatically in the build, and I have to create it every time. Otherwise, it is failing:

For now I'm going to generate empty VERSION file by script, however it is not a solution.

I suggest to use the approach numpy guys use:
They generate version.py file by setup.py:
https://github.com/numpy/numpy/blob/96abdedcb21df7e1a07a7e73a5f2103895a36faf/setup.py#L127

and then import it as __version__:
https://github.com/numpy/numpy/blob/b7c27bd2a3817f59c84b004b87bba5db57d9a9b0/numpy/__init__.py#L132

Hi,
I have got some problems to open an url with the prefix https. I just tried to fix it but the error message remains. My code is:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

profile = webdriver.FirefoxProfile()
profile.accept_untrusted_certs = True

o = Options()
o.add_argument('-private')

driver = webdriver.Firefox(firefox_options=o)
driver.get("https://www.python.org/")

And the message in Firefox is

Could anyone help me, please?
s.