Comments (1)
The classification loss and regression loss is combined together, however, the scale of two loss is different meaning, the binary crossentropy loss is often much smaller than regression loss. Shouldn't we compensate for this difference in scale?
I agree, this could likely be a hyperparameter for the cost weights that you could tune on your specific problem.
EDIT: I am going to update my opinion here as I think I have changed my mind. After thinking about it further, I realized this:
Our goal is to optimize the predicted ZILN distribution. Our model outputs the zero-inflation probability which is P(Y > 0) and it also outputs conditional log normal distribution which we use to calculate the conditional probability P(Y | Y > 0).
If you think about it, then we can calculate the probability of any observed value as:
P(Y) = P(Y > 0) * P(Y | Y > 0)
Now, if we want to optimize this probability distribution, we can take the negative log likelihood. But due to the log transformation, this becomes:
-log P(Y) = -log P(Y > 0) - log P(Y | Y > 0)
-log P(Y) = classification_loss + regression_loss
As you can see, by taking the log transformation of our ZILN likelihood, it becomes the sum of the 'classification loss" and the 'regression loss'.
So originally I thought we could use a hyperparameter to re-scale the loss weightings. However, after thinking about it further, I think it makes sense to leave it as is and simply add the losses together because it directly replicates what we want to optimize which is log P(Y) of the ZILN distribution.
In addition, why do we take the negative of regression loss?
This is because the model is computing the 'log probability' of the given label. When you compute the log probability of the true label against your estimated distribution, it will give you a number between negative infinity and zero. Where zero is a 'perfect' guess and negative infinity would be the worst possible guess.
Since we want to minimise our cost function, we take the negative of this so that the scale is from zero to positive infinity, and where a lower value is 'better' so that we can minimise it.
from lifetime_value.
Related Issues (7)
- In ZILN loss network, the three activation function are sigmoid, identity and softplus. HOT 1
- Why the ltv prediction part use probabilitity prediction multiply expectation as the final ltv prediction? HOT 2
- Why use the" y_eval" of the tuning parameter as the test set for evaluation, shouldn't it be separate? HOT 1
- Question regarding simultaneous objectives HOT 2
- How are LTV distributions calculated? HOT 1
- Upgrade to scikit-learn as opposed to sklearn
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lifetime_value.