arielf / weight-loss Goto Github PK

Machine Learning meets ketosis: how to effectively lose weight

License: Other

Makefile 15.08% R 21.83% Perl 17.84% Shell 2.91% HTML 2.16% Python 40.19%

weight-loss's Introduction

Discovering ketosis: how to effectively lose weight

Here is a chart of my weight vs. time in the past 16 months or so:

The chart was generated from a data-set weight.2015.csv by the script date-weight.r in this git repository. It requires R and ggplot2.

In the following I'll describe the thought process, some other people ideas, and the code I used to separate signal from noise. This separation was critical to help lead me in the right direction.

This github repository includes my code, a Q&A section, and links for further reading.

Disclaimers:

The below is what worked for me. Your situation may be different. Listen to your own body. The code here is designed to be used on your own data, not on mine.

Also: this was not a scientific experiment, or a "study"; rather, it was a personal journey of experimentation and discovery.

With these behind us, I'd like to channel Galileo in the face of the inquisition: evolution has been hard at work for 2 billion years shaping the chemistry of all eukaryotes, multi-cellular life and eventually mammals. The Krebs cycle, glucose metabolism, insulin spikes, glycogen in the liver, carnitine, lipase, are as real for you as they are for me. We may be very different in our genes and traits, some are more insulin resistant, for example, but we cannot be too different in our most fundamental metabolic chemistry. The chemistry which drives fat synthesis and break-up.

Salient facts & initial observations

I used to be a pretty thin person. My 1st DMV card below, says 143 lb.
Unfortunately, since moving to the US, I've been gaining more and more weight. I peaked in 2015, over 50 lbs higher.
The US is a country where obesity is an epidemic.
Poorer demographics in the US have higher levels of obesity.

Does a US typical lifestyle has anything to do with this epidemic? After reading on the subject, I could point at a few of the main suspects:

Fast food is highly available, and is very cheap compared to most alternatives
Most food we buy and eat is heavily processed -- watch Food, Inc. (documentary)
"No Fat" and "Low Fat" labels are everywhere on supermarket shelves
Many foods are enriched and sweetened with high-fructose corn-syrup -- watch Sugar Coated (documentary)

As in many other instances, I realized I need to think for myself. Ignore all "expert" advice. Question widely accepted ideas like the FDA "food pyramid". Start listening to my own body, my own logic & data I can collect myself and trust.

Once I did, the results followed.

What didn't work

In the past, I tried several times to change my diet. After reading one of Atkins' books, I realized, checked, and accepted the fact that excess carbs are a major factor in gaining weight. But that realization alone has not led to success.

My will power, apparently, was insufficient. I had too much love of pizza and bread. I would reduce my carb consumption, lose a few pounds (typically ~5 pounds), and then break-down, go back to consuming excess carbs, and gain all these pounds back, and then some. My longest diet stretch lasted just a few months.

It was obvious that something was missing in my method. I just had to find it. I could increase my physical activity, say start training for a mini-marathon, but that's not something I felt comfortable with.

I realized early on that I need to adopt a lifestyle that not just reduces carbs, or add exercise, but is also sustainable and even enjoyable so it can turn into a painless routine. Something that:

I could do for years

Never feel the urge to break habits

Is not hard, or unpleasant for me to do

Early insights & eureka moments

Early in the process I figured I could use machine learning to identify the factors that made me gain or lose weight. I used a simple method: every morning I would weigh myself, and record both the new weights and whatever I did in the past ~24 hours, not just the food I ate, but also whether I exercised, slept too little or too much, etc.

The file I kept was fairly simple. A CSV with 3 columns:

Date, MorningWeight, Yesterday's lifestyle/food/actions

The last column is a arbitrary-length list of word[:weight] items.

The (optional) numerical-weight following :, expresses higher/lower quantities. The default weight, when missing is 1:

#
# -- Comment lines (ignored)
#
Date,MorningWeight,YesterdayFactors
2012-06-10,185.0,
2012-06-11,182.6,salad sleep bacon cheese tea halfnhalf icecream
2012-06-12,181.0,sleep egg
2012-06-13,183.6,mottsfruitsnack:2 pizza:0.5 bread:0.5 date:3 dietsnapple splenda milk nosleep
2012-06-14,183.6,coffeecandy:2 egg mayo cheese:2 rice meat bread:0.5 peanut:0.4
2012-06-15,183.4,meat sugarlesscandy salad cherry:4 bread:0 dietsnapple:0.5 egg mayo oliveoil
2012-06-16,183.6,caprise bread grape:0.2 pasadena sugaryogurt dietsnapple:0.5 peanut:0.4 hotdog
2012-06-17,182.6,grape meat pistachio:5 peanut:5 cheese sorbet:5 orangejuice:2
# and so on ...

Then I wrote a script to convert this file to vowpal-wabbit training-set regression format. In the converted train-set the label (target feature) is the change in weight (delta) in the past 24 hours, and the input features are what I've done or ate in the ~24 hours leading to this delta -- a straight copy of the 3rd column.

I was not dieting at that time. Just collecting data.

The machine learning process error-convergence after partly sorting the lines descending, by abs(delta) to smooth it out and try to amplify very weak signals from the data, and 4-passes over the data, looks like this:

You can reproduce my work by compiling your own data-file, installing all prerequisites, and running make in this directory. I wrote a HOWTO file with more detailed instructions. Please open an issue, if anything doesn't work for you.

When you type make in this directory -- some magic happens.

Here's how a typical result looks like.

$ make

... (output trimmed for brevity) ...

FeatureName       HashVal   ...   Weight RelScore
nosleep            143407   ...  +0.6654 90.29%
melon              234655   ...  +0.4636 62.91%
sugarlemonade      203375   ...  +0.3975 53.94%
trailmix           174671   ...  +0.3362 45.63%
bread              135055   ...  +0.3345 45.40%
caramelizedwalnut  148079   ...  +0.3316 44.99%
bun                  1791   ...  +0.3094 41.98%

... (trimmed for brevity. Caveat: data is too noisy anyway) ...

stayhome           148879   ...  -0.2690 -36.50%
bacon               64431   ...  -0.2998 -40.69%
egg                197743   ...  -0.3221 -43.70%
parmesan             3119   ...  -0.3385 -45.94%
oliveoil           156831   ...  -0.3754 -50.95%
halfnhalf          171855   ...  -0.4673 -63.41%
sleep              127071   ...  -0.7369 -100.00%

The positive (top) relative-score values are life-style choices that make you gain weight, while the negative ones (bottom) make you lose weight.

And here's a variable-importance chart made from a similar data-set:

Disclaimer: please don't read too much into the particulars of this data. Working with this particular data set, was pretty challenging, since:

The number of original data-points (a bit over 100 days) may be too small to establish enough significance.
Typical daily changes in body weight are very small, often ~0.1 lb.
My scales are not accurate: you may note that my data has 0.2 pound resolution. This is not ideal. Getting scales with 0.1 pound resolution is highly recommended.
You may also note that the loss-convergence chart hits a hard floor at ~0.2 even when you do multiple-passes over the data (overfit the training-set) for a similar reason.
Items that make you lose and gain weight, often appear together on the same line so they cancel each other. This throws the automatic learning process off-course.
There were some misspellings in the original data (I hope I fixed all of these by now)

So I focused mostly on the extremes (start and end) of the list as presented above, and just used the hints as general guidance for further study, experimentation, and action.

Despite the noisy & insufficient data, and the inaccuracies in weighting, the machine-learning experiments made 4 facts pretty clear, pretty early:

Sleeping longer consistently appeared as the #1 factor in losing weight.
Lack of sleep did the opposite: too little sleep lead to weight gains.
Carbs made me gain weight. The worst were high-starch and sugary foods.
Fatty and oily foods tended to do the opposite: they were positively correlated with weight-loss.

The 'stayhome' lifestlye, which fell mostly on weekends, may have been a red-herring: I slept longer when I didn't have to commute to work, OTOH: my diet on stay-home days may have been different.

It took me a while to figure out the sleep part. When we sleep we don't eat. It is that simple.

Moreover: we tend to binge and snack while not particularly hungry, but we never do it during sleep.

Our sleeping time is our longest daily fasting time.

Please note that my explanations of the effects may not in fact be accurate or deeply scientific. The goal of all this was incremental discovery: experiment, check effect, rinse, repeat.

Further progress

You may note that in the top (date vs. weight) chart there's a notable acceleration in the rate of weight-loss. The cause was deeper insights and better ability to sustain the diet the more I understood the problem.

Extending the fasting time was one major accelerator of weight-loss rate. I did that by:

Skipping breakfast and

Stop eating earlier in the evening before going to bed.

This gave me 14-16 hours of fasting each day. Rather than the more typical 10-12 hours/day of fasting.

The 2nd accelerator was consuming fatty stuff (instead of carbs) in order to feel full.

The 3rd accelerator was understanding the concepts of Glycemic index and Glycemic Load, and shifting whatever I chose to eat towards lower Glycemic loads.

I now believe and hope that I can go all the way back to my original weight when I first landed on US soil.

If I can keep the present rate, it should take 1-2 years to completely reverse the damage of the past ~20 years.

It is important to stress that I also feel much better the more weight I lose. As a welcome side-effect, the few borderline/high levels in my blood tests, have moved significantly towards normal averages, during the period I lost weight.

What was my data and clear improvement in health saying?

Looking at my data, and reading more, convinced me that I should beware of doctors who push statins instead of suggesting a better diet. I started doubting anyone who told me I need to reduce fat. I run away if anyone now tells me "high cholesterol" in the diet is dangerous.

Cholesterol, by the way, is an essential building block for many essential body by-products. The liver produces as much cholesterol as we need.

Our body is an amazing machine. Billions of years of evolution have made it extremely adaptive.

It is not our high fat consumption, it is the storage of fat process that makes us acummulate fat in the tissues and become unhealthy.

An enzyme called Lipase breaks-up fat. Raise the levels of Lipase and our body fat gets consumed faster. To get there, we need to give the body fat as an alternative to carbohydrates. When the body has depleted both the blood sugar, and the glycogen (hydrated sugar) buffer in the liver, it has no other choice but to adapt and compensate. Our source of energy -- ATP synthesis -- switches from carbs to fats by producing more fat-breaking agents. The body is a "Flex Fuel" kind of machine, that has simply replaced one fuel (carbs) with another (fat).

When Lipase, and all other agents in the fat-to-ATP chemical path, aka Beta oxidation mobilize, and their levels are elevated, we burn more fat and lose weight over time.

In a low-carb/high-fat (LCHF) regime, our night sleep (fasting time) becomes our friend. The fat-breaking agents keep working while we sleep, breaking-up the stored fat. This leads to weight-loss, and a healthier state.

And when we push even further, and cut carbs to really low levels, we may reach a new steady state, called ketosis, in which practically all our energy comes from fat, and that's when we really win big in the weight-loss battle.

The above is a very simplified, and hopefuly easy to digest, version of what some diet books try to explain in hundreds of pages.

My bottom-line recipe:

The hardest part (especially at the beginning) is reducing carbs. The worst are starch rich foods (pizza, pasta, bread etc.), then processed foods with high sugar content (sweet sodas, no-pulp juices, etc). This doesn't mean no carbs. You may afford yourself carbs from time to time (say a pizza once a week). As it turns out, an occasional lapse isn't enough to completely reverse any steady-state. However, you need to make sure you consume much less carbs and less frequently than before. In particular, you must avoid binging on snacks like chips, pizza, doughnuts, pasta, and bread, or drinking sugar-rich drinks.
Look-up Glycemic index and Glycemic Load on wikipedia. Avoid foods with high glycemic load. This prevents the blood sugar spikes which lead to insulin spikes and tell the body chemical cycles to revert back from ketosis, or near ketosis, to fat-accumulation. Have a sweet tooth? Eat an orange instead of drinking orange juice. The two have vastly different glycemic loads and this makes a huge difference. If you must add sweetness to your cup of tea or coffee, use a Splenda (sucralose+dextrose) tablet, or a Stevia drop/tablet which typically weight just ~0.1 gram, rather than a tea-spoon of sugar (~4.2g, about 40x more). Result: similar sweetness effect, but much lower Glycemic load and resulting levels of blood-glucose.
High fat: I switched from milk to half-and-half and am considering heavy (and unsweetened) whipped cream. It has less carbs (lactose) and more fat; plus, it tastes better. Eat avocados, olive oil, mayo, coconut oil, nuts. I never worry about natural fat, I eat as much fat as I want. This is what makes it much easier to avoid carbs. When I stuff myself with fat I feel much less hungry and miss the carbs less. The body is very good at figuring this out: "I have too much fat in the blood, so let's increase the amount of enzymes which break-up fat" and this makes me lose weight in the long run. Most importantly, I always avoid any products labeled "low-fat" or "fat-free". The food industry usually replaces fat with sugar, so it tastes better - otherwise it tastes awful. You'll often hear about "bad" vs "good" fat. My take: as long as it is natural, it is ok. The worst trans-fat is fat that's artificially hydrogenated, to increase shelf-life, by the food industry. The less saturated fat is, the better. Mono-saturated (plant) liquid oil is the best, then come the poly-unsaturated fats, and finally near saturated (but not fully saturated) fats that come from animals. My buttery-spread spectrum is: Margarine: no; Butter: ok; Earth Balance: no problem. At any rate, even the most saturated fat, gets broken and depleted by the natural processes in the body.
A bit of exercise. Of course, more is better, but for many this may prove difficult. I don't excercise too much. I just bike to work and back about 20 min each way, meaning 40 min/day, 5 out of 7 days/week. You can try walking the dog (but walk faster), or Zumba dance to music. The trick is to find something that you don't find hard to do. Or find company to do it together. Then, do a little bit of it every day.
Longer fasting periods: This is the #1 contributor to weight-loss. sleep longer, stop eating as early as possible before going to sleep and start eating as late as possible after sleeping. Skip breakfast, after some time you won't feel hungry in the morning anymore. After long periods of fasting, the body chemistry adjusts. It needs ATP, but there's a too low level of glucose in the blood. The glycogen in the liver is fully consumed (this takes about 1-2 days of low or no carbs) so there's no other option, but to start looking for other sources, like stored fat. This elevates the enzymes that help with breaking up fat and the Krebs cycle reverses direction in the critical paths. Instead of transforming excess-carbs into stored fat, we break-up stored fat for energy.
Eat eggs. They are a wonderful combo of fat and protein with no carbs at all. I read an interview with a Japanese woman who reached 114 years and one of her secrets was to eat eggs daily. My favorite food is a scrambled egg with grilled onions (onions are a bit high on carbs, but too tasty to give up) and olives.
Eat slower, and chew longer... don't swallow just yet! Humans, just like dogs, tend to swallow too soon. Stop eating when you feel full. There's about 20 min delay before your brain registers that you are full so don't over-eat.

Acknowledgements

Big thanks to the following people for contributing to this project in myriad ways, comments, references, corrections, etc.

Anat Faigon, Ingrid Kane, Hans Lee, Steve Malmskog, Eyal Friedman, Shiri Shoham, Gabi Harel, Shingi, Noa

Update: 2016-08-12: this project made Hacker News and reached the top place for a while. Thanks for some great comments by benkuhn, aab0, zzleeper, and others which helped me make it better.

Special thanks to John Langford and the many other contributors to vowpal wabbit.

License:

This code and additional material are released under a permissive and simple 2-clause BSD licence. The one sentence summary of this is "as long as you don't sue me and not claim it as your own, you should be ok."

weight-loss's People

Contributors

Stargazers

Watchers

Forkers

tdaitx jeffcressman ttruga gregnwosu ddevaz techscientist domenicosolazzo vasili-zolotov robeastham up1 javaguy147 zbessinger yosoyubik braintrain neilbryant bkjackson mscherer mira5619 tedm johndpope lesthersk shimon1981 pjebs brspurri bradparks ismailmechbal midoriyakumo xuanhan863 thecharge xiaojay fisle lfthwjx daniel-perry chrisswk ranganaths wuatanabe neuromancer977 lambder anderslemke ml-lab murugespandian davidmilberg alfonsodev justinjim linearregression kumaranvpl lury disc5 wahyuinspire todun oiclid wubr2000 singlexu stefek99 tspannhw nikolayvoronchikhin martinbodocky annasedlar bastinrobin rp-mcguire vivian8725118 saveriosalatino warm-ice dorelljames xinyuwufei garyblankenship likeshumidity vsching zachphillips chrisemoulton aeyakovenko davidrichards sfarkov sheldonrampton camyyssa claudiouzelac anttiviljami sebastienwindal asleda gojira modulexcite enterstudio narain280493 prav10194 gcichosz alancpu 007v hengx sgrives allan2327 toast707 rusfearuth reloadbrain atouhou emersonk satyaam klgraham 2dpodcast datahack-ru clustersdata

weight-loss's Issues

Age constant

I was thinking if age matters on output ? If yes, how do you suggest to age constant into formula ?

Tooth brushing

I know this sounds silly and can be considered as uninteresting as it does not focus on the primary source for weight results, but I am interested to hear if you did record and/or change your tooth brushing behaviour over these three(?) years (e.g. frequency, time of day, using gums, flossing, etc)?

I can see (and frequently experience this as an effect similar to sleep: it expands the fasting times. In times where I only brushed one time per day (before going to bed) I could imagine eating more and feeling more the lust to eat over the day vs brushing my teeth now three times per day, which takes away roughly 2hrs per day I just don't feel like eating. If that would be true (i.e. I don't overcompensate during meals) adding this alongside the "trick" of sleeping enough/long could reduce the potential times of "false decisions" during the day.

Neat analysis! How many days worth of data did you use?

Hi, I run the Quantified Self Forum and we'd love to learn more about your experiment. How many days were in the data-set that led to the variable-importance chart? The CSV in the repo only has 120 days worth of data vs. a weight variation from 186.8lbs to 182.8, which over 4 months seems too insignificant to draw conclusions from.

Perhaps you've only shared a sample/excerpt of the data? Anyway, would be great if you could update the README on that point. Thanks!

Breakfast

First - thanks a lot for sharing that! This is very inspiring :)

Question: many people (and research) say breakfast (breaking fasting :) ) is the most important meal of the day. What are you thoughts on that?

confidence levels

Given the size of the data set and the outputted attributions/predictions it would be helpful to know what the confidence levels are.

Deriving importance factors on a per-week basis

Daily weight changes are considered a very unreliable metric, prone to unpredictable fluctuations. Additionally, some factors may not be reflected immediately within 24h, but have a more long-term effect.

It would be great to take the same data and aggregate both factors and weight delta on a per-week basis, and then see whether the resulting factors are different from the existing daily results. Using the data you already have, it may give some new insights.

cc @arielf

Carbs and water fluctuations might confound results.

Awesome project, as a strength athlete and master in data science, this project includes my two biggest interests! I'm interested in collaborating with this project. Right of the bat I see one major flaw in the way to interpret the results:

Weight alone might not be sufficient to measure the important properties of your body composition. Aside from fat, your weight is also composed of muscle and water. Weight loss in the form of pure muscle would in general not be desired from an aesthetical or health standpoint. Water is an even bigger problem: as carbs are transformed to glycogen and stored in the muscles, for every glycogen particle, 4 water particle are stored around it. Weight spikes after a carb meal are normal and don't have much to do with fat loss. Over the long term water levels stay relatively stable, so comparing high carb weeks with low carb weeks might be a way to cancel this factor out.

Ambiguity on Weight Quantity

I was wondering how one might deal with the differing type of quantities for items in the third column. For example, if I slept 8 hours and had 1 slice of pizza in a day, would it be best to put sleep:8 pizza:0.125 for example, or sleep pizza, etc... Looking more for your personal input on how to make the most of these weights.

Numerical-Weight

Can you please explain the (optional) numerical-weight? What is the unit?

For example from your journal:
bagel bread date:1.5 dietsnapple splenda pizza nosleep:0.2
cherry knish cheese salmon pistachio:0.7 peanut

Does this mean you ate 1.5 dates?

What does this "pistachio:0.7" represent? .7 of a cup?

OSError: No such file or directory

Hi there,

I'd followed your guides on executing the script, installed vw via brew but when I do make in the directory, I get the following error.

I looked it up on stackoverflow, someone says it's related to python subprocess.popen function that command (vw_cmd) should be in array of strings, actually it is in that form. I tried with shell=True option but then I get returncode=127.

The other thing I've tried is executing the command manually where the script breaks, vw-varinfo2 -k --loss_function squared --progress 1 -l 1.0 --l2 1.85201e-08 sercan.train | tee scores.txt it doesn't work but when it executed as ./vw-varinfo2, it works and creates score.txt.

I think it's related to my environment but a help could be great.

Thanks in advance

my fitness pal as data store

Is there a way to use myfitnesspal as a data store as well?
I keep track there of what I eat and how much.
I have started to track my weight there as well.
I'd like to learn which foods are helping and which not so much.
Is there a different app which has a better API?

I am using google fit to automatically record my walking and cycling.
I use Sleep as Android to record my sleep.

I realize that to test some of the current hypotheses, I may have to keep track of additional data, such as the time I eat my meals.

Are there clear enough APIs to be able to do this kind of machine learning?

Learning the policy which leads to balanced/healthy weight loss?

What if the problem losing weight could be cast as a problem of finding the best policy? Given the context (day eating habits, moving etc), the policy (to be learned) should output an action (what to do in that day). Am I correctly understanding the learning problem? This would be fun to explore.

How to do combinations?

Hey Ariel :) I saw that you're answering questions through issues. I hope this is the best way to reach you.

I love this project! I'm using it on Clash Royale like this

    2017-07-26,4600,goblin_archer13 fire_fireball9 goblins13 minion13 knight13 royal_giant13 hog_rider10 princess3
    2017-07-26,4628,giant10 firespirit_hut10 minion12 chr_witch7 barbarians12 miner3 ice_wizard3 zap12
    2017-07-26,4598,barbarians13 miner4 wizard10 chr_witch7 princess3 royal_giant13 the_log3 zap13

Everything's working great. I can generate a score chart and see the best cards overall. But what I noticed in username.train is something like this

    -1202.00 | giant10 valkyrie9 minion_horde12 rage_barbarian3 chr_witch7 lightning6 bomber12 zap12

I'm assuming those are the most commonly played cards? Or is that the best possible deck combination, because it has the greatest negative weight (-1202) meaning it wins against the player more frequently than any other combination?

I guess I'm asking the significance of the number on the left and how the features are chosen on the right side, and from there I can work toward finding the best 8 card combination. Thanks for any help you can provide!

Understanding the RelScore better

Hey, once I run the software with my csv, I get a chart along with some files.
one of those files is scroes file which is of following format

FeatureName,RelScore
1-3k_sleep_onset,-100.00
Turkey,-69.63
Boar's_Head,-69.63
Pepperoni,-69.63
Homemade,-64.10

What does RelScore mean here? One thing I understood is a negative score here means I lose weight but how does the number signify?

How-To Section

Hi there,

Great work, really admired what you've done here. But for the someone who doesn't know a thing about machine learning, it could be great to have step-by-step reproduction of this mechanism.

Thanks a lot

What does a unexpected "Constant" row mean in output file "scores.txt"?

Hi, Arielf, I got something like this(factors in chinese):

FeatureName HashVal MinVal MaxVal Weight RelScore
Constant 142054 1.00 1.00 -0.02 -100.00
花生 247270 1.00 10.00 -0.01 -73.55
素三鲜白菜 176662 1.00 20.00 -0.00 -23.43
韭菜鸡蛋饼 86006 1.00 20.00 -0.00 -22.81
......

I don't have any factor "Constant", is it a bug?

What is the cycling travel distance?

It should also be possible to cut fat and maintain carbs with adequate physical training. So I am curious, what is the cycling distance each day?

Specify what "half-and-half" means

Specifically in the part: "I switched from milk to half-and-half". My initial reaction was "half and half" of what? But Googling "half-and-half" yields this wikipedia page, so from the context I assumed you meant half-and-half milk and cream, however even those can range in the fat percentage quite widely.
Could you specify what you meant so that it's clearer for those of us outside of the US?
Thanks. Awesome work, by the way!

<redacted>

Please update your weight loss progress since July/Aug 2016

Love this N=1 work. I've done Atkins and took my weight from 232 to 208 - but since then I'm back around 214 and looking to kickstart my own journey.
Would love to hear what you've learned since mid 2016.

What does "nosleep" exactly mean?

What does "nosleep" exactly mean?
Haven't you slept in all continuous 24 hours?

And, how about record the sleep data in this pattern: "sleep:6.5" (sleep[:hours]) ?

Thank you!

semi-colon in Python

Hi,
First off thanks for the great initiative.

What is the purpose of the semi-colon in line 160 of vw-varinfo2 and in line 352 of same?

It's just that me and pylinter flag this as an error. But, I just wanted to make sure the semi-colon is not there for some higher purpose ;-)

Making sense of the results

I think I must have collected the data wrong, but here's the SO question.

https://stats.stackexchange.com/questions/298151/making-sense-of-the-results-of-machine-learning-feature-selection

Here's an example of the data format I tried to mimic (ariel.csv)

Date,MorningWeight,YesterdayFactors
2012-06-01,186.8,
2012-06-10,185.9,
2012-06-11,182.6,salad sleep:0.15 cheese egg halfnhalf:1.5
2012-06-12,181.0,sleep:0.5 tea grape
2012-06-13,183.6,bagel bread date:1.5 dietsnapple splenda pizza nosleep:0.2
2012-06-14,183.7,coffeecandy cheese rice salmon peanut bread

Here's what my data looks like

date,trophies,characters
# NescalRdical UQP2UYLR
2017-07-26,0,
# friendly battle 3200 3200
# friendly battle 3200 3200
# friendly battle 3200 3200
# friendly battle 3200 3200
# friendly battle 3200 3200
2017-07-26,0,miner2 minion_horde12 electro_wizard2 the_log2 hog_rider8 snow_spirits9 princess2 zap11
# ARTURO QRPRCU2J
2017-07-26,-25,
# friendly battle 4025 4025
# friendly battle 4025 4025
2017-07-26,-25,goblin_gang12 barbarians12 minion_horde12 wizard9 miner2 giant9 zap12 mini_pekka9
2017-07-26,0,skeleton_horde4 goblin_gang11 rocket9 minion12 prince6 goblin_barrel5 princess1 zap12
2017-07-26,-25,goblin_gang11 archers12 rocket8 rage5 wizard9 chr_balloon6 angry_barbarian12 zap11
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
# friendly battle 3996 3996
2017-07-26,-50,fire_fireball9 baby_dragon6 rage5 minion_horde12 freeze6 angry_barbarian13 zap11 inferno_dragon2
# friendly battle 3966 3966
2017-07-26,-75,executioner5 rage6 tornado6 electro_wizard2 angry_barbarian13 hog_rider9 bomber13 the_log2
2017-07-26,-50,skeleton_horde6 valkyrie9 electro_wizard2 the_log2 hog_rider9 goblin_barrel6 ice_wizard2 princess2
2017-07-26,-25,goblin_gang12 firespirit_hut9 minion11 valkyrie8 musketeer9 lightning6 royal_giant12 zap12
# tilted23 289CVLCY9
2017-07-26,-50,
# friendly battle 3075 3075
# friendly battle 3075 3075
# friendly battle 3075 3075
# friendly battle 3075 3075
2017-07-26,-50,executioner4 fire_fireball9 giant9 goblin_archer9 angry_barbarian11 mega_minion8 princess2 zap11
# friendly battle 3046 3046
2017-07-26,-75,skeleton_horde3 fire_fireball8 baby_dragon5 valkyrie8 musketeer8 hog_rider8 zap10 inferno_dragon2

Edit: Do you think I should reset the counter to 0 (it increases or decreases by 25 for win or lose) with every new player?

Example:

# playerName playerID
arbitrary-date, 0, # NOTE Erased characters on every first game because they impact the score of a game before. Might be easier to explain with math. No n characters on the first line because n-1 game is missing.
same-date, 25, a b c d e f g h
same-date, 50, a b c d e f g h
# playerName playerID
same-date, 0, #??????? Go back to zero here?
same-date,- 25, a b c d e f g h
same-date, 0, a b c d e f g h

Mac ggplot2 issue

Hi Ariel, thanks so much for your work on this.

I'm working on the Mac instructions for a pull request, and have gotten mostly everything working (it isn't intuitive or even all that Google-able) but for some reason, the chart it's printing out looks like this:

For the record, I had to change the font from FreeSans, so that could be part of it. I switched to generic Helvetica.

Errors with Python

This is a great project. I experienced some errors getting it to run on my laptop, the worst of which were due to my version of Python. I'm running Mac OSX 10.11.6, with Anaconda installed so my default Python version is Python 3.5.1.

This produced a number of syntax errors. For example, line 418 of file vw-varinfo2 throws an invalid syntax error for the line that says:

sorted_tuples = sorted(F_WEIGHT.items(), key=lambda(k,v): v)
                                                   ^

It didn't like using a tuple as the lambda parameter. I fixed that but then ran into some other syntax errors, so rather than fix them I switched to the version of Python that ships as the default with OSX (Python 2.7.10), and everything ran except that "make sc" still throws the following errors:

/usr/local/Cellar/r/3.3.0_1/R.framework/Versions/3.3/Resources/bin/R: line 209: /usr/local/Library/ENV/4.3/sed: No such file or directory
/usr/local/Cellar/r/3.3.0_1/R.framework/Versions/3.3/Resources/bin/R: line 209: /usr/local/Library/ENV/4.3/sed: No such file or directory
Fatal error: cannot open file '': No such file or directory
make: *** [sc] Error 2

Request for details about the specific algorithm used

This is not an issue but a request to get details about what specific machine learning algorithm was used. This would help in cases where one would want to consider using a different ml package perhaps. Also it would be of great help to the uninitiated if details could be provided about what does the algorithm do.

Thanks

RE: Please Update with most recent data

While recently listening to a number of people give contradictory health information, I had a co-worker and friend tell me I should start performing my own measurements based on my intake and exercise and find out what does and doesn't work.

I thought about this and decided, after a recent conference, that I needed to apply machine learning to the problem. I'd like to hear more about your current progress or regressions, and see what updates you have in the way of insight you've gained since your last commit.

Relevant Issue

Selfies?

Really awesome job reading the data. I like that you were able to consider fasting time and how it relates to sleep. Question: Before and after pictures? Maybe you have a photo around the time of acceleration as well? Thanks

Extending the fasting time vs. eating fewer calories

In the docs, you mention

Extending the fasting time was one major accelerator of weight-loss rate.

You also state

Sleeping longer consistently appeared as the #1 factor in losing weight.…It took me a while to figure out the sleep part. When we sleep we don't eat. It is that simple.…Our sleeping time is our longest daily fasting time.

I was surprised by your conclusion and, after thinking about it, thought, "That makes sense."

After thinking about it some more, I'm curious how you determined longer fasting was a factor in and of itself. There is more than one variable there, it seems. If you skip breakfast, you're not just fasting longer; assuming you ate at roughly the same rate throughout the day on a day you didn't skip breakfast, you're also eating fewer calories. That is, if you normally ate 3 x 800 calorie meals, but then switched to sleeping longer and skipping breakfast, then ate 2 x 800 calorie meals, you're also cutting your calorie consumption by ~33%.

Did you eat an amount of calories equal to what you would've eaten during breakfast throughout the day to be more confident that it was fasting time alone that lead to the weight loss?

If so, that would be both a surprising and valuable insight, and worth mentioning!

How do you know you're in ketosis?

Thanks for sharing this.

Can you give me any idea of how much carbs is too much carbs in any single day? And how do you know you're in ketosis? (I know about keto sticks, but having tried this in the past it becomes sort of an obsession that I'd like to avoid)

Related: you mention you could get a way with having a pizza once a week -- wouldn't that kick you off ketosis? If so, how long does it take you to get back into ketosis?

Coffee

Could you briefly explain why coffee is not in the graph? Do you entirely drink tea/no tea? If not, would you say anything significant about coffee vs tea?

It might not be a primary contributor to weight results, but as secondary I am interested to see if there are any possible conclusions to draw from.