Doesn't use both threads on each core when running on CPU about min-dalle HOT 3 OPEN

I tried running two runs at the same time, and I got about the performance hit I would expect from having something on the other thread in a core, and I saw my CPU get fully utilized. I don't think using both threads in each core would really double performance, but it would increase it by a fair bit.

time python image_from_text.py --text='alien life' --seed=8 --torch running alone:

real	0m59.319s
user	5m41.542s
sys	0m14.994s

And with another run also running at the same time:

real	1m36.393s
user	9m33.206s
sys	0m17.905s

User time counts time the same whether there's something on the other thread of your core stealing your CPU machinery or not, so look at the real time times. 59 seconds / (60 seconds + 36 seconds) = 61%, so if I run two runs simultaneously each runs at about 61% of the speed of a single run alone. If one single run could use all that compute it would be running at about 120% speed, or about 20% faster than it runs now.

from min-dalle.

I merged your pull request that adds the line torch.set_num_threads(os.cpu_count()) but it still doesn't seem to be using all the threads on my M1 macbook. I agree there should be a way to better utilize the cpu. I'm working on converting it to CoreML so that it can use the neural engine

from min-dalle.

Probably what we want is a --threads option to let the user specify thread count, and then a default detector that works right on both Linux and M1 Mac.

from min-dalle.