Cannot load ZIP gem under certain conditions about shoes3 HOT 42 CLOSED

Confirmed bug for Windows and 3.3.8. For the zip-2.0.2 gem, it's require 'zip/zip' . There is also a 'rubyzip' gem. I'm not sure which one is better but don't install both they may conflict. I'll test with zip-2.0.2. It's pure ruby gem to that makes things easier to debug, maybe. You could try Shoes 3.3.7 but that might have other issues for you.

from shoes3.

rubyzip appears to be more current than the zip gem. There us also a good chance that these gems can conflict with the built-in Ruby Zlib. Depending on what your doing with a zip using the builtin is going to be more stable.

from shoes3.

We have a workaround! The problem is 3.3.8 on windows is it's installing gems differently depending on Cobbler vs command line. Either one fails BUT - using both works.
(1) delete 'zip 2.0.2' with Cobbler.
(2) from Cobbler, search for 'rubyzip' and install. Quit Shoes
(3) from command line 'cshoes.exe -g install rubyzip'

Now try your code with a require 'zip' . Works for me on Windows 10. Make sure to use the API from rubyzip. There is a README.md at {HOME}:AppData\Local\Shoes\gems\rubyzip-1.2.3 and some examples that might help too.

from shoes3.

Oah.. I did not know that Shoes have its own archive handler.
I am on foot today but I will check this tomorrow and let you know.
Thank you for the tips and quick solution!

from shoes3.

It is all nice but zlib cannot handle .zip archive

I do tried with your approach to install rubyzip but I get this when trying to open archive :

Error in <unknown> line 0 | 2019-08-25 20:26:28 +0300
Zip end of central directory signature not found
C:/Program Files (x86)/Shoes/lib/ruby/gems/2.5.0/gems/rubyzip-1.2.3/lib/zip/central_directory.rb:143:in `get_e_o_c_d'
C:/Program Files (x86)/Shoes/lib/ruby/gems/2.5.0/gems/rubyzip-1.2.3/lib/zip/central_directory.rb:103:in `read_e_o_c_d'
C:/Program Files (x86)/Shoes/lib/ruby/gems/2.5.0/gems/rubyzip-1.2.3/lib/zip/central_directory.rb:136:in `read_from_stream'
C:/Program Files (x86)/Shoes/lib/ruby/gems/2.5.0/gems/rubyzip-1.2.3/lib/zip/file.rb:76:in `block in initialize'
C:/Program Files (x86)/Shoes/lib/ruby/gems/2.5.0/gems/rubyzip-1.2.3/lib/zip/file.rb:75:in `open'
C:/Program Files (x86)/Shoes/lib/ruby/gems/2.5.0/gems/rubyzip-1.2.3/lib/zip/file.rb:75:in `initialize'
C:/Program Files (x86)/Shoes/lib/ruby/gems/2.5.0/gems/rubyzip-1.2.3/lib/zip/file.rb:97:in `new'
C:/Program Files (x86)/Shoes/lib/ruby/gems/2.5.0/gems/rubyzip-1.2.3/lib/zip/file.rb:97:in `open'
main.rb:43:in `block in <main>'
eval:1:in `instance_eval'
eval:1:in `block in <main>'

from shoes3.

The good news is Shoes has done it's task - load the gem. What the gem does after that ? Not a Shoes problem ;-) Given it's all Ruby in the gem and you have a traceback and an error message from the gem, you could debug the gem. My guess is the file is not a zip format that the gem can read. There are a lot of zip formats. When I skimmed through the gem, I saw it tries to handle some really old format. That coupled with it's age (works in Ruby 1.9.2!) suggests no one is maintaining it.

My guess is the zip file doesn't have a good directory signature (if at all) and you need to set some flags or options before trying to open it.

from shoes3.

Thanks for the help. It was my bad.
I changed the .zip file with another extension to test zlib and forgot about it when installed rubyzip.

Final line:

• removed rubyzip
• used 'cshoes.exe -g install zip' to install zip 2.0.2

Now all is working well. Technically both libraries -rubyzip and zip 2.0.2 have the same issue through the cobler. This makes me assume that there may be many such issues with other gems.

What is the difference between cshoes gem installation and Cobler gem installation?
if they are the same it seems weird they have different output.

from shoes3.

What is the difference between cshoes gem installation and Cobler gem installation? if they are the same it seems weird they have different output.

That is the path to a solution. Obviously, they are different and shouldn't be. In times past they were not different. It has to do with keeping user loaded gems in HOME\APPDATA\LOCAL instead of Program Files (x86)\Shoes

from shoes3.

I see. Should I assume that the design is to move gems to HOME\APPDATA\LOCAL but it was not fully complete on all levels thus issue happens?

Is there anything I can help with or it is too much work in things I dont understand (probably)?

from shoes3.

It used to work properly. It's a regression. I have done work in that area for Shoes 3.3.8. It's a regression error. Probably a simple oversight whan I fixed something else.

from shoes3.

Thanks! Closing this one.

Cant wait for the new version :)

from shoes3.

I believe I have it working properly (again). New Windows beta here. The error actually goes back to fixing gem things for OSX and 'cleanings up while I'm there', Thanks for reporting the bug.

from shoes3.

Yay :)

Just tested it. Some feedback
Not only does it work but the cobler and manual are somehow more responsive.
But I have big issues with my app, every click interaction is very sluggish, I guess it may clog the CPU to some extent as it jumps to 40% for about a second. The more code is hidden under a click operation the sluggish it is. And the slowness is significant compared to the previous 3.3.8 build

from shoes3.

A 40% jump occasionally sounds like garbage collection but you have to be looking at memory stats to see it (and it shouldn't take a multiple seconds, IMO). It is possible that Ruby 2.5 has changed threading behavior vs the Ruby 2.3 in Shoes 3..3.7. You could play with settings.wintmo values (see the manual). It could be that a low tmo value like 10 isn't giving the script enough time to run before pausing it to check for gui events. Which makes it look slow to respond. Maybe set a value of 50 or 80 to see if that helps. It would hurt download performance.

There is a ruby 2.5.5 threading problem with download method that I know about. There could be others.

from shoes3.

@ccoupe it is not threading (unless Shoes do some threading), the code the hangs is sequential, it is just the more the code and things it do on the canvas, the longer the slug. I will test what you say tomorrow and let you know.

from shoes3.

Gtk has a thread and ruby has it's threads and sometimes Shoes has a new thread. It's a weakness in how Shoes 3.2+ is done on Windows. For your testing pleasure and my curiosity I just put up a 32 bit beta. I see a slight difference in loading speed. It also uses ruby 2.4.5 instead of 2.5.5 in the 64bit beta.

from shoes3.

Sadly not much of an improvement on my side :(.

I do played with the value but it kind of does not make any difference whatever it is (1-100 range). Isnt that weird?

from shoes3.

(1-100 range)

That says the problem is not Shoes/Gtk threading or 32 vs 64. Two candidates down - it's progress, I did read about a MSFT 1903 bug that some people have but would likely affect Shoes 3.3.7 as well as .8, Probably not your problem. The changes to shoes 3.3.8 are mostly the newer Rubies. Some restructured C code but those are more cosmetic than functional changes. If you are using menus that could be different. Or possibly the touch screen changes -- do you have a touch pad/screen ?

from shoes3.

Btw the exe i got from the link was 32. I have not done any 64 bit installations so far.
No touchpad just a classical click do stuff. Tomorrow I may do some troubleshooting using the performance gadjet :). I will let you know how it goes.

from shoes3.

Huh? Most of the 3.3.8 beta's are 64 bit.

from shoes3.

@ccoupe regarding the 3.3.8 slug, I did some research. Unfortunately it is GTK3 issue. Check this thread here as well.

May be this can shed further light. Let me know if there is anything else I can test or I will have to leave 3.3.8 :(.

from shoes3.

The changes to shoes 3.3.8 are almost entirely for Ruby 2.4+. That's not to say the gtk3 and Windows is good - it's not. But Gtk3 is not as bad as some 'internet' people claim. We went through the gtk3 transition back in Shoes 3.3.0 and since. The suggestion to run a Windows profiler is good but probably a lot of work - I for one don't know that level of windows tooling. Shoes used to have a way to run 'gprof' so I can look into that but it's misleading since everything slows down under profiling so you need good test cases and I may not be able to rebuild 3.3.7 with gprof on so the base case would be missing.

I can also try to build Shoes 3.3.8 with Ruby 2.3.

from shoes3.

I have two tests you can try. At the beta site are two shoes exe. One is 3.3.7 an one is 3.3.8, both are built with Ruby 2.3, 32bit,debug symbols included - 'Rb23-shoes-3.3.7-gtk3-32.exe' and 'Rb23-shoes-3.3.8-gtk3-32.exe' Both install as Shoes by default so be careful. It's not quite the 3.3.7 you have but it should be very close. 3.3.8 also has a different cobbler menu scheme, just so you know.

We need to follow a protocol to narrow this down.

1. Write down what you do to get the performance problem with 3.3.8 that you reported.
2. Install the beta named Rb23-shoes-3.3.7 and test. If OK proceed to #3.
   If not OK. STOP and report your finding here.
3. Install the beta named Rb23-shoes-3.3.8 and test. Please report. If rb23-3.3.8 is OK then we know know the problem is not the new ruby api used in ruby 2.4. If the problem shows up then it is the new ruby API or something I added to 3.3.8

from shoes3.

Thank you. Will update you as soon as I collect all the data :)!

from shoes3.

Just to hammer a point about reproducibility, I made a mistake generating Rb23-shoes-3.3.8. Sigh. That has been corrected. I also provided a 'Rb24-shoes-3.3.8-gtk3-32.exe' and ' Rb25-shoes-3.3.8-gtk3-64.exe' which use different Ruby versions and the Rb25 is 64 bit. Again, only proceed from one version to the next if everything i(performance wise) is OK.

Garbage collection has always been a focus for versions of Ruby so I would be no great surprise for something to change although yours is extreme. I know you use lots of Images which will cause GC to occur sooner and depending on how they are used may not be eligible for GC which could cause a long delay when ruby goes looking for memory and it can't find enough, Other things to consider or note:

1. How much memory is in your Windows machine?
2. What other apps are loaded in Windows? Are they memory hogs and is Shoes fighting with them to get enough memory. When that happens Windows can get really slow (Linux and OSX too but Windows is worse).

from shoes3.

No worries I have not tested it yet. I did the 3.3.8 tests on 2 separate laptops with windows 10. They have respectively 12 and 16 GB or memory.

from shoes3.

Windows 10 tests:

Rb23-shoes-3.3.7 - Has mouse offset.
Rb23-shoes-3.3.8 - works great!
Rb24-shoes-3.3.8 - sluggish
Rb25-shoes-3.3.8 - sluggish

from shoes3.

Now we are getting somewhere! It appears the the switch from Ruby 2.3 to 2.4 is causing the problem. This may be challenging because I or you are using a ruby method or API that changed behavior for the worse in 2.4 (and 2.5). A scan of the 2.4 release notes is needed by both of us and the more detailed Changelog.

from shoes3.

Or fixes like https://bugs.ruby-lang.org/issues/12690

from shoes3.

@ccoupe FYI I managed to do a clean release with the 3.3.7.
I am glad that things are now clear on what is the root cause. I do organized the manual repo.

If you want to use it for tests, just download it and run the main.rb under shoes. It should be working.
However I never run it on Linux so if it doesn't work dont mind to bug me so I can adjust the code properly.

from shoes3.

@dredknight, Linux has a case sensitive file system - I had to rename code/readskills.rb to code/ReadSkills.rb to match the 'require on line 6 of main.rb,

On linux(Mint), it seems to work for me - but I was just clicking around (using shoes 3.3.8) - Is there a sequence of things to do to get the slowdown?

I'm having trouble getting the zip gem to load on my raspberry pi and MacOS -- although different errors -- Odd but very on topic.

I also get warnings about class vars used at Toplevel. These could be a problem. I'll poke around with my clone of the code.

from shoes3.

Thank you, I will update the github filenames accordingly. Feel free to do as you wish :).
Nothing to poke around, just decided to share it as a testing data.

The most cumbersome part (as a Windows user) is when you change heroes with the left/right button at the "Hero pane" (the one with the skill circle).

from shoes3.

Ahh, the gem to install is rubyzip not 'zip'. We're OK on linux (x86 and raspberry). OSX claims the settings.db is locked but I'm not going to worry about that problem. On Linux and Win 10 (shoes 3.3.8 64 bit Ruby 2.5.5), I don't see the slowdown/pause - how many clicks does it take?

from shoes3.

@ccoupe this is the one that I do tested with the slowness? Btw I just now recalled that @backorder pinned each x.x.9 release to be for performance fixes. Let see if he summons around ;).

I will make a short video to show the slowness and post it.

from shoes3.

@dredknight , Don't believe future anything about 3.3.9. I'm not sure there will be one.

from shoes3.

Sure I understand the state of the project, decided to see if I can recall @backorder :).

from shoes3.

Is there a chance that during the slow down that the sqlite3 db is being accessed? I ask because I don't see any index'es in the schema.

The pics are 14MB on disk so probably double or for main memory. The image cache.probably has all 14MB and untold number of Shoes image objects to be collected plus the Gtk3 image (may be an internal copy there too?)

This needs profiling but I'm not familiar with the windows tools.

from shoes3.

@ccoupe, indeed it is accessed during that phase!

What the code do is while scrolling between heroes, based on the hero index it gets a table with all his skills and perks, one by one, then based on skill/perk name it draws the image and prepares the hover pop up.

from shoes3.

Creating an index for the table(s) and columns involved may speed up many things and is something you can do w/o touching Shoes or Ruby. Just make a backup first. syntax:

I did notice that moving the cursor over icons (to get tooltips) took 25% of a fast processor which I though was odd. Lack of indices could explain it.

from shoes3.

Thank you for the idea!
I will study what you sent me carefully and update with details :). Even though there is nothing to do on the wheel at the moment this seems like really fun project! I will keep you posted and may even do a wiki about this!

from shoes3.

Here's an example of a script I used to create a DB, some tables, and index's. The app analyzes downloads on my shoes download logs.

from shoes3.

There is a feedback to return here - indexes help to speed the wheel!
Though as we spoke before Shoes with old Ruby is working way better so I suppose they've done something to it.

Also closing this issue as you did add fixes to the core.

from shoes3.