JSON parsing error in `zmq--download-module` about emacs-zmq HOT 11 CLOSED

Hmmm...that is weird. Looking at url-http-end-of-document-sentinel, it could be that the server is closing the connection on us. Can you change the url to http and see if that goes through more easily.

Either way, it would be a good idea to handle failures instead of expecting success.

from emacs-zmq.

Changing the url to http doesn't fix it -- I still get the empty string "".

from emacs-zmq.

I did a git bisect, and the error (or related ones) start appearing in eb96b39, when zmq--download-module was introduced. Before that commit, make test runs fine for me.

On a related note -- copying over an older, installed copy of emacs-zmq from my .emacs.d/elpa to the .cask directory allowed me to get tests working for emacs-jupyter.

from emacs-zmq.

I just did a cask update and the download went through on OS X and it works on Travis most of the time. On Travis there is never an error, but sometimes it just doesn't download anything and builds the module instead. I have seen this error when testing emacs-jupyter on Appveyor but I didn't bother looking into it much and started downloading the module manually when testing on Windows, it may be the same problem.

Are you trying the url-retrieve-synchronously call multiple times and it fails each time until you call url-retrieve? url-retrieve is used internally by url-retrieve-synchronously so I'm thinking it has something to do with how url-retrieve handles bad connections. Can you try downloading something else, e.g. https://ftp.gnu.org/gnu/emacs/emacs-lisp-intro-2.04.tar.gz, and see if you get the same problem.

I think what we can do is check for common programs for downloading files like curl or wget, use those for downloading, and fall back to url-retrieve-synchronously if needed. Then if we have to fallback to url-retrieve-synchronously, resort to building the module if we get a JSON parsing error.

from emacs-zmq.

Here's a summary of some URLs that do and don't work with url-retrieve-synchronously on my system:

#+begin_src emacs-lisp :results silent
  (setq url-debug t)

  ;; call url-retrieve-synchronously and print a substring of it
  (defun my-retrieve-synchronous (addr)
    (with-temp-buffer
       (url-insert
        (url-retrieve-synchronously addr))
       (buffer-substring-no-properties (point-min) (min (point-max) 100))))
#+end_src

Request to api.github.com returns nothing

#+begin_src emacs-lisp
  (my-retrieve-synchronous
   "https://api.github.com/repos/dzop/emacs-zmq/releases/tags/v0.10.9")
#+end_src

#+RESULTS:

Request to another https address works

#+begin_src emacs-lisp
  (my-retrieve-synchronous
   "https://ftp.gnu.org/gnu/emacs/emacs-lisp-intro-2.04.tar.gz")
#+end_src

#+RESULTS:
: ��\0{F�<��<kwj
: ]Qd;NW#'{l,��m\0C"MwFM-���3y��-[+��Qh�

Request to api.github.com still returns nothing

#+begin_src emacs-lisp
  (my-retrieve-synchronous
   "https://api.github.com/repos/dzop/emacs-zmq/releases/tags/v0.10.9")
#+end_src

#+RESULTS:

Request to HTTP version still returns nothing

#+begin_src emacs-lisp
  (my-retrieve-synchronous
   "http://api.github.com/repos/dzop/emacs-zmq/releases/tags/v0.10.9")
#+end_src

#+RESULTS:

Request to a different URL on api.github.com still returns nothing

#+begin_src emacs-lisp
  (my-retrieve-synchronous
   "https://api.github.com/repos/conda/conda/releases/tags/4.6.14")
#+end_src

#+RESULTS:

Note I was having some trouble consistently reproducing the effect where calling url-retrieve once will allow subsequent url-retrieve-synchronously to work -- sometimes it works, sometimes it doesn't. The requests started giving me an API rate limit exceeded preventing me from further testing to figure out reproducible conditions.

from emacs-zmq.

One thing I noticed when comparing to your failed download to your successful download is the (:peer (:certificate ... stuff on the http -> Activating callback in buffer line in the successful download output. This is nil for the failed download. Maybe it has something to do with network security? Can you try setting network-security-level to low (https://www.gnu.org/software/emacs/manual/html_node/emacs/Network-Security.html) so we can rule that out.

from emacs-zmq.

Just tried this and unfortunately didn't work :/

from emacs-zmq.

This bug report makes me think it still has something to do with network security since it mentions a problem where only the HTTP headers are returned. Does the variable system-configuration-features contain GNUTLS on your system?

from emacs-zmq.

Yes it does.

from emacs-zmq.

OK, I figured out why Travis sometimes downloads the module and sometimes doesn't. Its because the API rate limit is hit on Travis sometimes. On Appveyor, its because Emacs on Windows doesn't come with built-in support to handle HTTPS links.

I've just pushed a change that tries to download the module using curl, then wget and falls back to url-retrieve as a last resort.

from emacs-zmq.

Thanks, it works now :)

jack@jaheira:~/src/emacs-zmq (git)-[master]- % make test
emacs -nw -Q -batch -L . -l ert -l zmq-tests.el \
	--eval "(ert-run-tests-batch-and-exit)"
Checking for compatible module binary to download
Downloading https://github.com/dzop/emacs-zmq/releases/download/v0.10.9/emacs-zmq-x86_64-linux-gnu.tar.gz
Verifying sha256 signature of emacs-zmq-x86_64-linux-gnu.tar.gz
uncompressing emacs-zmq-x86_64-linux-gnu.tar.gz...
uncompressing emacs-zmq-x86_64-linux-gnu.tar.gz...done
Parsing tar file...
Parsing tar file...done
Extracting emacs-zmq-x86_64-linux-gnu/
Extracting emacs-zmq-x86_64-linux-gnu/emacs-zmq.so
ZMQ Version: 4.3.1
Running 11 tests (2019-05-16 15:00:38-0700)
...

from emacs-zmq.