karpathy / arxiv-sanity-lite Goto Github PK

With running the arxiv_daemon, I mostly am getting the response from arxiv "Connection reset by peer", which loops and loops for 1000 times before I get the message

"ok we tried 1,000 times, something is srsly wrong. exiting."

Is there a reason you set it to 1000? Why is this looped in the first place, is arxiv supposed to be finicky about this? Regardless, I feel like hammering arxiv so much is probably not preferred. Perhaps set it to a lower value?

Strange thing is, it doesn't always happen. Sometimes, I do get a connection immediately and a proper response from arxiv. That never happens after a few loops of "Connection reset". Then, a minute later if I try it would loop for the full 1000 times again. Is this an issue on arxiv side (like I'm on a blocklist of one of their load-balancing servers), or is this an arxiv-sanity-lite issue? Any ideas?

Hi @karpathy, thank you for introducing arXiv Sanity Lite!

Few of my peers and I are developing Skim https://skimhq.tech - Spotify for ML World - inspired by arXiv Sanity.

Currently it supports:

• Creating a list of papers as "Rack" (similar to Spotify playlist)
• See similar papers based on TF-IDF based features
• See popular conferences and their racks (arXiv papers bundled into racks based on their yearly proceedings) and conference statistics as well - giving you complete information about a conference :)
• Search across all papers, racks, conferences and user base

We would like to discuss more and share an invite to you - so that we can collaborate on this and improve over time.
Please let us know - [email protected]

Hi @karpathy,

We built papers.labml.ai in May (introductory tweet) to discover research papers based on popularity on Twitter. We were using arxiv-sanity to discover papers and I started this as a side project inspired by it (partly because it was down from time to time).

We worked on it on and off since May and have added a bunch of features, such as:

• Popular papers based on Tweets
• Link source codes, annotated implementations, videos, Reddit and Hackernews discussions, and other resources related to the paper
• Conferences (iclr 2022, neurips 2021)
• Short two-line summaries of the papers to quickly browse through lists of papers
• Similar papers based on language model embeddings

And we are working on something very similar to tags on sanity-lite (which we call lists).

We love to hear your feedback and suggestions. Thanks for releasing your work.

Hi, great site :) would there be capacity to integrate bioarxiv articles in the future. I am aware of forks which have done this but they seem to be offline.

Add a line under each entry on the home page stating where the article was published. "Comments" on the arxiv.org website.

If I've added e-mail to my account and managed to forget the login than there is no way to unsubscribe from what I can tell. One simple solution is to add login to e-mail body.

I noticed that out paper is missing from arxiv-sanity. It was stuck in moderation for a while so maybe couldn't be indexed properly? I assume there might be other papers affected by the same issue.

Link to missing paper