nyaapantsu / nyaa Goto Github PK
View Code? Open in Web Editor NEWNyaa.se replacement written in golang
License: MIT License
Nyaa.se replacement written in golang
License: MIT License
There can be another shutdown situation.
and this will be ultimate way to block the shutdown. TOR-ONION DISTRIBUTION
, or i2p.
I know this can be looked like little illegal (which is absolutely not), but should be done for now days.
So many censorship, blocks, bullshit shutdowns. needs some solutions for that.
Earlier i read somewhere about people not being able to get past page 1000 when scraping stuff from before 2010.
So i raise you these in the hopes that you may be able to scrape some recent stuff and old stuff from it.
https://web.archive.org/web/*/nyaa.se
https://web.archive.org/web/*/sukebei.nyaa.se
https://web.archive.org/web/*/nyaatorrents.org
https://web.archive.org/web/*/sukebei.nyaatorrents.org
https://web.archive.org/web/*/nyaa.eu
https://web.archive.org/web/*/sukebei.nyaa.eu
All of the torrent links and magnet hashes appear to work (i.e. torrent files are downloadable).
Earliest snapshot of nyaa.se is 2012 and the latest is april 30 2017.
Earliest snapshot of sukebei.nyaa.se is 2013 and latest is april 30 2017.
Earliest snapshot of nyaatorrents.org is 2006 and latest is april 30 2017.
Earliest snapshot of sukebei.nyaatorrents.org is 2008 and latest is april 30 2017.
Earliest snapshot of nyaa.eu is 2011 and latest is april 30 2017.
Earliest snapshot of sukebei.nyaa.eu is 2011 and latest is april 30 2017.
Not much of an issue, just a suggestion. Dark green just doesn't look as good.
On IRC we discussed making a custom 404 page. Original nyaa had a 403 page with a cute miku (probably something similiar on 404 page but i'm not able to retrieve that one); As the new page is more Megumin/Renchon themed, we should pick a new one. Please send your suggestions in this issue.
My suggestion:
(ofc this would need to be completely redone from scratch, but i can do it)
For people who are not so familiar with Go to be able to figure it out themselves.
On every page load the following is printed to console:
(can't preload field Sub_Categories for main.Categories)
Although we aren't doing it yet, ostensibly this means displaying subcategory information on the main page will fail. This is related to all the query functions with Preload("Subcategories") in them. No idea what the exact problem is.
(/root/go/src/github.com/ewhal/nyaa/db/gorm.go:33)
[2017-05-06 15:34:09] [5.39ms] CREATE TABLE "torrents" ("torrent_id" integer,"torrent_name" varchar(255),"category_id" integer,"sub_category_id" integer,"status_id" integer,"torrent_hash" varchar(255),"date" bigint,"downloads" integer,"filesize" varchar(255),"description" blob,"comments" blob )
(/root/go/src/github.com/ewhal/nyaa/db/gorm.go:33)
[2017-05-06 15:34:09] [5.19ms] CREATE TABLE "categories" ("category_id" integer,"category_name" varchar(255) )
(/root/go/src/github.com/ewhal/nyaa/db/gorm.go:33)
[2017-05-06 15:34:09] [4.09ms] CREATE TABLE "sub_categories" ("sub_category_id" integer,"sub_category_name" varchar(255),"parent_id" integer )
(/root/go/src/github.com/ewhal/nyaa/db/gorm.go:33)
[2017-05-06 15:34:09] [4.35ms] CREATE TABLE "statuses" ("status_id" integer,"status_name" varchar(255) )
Sorting based on various parameters, such as size, seeders, etc.
Before enabling upload, there NEEDS to be a report system to prevent random people from uploading trash to the site, and a Moderator Team to oversee said system.
Preferably not the old team from Nyaa, they are quite "famous", and some torrents they marked as "A+" (blue) are actually troll translations.
cough maybe someone willing to comb through reports cough
On nyaa there were a handful of primary uploaders, this feature would allow an uploader to link their old user with their new one or restore their old account.
An uploader is required to verify that they own the old user by release website, forum profile (anime-sharing, 2djgame, mikocon, etc), IRC, twitter, etc by looking at their history and require a verification post or message on said website. The verification process will be verified by 2 or more moderators.
The uploader can request this feature via IRC. (help channel?)
All Existing torrents should be locked to would prevent malicious users that got past the verifying process or are now malicious from modifying torrents.
Methods
Whats your ideas for displaying older torrents for users that "imported" their accounts? (2,3)
A step into the future by going back.
Yea I know IE is shitty browser but some still uses it (I personally mix browsers depend on what I going to do as sometimes IE surprise me and do things that other browsers don't), and iirc it worked just fine in old nyaa and other sites, is there any chance you will make it work with magnets? it not that big of a deal really, but if you try your best to bring back site that be better than old nyaa (which I really thankful for it btw... don't think many really said it despite being thankful themselves) it will be nice to see it beat it in any aspect :P
Custom 404 works, but only when you try a page on root that doesn't exist (such as "/asdfasfasf"). Old 404 appears, if you try a page in directory css, js or img(such as "/img/asdfasdfasfd").
You have chosen to use a .cat domain. Since the website is not directly related to the Catalan community nor translated into Catalan, the domain is likely to be suspended in the future, as the registry authority has already done in the past with other domains.
However, if you need a Catalan translation, we can help! The site should be prepared to accept multilanguage, and then we can translate anything you need! ;)
https://sukebei.pantsu.cat is showing a wrong date in the Date column.
The idol png pics previously on nyaa were archived. Here's a mega link with the 14 images that used to display on the info pages
Just a thought but I am sure many would not mind hosting p2p type docker containers on our seedbox's?
DBParams seem to always be "./nyaa.db" even when passing -dbparams
EDIT: Even passing -conf doesn't seem to work.
So there were some SDF Macross batches some hours ago, but now I'm looking for them and they aren't there. is this a problem with the database (sorry, not informed on the actual naming) or what? They are like the only ones working anywhere for this series (>;_;)>
I can make an elastic search api server which would help with storing and finding torrents.
Reply to this issue to give me the go or just close it :3
(/opt/golang/src/github.com/ewhal/nyaa/db/gorm.go:34)
[2017-05-06 12:22:39] [2.56ms] CREATE TABLE "torrents" ("torrent_id" integer,"torrent_name" varchar(255),"category_id" integer,"sub_category_id" integer,"status_id" integer,"torrent_hash" varchar(255),"date" bigint,"downloads" integer,"filesize" varchar(255),"description" blob,"comments" blob )
(/opt/golang/src/github.com/ewhal/nyaa/db/gorm.go:34)
[2017-05-06 12:22:39] [2.37ms] CREATE TABLE "categories" ("category_id" integer,"category_name" varchar(255) )
(/opt/golang/src/github.com/ewhal/nyaa/db/gorm.go:34)
[2017-05-06 12:22:39] [1.82ms] CREATE TABLE "sub_categories" ("sub_category_id" integer,"sub_category_name" varchar(255),"parent_id" integer )
(/opt/golang/src/github.com/ewhal/nyaa/db/gorm.go:34)
[2017-05-06 12:22:39] [1.97ms] CREATE TABLE "statuses" ("status_id" integer,"status_name" varchar(255) )
then no response
could not open ip:9999 as well
PS. I run it on the server.
Can someone put a link somewhere for the sql database actually used by the website? Would be more easy for the future add-on (like comments, users, ...)
someone on the IRC suggested https://github.com/blevesearch/bleve, which might be a good choice since base is already in Go.
Currently, the search takes everything for granted, so typing 'horrible kemono 1080', for example, would return no search results.
Mostly due to the HomeTemplateVariable
object not being passed the correct parameter types, specifically:
Navigation
url.URL
mux.Route
Anyone interested in assisting with any aspect of the frontend development pertaining to Nyaa's revival should mail [email protected] so we can better coordinate efforts. Communications will most likely take place through discord.
For easier reference by the frontend team as a whole, I'd like to request that all new feature requests / upgrades from old nyaa / fixes be commented to this document.
Individuals should still make separate issue threads for each new idea, feature, upgrade request etc so they can easily be seen by the community here.
Cheers for free media.
I think it should have this following features
Then improve performance I think Docker is a good choice to made it scalable.
Thanks,
Wiput Pootong
There needs to be a SPAM filter before enabling upload, to prevent people and bots from uploading garbage into the site.
Here are some ideas for that, along with the moderation system mentioned on #35
The main problem is that this being an OSS project, you'd need to either:
IMO the second solution is best, while also lending your own version of the filters to other (trusted) people for them to host their version of nyaa.
Not really an issue but functionality.
Based on >>60217123
can you put page number 1 and make it bold on when you are on that page, also make it so its like 1 2 3 4 5 .... (last page number) and give an option so you can type in a page number you want to go to
So I assume that subaki still underwork since it under heavy load most of the time and still don't have updated database (and may never will :( )
But I noticed some bug that may be worth reporting. it seem that the image category don't fit the file, let say I look for random H game, it will find it but will be under looseless audio categorey
I think the serach option is also a bit buggy but I saw that it was already mentioned in other thread
The date at the torrent details is a timestamp.
We need better logging and error handling in the code.
For example, if the app can't open a connection to the database, it just panics and crashes. Crashing is fine but it'd be better if we had clear error messages printed to log files / stdout.
So anyone that wanna contribute that's one thing that can be done.
So my Utorrent is not in English (and so my system) and each time I add magnet link only from you guys, each line that been wrote in Japanese latters will be shown as question mark in my uTorrent,
It seem to happend only for the magnet part (at least for now, need to check stuff with other torrents) as once it changed to the torrent everything look alright.
The thing is that it never happened to me before on any site, is this thing can be fixed or it just how things will be from today onward?
RSS feeds for specific search queries.
So it seem you managed to get most of nyaa back, but my main question is, if you going to let people upload torrents again in the future or at least add magnet links, TPB proved that you can still show seeders/peers and files previews (at least before their last fall in 2015) even when you don't use trackers (I think...).
As for the FAQ it saying in the last line that "nyaa.pantsu.cat and sukebei.pantsu.cat do not host any files."..... is it mean that the FAQ still not finish (as it seem to be only title of unfinished subject) or that you completely gave up on let people upload stuff and pantsu nyaa will be simple magnet archive for old torrents without even having the option to add new contact?
This is for Subekei
The trackers provided with the magnet link for Files 712278 and 711897 (picture 1) are showing incorrect (fake) seeds for the swarm and also the incorrect number of peers (pictures 2 and 3). DHT seems to have it correct.
However the same trackers for 1332531 (picture 4) are behaving properly (picture 5). That leads me to believe that the trackers and torrent client are OK, but that there is something odd about how some of the files in the DB are being complied or indexed.
Windows 7, uTorrent 2.2.1, same results w/ and w/o VPN active.
I noticed this while searching for older series, but the search function doesn't function properly. I tried searching for the latest torrent that was uploaded before the ship sank, and it can't even return at least that result.
http://puu.sh/vIEzq/f0a306a9bd.png
I don't have much experience with coding, so I don't know what to test for, but I am willing to help.
1)The date column breaks the characters whenever there's a name that is longer than 1 line and the height of the rows is increased. 2)Also in some pages (i.e. page 4) the name column moves 10px to the right. Kind of trivial but annoying.
Where available additional metadata such as number of files, file names, size, etc. would be very useful.
File names are important in the case of multi-file torrents, such as music albums or batch releases to see whether they include certain bonus material. Sizes give a rough hint at quality, absent other information.
22:42 $ go build
_/home/tomleb/dev/go/nyaa
./main.go:224: cannot use "" (type string) as type Navigation in field value
./main.go:224: cannot use "" (type string) as type *url.URL in field value
./main.go:224: cannot use 1 (type int) as type *mux.Route in field value
./main.go:224: too many values in struct initializer
./main.go:226: undefined: templates in templates.ExecuteTemplate
./main.go:285: undefined: singleapiHandler
./models.go:210: t.Status undefined (type *Torrents has no field or method Status)
After a clean go get github.com/ewhal/nyaa.
Magnets and DHT do not work for torrents that contain the private flag. Since people occasionally create those even for public distribution there needs to be a way to deal with them.
At least we need to survey what percentage of the data is affected.
When I'm browsing, it'd be nice to not have to scroll all the way back up to go to the next page.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.