maflcko / wiki-java-tools Goto Github PK

View Code? Open in Web Editor NEW

This project forked from mer-c/wiki-java

25.0 25.0 21.0 2.67 MB

Collection of tools for MediaWiki in Java

wiki-java-tools's People

Stargazers

Watchers

Forkers

hajer1991 arturmafezzoli steinsplitter edgarskos mrcity cjbtokyo amitbd1508 abdeaitali zeusnightbolt nirvananimbusa testler-c-2 guardian820 dheri 7ofnine

wiki-java-tools's Issues

[Imker] Exception

Sadly, the tool breaks when encountering quotation marks in a file name. How to reproduce? Try to download WLE 2014 files:

Both downloading from the page and from the category break, most likely after encountering https://commons.wikimedia.org/wiki/File:%22A%C4%9F_qaya%22_%C5%9F%C9%99lal%C9%99si.JPG.

@MarcoFalke Wikimedia Commons is on a unixoid system, so a.ext, A.ext, a.EXT and A.EXT are four different files (not to think about potential mixed case file extensions). But in Windows only one of these can actually be saved in the same directory/folder, because letter case doesn’t matter for this OS. When downloading a full category this leaves to errors and there is no possibility to download the duplicates (for Windows) with IMKER.

This could be avoided if there would be a possibility to add a string to the filename in these cases. This optional setting should be active for Windows by default, if possible. String proposal:

name-edit-by-IMKER

Even shorter would be, of course, the simple edit, but the former makes the intentional edit visible.

[Imker] Exception

While crawling the category "Images from the National Gallery of Art" I encounter this error.

Version: v16.09.13
Stack trace:

java.lang.NoClassDefFoundError: javax/xml/bind/annotation/adapters/HexBinaryAdapter
	at app.ImkerBase.calcSHA1(Unknown Source)
	at app.ImkerBase.checksumLoop(Unknown Source)
	at app.ImkerGUI$5.doInBackground(Unknown Source)
	at app.ImkerGUI$5.doInBackground(Unknown Source)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:304)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:343)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

perhaps related to #21

[Imker] mojibake

OS: Windows 10
Locale: Japanese

[Imker] Exception on proxy usage

If you have a proxy set up, the imker GUI catches an exception "Unknown Host Exception". Unfortunately, I don't have a stack trace at hand.
Even if you set the environment variables, the download fails

ImkerBase.java:22: error: package javax.xml.bind.annotation.adapters is not visible

$ gradle
Starting a Gradle Daemon (subsequent builds will be faster)
The Task.leftShift(Closure) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use Task.doLast(Action) instead.
        at build_2k5gc5er999n13dlu1zukgvsi.run(/home/nico/src/bean-or-not/download-commons-categories/wiki-java-tools/build.gradle:22)
:copy_deps_for_ant
Download https://repo1.maven.org/maven2/com/beust/jcommander/1.48/jcommander-1.48.pom
Download https://repo1.maven.org/maven2/com/beust/jcommander/1.48/jcommander-1.48.jar
:compile
[ant:javac] : warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[ant:javac] /home/nico/src/bean-or-not/download-commons-categories/wiki-java-tools/src/app/ImkerBase.java:22: error: package javax.xml.bind.annotation.adapters is not visible
[ant:javac] import javax.xml.bind.annotation.adapters.HexBinaryAdapter;
[ant:javac]                                 ^
[ant:javac]   (package javax.xml.bind.annotation.adapters is declared in module java.xml.bind, which is not in the module graph)
[ant:javac] Note: /home/nico/src/bean-or-not/download-commons-categories/wiki-java-tools/src/wiki/Wiki.java uses or overrides a deprecated API.
[ant:javac] Note: Recompile with -Xlint:deprecation for details.
[ant:javac] Note: /home/nico/src/bean-or-not/download-commons-categories/wiki-java-tools/src/wiki/Wiki.java uses unchecked or unsafe operations.
[ant:javac] Note: Recompile with -Xlint:unchecked for details.
[ant:javac] 1 error
:compile FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/home/nico/src/bean-or-not/download-commons-categories/wiki-java-tools/build.gradle' line: 26

* What went wrong:
Execution failed for task ':compile'.
> Compile failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Using Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)

[Imker] Exception "java.util.zip.ZipException: Not in GZIP format"

I get this error no mattter what I try to download - worked fine a couple of days ago.

The software can read file lists just fine, both ones I made myself or loading a category, but when actually attempts to download anything (of Commons) this error occurs:

Version: v16.09.13
Stack trace:

java.util.zip.ZipException: Not in GZIP format
	at java.util.zip.GZIPInputStream.readHeader(Unknown Source)
	at java.util.zip.GZIPInputStream.<init>(Unknown Source)
	at java.util.zip.GZIPInputStream.<init>(Unknown Source)
	at wiki.Wiki.fetch(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at app.ImkerBase$1.fetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.ImkerBase.downloadLoop(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at javax.swing.SwingWorker$1.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at javax.swing.SwingWorker.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

[Imker] Exception

Version: v16.09.13
Stack trace:

java.util.zip.ZipException: Not in GZIP format
	at java.util.zip.GZIPInputStream.readHeader(Unknown Source)
	at java.util.zip.GZIPInputStream.<init>(Unknown Source)
	at java.util.zip.GZIPInputStream.<init>(Unknown Source)
	at wiki.Wiki.fetch(Unknown Source)
	at wiki.Wiki.getImagesOnPage(Unknown Source)
	at app.ImkerBase$2.fetch(Unknown Source)
	at app.ImkerBase$2.fetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.ImkerBase.getImagesOnPage(Unknown Source)
	at app.ImkerGUI.fetchFileNames(Unknown Source)
	at app.ImkerGUI$1.doInBackground(Unknown Source)
	at app.ImkerGUI$1.doInBackground(Unknown Source)
	at javax.swing.SwingWorker$1.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at javax.swing.SwingWorker.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

Caused when scraping with the following argument in page field: https://commons.wikimedia.org/wiki/Sovereign-state_flags

[Imker] Add license information to filename

When downloading files, it would be cool if I could get their licensing information as well.

For instance De MARIA-JOHANNA uit 1926 voor de wal bij de erfgoed manifestatie 2017 van de LVBHB in Hasselt (02).JPG would get the name De MARIA-JOHANNA uit 1926 voor de wal bij de erfgoed manifestatie 2017 van de LVBHB in Hasselt (02) - Copyright S.J. de Waard, license CC-BY-SA-3.0.JPG.

Inconvenient: Names get a bit longer
Advantage: Easier to stay legal.

Imker: Unique identifier?

Hello,

I'm running into some issues with the file name; I have a script that will obtain an image's meta data from the mediawiki API using the file name of the downloaded image.

This can cause me trouble as some of the image editing tools (i.e., Lightroom) export images with normalized extension of ".jpg". I run into issues if the original file name was ".jpeg" or ".JPG".

Rather than depend on on the image name, is there a unique ID property that could be used for a file name when Imker downloads an image? Heck, do WIKIs, like mediawiki, provide unique IDs for images?

Thanks!

[Imker] CLI not exiting when child process in node.js

I am using node.js to run the jar as a child process.

Nether close or exit fire.

Looking at:

https://github.com/MarcoFalke/wiki-java-tools/blob/18ecd9ab0f69d7e528cb442f081bbdcf3397390e/src/app/ImkerCLI.java#L164-L178

Is there a need to System.exit(1) at the end of download to send the proper exit code?

From what I can see, my script is not getting any exit code, so the Java CLI just stops without error or success and I have to CTRL+C (which kill my parent process which I would like to avoid).

I could be totally barking up the wrong tree here, but do you think there's a way to make sure the CLI returns an exit code?

[Imker] [Settings] wiki url

Allow the user to set the wiki url like commons.wikimedia.org or any other wiki.

[Imker] Exception

Version: v16.09.13
Stack trace:

java.util.zip.ZipException: Not in GZIP format
	at java.util.zip.GZIPInputStream.readHeader(Unknown Source)
	at java.util.zip.GZIPInputStream.<init>(Unknown Source)
	at java.util.zip.GZIPInputStream.<init>(Unknown Source)
	at wiki.Wiki.fetch(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at app.ImkerBase$1.fetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.ImkerBase.downloadLoop(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at javax.swing.SwingWorker$1.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at javax.swing.SwingWorker.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

Imker: Add --yes option to cli

See #13 (comment)

Imker Exception for files with a ? in filename

Imker fails with a Java.io.FileNotFound exception when attempting to download the following file:

https://commons.wikimedia.org/wiki/File:Attributed_to_Andr%C3%A9s_Lagarto,_Mexican_-_Nun%27s_Shield_showing_the_Virgin_and_Child_with_Saints_John_the_Baptist_and_Catherine_of_Alexandria_(%3F)_-_Google_Art_Project.jpg

This is likely because of the presence of the (?) in the image title, which is an invalid character for Windows filenames.

[Imker] Exception (FileNotFoundException)

Version: v16.09.13
Stack trace:

java.io.FileNotFoundException: https://upload.wikimedia.org/wikipedia/commons/a/a3/Historical_map_of_Moldova_AD_1940.svg
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1890)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
	at wiki.Wiki.getImage(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at app.ImkerBase$1.fetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.ImkerBase.downloadLoop(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at javax.swing.SwingWorker.run(SwingWorker.java:334)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I wanted to download the images from SVG_maps_showing_history_in_Russian, but saidly there is one Image not found in Wikimedia:
https://commons.wikimedia.org/wiki/File:Historical_map_of_Moldova_AD_1940.svg

And I get following error message.

java.io.FileNotFoundException: https://upload.wikimedia.org/wikipedia/commons/a/a3/Historical_map_of_Moldova_AD_1940.svg

java.io.FileNotFoundException: https://upload.wikimedia.org/wikipedia/commons/a/a3/Historical_map_of_Moldova_AD_1940.svg
You may submit a bug report to the GitHub issue tracker:
https://github.com/MarcoFalke/wiki-java-tools/issues/new?title=%5BImker%5D+Exception&body=Version%3A+v16.09.13%0AStack+trace%3A%0A%60%60%60java%0Ajava.io.FileNotFoundException%3A+https%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2Fa%2Fa3%2FHistorical_map_of_Moldova_AD_1940.svg%0A%09at+sun.net.www.protocol.http.HttpURLConnection.getInputStream0%28HttpURLConnection.java%3A1890%29%0A%09at+sun.net.www.protocol.http.HttpURLConnection.getInputStream%28HttpURLConnection.java%3A1492%29%0A%09at+sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream%28HttpsURLConnectionImpl.java%3A263%29%0A%09at+wiki.Wiki.getImage%28Unknown+Source%29%0A%09at+wiki.Wiki.getImage%28Unknown+Source%29%0A%09at+app.ImkerBase%241.fetch%28Unknown+Source%29%0A%09at+app.App.attemptFetch%28Unknown+Source%29%0A%09at+app.App.attemptFetch%28Unknown+Source%29%0A%09at+app.App.attemptFetch%28Unknown+Source%29%0A%09at+app.ImkerBase.downloadLoop%28Unknown+Source%29%0A%09at+app.ImkerGUI%244.doInBackground%28Unknown+Source%29%0A%09at+app.ImkerGUI%244.doInBackground%28Unknown+Source%29%0A%09at+javax.swing.SwingWorker%241.call%28SwingWorker.java%3A295%29%0A%09at+java.util.concurrent.FutureTask.run%28FutureTask.java%3A266%29%0A%09at+javax.swing.SwingWorker.run%28SwingWorker.java%3A334%29%0A%09at+java.util.concurrent.ThreadPoolExecutor.runWorker%28ThreadPoolExecutor.java%3A1149%29%0A%09at+java.util.concurrent.ThreadPoolExecutor%24Worker.run%28ThreadPoolExecutor.java%3A624%29%0A%09at+java.lang.Thread.run%28Thread.java%3A748%29%0A%0A%60%60%60%0A
And include the following details:
Version: v16.09.13
Stack trace:
``java
java.io.FileNotFoundException: https://upload.wikimedia.org/wikipedia/commons/a/a3/Historical_map_of_Moldova_AD_1940.svg
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1890)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
	at wiki.Wiki.getImage(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at app.ImkerBase$1.fetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.ImkerBase.downloadLoop(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at javax.swing.SwingWorker.run(SwingWorker.java:334)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

``

So far so good, but now I click ok and Imker does not proceed with the other images. (Or at least I don't know how to tell Imker to do so.)

Maybe try to Bach-download, with imker:
Files_with_404_errors
(less data to download)

[Imker] Exception

When downloading images from the category "Vehicles in Tyresö Municipality" to the folder "/Users/jonatanglad/Downloads/namnlös mapp" on a Mac.

Version: v16.09.13
Stack trace:

java.lang.NoClassDefFoundError: javax/xml/bind/annotation/adapters/HexBinaryAdapter
	at app.ImkerBase.calcSHA1(Unknown Source)
	at app.ImkerBase.checksumLoop(Unknown Source)
	at app.ImkerGUI$5.doInBackground(Unknown Source)
	at app.ImkerGUI$5.doInBackground(Unknown Source)
	at java.desktop/javax.swing.SwingWorker$1.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.desktop/javax.swing.SwingWorker.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.annotation.adapters.HexBinaryAdapter
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
	at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
	... 10 more

[yacbot] Use crfs token

switch bot to use crfs (https://github.com/MarcoFalke/wiki-java-tools/blob/yacbot/src/wiki/Wiki.java#L1378) token.

{
"batchcomplete": "",
"warnings": {
"info": {
"*": "The intoken parameter has been deprecated."
}
}
}

[Imker] Exception when creating files with forbidden characters in Windows

E.g. downloading http://commons.wikimedia.org/wiki/File:Windows-forbidden-chars*%22%3F.png gives java.io.FileNotFoundException: C:\Users\marco\Downloads\temp\4\Windows-forbidden-chars*"?.png (The filename, directory name, or volume label syntax is incorrect)

[Imker] How to use the Java CLI?

Not seeing any instructions on this. Just curious if anyone has instructions or tips? I'd like to use the command line over the GUI. 👍

Imker:

java.io.FileNotFoundException: P:\Biblioteca\Art.Wikimedia.public domain\PD-Art (PD-old-100-1923)\invalid_names_encoded\%28Recto%29+Sketch+of+two+young+women%2C+one+sitting+and+reading+the+other+stretching+and+yawning+%28drawn+by+Lord+Bury%29+with+a+drawing+of+a+tree+alongside%3B+%28Verso%29+HMS+%27Queen%27+off+the+%27Island+of+Brada%2C+Ivisa%27+%28Ibiza%3F%29+26+RMG+PZ0859-001.tiff (The filename, directory name, or volume label syntax is incorrect)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(Unknown Source)
	at java.io.FileOutputStream.<init>(Unknown Source)
	at java.io.FileOutputStream.<init>(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at app.ImkerBase$1.fetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.ImkerBase.downloadLoop(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at javax.swing.SwingWorker$1.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at javax.swing.SwingWorker.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

Flock:

[yacbot] do not do interwiki cleanup in licensing section

do not do interwiki cleanup in licensing section https://commons.wikimedia.org/w/index.php?title=File:Portrait_of_user_Code,_161002,_ako.jpg&diff=211182399&oldid=209191409

Otherwise the link in the license section will not appear.

btw: i think we have to revert such changes, is there a way to find affected files via dbdump or just changing the template?

[yacbot] token deprecation

WARNING: getPageInfo: watchtoken is deprecated and will be removed shortly.

[Imker] java.lang.NullPointerException

when trying to download files from files(local files).I get this error
OS:linux(ubuntu)
Version: v16.09.13
Stack trace:

    java.lang.NullPointerException
	at app.ImkerBase.downloadLoop(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:304)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:343)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

[Imker] IOException

When trying to download pictures from the Category Pictures of the Year (finalist) I get:

java.io.IOException: Server returned HTTP response code: 503 for URL: https://commons.wikimedia.org/w/api.php?format=xml&maxlag=3&action=query&prop=imageinfo&iiprop=url&titles=File%3AAgamaSinaita01+ST+10.jpg&iiurlwidth=-1&iiurlheight=-1
You may submit a bug report to the github issue tracker:
https://github.com/MarcoFalke/wiki-java-tools/issues/new

Imker aborts whole category download whenever a single download fails

Version: v16.09.13
Stack trace:

java.lang.UnknownError: MW API error. Server response was: <?xml version="1.0"?><api servedby="mw2283"><error code="maxlag" info="Waiting for 10.192.32.167: 3.3404757976532 seconds lagged." host="10.192.32.167" lag="3.3404757976532" type="db" xml:space="preserve">See https://commons.wikimedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &amp;lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&amp;gt; for notice of API deprecations and breaking changes.</error></api>

	at wiki.Wiki.fetch(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at wiki.Wiki.getImage(Unknown Source)
	at app.ImkerBase$1.fetch(Unknown Source)
	at app.App.attemptFetch(Unknown Source)
	at app.ImkerBase.downloadLoop(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at app.ImkerGUI$4.doInBackground(Unknown Source)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:295)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:334)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:844)

That happened after 1117 files (out of many more) got downloaded. 3 seconds of lagging does not sound like a very serious problem that would requires to abort the whole category download.

What happens currently: Imker aborts the whole download, so I have to run again the whole category upload
What I would expect: Imker wait a few seconds and retries the download, and abort that particular file if it fails again, then proceeding with the rest of the files.

[Imker] java.net.UnknownHostException: commons.wikimedia.org

I keep receiving UnknownHostExceptions. Internet is working and I can access commons.wikimedia.org via browser.

My guess would be that Imker has problems getting past our company's firewall. Do you have any plans of adding proxy support to the tool?

Full error message:
Version: v16.09.13
Stack trace:

java.net.UnknownHostException: commons.wikimedia.org
	at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
	at java.net.PlainSocketImpl.connect(Unknown Source)
	at java.net.SocksSocketImpl.connect(Unknown Source)
	at java.net.Socket.connect(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
	at sun.net.NetworkClient.doConnect(Unknown Source)
	at sun.net.www.http.HttpClient.openServer(Unknown Source)
	at sun.net.www.http.HttpClient.openServer(Unknown Source)
	at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
	at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown Source)
	at wiki.Wiki.fetch(Unknown Source)
	at wiki.Wiki.populateNamespaceCache(Unknown Source)
	at wiki.Wiki.ensureNamespaceCache(Unknown Source)
	at wiki.Wiki.namespaceIdentifier(Unknown Source)
	at app.ImkerBase.setWiki(Unknown Source)
	at app.ImkerGUI.initialize(Unknown Source)
	at app.ImkerGUI.handleAction(Unknown Source)
	at app.ImkerGUI$16.actionPerformed(Unknown Source)
	at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
	at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
	at java.awt.Component.processMouseEvent(Unknown Source)
	at javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.awt.Component.processEvent(Unknown Source)
	at java.awt.Container.processEvent(Unknown Source)
	at java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Window.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.awt.EventQueue.access$500(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue$4.run(Unknown Source)
	at java.awt.EventQueue$4.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)

[Imker] Downloaded media meta data

Hi,

Love this app! Really useful.

I think it would be helpful to somehow obtain a photo's description/meta data as, maybe, a sidecar text file or some sort of text file data set? YAML perhaps? Something to match file name to it's meta data could be useful.

Thanks!

maflcko / wiki-java-tools Goto Github PK

wiki-java-tools's People

Stargazers

Watchers

Forkers

wiki-java-tools's Issues

Recommend Projects

Recommend Topics

Recommend Org