Dear Jonathan,
please, rename variable "version" to "version of an extractor" or something similar.
Since there is "version of format" variable at list of SP.

Next, please, add test of this SP into each test.

As I told that you have to run extractor by hand, I was wrong.
Since raw output is generated you do not have to run external extractor anymore.

I hope all extractors say its own version in a raw output.

doesnot show mime type
the same for "Type of format"

And all other extractors.

it should be at identification.

Each property should be handled some way.

Someone is ignored, other one is ignored.

Noone can be unhandled.

So, please check that list of unhandled properties is empty.

Differ-common uses:
https://github.com/Differ-GSOC/differ/blob/master/differ-common/src/main/resources/kakaduMetadataExtractor.xml#L62

It would be great to have 0install package for this great tool.

See http://0install.net/
Jan Stavel

From Bedrich:

FORM:DJVU [901662]
INFO [10] DjVu 3776x2520, v21, 200 dpi, gamma=2.2
BG44 [103431] IW4 data #1, 76 slices, v1.2 (color), 3776x2520
BG44 [150233] IW4 data #2, 13 slices
BG44 [336086] IW4 data #3, 12 slices
BG44 [311856] IW4 data #4, 6 slices

Significant properties for this case:

Name of format: DjVu

Image width: 3776

Image height: 2520

Version of DjVu: v21

Resolution: 200 ppi

Version of second algorithm JB2: v1.2

Profile: color

Gamma: 2.2

Used compression algorithm: for particular layers: BG44 (in this case is used just BG (background layer) and number 44 indicates that discrete wavelet transformation IW44 was used)

Non-significant properties for this case:

Information about slides is not significant for us as long as I know.

More info about DjVu: http://www.djvuzone.org/techpapers/index.html

For different images the tool give you more significant properties, for instance information about background (BG), foreground (FG), mask (Sjbz), shared layer (Djbz), background using JPEG2000 compression (BG2k), background using JPEG compression (BGjp), etc.

That is why web application cannot use the same extractors and properties that cmdline uses.

One image we have is at a directory images:
https://github.com/Differ-GSOC/differ/blob/master/docs/images/05.djvu

Please try to make test for this image.

Do know be afraid, it will be difficult a little.

Thanks,
Jan Stavel

At some case the binary returs invalid xml.
It looks like:

READBOX=yes

djvu should valid

Please, create new test for exiftool. With image 14.

Thank you,
Jan Stavel

• test whether GET differ.nkp.cz:80 returns

  some title

• test whether there is enought free space at a disc
• test whether mysql dabatase is running at a server

manage the merging.
will be:

• tested by Jonathan, Thomas
• installed by Vasek

Great,
Jan

Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'imageProcessor' defined in class path resource [pureImageProcessor.xml]: Cannot resolve reference to bean 'metadataExtractors' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'metadataExtractors' defined in class path resource [appCtx-differ-common.xml]: Cannot resolve reference to bean 'imagemagickMetadataExtractor' while setting bean property 'extractors' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'imagemagickMetadataExtractor' defined in class path resource [imagemagickMetadataExtractor.xml]: Cannot resolve reference to bean 'units' while setting bean property 'units'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'units' is defined
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106)
at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:630)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:148)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1035)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:939)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:609)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:469)
at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:139)
at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:93)
at cz.nkp.differ.cmdline.Main.main(Main.java:152)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'metadataExtractors' defined in class path resource [appCtx-differ-common.xml]: Cannot resolve reference to bean 'imagemagickMetadataExtractor' while setting bean property 'extractors' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'imagemagickMetadataExtractor' defined in class path resource [imagemagickMetadataExtractor.xml]: Cannot resolve reference to bean 'units' while setting bean property 'units'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'units' is defined
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveManagedList(BeanDefinitionValueResolver.java:353)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:153)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1360)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1118)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:322)
... 17 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'imagemagickMetadataExtractor' defined in class path resource [imagemagickMetadataExtractor.xml]: Cannot resolve reference to bean 'units' while setting bean property 'units'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'units' is defined
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1360)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1118)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:322)
... 29 more
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'units' is defined
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:553)
at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:1095)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:277)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:322)
... 39 more
jan@jan-XPS-L421X:/opt/differ/differ-cmdline$

Dear Jonathan,
please, try to run profiler when running batch.

We think that a lot of slow processes are run just once at the start of cmdline application.

So batch processing should be faster/more eficient.

I looking forward to a result,
Jan Stavel

Login as test
click on P1160752.jpg
click on Delete images.

red message appears:
Internal error, please contact system administrator.

log from catalina.log:

Oct 14, 2013 9:41:39 PM com.vaadin.Application terminalError
SEVERE: Terminal error:
com.vaadin.event.ListenerMethod$MethodException: Invocation of method componentEvent in cz.nkp.differ.gui.components.DifferProgramTabButtonPanel$3 failed.
at com.vaadin.event.ListenerMethod.receiveEvent(ListenerMethod.java:530)
at com.vaadin.event.EventRouter.fireEvent(EventRouter.java:164)
at com.vaadin.ui.AbstractComponent.fireEvent(AbstractComponent.java:1219)
at com.vaadin.ui.AbstractComponent.fireComponentErrorEvent(AbstractComponent.java:1257)
at com.vaadin.ui.AbstractComponent.setComponentError(AbstractComponent.java:599)
at com.vaadin.Application.terminalError(Application.java:1226)
at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.handleChangeVariablesError(AbstractCommunicationManager.java:1564)
at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.handleVariableBurst(AbstractCommunicationManager.java:1425)
at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.handleVariables(AbstractCommunicationManager.java:1329)
at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.doHandleUidlRequest(AbstractCommunicationManager.java:761)
at com.vaadin.terminal.gwt.server.CommunicationManager.handleUidlRequest(CommunicationManager.java:296)
at com.vaadin.terminal.gwt.server.AbstractApplicationServlet.service(AbstractApplicationServlet.java:501)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:200)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at cz.nkp.differ.io.ImageManager.deleteImage(ImageManager.java:144)
at cz.nkp.differ.gui.components.UserFilesWidget.removeSelectedImages(UserFilesWidget.java:126)
at cz.nkp.differ.gui.components.DifferProgramTabButtonPanel$3.componentEvent(DifferProgramTabButtonPanel.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.vaadin.event.ListenerMethod.receiveEvent(ListenerMethod.java:510)
... 52 more

load png file -> Process
no information about progress.

Just "loading bar" appears.

Results page is not working - clicking on result show exclamation mark and result is not shown.

Dear Jonathan,
please, create new SP that comes to "extractorProperties".

It will tell how much time a running of an extractor takes.

You can name as you want. Important is the name is concises and everyone understand easy what the property means.

Thank you,
Jan Stavel

Please make a test for image 14 for kdu_expand tool.

Thank you.
Jan Stavel

Please, make new test for exiftool extractor.
Use image 01.

Thank you,
Jan Stavel

http://differ.readthedocs.org/en/latest/#arguments-of-the-application

configuration is is at file:
common.properties

https://github.com/Differ-GSOC/differ/blob/master/differ-cmdline/src/main/java/cz/nkp/differ/cmdline/ReportHTTPSender.java

Fri Jan 11 14:47:36 2013 -> 2013-01-11 14:47:36+01:00

Please use some common way:
http://docs.oracle.com/javase/1.4.2/docs/api/java/text/SimpleDateFormat.html

Date startDate = null;
Date endDate = null;
try {
if (validDateStart!= null) startDate = new SimpleDateFormat("MM/dd/yyyy HH:mm", Locale.ENGLISH).parse(validDateStart + " " + validDateStartTime);
if (validDateEnd!= null) endDate = new SimpleDateFormat("MM/dd/yyyy HH:mm", Locale.ENGLISH).parse(validDateEnd + " " + validDateEndTime);
} catch (ParseException e) {
e.printStackTrace();
}

Jan Stavel

It is important to add special code to bypass captcha.
Since selenium tests cannot pass captcha.

The code should be configurable in some properties.

Will be moved to link.

it is in a value "Version"

It is important for selenium tests. Since it is difficult to get path for elements.

Result area disappear after compare process finishes at the moment.
It would be great the result area to be persistent.
User would take a look at the area each time he want even in a future.

Thanks,
Jan

definition of jhove stuff is at jhoveMetadataExtractor.xml

Good luck,
Jan Stavel

It is necessary to have information about tests that are provided.
I think it can be table:

image | extractor | passed
examples/example_01/01.jpg | imagemagik | OK

Please, create new rst file in a documentation and try to write information about all tests we made.

Thank you,
Jan

mime/type
version of format
type of format

je to v polozce "Version"

Please use JasperReports to make pdf/html report of results.
I think you can use drep output file of a cmdline application, since it is xml file.

Please, try to create at least basic report. I will extend it more after you end our cooperation.

Use layout similar to text output with using more typographic features that graphic output offers.

Thanks,
Jan Stavel

There are a lot of rows in comparison siginificant properties list.
Application should write just first occurence of SP.
Other occurences of SP name would be left empty.

see text output:
http://differ.readthedocs.org/en/latest/_downloads/cmdline.txt

e.g.:

MIME type exiftool image/jpeg
exiv2 image/jpeg
fits image/jpeg
jhove image/jpeg

So If some text is repeating, next occurrence of the text is hidden.

It helps text to be readable a lot.

daits "type of format" too.
exiv2 - Mime type for jp2000 should show
fits - Type of format/jp2000 should show

Dear Jonathan,
please, create around advice:
http://static.springsource.org/spring/docs/3.0.x/reference/aop.html#aop-ataspectj-around-advice

around method:
https://github.com/Differ-GSOC/differ/blob/master/differ-common/src/main/java/cz/nkp/differ/compare/metadata/external/ExternalMetadataExtractor.java#L68

It will compute time consumed when calling an extractor.
Thank you,
Jan Stavel

Differ - app. does not get well formed and valid for tiff image.

Dear Jonathan,
please create test for imagemagick.

An image it will run for is examples/images_01/01.jpg

Thank you,
Jan

google: tcpdump sniff http
or google: justsniffer http

Main goal is to compare communication using curl with communication using cmdline app.

Since curl communication works well we can compare what http headers arent presented in cmdline http communication.

curl -i -H "Content-type: application/xml" -X POST --user test:test -d
"@result.xml" http://differ.nkp.cz:8080/rest/results

Please, give here even description how you run sniffer (to use it in a future the same way).

If you need more help, give me an anwer as you want.

Thank you,
Jan

Load jpf file.
You can see lines with versions of used extractors.

The table with informations are not well formed.

Results are not shown when formats on images differ - for example JPEG or PNG. It throws:

Sep 17, 2013 7:25:11 PM com.vaadin.Application terminalError
SEVERE: Terminal error:
java.lang.IndexOutOfBoundsException: Index: 124, Size: 117
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at cz.nkp.differ.compare.io.generators.ImageMetadataComponentGenerator$2.getStyle(ImageMetadataComponentGenerator.java:236)
at com.vaadin.ui.Table.paintRowAttributes(Table.java:3410)
at com.vaadin.ui.Table.paintRow(Table.java:3306)
at com.vaadin.ui.Table.paintRows(Table.java:3152)
at com.vaadin.ui.Table.paintContent(Table.java:2764)

rename at averything.

Dear Jonathan,
please, find out all significant properties for this image.

The image is at a directory docs/examples/images_01/01.jpg.

sudgested plan:

• create new test configuration (example image14SignificantProperties.xml)
• copy all significant properties from significantProperties.xml into new test configuration
• run some extractor (for e.g. imagemagick)
• fill values in the test configuration with values that you decide
• IF you are unsure, you can use report in examples examples/images_01/01/01-01-report.txt, but be careful - there can be mistakes in the report