Mozilla/5.0 (Linux; Android 4.0.3; GT-P5100 Build/IML74K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.92 Safari/537.36

Galaxy S3 Mozilla/5.0 (Linux; Android 4.1.2; GT-I9300 Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.64 Mobile Safari/537.36

Galaxy S4 Mozilla/5.0 (Linux; Android 4.2.2; GT-I9500 Build/JDQ39) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.169 Mobile Safari/537.22

Galaxy S4 Native: Mozilla/5.0 (Linux; Android 4.2.2; en-gb; SAMSUNG GT-I9500 Build/JDQ39) AppleWebKit/535.19 (KHTML, like Gecko) Version/1.0 Chrome/18.0.1025.308 Mobile Safari/535.19

Nexus 10 Mozilla/5.0 (Linux; Android 4.3; Nexus 10 Build/JWR66V) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.94 Safari/537.36

NoteII Mozilla/5.0 (Linux; Android 4.1.2; GT-N7100 Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.82 Mobile Safari/537.36

Of all these UA strings above only Galaxy S4 Native was recognized as MOBILE_BROWSER, the rest was classified as BROWSER.

Could you please fix this?

Could you add methods so that we can access the icon names in the XML data (for both the operating system and the user agent)?

When we have user agent as one of:

• Mozilla/4.0 (compatible; MSIE 5.0; Mac_PowerPC)
• Mozilla/4.0 (compatible; MSIE 5.0; Mac_PowerPC; Alexa Toolbar)
• Mozilla/4.0 (compatible; MSIE 4.5; Mac_PowerPC)

uadetector return to us :

UserAgent [family=IE, icon=msie.png, name=IE, operatingSystem=OperatingSystem [family=MAC_OS, familyName=Mac OS, icon=macos.png, name=Mac OS, producer=Apple Computer, Inc., producerUrl=http://www.apple.com/, url=http://en.wikipedia.org/wiki/Mac_OS, versionNumber=VersionNumber [groups=[, , ], extension=]], producer=Microsoft Corporation., producerUrl=http://www.microsoft.com/, type=BROWSER, typeName=Browser, url=http://en.wikipedia.org/wiki/Internet_Explorer, versionNumber=VersionNumber [groups=[5, 0, ], extension=]]

Look at Operation System, its name=Mac OS which is wrong.
As all we know :

Mozilla/MozVer (compatible; MSIE IEVer[; Provider]; Platform[; Extension]*) [Addition]

where:

MozVer: 4.0 - is version compatibility with Netscape: 4.0: Internet Explorer 4.x and higher

IEVer: MSIE 4.5 or MSIE 5.0 - is Internet Explorer version

Platform: Mac_PowerPC mean that used in IE 5.x and above

So Operating System name definitely can't be Mac OS

Getting information of device and device family

Is there a code maintenance benefit provided by using sub-modules in uadetector? It would be nice if at least uadetector-core and uadetector-resources were combined to fix a Netbeans issue. Netbeans has excellent maven support, but it can't figure out the dependencies correctly when two sub-modules share the same package names. Combining them would presumably make the dependencies more obvious to developers as well.

Hi,

The current bundled uas.xml file only includes Adidxbot version 1.1, not version 2.0. The user agent string of the new version is:

adidxbot/2.0 (+http://search.msn.com/msnbot.htm)

Thanks.

I need to persist the version of the user agent definition file when making a bot determination so I can go back and re-run old determinations on historic data.

I dug in and copied code from UADetectorServiceFactory but to me this would be much more elegant if the parser exposed this version String itself.

    private transient DataStore parserDataStore;
    private transient UserAgentStringParser parser;

    private void createParser( long updateInterval ) {
            OnlineXmlDataStore privateDataStore = new OnlineXmlDataStore();

            UpdatingUserAgentStringParserImpl privateParser = new UpdatingUserAgentStringParserImpl( privateDataStore );
            privateParser.setUpdateInterval( updateInterval );

            parserDataStore = privateDataStore;
            parser = privateParser;
    }

    public String getBotModelVersion() {
        String version = null;

        if (parserDataStore != null) {
            Data data = parserDataStore.getData();
            if (data != null) {
                // TODO: Convince 3rd party devs to expose data file version to parser
                version = data.getVersion();
            }
        }
        return version != null ? version : "unknown";
    }

    public boolean isBot( String userAgent ) {
        UserAgent agent = parser.parse( userAgent );
        return agent != null && agent.getType() != null && agent.getType() == UserAgentType.ROBOT;
    }

The following is a user-agent string we came across that is used by one of Google's "adsbots":

• User-agent string: AdsBot-Google-Mobile (+http://www.google.com/mobile/adsbot.html) Mozilla (iPhone; U; CPU iPhone OS 3 0 like Mac OS X) AppleWebKit (KHTML, like Gecko) Mobile Safari

UADetector returns the following result for this user-agent string:

• UADetector result: Mobile Safari / OperatingSystem [family=IOS, familyName=iOS, icon=iphone.png, name=iOS, producer=Apple Inc., producerUrl=http://www.apple.com/, url=http://en.wikipedia.org/wiki/IOS, versionNumber=VersionNumber [groups=[, , ], extension=]]

The Google adsbot is probably masquerading as Safari on iPhone so that specific content / format is returned, so I think the identification is appropriate. Returning content other than what you would return to Safari on iPhone could have unforeseen consequences, such as deterioration in Google page rankings or search results.

However, it would be useful to have a flag or some other indication that this is a bot, so these user-agents could be filtered or logged appropriately.

Test set: net.sf.uadetector.internal.util.UrlUtilTest_open

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.686 sec <<< FAILURE!
open_withIOException(net.sf.uadetector.internal.util.UrlUtilTest_open) Time elapsed: 353 sec <<< ERROR!
java.lang.Exception: Unexpected exception, expected<net.sf.uadetector.exception.CanNotOpenStreamException> but was<java.lang.IllegalArgumentException>
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.handleException(PowerMockJUnit44RunnerDelegateImpl.java:363)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.handleException(PowerMockJUnit47RunnerDelegateImpl.java:107)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.handleInvocationTargetException(PowerMockJUnit44RunnerDelegateImpl.java:339)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:317)
at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:86)
at org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:94)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(PowerMockJUnit44RunnerDelegateImpl.java:296)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestInSuper(PowerMockJUnit47RunnerDelegateImpl.java:112)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(PowerMockJUnit47RunnerDelegateImpl.java:73)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThenTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:284)
at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:84)
at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:49)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.invokeTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:209)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:148)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:122)
at org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:34)
at org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:120)
at org.powermock.modules.junit4.common.internal.impl.JUnit4TestSuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:102)
at org.powermock.modules.junit4.common.internal.impl.AbstractCommonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53)
at org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner.java:42)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.IllegalArgumentException: Class mocking is currently disabled. Change easymock.disableClassMocking to true do modify this behavior
at org.easymock.internal.MocksControl.createProxyFactory(MocksControl.java:134)
at org.easymock.internal.MocksControl.createMock(MocksControl.java:58)
at org.powermock.api.easymock.PowerMock.doCreateMock(PowerMock.java:2212)
at org.powermock.api.easymock.PowerMock.doMock(PowerMock.java:2134)
at org.powermock.api.easymock.PowerMock.createMock(PowerMock.java:89)
at net.sf.uadetector.internal.util.UrlUtilTest_open.open_withIOException(UrlUtilTest_open.java:21)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:66)
at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:312)
... 29 more

Right now the cache file is just called "uas". Can we add a ".xml" file extension so that editors and other tools know that it is an xml file?

We should provide a XML exporter to create (restore) a uas.xml out of a Data instance.

This functionality is a precondition to be able to create a patching system on top of the downloadable version from http://user-agent-string.info and allows us to provide an own enhanced uas.xml.

A number of Dependency Injection containers now support the PreDestroy annotation which can be useful for indicating methods that should be called when the container is shutting down. This is the case for the Spring container that I use in a number web applications that I maintain. I was doing some reading during the day today and determined that the PreDestroy annotation is now available by default in Java SE 7 and can be made available through the jsr250-api-1.0.jar for earlier versions of java.

In our consumption of the UserAgentStringParser implementation that was updated to expose a shutdown method we have to explicitly inform the container of the shutdown method. However, if the method where annotation with PreDestory then the container would by convention find this on the UserAgentStringParser bean that we have registered and invoke it during the shutdown lifecycle of the container.

I was able to reproduce the same issue here:
#23

with the latest release:
dependency/uadetector-core-0.9.6.jar
dependency/uadetector-resources-2013.06.jar

Bad file:
$ cat /tmp/uas.xml

Error - Connect failed:
Too many connections

The error output below is when calling this method: UADetectorServiceFactory.getCachingAndUpdatingParserHolder():

Exception in thread "main" java.lang.ExceptionInInitializerError
        at net.sf.uadetector.service.UADetectorServiceFactory.getCachingAndUpdatingParser(UADetectorServiceFactory.java:129)
        at com.eyeview.logsparser.LogParserServer.main(LogParserServer.java:33)
Caused by: java.lang.IllegalStateException: Argument 'data' must not be empty.
        at net.sf.uadetector.datastore.AbstractDataStore.checkData(AbstractDataStore.java:48)
        at net.sf.uadetector.datastore.AbstractDataStore.<init>(AbstractDataStore.java:124)
        at net.sf.uadetector.datastore.CachingXmlDataStore$CacheFileDataStore.<init>(CachingXmlDataStore.java:52)
        at net.sf.uadetector.datastore.CachingXmlDataStore.createCachingXmlDataStore(CachingXmlDataStore.java:167)
        at net.sf.uadetector.datastore.CachingXmlDataStore.createCachingXmlDataStore(CachingXmlDataStore.java:126)
        at net.sf.uadetector.datastore.CachingXmlDataStore.createCachingXmlDataStore(CachingXmlDataStore.java:104)
        at net.sf.uadetector.service.UADetectorServiceFactory$CachingAndUpdatingParserHolder.<clinit>(UADetectorServiceFactory.java:40)
        ... 2 more

Workaround:
remove /tmp/uas.xml.

Since http://user-agent-string.info is down we get an exception which describes why the UAS data can not be imported, but UADetector works like expected. If the corresponding thread have a meaningful name the user get an idea that's not a big issue. Additionally we should improve to suppress the stacktrace and write a nice log message.

16:14:08 [qtp2136269707-22] DEBUG n.s.u.internal.data.Data$Builder - No patterns for operating system entry (with id '114') available.
16:14:08 [Thread-14] DEBUG n.s.u.d.AbstractUpdateOperation - An update is available. Current version is '20130301-01' and remote version is '<h1>Error - Connect failed:<br /> <u>Too many connections</u></h1>'.
16:14:08 [Thread-14] WARN  n.s.u.internal.data.XmlDataHandler - Error while reading UAS data: Document root element "h1", must match DOCTYPE root "null". (line: 1)
16:14:08 [Thread-14] WARN  n.s.u.datareader.XmlDataReader - Document root element "h1", must match DOCTYPE root "null".
Exception in thread "Thread-14" java.lang.IllegalStateException: Argument 'data' must not be empty.
    at net.sf.uadetector.datastore.AbstractRefreshableDataStore.checkData(AbstractRefreshableDataStore.java:57)
    at net.sf.uadetector.datastore.AbstractRefreshableDataStore.setData(AbstractRefreshableDataStore.java:237)
    at net.sf.uadetector.datastore.UpdateOperationTask.call(UpdateOperationTask.java:21)
    at net.sf.uadetector.datastore.AbstractUpdateOperation$1.run(AbstractUpdateOperation.java:157)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:680)

UserAgent [
deviceCategory=DeviceCategory [
category=TABLET,
icon=tablet.png,
infoUrl=/list-of-ua/device-detail?device=Tablet,
name=Tablet
],
family=CHROME_MOBILE,
icon=chrome.png,
name=Chrome Mobile,
operatingSystem=OperatingSystem [
family=ANDROID,
familyName=Android,
icon=android.png,
name=Android 4.3 Jelly Bean,
producer=Google, Inc., producerUrl=http://www.google.com/,
url=http://en.wikipedia.org/wiki/Android_%28operating_system%29,
versionNumber=VersionNumber [groups=[4, 3, ], extension=]
],
producer=Google Inc.,
producerUrl=http://www.google.com/,
type=MOBILE_BROWSER,
typeName=Mobile Browser,
url=http://www.google.com/intl/en/chrome/browser/mobile/,
versionNumber=VersionNumber [groups=[31, 0, 1650, 59], extension=]
]

Hi Team,

Is there any way to find out the Good bots like google/ MSN ping bots based on the ip address retrieved in the browser request ?

This is because, as useragents can be spoofed up easily by the scrappers.

Please share your thoughts on how to achieve this.

Regards
Srikanth

Here's an example user agent string. The older default "Browser" app is properly detected, but newer tablets like the Nexus 7 have replaced older "Browser" app with Chrome.

Mozilla/5.0 (Linux; Android 4.1.1; ASUS Transformer Pad TF700T Build/JRO03C) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Safari/535.19

Chrome has an option to "Request Desktop Site" and when that is checked the user agent changes to looks like this:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.45 Safari/535.19

The second user agent should still be flagged as type "BROWSER" while the first one needs to be "MOBILE_BROWSER"

If this library would implement a reordering system so that the most used user agent strings would be checked first, there would probably be major speed improvements. Just a simple hitcounter which is reset at regular intervals would probably be enough.

The current background updater doesn't have a public interface for shutdown.

I created a stand-alone example that shows the issue:
https://github.com/dimalinux/uadetector-example

See the comments at the end of this file:
https://github.com/dimalinux/uadetector-example/blob/master/src/main/java/to/noc/uadetector/example/Main.java

Restore original state of field after test run of class FinalStaticFieldValueChangerTest.

If the user-agent-string.info website is down for maintenance (or hardware failure, DOSed, ...), the OnlineUpdatingParser should log the problem, but not fail. It should use the most recently cached data file that it has available, but instead it's propogating CanNotOpenStreamException upwards.

I updated my example program to maven release '2012.09': https://github.com/dimalinux/uadetector-example

To see the issue, run the program once normally and note the version number. Then run the program a second time after adding the following line to /etc/hosts:

127.0.0.1 user-agent-string.info

Here's the output from the second run:

% java -jar target/uadetector-example-0.2-SNAPSHOT.jar
Exception in thread "main" java.lang.ExceptionInInitializerError
at net.sf.uadetector.service.UADetectorServiceFactory.getOnlineUpdatingParser(UADetectorServiceFactory.java:111)
at to.noc.uadetector.example.Main.main(Main.java:15)
Caused by: net.sf.uadetector.exception.CanNotOpenStreamException: Can not open stream to the given URL: http://user-agent-string.info/rpc/get_data.php?key=free&format=xml
at net.sf.uadetector.internal.util.UrlUtil.open(UrlUtil.java:86)
at net.sf.uadetector.datareader.XmlDataReader.readXml(XmlDataReader.java:92)
at net.sf.uadetector.datareader.XmlDataReader.read(XmlDataReader.java:125)
at net.sf.uadetector.datastore.AbstractDataStore.readData(AbstractDataStore.java:68)
at net.sf.uadetector.datastore.AbstractDataStore.(AbstractDataStore.java:163)
at net.sf.uadetector.datastore.AbstractDataStore.(AbstractDataStore.java:145)
at net.sf.uadetector.datastore.OnlineXmlDataStore.(OnlineXmlDataStore.java:39)
at net.sf.uadetector.service.UADetectorServiceFactory$OnlineUpdatingParserHolder.(UADetectorServiceFactory.java:39)
... 2 more

In my Windows test server I see the following exception on server startup:

2013-06-19 16:25:29,696 WARN   [UpdateOperationWithCacheFileTask] The read content is faulty and can not be processed correctly.
net.sf.qualitycheck.exception.IllegalStateOfArgumentException: Renaming file from 'C:\Users\foo\AppData\Local\Temp\uas.xml.temp' to 'C:\Users\foo\AppData\Local\Temp\uas.xml' failed.
    at net.sf.qualitycheck.Check.stateIsTrue(Check.java:2198)
    at net.sf.uadetector.datastore.UpdateOperationWithCacheFileTask.renameFile(UpdateOperationWithCacheFileTask.java:160)
    at net.sf.uadetector.datastore.UpdateOperationWithCacheFileTask.readAndSave(UpdateOperationWithCacheFileTask.java:130)
    at net.sf.uadetector.datastore.UpdateOperationWithCacheFileTask.readDataIfNewerAvailable(UpdateOperationWithCacheFileTask.java:193)
    at net.sf.uadetector.datastore.UpdateOperationWithCacheFileTask.call(UpdateOperationWithCacheFileTask.java:183)
...

As I can see, the reason for this is the output stream in the UpdateOperationWithCacheFileTask.readAndSave(), which is still open at the moment of the call to renameFile(tempFile, file).

Since October 2013 the UAS data provides device types. We should support this device data and provide it via ReadableUserAgent.

This is the exception I get when app starts

[2013-06-20 20:17:13] WARN  [ernal.data.XmlDataHandler] :265 - Error while reading UAS data: Document root element "h1", must match DOCTYPE root "null". (line: 1)
[2013-06-20 20:17:13] WARN  [.datareader.XmlDataReader] :112 - Document root element "h1", must match DOCTYPE root "null".
[2013-06-20 20:17:13] WARN  [perationWithCacheFileTask] :203 - The read content is faulty and can not be processed correctly.
java.lang.IllegalStateException: The read in content can not be transformed to an instance of 'Data'.
    at net.sf.uadetector.datastore.UpdateOperationWithCacheFileTask.readAndSave(UpdateOperationWithCacheFileTask.java:115)
    at net.sf.uadetector.datastore.UpdateOperationWithCacheFileTask.readDataIfNewerAvailable(UpdateOperationWithCacheFileTask.java:193)
    at net.sf.uadetector.datastore.UpdateOperationWithCacheFileTask.call(UpdateOperationWithCacheFileTask.java:183)
    at net.sf.uadetector.datastore.AbstractUpdateOperation$1.run(AbstractUpdateOperation.java:161)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
[2013-06-20 20:17:13] INFO  [perationWithCacheFileTask] :212 - Reading fallback data...

Using version *core-0.9.5 and *resources-2013.05

For example, fails to parse the following UA string:

Y!J-BRO/YFSJ crawler (compatible; Mozilla 4.0; MSIE 5.5; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html; YahooFeedSeekerJp/2.0; users 1; views 462)

given this user agent strings:

Mozilla/5.0 \[en] (PalmOS; U; WebPro/3.5; Palm-Zi72)

Mozilla/5.0 (webOS/1.3; U; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Version/1.0 Safari/532.2 Desktop/1.0

Mozilla/5.0 (SymbianOS/9.1; U; [en-us]) AppleWebKit/413 (KHTML, like Gecko) Safari/413

uadetector does not recognize them as mobile type and parse.getType returns UNKNOWN or BROWSER

I think MOBILE_BROWSER would be a better response

User-Agent string exposes the mobile device type, but uadetector does not seem to capture this information.

For example, if a user connects to a site with an iPad:
"Mozilla/5.0 (iPad; CPU OS 5_0 like Mac OS X) ......."

"Mobile Browser" is not terribly descriptive for the device type.

Might not be in the scope of this project and/or difficult to implement, but having just started using it, the lack of device type came up pretty quickly.

Nice library, btw ;-)

Thanks

Using uadetector lib, I have noticed that sometimes browser version is detected as 535, I have tried to localize the issue and find a user agent which cause the issue. The details are below:

UADetector versions:
uadetector-core-0.9.3
uadetector-resources-2013.03

User agent:
Mozilla/5.0 (Linux; U; en-gb; KFOT Build/IML74K) AppleWebKit/535.19 (KHTML, like Gecko) Silk/2.8 Safari/535.19 Silk-Accelerated=true

As far as I understand - KFOT means the Kindle Fire device.

Actual result:
UserAgent [family=SAFARI, icon=safari.png, name=Safari, operatingSystem=OperatingSystem [family=LINUX, familyName=Linux, icon=linux.png, name=Linux, producer=, producerUrl=, url=http://en.wikipedia.org/wiki/Linux, versionNumber=VersionNumber [groups=[, , ], extension=]], producer=Apple Inc., producerUrl=http://www.apple.com/, type=BROWSER, typeName=Browser, url=http://en.wikipedia.org/wiki/Safari_%28web_browser%29, versionNumber=VersionNumber [groups=[535, 19, ], extension=]]

In particular:
version = 535.19 (I guess it comes from "AppleWebKit/535.19" in user agent string)
name = Safari

Expected result:
Browser version (in particular, major version) shouldn't be identified as engine's version, So, the expected result is:
version= 2.8 (should come from "Silk/2.8" in user agent string)
name = Silk - (should come from "Silk/2.8" in user agent string)

I'm using UADetector in Google App Engine to parser this user agent like this:

Mozilla/5.0 (Linux; U; Android 2.2; en-ca; GT-P1000M Build/FROYO) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

I can getOperatingSystem(), but i cannot get Android device name like GT-P1000M

How can i do this?

Thanks

I'm trying to detect an ipad device but that part of the user agent seems not to be accessible from a UserAgent object.

Having access to the originally provided user agent could help

Mozilla/5.0 (iPad; CPU OS 5_0 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A334 Safari/7534.48.3"

Presently parse method returns an instance of the final UserAgent class and cannot be used with mock-up testing. It should likely be returning ReadableUserAgent interface instance instead.

The current implementation of CachingXmlDataStore and OnlineXmlDataStore has no fallback implementation if the UAS data can not be read from the passed URL. This leads to an error during initialization of an instance (which is not bug).

The idea is to build a fallback into CachingXmlDataStore and OnlineXmlDataStore that can be passed during construction. This allows us to pass an instance of ResourceModuleXmlDataStore into when instantiating the other DataStoreimplementation in the UADetectorServiceFactory (in the uadetector-resources module). Thus avoids vulnerability of UADetectorServiceFactory and protects an application during startup with no internet connection.

The user agent as of now contains the device information as in device types can be the following,

The device types must be categorized into below categories.

1. Mobile - A handheld mobile device
2. Tablet - A tablet
3. Desktop - A desktop web browser
4. Bot - A search engine crawler or other known bot
5. Unknown - Could not be classified as one of the above
6. Undefined - Shouldn't happen. Indicates an error, please inform the maintainer.

So when we do... userAgent.getDevice() it should return the Mobile for mobile devices.... this will be a convenient method and will be used to categorized based on devices.

Followed usage directions on http://uadetector.sourceforge.net/usage.html, downloaded package successfully from the Central Repository, however unable to use due to the following error:

@sanity

java.lang.NoClassDefFoundError: net/sf/uadetector/datastore/DataStore
    at net.sf.uadetector.UADetectorServiceFactory.getResourceModuleParser(UADetectorServiceFactory.java:158)
        ...
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
    at org.testng.TestRunner.privateRun(TestRunner.java:767)
    at org.testng.TestRunner.run(TestRunner.java:617)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
    at org.testng.SuiteRunner.run(SuiteRunner.java:240)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1197)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1122)
    at org.testng.TestNG.run(TestNG.java:1030)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
Caused by: java.lang.ClassNotFoundException: net.sf.uadetector.datastore.DataStore
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    ... 27 more

The line in question:

UserAgentStringParser parser = UADetectorServiceFactory.getResourceModuleParser();

It would be nice if UADetector failed back to its bundled data file if the cache file fails to parse. I'm not too worried about one-time overhead at startup. At least for my uses, I'd be fine if it used the bundled data file, then switched to the cached file if it's newer and parses successfully. Whatever solution you think is best. Any solution that doesn't fail and makes a best effort to get the latest data file works great for my needs.

The error output below is when running uadetector-example which calls UADetectorServiceFactory.getCachingAndUpdatingParserHolder():

Version:

$ ls target/lib/*uadetector*
target/lib/uadetector-resources-2013.02.jar
target/lib/uadetector-core-0.9.2.jar

The bad data file:

$ cat /tmp/uas.xml 
<h1>Error - Connect failed:<br /> <u>Too many connections</u></h1>

Exception:

18:58:49 [to.noc.uadetector.example.Main.main()] WARN  n.s.u.internal.data.XmlDataHandler - Error while reading UAS data: Document root element "h1", must match DOCTYPE root "null". (line: 1)
18:58:49 [to.noc.uadetector.example.Main.main()] WARN  n.s.u.datareader.XmlDataReader - Document root element "h1", must match DOCTYPE root "null".
[WARNING] 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
    at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.ExceptionInInitializerError
    at net.sf.uadetector.service.UADetectorServiceFactory.getCachingAndUpdatingParserHolder(UADetectorServiceFactory.java:134)
    at to.noc.uadetector.example.Main.main(Main.java:47)
    ... 6 more
Caused by: java.lang.IllegalStateException: Argument 'data' must not be empty.
    at net.sf.uadetector.datastore.AbstractDataStore.checkData(AbstractDataStore.java:45)
    at net.sf.uadetector.datastore.AbstractDataStore.<init>(AbstractDataStore.java:136)
    at net.sf.uadetector.datastore.CachingXmlDataStore$CacheFileDataStore.<init>(CachingXmlDataStore.java:48)
    at net.sf.uadetector.datastore.CachingXmlDataStore.createCachingXmlDataStore(CachingXmlDataStore.java:170)
    at net.sf.uadetector.datastore.CachingXmlDataStore.createCachingXmlDataStore(CachingXmlDataStore.java:120)
    at net.sf.uadetector.datastore.CachingXmlDataStore.createCachingXmlDataStore(CachingXmlDataStore.java:99)
    at net.sf.uadetector.service.UADetectorServiceFactory$CachingAndUpdatingParserHolder.<clinit>(UADetectorServiceFactory.java:40)
    ... 8 more

Minor documentation change, the servlet example on http://uadetector.sourceforge.net/usage.html still imports

net.sf.uadetector.UADetectorServiceFactory

instead of net.sf.uadetector.service.UADetectorServiceFactory

Since device categorization is available beginning October 2013 all version lower or equal to version uadetector-core 0.9.9 and uadetector-resources 2013.09 print the following log message:

22:35:02 [update-operation] WARN  n.s.u.internal.data.XmlDataHandler - Error while reading UAS data: Element type "devices" must be declared. (line: 20917)
22:35:02 [update-operation] WARN  n.s.u.datareader.XmlDataReader - Element type "devices" must be declared.
Exception in thread "update-operation" net.sf.qualitycheck.exception.IllegalStateOfArgumentException: The passed arguments have caused an invalid state: Argument 'data' must not be empty.
    at net.sf.uadetector.datastore.AbstractRefreshableDataStore.checkData(AbstractRefreshableDataStore.java:61)
    at net.sf.uadetector.datastore.AbstractRefreshableDataStore.setData(AbstractRefreshableDataStore.java:229)
    at net.sf.uadetector.datastore.UpdateOperationTask.call(UpdateOperationTask.java:36)
    at net.sf.uadetector.datastore.AbstractUpdateOperation$1.run(AbstractUpdateOperation.java:197)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:695)

It means that UADetector cannot get the most recent version from http://user-agent-string.info, because the XML schema changed (and unsupported by this library). In this case the library falls back to the cached file or the delivered database within uadetector-resources.

Stay cool, the library continues to work normally but without updates.

Since October 2013 an update of UADetector support device categorization. Please update to uadetector-resources version 2013.10 or uadetector-core version 0.9.10.

I use uadetector-core-0.9.8.jar and uadetector-resources-2013.08.jar.

Parse following user agent strings:
Mozilla/5.0 (BlackBerry; U; BlackBerry 9900; en) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.1.0.346 Mobile Safari/534.11+
BlackBerry 9000: BlackBerry9000/4.6.0.65 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/102
No version info for OS and Browser

Mozilla/5.0 (BB10; Touch) AppleWebKit/537.10+ (KHTML, like Gecko) Version/10.1.0.4633 Mobile Safari/537.10+
It's black berry 10 user agent string. Detector parse this as unknown OS with Mobile Safari browser.

Fest Assertions follows many best practices in software engineering and development and we want to replace all JUnit assertions with it. We switch to the current stable version 1.4 since 2.0 is still under development and not finished yet.

Patterns to convert with Eclipse IDE

1. Converting import statement
   import org\.junit\.Assert; => import static org.fest.assertions.Assertions.assertThat;
2. Converting assertEquals(1, myComparable1.compareTo(myComparable2)) to assertThat(myComparable1.compareTo(myComparable2)).isEqualTo(1)
Assert.assertEquals\(\s?(.*),\s?(.*).compareTo\((.*)\)\);=> assertThat($2.compareTo($3)).isEqualTo($1);
3. Converting assertEquals(-1, myComparator.compare(myObject1, myObject2)) to assertThat(myComparator.compare(myObject1, myObject2)).isEqualTo(-1)
Assert.assertEquals\(\s?(.*),\s?(.*).compare\((.*),\s?(.*)\)\);=> assertThat($2.compare($3, $4)).isEqualTo($1);
4. Converting assertEquals(0, myList.size()) to assertThat(myList).isEmpty()
Assert.assertEquals\(0,\s?(.*).size\(\)\); => assertThat($1).isEmpty();
5. Converting assertEquals(expectedSize, myList.size()) to assertThat(myList).hasSize(expectedSize)
Assert.assertEquals\(\s?(.*),\s?(.*).size\(\)\); => assertThat($2).hasSize($1);
6. Converting assertEquals("text", valueUnderTest) to assertThat(valueUnderTest).isEqualTo("text")
Assert.assertEquals\(\s?"(.*)",\s?(.*)\); => assertThat($2).isEqualTo("$1");
7. Converting assertEquals(value, valueUnderTest) to assertThat(valueUnderTest).isEqualTo(value)
Assert.assertEquals\(\s?(.*),\s?(.*)\); => assertThat($2).isEqualTo($1);
8. Converting assertNotEquals(value, valueUnderTest) to assertThat(valueUnderTest).isNotEqualTo(value)
Assert.assertNotEquals\(\s?(.*),\s?(.*)\); => assertThat($2).isNotEqualTo($1);
9. Converting assertSame(value, valueUnderTest) to assertThat(valueUnderTest).isSameAs(value)
Assert.assertSame\(\s?(.*),\s?(.*)\); => assertThat($2).isSameAs($1);
10. Converting assertNotSame(value, valueUnderTest) to assertThat(valueUnderTest).isNotSameAs(value)
Assert.assertNotSame\(\s?(.*),\s?(.*)\); => assertThat($2).isNotSameAs($1);
11. Converting assertNull(objectUnderTest) to assertThat(objectUnderTest).isNull()
Assert.assertNull\(\s?(.*)\); => assertThat($1).isNull();
12. Converting assertNotNull(objectUnderTest) to assertThat(objectUnderTest).isNotNull()
Assert.assertNotNull\(\s?(.*)\); => assertThat($1).isNotNull();
13. Converting assertTrue(logicalCondition) to assertThat(logicalCondition).isTrue()
Assert.assertTrue\(\s?(.*)\); => assertThat($1).isTrue();
14. Converting assertFalse(logicalCondition) to assertThat(logicalCondition).isFalse()
Assert.assertFalse\(\s?(.*)\); => assertThat($1).isFalse();

If a regular expression is invalid and cannot be compiled by java.util.regex.Pattern.compile(String, int) the whole parsing process will be stopped.

18:41:38 [main] WARN  n.s.u.datareader.XmlDataReader - Illegal/unsupported escape sequence near index 35
^Mozilla.*Android.*LG\-F200K|F200L\F200S
                                   ^
java.util.regex.PatternSyntaxException: Illegal/unsupported escape sequence near index 35
^Mozilla.*Android.*LG\-F200K|F200L\F200S
                                   ^
    at java.util.regex.Pattern.error(Pattern.java:1713) ~[na:1.6.0_51]
    at java.util.regex.Pattern.escape(Pattern.java:2177) ~[na:1.6.0_51]
    at java.util.regex.Pattern.atom(Pattern.java:1952) ~[na:1.6.0_51]
    at java.util.regex.Pattern.sequence(Pattern.java:1885) ~[na:1.6.0_51]
    at java.util.regex.Pattern.expr(Pattern.java:1752) ~[na:1.6.0_51]
    at java.util.regex.Pattern.compile(Pattern.java:1460) ~[na:1.6.0_51]
    at java.util.regex.Pattern.<init>(Pattern.java:1133) ~[na:1.6.0_51]
    at java.util.regex.Pattern.compile(Pattern.java:847) ~[na:1.6.0_51]
    at net.sf.uadetector.internal.util.RegularExpressionConverter.convertPerlRegexToPattern(RegularExpressionConverter.java:315) ~[classes/:na]
    at net.sf.uadetector.internal.util.RegularExpressionConverter.convertPerlRegexToPattern(RegularExpressionConverter.java:286) ~[classes/:na]
    at net.sf.uadetector.internal.data.domain.DevicePattern$Builder.setPerlRegularExpression(DevicePattern.java:142) ~[classes/:na]
    at net.sf.uadetector.internal.data.XmlDataHandler.addToDevicePatternBuilder(XmlDataHandler.java:434) ~[classes/:na]
    at net.sf.uadetector.internal.data.XmlDataHandler.transferToSpecificBuilderAndReset(XmlDataHandler.java:697) ~[classes/:na]
    at net.sf.uadetector.internal.data.XmlDataHandler.endElement(XmlDataHandler.java:502) ~[classes/:na]
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:601) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endNamespaceScope(XMLDTDValidator.java:2077) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.handleEndElement(XMLDTDValidator.java:2028) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endElement(XMLDTDValidator.java:901) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2939) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205) ~[na:1.6.0_51]
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522) ~[na:1.6.0_51]
    at javax.xml.parsers.SAXParser.parse(SAXParser.java:395) ~[na:1.6.0_51]
    at javax.xml.parsers.SAXParser.parse(SAXParser.java:198) ~[na:1.6.0_51]
    at net.sf.uadetector.datareader.XmlDataReader$XmlParser.parse(XmlDataReader.java:61) ~[classes/:na]
    at net.sf.uadetector.datareader.XmlDataReader.readXml(XmlDataReader.java:111) [classes/:na]
    at net.sf.uadetector.datareader.XmlDataReader.read(XmlDataReader.java:170) [classes/:na]
    at net.sf.uadetector.datastore.AbstractDataStore.readData(AbstractDataStore.java:72) [classes/:na]
    at net.sf.uadetector.datastore.AbstractDataStore.<init>(AbstractDataStore.java:168) [classes/:na]
    at net.sf.uadetector.service.UADetectorServiceFactory$ResourceModuleXmlDataStore.<init>(UADetectorServiceFactory.java:84) [classes/:na]
    at net.sf.uadetector.service.UADetectorServiceFactory.<clinit>(UADetectorServiceFactory.java:92) [classes/:na]
    at net.sf.uadetector.UserAgentStringParserIntegrationTest.<clinit>(UserAgentStringParserIntegrationTest.java:74) [test-classes/:na]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [na:1.6.0_51]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) [na:1.6.0_51]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) [na:1.6.0_51]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513) [na:1.6.0_51]
    at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:195) [junit-4.11.jar:na]
    at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:244) [junit-4.11.jar:na]
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.11.jar:na]
    at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:241) [junit-4.11.jar:na]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) [junit-4.11.jar:na]
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) [junit-4.11.jar:na]
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) [junit-4.11.jar:na]
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) [junit-4.11.jar:na]
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) [junit-4.11.jar:na]
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) [junit-4.11.jar:na]
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) [junit-4.11.jar:na]
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309) [junit-4.11.jar:na]
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) [.cp/:na]
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197) [.cp/:na]

We should avoid this and log a specific warning message.

The online updating and caching functionality is already in the uadetector-core package but not available through UADetectorServiceFactory. We need a new method to get the updating and caching parser via the service factory.

Hi there,

it would be great if there was a simple and compact serialization (long/bitmask/binary for example) and deserialization/constructor for serialized objects.
I see that you're implementing Serializable but the resulting byte array does not seem that compact so far.
The object I just serialized is 1212 bytes long, which is far bigger than the UA String itself.
Is there anything I might have missed?

Best,

Tobias

For this user-agent strings (taken from http://www.useragentstring.com/pages/Safari/) uadetector says it is Mozilla instead of Safari:

Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.1b3pre) Gecko/20081212 Mozilla/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/526.9 (KHTML, like Gecko) Version/4.0dp1 Safari/526.8

Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.1b3pre) Gecko/20081212 Mozilla/5.0 (Windows; U; Windows NT 5.1; en) AppleWebKit/526.9 (KHTML, like Gecko) Version/4.0dp1 Safari/526.8

now user-agent-string.info claims it to be Mozilla, but it looks more like Safari to me

tested with uadetector-resources-2013.09 and uadetector-resources-2013.08

Recently I have noticed that my application fails when tries to parse user agent string "Eudora". The error looks like:

java.lang.IllegalArgumentException: Argument 'version' must not be null.
    at net.sf.uadetector.internal.util.VersionParser.parseVersion(VersionParser.java:345)
    at net.sf.uadetector.parser.AbstractUserAgentStringParser.examineAsBrowser(AbstractUserAgentStringParser.java:53)
    at net.sf.uadetector.parser.AbstractUserAgentStringParser.parse(AbstractUserAgentStringParser.java:128)....

I even don't try to argue - I agree that "Eudora" is a bad user agent string since it doesn't contain any information about version. But.... I have tried to dig deeper into uadetector source code. If you take a look at AbstractUserAgentStringParser.examineAsBrowser method, you can find the line there:

// try to get the browser version from the first subgroup
version = VersionParser.parseVersion(matcher.groupCount() > 0 ? matcher.group(1) : "");

In case if ua string is "Eudora", matcher.groupCount() > 0 returns true, but matcher.group(1) returns null. After that VersionParser.parseVersion() throws an exception (mentioned above) in case if version is null.

My question is: probably it is worth to change the implementation of AbstractUserAgentStringParser.examineAsBrowser in the following way:

replace
version = VersionParser.parseVersion(matcher.groupCount() > 0 ? matcher.group(1) : "");

with
version = VersionParser.parseVersion(matcher.groupCount() > 0 && matcher.group(1) != null ? matcher.group(1) : "");

According to this implementation a "" value will be passed to VersionParser instead of null. Or could you tell me please how such kind of error can be handled in an alternative way? I'm not sure it is a good idea to catch IllegalArgumentException in my code, in case if uadetector would thrown some special exception (e.g. net.sf.uadetector.exception.VersionParsingException) it wouldn't have been a problem for me to catch such kind of exceptions in my application

The implementations including core 0.9.9 have an issue when an identified browser is not in maintained in our enum UserAgentFamily that the family name holds the text "unknown" instead of the determined name.

I'm using OnlineUpdatingParser, and some times in the log i find this kind of trace:

...
INFO: Illegal access: this web application instance has been stopped already.  Could not load META-INF/services/javax.xml.parsers.SAXParserFactory.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
Oct 03, 2013 12:42:49 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already.  Could not load com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
    at javax.xml.parsers.FactoryFinder.getProviderClass(FactoryFinder.java:112)
    at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:178)
    at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:147)
    at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:265)
    at javax.xml.parsers.SAXParserFactory.newInstance(SAXParserFactory.java:126)
    at net.sf.uadetector.datareader.XmlDataReader$XmlParser.parse(XmlDataReader.java:57)
    at net.sf.uadetector.datareader.XmlDataReader.readXml(XmlDataReader.java:111)
    at net.sf.uadetector.datareader.XmlDataReader.read(XmlDataReader.java:170)
    at net.sf.uadetector.datastore.UpdateOperationTask.call(UpdateOperationTask.java:35)
    at net.sf.uadetector.datastore.AbstractUpdateOperation$1.run(AbstractUpdateOperation.java:178)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)

Oct 03, 2013 12:42:49 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already.  Could not load com.sun.org.apache.xerces.internal.impl.dv.dtd.DTDDVFactoryImpl.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
    at com.sun.org.apache.xerces.internal.utils.ObjectFactory.findProviderClass(ObjectFactory.java:324)
    at com.sun.org.apache.xerces.internal.utils.ObjectFactory.newInstance(ObjectFactory.java:269)
    at com.sun.org.apache.xerces.internal.utils.ObjectFactory.newInstance(ObjectFactory.java:255)
    at com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory.getInstance(DTDDVFactory.java:63)
    at com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory.getInstance(DTDDVFactory.java:49)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.<init>(XML11Configuration.java:565)
    at com.sun.org.apache.xerces.internal.parsers.XIncludeAwareParserConfiguration.<init>(XIncludeAwareParserConfiguration.java:130)
    at com.sun.org.apache.xerces.internal.parsers.XIncludeAwareParserConfiguration.<init>(XIncludeAwareParserConfiguration.java:91)
    at com.sun.org.apache.xerces.internal.parsers.SAXParser.<init>(SAXParser.java:106)
    at com.sun.org.apache.xerces.internal.parsers.SAXParser.<init>(SAXParser.java:91)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.<init>(SAXParserImpl.java:377)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.<init>(SAXParserImpl.java:127)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl.newSAXParser(SAXParserFactoryImpl.java:81)
    at net.sf.uadetector.datareader.XmlDataReader$XmlParser.parse(XmlDataReader.java:59)
    at net.sf.uadetector.datareader.XmlDataReader.readXml(XmlDataReader.java:111)
    at net.sf.uadetector.datareader.XmlDataReader.read(XmlDataReader.java:170)
    at net.sf.uadetector.datastore.UpdateOperationTask.call(UpdateOperationTask.java:35)
    at net.sf.uadetector.datastore.AbstractUpdateOperation$1.run(AbstractUpdateOperation.java:178)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)

Oct 03, 2013 12:42:49 PM org.apache.catalina.loader.WebappClassLoader findResourceInternal
INFO: Illegal access: this web application instance has been stopped already.  Could not load uadetector/uasxmldata.dtd.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
Exception in thread "update-operation" net.sf.qualitycheck.exception.IllegalStateOfArgumentException: The passed arguments have caused an invalid state: Argument 'data' must not be empty.
    at net.sf.uadetector.datastore.AbstractRefreshableDataStore.checkData(AbstractRefreshableDataStore.java:61)
    at net.sf.uadetector.datastore.AbstractRefreshableDataStore.setData(AbstractRefreshableDataStore.java:229)
    at net.sf.uadetector.datastore.UpdateOperationTask.call(UpdateOperationTask.java:36)
    at net.sf.uadetector.datastore.AbstractUpdateOperation$1.run(AbstractUpdateOperation.java:178)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
...

I was wondering if this is a known issue or someone has a clue why this can happened, this will be really helpfull.
Maybe I better use CachingAndUpdatingParser..?

One of the often requested feature is to detect devices too. To be able to implement this, we need a large list of User-Agent strings with informations about the corresponding device. I do not think we get this list, but we are able to implement it partly (on demand).

I've thought about that and playing with the idea to implement a set of known fragments, so that you can write something like this:

UserAgentStringParser parser = UADetectorServiceFactory.getResourceModuleParser();
String uaString = "Mozilla/5.0 (iPad; CPU OS 5_0 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A334 Safari/7534.48.3";
UserAgent ua = parser.parse(uaString);
if (ua.getOperatingSystem().getFamily() == OperatingSystemFamily.IOS) {
    if (ua.getKnownFragments().contains(KnownFragments.IPAD)) {
        System.out.println("Seems to be an iPad.");
    } else if (ua.getKnownFragments().contains(KnownFragments.IPHONE)) {
        System.out.println("Seems to be an iPhone.");
    }
}

Would that help? Please provide some feedback as comment below.

I updated uadetector-example today which uses: UADetectorServiceFactory.getCachingAndUpdatingParserHolder()

At the time of my testing, connections to http://user-agent-string.info/rpc/get_data.php?key=free&format=xml were timing out. My test program had to wait for a full TCP timeout period (over a minute) before it started parsing user agents.

My question is: Should updates that involve network connections always run in the background? (i.e. never cause a bottleneck before parsing the first user agent). I had a /tmp/uas.xml on top of the data file in uadetector-resources which could have been used until any network obtained update was ready.

Thanks,
Dmitry

Version 2012.12

Given the user agent string: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.14 (KHTML, like Gecko) Version/6.0.1 Safari/536.26.14"

I would expect getVersionNumber().getMajor() to be "6" and getVersionNumber().getMinor() to be "0". Instead, they are "536" and "26", respectively.

EDIT: It appears this works as expected in 2012.11

getFamily(): UserAgentFamily  = net.sf.uadetector.UserAgentFamily@4a4a04aa[name=Safari,pattern=Safari,name=SAFARI,ordinal=615]
getIcon(): String             = safari.png
getName(): String             = Safari
getOperatingSystem(): OperatingSystem= net.sf.uadetector.OperatingSystem@2c6d1158[family=OS_X,familyName=OS X,icon=macosx.png,name=OS X 10.8 Mountain Lion,producer=Apple Computer, Inc.,producerUrl=http://www.apple.com/,url=http://www.apple.com/osx/,versionNumber=VersionNumber [groups=[10, 8, 2], extension=]]
getProducer(): String         = Apple Inc.
getProducerUrl(): String      = http://www.apple.com/
getType(): UserAgentType      = net.sf.uadetector.UserAgentType@e0340d1[name=Browser,name=BROWSER,ordinal=0]
getTypeName(): String         = Browser
getUrl(): String              = http://en.wikipedia.org/wiki/Safari_%28web_browser%29
getVersionNumber(): VersionNumber= net.sf.uadetector.VersionNumber@2e4d3abf[groups=[536, 26, 14],extension=]

------------- class net.sf.uadetector.VersionNumber
toString(): VersionNumber [groups=[536, 26, 14], extension=]
getBugfix(): String           = 14
getClass(): Class             = java.lang.Class@504be958[annotationType=<null>]
getExtension(): String        = 
getGroups(): List             = java.util.Collections$UnmodifiableRandomAccessList@1bb41390[list=java.util.ArrayList@5c244f19,c=java.util.ArrayList@5c244f19]
getMajor(): String            = 536
getMinor(): String            = 26

If a an user of UADetector whats to build an own parser with updating functionality he/she needs to create an implementation of an update operation compeletly for its own if a customization is needed.
Therefore we should provide our net.sf.uadetector.datastore.AbstractUpdateOperation to the public to make it easier for others to extend and customize it.