lcosmin / boardgamegeek Goto Github PK

When investigating the response from this library, the game Port Royal does not include a min/max playing time, but instead only includes the singular property playing_time. In the case of Port Royal, the playing_time property only includes the max playing time, and not the min.

Reference the raw XML response for additional details:

<playingtime value="50"/>
<minplaytime value="20"/>
<maxplaytime value="50"/>

This library does include min/max for player count, and min for age.

import boardgamegeek

bgg = BGGClient()

NameError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_908\3723979965.py in
1 import boardgamegeek
2
----> 3 bgg = BGGClient()

NameError: name 'BGGClient' is not defined

The old BGG API had a call to be able to collect the field "conditiontext" (aka Trade Condition). It still exists as I can see it when I export my game collection list from the UI. Has this been deprecated from the API?

Allow the application to pass options (e.g. timeout) to the requests library.

For some reason BGG will always hit the default timeout (5 seconds) on some games. I assume ther is some backend processing that takes longer in some cases. "Go" is an example.

> python ./boardgamegeek/main.py -g go --debug

I added a recursive call to get_parsed_xml_response() to handle this. On a timeout, get_parsed_xml_response() will call itself again 'retries' times, backing off the timeout by 1.5 times.

github wouldn't let me fork this repo possibly because it is a fork of one of my repos. :) So I couldn't do the pull-request thing. If you want it, here's the diff:

diff --git a/boardgamegeek/utils.py b/boardgamegeek/utils.py
index 671caf2..be81ffe 100644
--- a/boardgamegeek/utils.py
+++ b/boardgamegeek/utils.py
@@ -16,6 +16,7 @@ import xml.etree.ElementTree as ET
 from xml.etree.ElementTree import ParseError as ETParseError
 import requests_cache
 import requests
+import logging

 try:
     import urllib.parse as urlparse
@@ -24,6 +25,7 @@ except:

 from .exceptions import BoardGameGeekAPIError, BoardGameGeekAPIRetryError, BoardGameGeekError, BoardGameGeekAPINonXMLError

+log = logging.getLogger("boardgamegeek.api")

 class DictObject(object):
     """
@@ -146,7 +148,7 @@ def xml_subelement_text(xml_elem, subelement, convert=None):
     return text


-def get_parsed_xml_response(requests_session, url, params=None, timeout=5):
+def get_parsed_xml_response(requests_session, url, params=None, timeout=5, retries=5):
     """
     Downloads an XML from the specified url, parses it and returns the xml ElementTree.

@@ -157,6 +159,9 @@ def get_parsed_xml_response(requests_session, url, params=None, timeout=5):
     :raises: :class:`BoardGameGeekAPIRetryError` if this request should be retried after a short delay
     :raises: :class:`BoardGameGeekAPIError` if the response couldn't be parsed
     """
+    if not retries:
+        raise BoardGameGeekError('maximum retries when getting response')
+
     try:
         r = requests_session.get(url, params=params, timeout=timeout)

@@ -176,7 +181,10 @@ def get_parsed_xml_response(requests_session, url, params=None, timeout=5):
             root_elem = ET.fromstring(utf8_xml)

     except requests.exceptions.Timeout:
-        raise BoardGameGeekError("API request timeout")
+        timeout *= 1.5
+        retries -= 1
+        log.debug("API request timeout, retrying {} more times w/timeout {}".format(retries, timeout))
+        return get_parsed_xml_response(requests_session, url, params, timeout, retries)

     except ETParseError as e:
         raise BoardGameGeekAPIError("error decoding BGG API response: {}".format(e))

The current name property in BoardGameVersion class is what is actually called "nickname" in the web site.

(For example, the main name for game 41052 is "Loco Motive", but the name of the version 337221 is "Tokyo Train" while its nickname is "Cocktail Games French edition".)

Unfortunately, the actual name ("Tokyo Train" in the example) is not returned by the BGG XMLAPI2 (it returns "Cocktail Games French edition"). And the legacy API does not help for this problem.

The only way I could find to be able to retrieve this information is through a hidden :-( API.
E.g.:
https://api.geekdo.com/api/geekitem/linkeditems?ajax=1&linkdata_index=boardgameversion&nosession=1&objectid=41052&objecttype=thing&pageid=1&showcount=10&sort=yearpublished&subtype=boardgameversion
returns:

// ...
{
    "yearpublished": "2013",
    // ...
    "linktype": "boardgameversion",
    "objecttype": "version",
    "objectid": "337221",
    // ...
    "linkedname": "Tokyo Train",
    "links": {
        // ...
        "languages": [
            {
                "name": "French",
                // ...
            }
        ]
    },
    "href": "\/boardgameversion\/337221\/cocktail-games-french-edition",
    "versionname": "Cocktail Games French edition",
    "images": {
        // ...
    }
},
// ...

(See also: https://boardgamegeek.com/thread/1616315/getting-structured-game-version-and-publisher-data.)

Though, I'm really not sure if using undocumented (and even less supported) API's is in the scope of this project... :-/

?? Or should I start a side project ??

I was looking at the documentation, and it seems that there are no options for user data via this library. http://lcosmin.github.io/boardgamegeek/modules.html#module-boardgamegeek.user Is this true? or is this just a fluke of the documentation? I interested in playing with user data, but need a tool to access it. I started messing around with this tool some weeks ago when I was looking at game data.

The BGG site seems to have added a mechanism to throttle requests (you start getting HTTP 503 if doing too many requests too fast); this forces the application using boardgamegeek to add delays in the code to make sure it doesn't trigger BGG's protection.

However, these delays are useless if the requested information is already in the cache. A solution would be to add a throttling mechanism inside the library itself (it would be aware of what's in the cache and what not).

In the XML API, a playingtime and min-, max-playingtime are available at the top level. Would you consider adding this to the API too?

Example: https://www.boardgamegeek.com/xmlapi/boardgame/68448

<playingtime>30</playingtime>
<minplaytime>30</minplaytime>
<maxplaytime>30</maxplaytime>

The search API is not exposed by the module, just used internally for converting game names to game ids. Allow the user to call the search API.

Is it possible to allow manual changing of the api_endpoint variable? RPGGeek is still on xmlapi so forcing xmlapi2 means you can't access RPGGeek data using boardgamegeek2

Something like allowing this would be helpful:

bgg = BGGClient(api_endpoint="https://boardgamegeek.com/xmlapi")

Hello,
I think, there is a issue when searching a location in a played game session.
The location information is empty when you get a bgg play session

In [54]: t.plays(name=username, min_date=mindate.date())[0].location

In [55]: t.plays(name=username, min_date=mindate.date())[0].players[0].location

In [56]:

The procedure add_play_from_xml seems to search data on the player item.

    player_data = {"username": player.attrib.get("username"),
                       "user_id": int(player.attrib.get("userid", -1)),
                       "name": player.attrib.get("name"),
                       "startposition": player.attrib.get("startposition"),
                       "new": player.attrib.get("new"),
                       "win": player.attrib.get("win"),
                       "rating": player.attrib.get("rating"),
                       "score": player.attrib.get("score"),
                       "color": player.attrib.get("color"),
                       "location": player.attrib.get("location")}

I 've changed the loaders/play.py to avoid this issue.
line 60 : add "location":play.attrib["location"], in the add_play_from_xml procedure

    data = {"id": int(play.attrib["id"]),
            "date": play.attrib["date"],
            "quantity": int(play.attrib["quantity"]),
            "duration": int(play.attrib["length"]),
            "location":play.attrib["location"],
      "incomplete": int(play.attrib["incomplete"]),
            "nowinstats": int(play.attrib["nowinstats"]),
            # for User plays, will be overwritten with the user id when adding the play.
            "user_id": int(play.attrib.get("userid", -1)),
            "game_id": xml_subelement_attr(play, "item", attribute="objectid", convert=int),
            "game_name": xml_subelement_attr(play, "item", attribute="name"),
            "comment": xml_subelement_text(play, "comments"),
            "players": player_list}`

In [8]: t=BGGClient()

In [9]: BGGClient().plays(name=username, min_date=mindate.date())[0]
Out[9]: <boardgamegeek.objects.plays.PlaySession at 0x6c839e8>

In [10]: BGGClient().plays(name=username, min_date=mindate.date())[0].location
Out[10]: 'XXXXXXX'

What would your thoughts be on making the tests true unit tests (i.e. ones which don't depend on any external systems)? For example, changing test_get_unknown_game_info in test_game.py to something like:

class MockResponse():
    def __init__(self, text):
        self.headers = {'content-type': 'text/xml'}
        self.status_code = 200
        self.text = text

def test_get_unknown_game_info(bgg, mocker):
    mock_get = mocker.patch('requests.sessions.Session.get')
    mock_get.return_value = MockResponse('<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="http://boardgamegeek.com/xmlapi/termsofuse"></items>')

    with pytest.raises(BGGItemNotFoundError):
        game = bgg.game(TEST_INVALID_GAME_NAME)

The advantages of this would be twofold:

1. Remove the dependency on the specific text being returned by BGG (as we're mocking it out) - therefore the tests would no longer start failing every time someone adds a new mechanic to Agricola or similar changes.
2. Make the tests run much quicker - no network access, and no need to add that 15 second throttle to avoid BGG getting grumpy with us.

Most functions raise exceptions related to various error conditions, but also can return None for others. Exceptions should be raised in all error cases.

Hello, it's me again. :)

When you request the game 'Bang!' with python 3.4, the library raises a Type exception:

[glawler@seek42:~/src/boardgamegeek/boardgamegeek]$ python3 -m main -g 'Bang!'
Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/glawler/src/boardgamegeek/boardgamegeek/main.py", line 89, in <module>
    main()
  File "/home/glawler/src/boardgamegeek/boardgamegeek/main.py", line 47, in main
    game = bgg.game(args.game)
  File "/home/glawler/src/boardgamegeek/boardgamegeek/api.py", line 653, in game
    game_id = self.get_game_id(name)
  File "/home/glawler/src/boardgamegeek/boardgamegeek/api.py", line 633, in get_game_id
    return self._get_game_id(name, "boardgame")
  File "/home/glawler/src/boardgamegeek/boardgamegeek/api.py", line 113, in _get_game_id
    elif year > _year:
TypeError: unorderable types: NoneType() > int()

This can be fixed by making sure year is an int by adding this line after you fetch the year on line 106 in api.py. Here's a snippet that fixes it:

        for game in root.findall(".//item[@type='{}']".format(game_type)):
            year = xml_subelement_attr(game, "yearpublished", convert=int)
            year = 0 if not year else int(year)
            if _year is None:
                _year = year

http://www.boardgamegeek.com/wiki/page/BGG_XML_API2#toc10

The way the XML is currently parsed will make it impossible for the games' BGG ratings to be populated since it is looking in the STATS tag while they reside in the RATING tag withing the STATS tag.

Have fixed it myself on my own library, I can help if you wish

I am running Python 2.7.6 - I get the following error when trying the example:

>>> g = bgg.game("Android: Netrunner")
/home/derek/.virtualenvs/boardgamegeek/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/derek/.virtualenvs/boardgamegeek/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning

What do I need to do to overcome this?

It looks like BGG is using unsigned ints for years. So when you search for Go, which has a published year of -2200, the API returns:

That value gets cast to an "int" in api.py, which python happily converts to a long. Then the boardgamegeek library checks the type of yearpublished in search.py (line 27 or so), finds the value is a long and not an int, then throws an exception.

This can be "fixed" in the boardgamegeek library by checking for a large year and converting to a negative int if found. Annoying and hacky but it should work.

year = int(4294965096)
....
if year > 99999: # or something large
year = int(-(pow(2, 32)-year))
...

I was planning on cloning the repo on my Windows PC, but when I try to do so, I get several errors about being unable to create files in the test/xml directory. My guess is that it's because the filenames there have the '?' character in the name, which is apparently not allowed on Windows. Is there anyway to rename these files to allow development on Windows? I'm not sure what the repercussions of this would be, so I figured I'd post about it and ask.

Thanks!

python boardgamegeek/main.py -g illuminati

Does not return the first in the list at BGG, game 859. It returns game 17687, which is only connected to the game "Illuminati" because that is listed as one of it's alternate names.

I have the following code which I am running in terminal on OS X.
boardgamegeek-0.13.2
Python 2.7.9 (default, Jan 29 2015, 06:28:58)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin

from boardgamegeek import BoardGameGeek
>>> bgg = BoardGameGeek()
>>> g = bgg.game("carcassonne")
>>> g.name 'carcassonne'
>>> for n in g.expansions: print n.id, n.name

Which works as expected.

however when I try the following I get an error.

for n in g.expansions: print n.id, n.name, n.alternative_names

> 167903 20 Jahre Darmstadt Spielt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/boardgamegeek/utils.py", line 102, in __getattr__
    raise AttributeError

I am both new to python and to the bgg module. Any suggestions on what my problem might be?

Is this at all because there is many to one relationship? I mean some things have more than one alternative name...

I had to do some investigation following the discussion in PR #55 and tested the actual HTTP codes for all the URLs simulated by the unit tests.

The reason is that the implementation of the legacy api required to trap the status code 404,
to detect item not found errors.
We had to verify if this have impact on existing requests for the (non-legacy) API.

Some objects don't have a __repr__() function, others should have more info.

I'm having some issues getting data of out RPGGeek with this. I can't use game(), search(), or get_game_id() for RPGs. It looks like searching for boardgames is hardcoded in.

Hi,
I was wondering if it is possible to access the private information of the collections. I understand that we need to be login but I am not sure how to do it from your API.

Thanks

Hi,

I'm trying to follow the usage example on this page. However this runs into trouble fast.

$ pip install boardgamegeek -U
Requirement already up-to-date: boardgamegeek in c:\anaconda3\lib\site-packages
Requirement already up-to-date: requests-cache>=0.4.4 in c:\anaconda3\lib\site-packages (from boardgamegeek)
Requirement already up-to-date: requests>=2.3.0 in c:\anaconda3\lib\site-packages (from boardgamegeek)
Requirement already up-to-date: idna<2.7,>=2.5 in c:\anaconda3\lib\site-packages (from requests>=2.3.0->boardgamegeek)
Requirement already up-to-date: chardet<3.1.0,>=3.0.2 in c:\anaconda3\lib\site-packages (from requests>=2.3.0->boardgamegeek)
Requirement already up-to-date: certifi>=2017.4.17 in c:\anaconda3\lib\site-packages (from requests>=2.3.0->boardgamegeek)
Requirement already up-to-date: urllib3<1.23,>=1.21.1 in c:\anaconda3\lib\site-packages (from requests>=2.3.0->boardgamegeek)

$ python
Python 3.6.0 |Anaconda 4.3.1 (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

from boardgamegeek import BGGClient
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'BGGClient'

Any advise?

Shows up as warnings when running the unit tests with Python 3:

$ python3 setup.py test
[...]
/home/philip/boardgamegeek/boardgamegeek/utils.py:278: DeprecationWarning: The unescape method is deprecated and will be removed in 3.5, use html.unescape() instead.
text = convert(text)

The issue isn't actually in utils.py but at loaders/game.py:32 and loaders/guild.py:23 both of which set convert to html_parser.unescape:

"description": xml_subelement_text(xml_root, "description", convert=html_parser.unescape, quiet=True)}

See http://boardgamegeek.com/boardgamefamily/4311/arkham-horror for example

Attempting to fetch "Clash of the Princes" (ID 54165) results in an exception; trivial test case is BGGClient().game(game_id = 54165):

Traceback (most recent call last):
File "./get-bgg-game.py", line 5, in
BGGClient().game(game_id = 54165)
File "/home/philip/.local/lib/python3.6/site-packages/boardgamegeek/api.py", line 845, in game
html_parser=html_parser)
File "/home/philip/.local/lib/python3.6/site-packages/boardgamegeek/loaders/game.py", line 217, in create_game_from_xml
return BoardGame(data)
File "/home/philip/.local/lib/python3.6/site-packages/boardgamegeek/objects/games.py", line 924, in init
super(BoardGame, self).init(data)
File "/home/philip/.local/lib/python3.6/site-packages/boardgamegeek/objects/games.py", line 567, in init
self._year_published = fix_unsigned_negative(data["yearpublished"])
File "/home/philip/.local/lib/python3.6/site-packages/boardgamegeek/utils.py", line 389, in fix_unsigned_negative
if value > 0x7FFFFFFF:
TypeError: '>' not supported between instances of 'NoneType' and 'int'

From eyeballing the XML, I suspect this is because the release year is just entirely unset: <yearpublished value="" />.

Thanks for your time on this library. It's very well written. I'm using it locally to format plays for blog posts that I write weekly

Here:
http://www.meeplemountain.com/articles/game-every-day-journal-june-16th-2016/
and here:
http://www.meeplemountain.com/articles/2016/01/game-every-day/

I also wrote a quick script to compare the collections of two users:
https://gist.github.com/commadelimited/c2b76b16252017aa1a519fb82c8c2d5b

I'm wondering if you've ever considered including methods like this in a syntactic sugar type module. One which doesn't tie directly to endpoints in the BGG API, but instead includes additional things that users might want.

According to PyPi, the last release for this package was in 2015 (ver 0.13.2) but there have been changes and patches since then.

Is there any chance for an update - preferable one working with Python 3.9 and 3.10?

Some boardgamegeek functions don't expose all parameters that are supported by the XML2 API; add the missing ones.

I get the following error message:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/irazall/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/211.7142.13/plugins/python/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "/home/irazall/promotion/it/python-projects/envs/dynamic_pricing/lib/python3.8/site-packages/boardgamegeek/__init__.py", line 10, in <module>
    from .api import BGGClient, BGGChoose, BGGRestrictDomainTo, BGGRestrictPlaysTo, BGGRestrictSearchResultsTo, BGGRestrictCollectionTo
  File "/home/irazall/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/211.7142.13/plugins/python/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "/home/irazall/promotion/it/python-projects/envs/dynamic_pricing/lib/python3.8/site-packages/boardgamegeek/api.py", line 703, in <module>
    class BGGClient(BGGCommon):
  File "/home/irazall/promotion/it/python-projects/envs/dynamic_pricing/lib/python3.8/site-packages/boardgamegeek/api.py", line 727, in BGGClient
    def __init__(self, cache=CacheBackendMemory(ttl=3600), timeout=15, retries=3, retry_delay=5, disable_ssl=False, requests_per_minute=DEFAULT_REQUESTS_PER_MINUTE):
  File "/home/irazall/promotion/it/python-projects/envs/dynamic_pricing/lib/python3.8/site-packages/boardgamegeek/cache.py", line 23, in __init__
    self.cache = requests_cache.core.CachedSession(backend="memory", expire_after=ttl, allowable_codes=(200,))
AttributeError: module 'requests_cache' has no attribute 'core'

I assume that some dependencies are deprecated. Thanks anyway for implementing this tool!

Hello, I would like to be able to access player comments in the Collection object, and I see it's in the xml but not in the object. Any reason it was excluded? If not, I can try to implement it.

Hi! Thanks for a great library. The BoardGame class has, according to the docs, a player_suggestions property. But I get KeyError trying to access it. I do get info back with the name suggested_numplayers, could that have changed, without the docs being updated?

• Retrieve the other items (expansions, etc.) when listing plays, not only boardgames
• Add support for getting the players of a play session

When retriving a game by title and there is more than one game which has that title, only one game is returned. You may want to return a list of games or allow the user to specify in some way which of the specific games he or she wants.

See Trains, Coup, and Mascarade for examples. In each case the older game is returned when most of the time (I would think) the newest one is the one desired.

In testing the call for a game to obtain marketplace data it seems that marketplace data is not returned (i.e. previous sales of the game that includes date, amount sold for, trade condition). I am hoping to get historical selling values for specific games to calculate the value of my collection. This worked in the older API call.

x.game(game_id=1234,marketplace=True)