sanjacob / blackboardsync Goto Github PK
View Code? Open in Web Editor NEWDownload your Blackboard Learn content automatically
Home Page: https://bbsync.app
License: GNU General Public License v2.0
Download your Blackboard Learn content automatically
Home Page: https://bbsync.app
License: GNU General Public License v2.0
University name: The Chinese University of Hong Kong, Shenzhen
Short name or abbreviation: CUHK-Shenzhen or CUHKSZ
Link to Blackboard portal: bb.cuhk.edu.cn
Landing URL after successfully logging in: https://bb.cuhk.edu.cn/webapps/portal/execute/tabs/tabAction?tab_tab_group_id=_1_1
I can see that this tool can support CUHK, and I think CUHK-Shenzhen works similarly. Thanks!
A response may contain "x-bb-lesson"
in the id
field, which fails to validate as the BBResourceType
enum does not have a corresponding value. https://github.com/jacobszpz/BlackboardSync/blob/8e7acee46199e01596576155ec9b55458516fce9/blackboard_sync/blackboard/blackboard.py#L106-L120
These response values are observed for SHSU's Blackboard service.
Since session is externally handled by Qt, cookie store must be cleaned out somehow
The domain for SHSU's blackboard service will change to blackboard.shsu.edu
at 8:00PM CST Nov 11, 2023 according to SHSU's own communications. I will monitor this change to see if the niversities.json
file must change to reflect this.
Bug only present on Linux, where the file path of the link is previously created as a directory, thus preventing the later creation of the web link.
University name (University of Connecticut).
Short name or abbreviation (UCONN).
Link to Blackboard portal (https://huskyct.uconn.edu/ or https://lms.uconn.edu/).
Both work from browser but unsure which one is correct
Landing URL after successfully logging in (https://huskyct.uconn.edu/ultra/).
Further investigation is needed
With the goal of decoupling things and allowing these two projects to grow on their own, I have begun decoupling BlackboardSync from the Blackboard API Client (module blackboard), the latter now exists in its own repo bblearn.
The current goals for BlackboardSync are to improve error prevention, static verification, and to refactor.
Furthermore, also to set the scene for future features such as improved content filtering.
This behaviour is not handled yet
This should help the user realise where the app is located.
The exception handling in _sync_task
incorrectly assumes that any ValueError
is the result of an expired session. https://github.com/jacobszpz/BlackboardSync/blob/8e7acee46199e01596576155ec9b55458516fce9/blackboard_sync/sync.py#L174-L178
pydantic.ValidationError
inherits from ValueError
, causing _sync_task
to handle validation errors with the above block. The result is that the application will assume that the session has expired when it has not. The application will then restart synchronization, validation will fail again, rinse, repeat. This loop does not break, and the user is provided no indication this is happening outside logs.
For SHSU specifically, the application:
Sam Houston State University
SHSU
https://shsu.blackboard.com/ultra
No response
No response
The "login page" provides a list of options for universities to log in with. I have confirmed that at least one of these universities redirects back to the original shsu.blackboard.com
domain, and I presume the API routes are likely the same.
For reference, here's my testing entry in universities.json
:
{
"name": "Sam Houston State University",
"short_name": "SHSU",
"login": {
"start_url": "https://shsu.blackboard.com/",
"target_url": "https://shsu.blackboard.com/ultra/"
},
"api_url": "https://shsu.blackboard.com/"
}
When I run the 'blackboard_sync' module using 'python -m blackboard_sync', the following error occurs:
Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Python311\Lib\site-packages\blackboard_sync_main.py", line 32, in
raise SystemExit(main())
^^^^^^
File "C:\Python311\Lib\site-packages\blackboard_sync_main.py", line 27, in main
BBSyncController()
File "C:\Python311\Lib\site-packages\blackboard_sync\sync_controller.py", line 58, in init
self._build_login_window(self.model.university.login)
File "C:\Python311\Lib\site-packages\blackboard_sync\sync_controller.py", line 96, in _build_login_window
self.login_window = LoginWebView(start_url=uni_login_info.start_url,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\blackboard_sync\qt\qt_elements.py", line 475, in init
self._init_ui()
File "C:\Python311\Lib\site-packages\blackboard_sync\qt\qt_elements.py", line 480, in _init_ui
self.web_view.load(QUrl.fromUserInput(self.start_url))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: arguments did not match any overloaded call:
fromUserInput(userInput: str): argument 1 has unexpected type 'pydantic_core._pydantic_core.Url'
fromUserInput(userInput: str, workingDirectory: str, options: Union[QUrl.UserInputResolutionOptions, QUrl.UserInputResolutionOption] = QUrl.DefaultResolution): argument 1 has unexpected type 'pydantic_core._pydantic_core.Url'
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
httplib_response = conn.getresponse()
File "/usr/local/lib/python3.9/http/client.py", line 1347, in getresponse
response.begin()
File "/usr/local/lib/python3.9/http/client.py", line 307, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.9/http/client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/local/lib/python3.9/socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.9/ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "/usr/local/lib/python3.9/ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/util/retry.py", line 531, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback(tb)
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
httplib_response = conn.getresponse()
File "/usr/local/lib/python3.9/http/client.py", line 1347, in getresponse
response.begin()
File "/usr/local/lib/python3.9/http/client.py", line 307, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.9/http/client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/local/lib/python3.9/socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.9/ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "/usr/local/lib/python3.9/ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/threading.py", line 950, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.9/threading.py", line 888, in run
self._target(*self._args, **self._kwargs)
File "/home/jacob/Development/Python/BlackboardSync/blackboard_sync/sync.py", line 187, in sync_task
self.last_sync = new_download.download()
File "/home/jacob/Development/Python/BlackboardSync/blackboard_sync/download.py", line 137, in download
all_memberships = self._sess.fetch_user_memberships(user_id=self.user_id,
File "/home/jacob/Development/Python/BlackboardSync/blackboard_sync/blackboard/api.py", line 147, in get_wrapper
response = self._bb_session.get(endpoint_format, params=kwargs, **g_kwargs)
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/home/jacob/.local/share/virtualenvs/BlackboardSync-jhqPDJ_d/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
First of all, thanks for creating this project. It would be great to see my own institution on this list if possible.
Name: University of Lincoln
Abbr.: UoL
Portal URL: https://blackboard.lincoln.ac.uk/
Landing URL: https://blackboard.lincoln.ac.uk/ultra
dateSourceID: _570_1
A response can contain "Disabled"
or "Term"
in the available
field, which fails to validate as BBAvailability
expects a bool
. https://github.com/jacobszpz/BlackboardSync/blob/8e7acee46199e01596576155ec9b55458516fce9/blackboard_sync/blackboard/blackboard.py#L84-L85
These response values are observed for SHSU's Blackboard service.
Hello, here is the data for the University of Pretoria
University name: University of Pretoria
Short name or abbreviation: UP
Link to Blackboard portal: https://clickup.up.ac.za
Landing URL after successfully logging in: https://clickup.up.ac.za/ultra
I have tested this on my machine and it does download the content.
The university is in Africa if you want to add it to the supported universities list
Thanks
Previously, a 403 response error would generate a ValueError
which was caught by BlackboardSync._sync_task(...)
and handled as an authorization error. Because the subsequent call to BlackboardSync.auth(...)
simply recreated the BlackboardSession
and returned (effectively a no-op), the application would simply (attempt to) download the next item.
With the new, differentiated BBAuthError
, this response code is no longer handled with this no-op. This will place the BlackboardSync
object into an error state. Unfortunately, the response that I receive does not have any fields other than a message field; there is nothing to check for.
I would like to handle this type of response code explicitly instead of resetting the session, like is done with a 403 response for private courses. Since there is no clear way to differentiate different 403 responses, a base exception class titled BBForbiddenError
should suffice. BBPrivateCourseError
will inherit from this, as it is a 403-related error. I will edit this issue to reference the PR once the new exceptions are completely integrated.
Hi,
Thank you for adding UoM support. Please could you provide a new .exe with the University of Manchester included?
Thanks!
University of Antwerp
UA
https://lms.uantwerpen.be/ultra/
{
"name": "University of Antwerp",
"short_name": "Universiteit Antwerpen",
"login": {
"start_url": "https://lms.uantwerpen.be",
"target_url": "https://lms.uantwerpen.be/ultra/"
},
"api_url": "https://lms.uantwerpen.be/"
}
University of Antwerp
Value error, 'x-bb-bltiplacement-mediasite.lesopnames' is not a valid BBResourceType [type=value_error, input_value='resource/x-bb-bltiplacement-mediasite.lesopnames', input_type=str]
I get an error when it tries to download the course video's (lesopnames) from blackboard.
Below is the log file. For courses without course video's, it works.
sync_error_2023-11-14T16_00_48_646488.log
By creating a fake API which returns pre-determinate results, and hosting it locally, it will be possible to test the download module.
0.11.2 Latest
windows and python
3.11
windows 11
schoolcraft college
hey bro, I am sorry. I wasn't trying to be rude. I think we have a timezone difference thats why my messages come unorganized. Please man, I need this fixed before my class ends, that's why I was tagging you multiple times. Please help.
No response
Latest
MacOS build
No response
macOS
UAntwerpen
Somehow UAntwerpen formatted specific folders as "Documents" with tags referring to the actual documents. This raises a ValueError in download.py which is not intercepted. Adding a pass for ValueError solves this issue for me:
` elif res in (BBResourceType.file, BBResourceType.document) and has_changed:
attachments = []
try:
attachments = self._sess.fetch_file_attachments(course_id=course_id,
content_id=content.id)
except RequestException:
self.logger.warn(f"Error while getting attachments for {course_id}")
except ValueError:
pass`
Since its probably only a UAntwerpen thing and not applicable for other universities (and might break stuff) I decided to just let you know and make it a pull request.
No response
Hanze University of Applied Sciences
Hanze UAS
https://blackboard.hanze.nl/ultra
No response
No response
No response
Hi, here is the data for University of Minho. I already tried it and it seems be working.
name
: University of Minho
short_name
: UM
start_url
: https://elearning.uminho.pt
target_url
: https://elearning.uminho.pt/ultra/
api_url
: https://elearning.uminho.pt
Data sourceIDs is always "_2_1"
University name: Centro Universitario de Tecnología y Arte Digital
Short name or abbreviation: U-tad
Link to Blackboard portal: https://u-tad.blackboard.com/
Landing URL after successfully logging in: https://u-tad.blackboard.com/ultra/
All DataSourceIDs are the same: "dataSourceId": "_2_1"
From source
Python 3.11
Linux 6.6.30-2-lts
University of Pretoria
The UI limits the download year to be from 2023 or before meaning I cannot download only the present year's content. Changing the maximum value to 2024 in qt/SetupWizard.ui seems to make it work as expected.
No response
Hello,
I'd like to add support for Blackboard Sync for Griffith University, Australia. Details are as follows:
University name: Griffith University
Short name or abbreviation: GriffUni
Link to Blackboard portal: http://bblearn.griffith.edu.au/
Landing URL after successfully logging in: https://bblearn.griffith.edu.au/webapps/portal/execute/tabs/tabAction?tab_tab_group_id=_1_1
And is there an ETA on the next release? :)
Kind regards,
Kristina
Learning and Teaching Solutions
Griffith University
Probably implemented through windows installer. However, on macos this will require additional work.
University name: University of Manchester
Short name or abbreviation: UoM
Link to Blackboard portal: https://online.manchester.ac.uk/
Landing URL after successfully logging in: https://online.manchester.ac.uk/ultra/
University name: University of Otago
Short name: UOO
Link to Blackboard portal: https://blackboard.otago.ac.nz/
Landing URL: https://blackboard.otago.ac.nz/ultra/institution-page
Schoolcraft College
No response
https://bb.schoolcraft.edu/ultra/institution-page
_2_1
No response
No response
Some courses may be structured in a way where some or most of the contents are only listed in one of the file attachments which actually exist within webdav. To download these, extra effort must be made to check for this webdav redirection and parse such files for links. Note that this is different from the current parsing for webdav links, as that happens within the body property of the API response, and this proposal relates to parsing the actual content of files downloaded, which means file content must be kept in memory temporarily to parse it out.
This scenario is quite rare and it isn't worth the trouble in my opinion, since the vast majority of the courses I have seen are not of this nature. However, if you would like to offer your comments on this proposal, please let me know here. If enough support for this proposal is gathered, I will consider implementing the feature.
Following the release of pyinstaller 6.* ( #90) which changed library linking, the github actions workflow should be adapted to no longer perform this step and instead letting pyinstaller do it.
University of Antwerp
UA
lms.uantwerpen.be
lms.uantwerpen.be/ultra
No response
No response
The green folders are downloaded, the red one not
Some tests were skipped intentionally since the API has changed since they were written. Extra work is needed to adapt these tests for the current interface.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.