webarchivcz / wa-kat Goto Github PK
View Code? Open in Web Editor NEWCatalogization tool for the czech webarchive.
License: MIT License
Catalogization tool for the czech webarchive.
License: MIT License
To let users know, that it was webarchive crawler.
Implement language tag normalization?
en -> english
eng -> english
And so on. Which format does the output data require?
REST for translating URL to ISSN.
Submodule to allow aleph keywords.
Look at carbon, search webarchive, metadata, maybe whois?
návrh z meta (author), DC, dtb Nár. autorit || návrh z meta (author, copyright), dtb Nár. autorit - bude staženo z adminu // kontrola oproti bázi autorit
Research how to fix the problem with requiring default values during documentation buiding.
Possible solutions:
Maybe use paralelization.
Use https://github.com/Anorov/cloudflare-scrape for cloudflare protected sites?
Research how to merge together bottle, form library and asynchronous ajax calls.
Depends on Bystroushaak/bottle-rest#6
From the <title>
tag, meta & DC. May be sent from admin.
Whois, metadata.
Make all analyzers available as REST services.
Add full unittest coverage.
Look for annotation metadata. Maybe summary from the google?
database conflict error Traceback (most recent call last): File "bin/../src/wa_kat/rest_api/init.py", line 40, in get_result ri.paralel_processing() File "bin/../src/wa_kat/zeo/request_info.py", line 109, in paralel_processing self.processing_started_ts = time.time() File "/usr/local/lib/python2.7/dist-packages/transaction/_manager.py", line 120, in exit self.commit() File "/usr/local/lib/python2.7/dist-packages/transaction/_manager.py", line 111, in commit return self.get().commit() File "/usr/local/lib/python2.7/dist-packages/transaction/_transaction.py", line 280, in commit reraise(t, v, tb) File "/usr/local/lib/python2.7/dist-packages/transaction/_transaction.py", line 271, in commit self._commitResources() File "/usr/local/lib/python2.7/dist-packages/transaction/_transaction.py", line 417, in _commitResources reraise(t, v, tb) File "/usr/local/lib/python2.7/dist-packages/transaction/_transaction.py", line 394, in _commitResources rm.tpc_vote(self) File "/usr/local/lib/python2.7/dist-packages/ZODB/Connection.py", line 786, in tpc_vote s = vote(transaction) File "/usr/local/lib/python2.7/dist-packages/ZEO/ClientStorage.py", line 1107, in tpc_vote return self._check_serials() File "/usr/local/lib/python2.7/dist-packages/ZEO/ClientStorage.py", line 937, in _check_serials raise s ConflictError: database conflict error (oid 0x0192, class wa_kat.zeo.request_info.RequestInfo, serial this txn started with 0x03b3a6fe24266baa 2015-12-08 17:02:08.472675, serial currently committed 0x03b3a6fe24d0cd99 2015-12-08 17:02:08.628665)
Add validation of data in the forms.
Related to #32
Add docstrings for all of the
.
├── analyzers
│ ├── annotation_detector.py
│ ├── author_detector.py
│ ├── creation_date_detector.py
│ ├── __init__.py
│ ├── keyword_detector.py
│ ├── language_detector.py
│ ├── place_detector.py
│ ├── shared.py
│ ├── source_string.py
│ └── title_detector.py
├── connectors
│ ├── aleph.py
│ ├── __init__.py
│ └── seeder.py
├── convertors
│ ├── __init__.py
│ ├── iso_codes.py
│ ├── mrc.py
│ └── to_dc.py
├── rest_api
│ ├── aleph_api.py
│ ├── __init__.py
│ ├── keywords.py
│ ├── shared.py
│ ├── to_output.py
│ └── virtual_fs.py
├── templates
│ └── static
│ ├── css
│ ├── fonts
│ └── js
│ └── Lib
│ └── site-packages
│ ├── components
│ │ ├── author_picker.py
│ │ ├── author_switcher.py
│ │ ├── conspect_handler.py
│ │ ├── dropdown_handler.py
│ │ ├── errorbox.py
│ │ ├── __init__.py
│ │ ├── input_controller.py
│ │ ├── keyword_handler.py
│ │ ├── log_view2.py
│ │ ├── output_picker.py
│ │ ├── overlay_controller.py
│ │ ├── placeholder_handler.py
│ │ ├── progressbar.py
│ │ └── shared.py
│ ├── descriptors.py
│ ├── rules_view.py
│ ├── view.py
│ └── wa_kat_main.py
├── zeo
│ ├── __init__.py
│ ├── request_database.py
│ ├── request_info.py
│ └── worker.py
├── bottle_index.py
├── data_model.py
├── __init__.py
└── settings.py
Use settings.
Handle case when user is supposed to come from seeder, but seeder API is not available.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.