Comments (10)
jeff1evesque
commented on May 12, 2024
The above remarks are no longer valid, for the same reasons as follows:
Therefore, we need to create generic logic, that will parse the contents of a webpage, for the following cases:
- csv
- xml
- json
from machine-learning.
jeff1evesque
commented on May 12, 2024
When we submit a data new session:
with the following url references:
- https://raw.githubusercontent.com/jeff1evesque/machine-learning/master/interface/static/data/json/web_interface/svm-1.json
- https://raw.githubusercontent.com/jeff1evesque/machine-learning/master/interface/static/data/json/web_interface/svm-1.json
- https://raw.githubusercontent.com/jeff1evesque/machine-learning/master/interface/static/data/json/web_interface/svm.json
we get the following traceback error upon form submission:
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python app.py
[{'type': u'json', 'file': <Response [200]>, 'filename': u'svm.json'}, {'type': u'json', 'file': <Response [200]>, 'filename': u'svm.json'}]
[2016-09-10 19:41:53,177] ERROR in app: Exception on /load-data/ [POST]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/vagrant/interface/views.py", line 122, in load_data
response = loader.load_data_new()
File "/vagrant/brain/load_data.py", line 70, in load_data_new
session.dataset_to_dict(session_id)
File "/vagrant/brain/session/base_data.py", line 173, in dataset_to_dict
response = dataset_dictionary(id_entity, self.model_type, self.upload)
File "/vagrant/brain/session/data/dataset_to_dict.py", line 29, in dataset_dictionary
json_upload = upload['dataset']['json_string']
KeyError: 'json_string'from machine-learning.
jeff1evesque
commented on May 12, 2024
18e1326: when we submit the same form submission as above, we get the following traceback error:
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python app.py
'Response' object has no attribute 'seek'
[AttributeError("'Response' object has no attribute 'seek'",)]which means we need to define additional / conditional logic prior to the val['file'].seek(0), within dataset_to_dict.py.
from machine-learning.
jeff1evesque
commented on May 12, 2024
defe3c1: when we submit the same form submission as above, we get the following traceback error:
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python app.py
[2016-09-11 13:09:02,139] ERROR in app: Exception on /load-data/ [POST]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/vagrant/interface/views.py", line 122, in load_data
response = loader.load_data_new()
File "/vagrant/brain/load_data.py", line 75, in load_data_new
session.save_observation_label('data_new', session_id)
File "/vagrant/brain/session/base_data.py", line 155, in save_observation_label
self.premodel_data['data']['dataset']['file_upload']
TypeError: 'NoneType' object has no attribute '__getitem__'from machine-learning.
jeff1evesque
commented on May 12, 2024
When we implement a print statement within brain/session/base_data.py as follows:
...
def save_observation_label(self, session_type, session_id):
'''save_observation_label
This method saves the list of unique independent variable labels,
which can be expected in any given observation, into the sql
database. This list of labels, is predicated on a supplied session
id (entity id).
@self.observation_labels, list of features (independent variables),
defined after invoking the 'dataset_to_dict' method.
@session_id, the corresponding returned session id from invoking the
'save_entity' method.
'''
print self.premodel_data
# save observation labels
response = observation_label(
session_type,
session_id,
self.observation_labels[0],
self.premodel_data['data']['dataset']['file_upload']
)
...then, submit the same form submission as above, we get the following traceback error:
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python app.py
{'data': {'dataset': None, 'settings': {'model_type': u'svm', 'session_name': u'test', 'dataset_type': u'dataset_url', 'dataset[]': [{'type': u'json', 'file': <cStringIO.StringI object at 0x7fbb4e319360>, 'filename': u'svm.json'}, {'type': u'json', 'file': <cStringIO.StringI object at 0x7fbb4e3192d8>, 'filename': u'svm.json'}], 'session_type': u'data_new'}}, 'error': None}
[2016-09-11 16:40:57,728] ERROR in app: Exception on /load-data/ [POST]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/vagrant/interface/views.py", line 122, in load_data
response = loader.load_data_new()
File "/vagrant/brain/load_data.py", line 75, in load_data_new
session.save_observation_label('data_new', session_id)
File "/vagrant/brain/session/base_data.py", line 156, in save_observation_label
self.premodel_data['data']['dataset']['file_upload']
TypeError: 'NoneType' object has no attribute '__getitem__'However, if we submit a trivial svm data new session:
we get the following (no error) traceback statement associated with the above print statement:
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python app.py
{'data': {'dataset': {'file_upload': [{'type': u'json', 'file': <FileStorage: u'svm-1.json' ('application/json')>, 'filename': u'svm-1.json'}], 'json_string': None, 'upload_quantity': 1}, 'settings': {'model_type': u'svm', 'session_name': u'test', 'dataset_type': u'file_upload', 'session_type': u'data_new'}}, 'error': None}From above, we immediately notice the difference between the two session instances, namely the existence of the 'file_upload' key during the latter session. Specifically, the missing file_upload key, may contribute to the former traceback error:
...
File "/vagrant/brain/session/base_data.py", line 156, in save_observation_label
self.premodel_data['data']['dataset']['file_upload']
TypeError: 'NoneType' object has no attribute '__getitem__'from machine-learning.
jeff1evesque
commented on May 12, 2024
The web-interface can now successfully execute the above form submission. Specifically, successive model generate, and model predict will succeed using the latter data new session. However, since the above form submission only include links to json dataset, we need to check both csv, and xml datasets. Then, we need to implement the corresponding solution for the programmatic interface.
from machine-learning.
jeff1evesque
commented on May 12, 2024
7d31a75: all cases, with the exception of the programmatic-interface has been successfully tested.
from machine-learning.
jeff1evesque
commented on May 12, 2024
b3da19c, 5c1ad52: resolves the following import file mismatch error traceback:
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ (cd test && pytest manual)
========================================= test session starts =========================================
platform linux2 -- Python 2.7.6, pytest-3.0.2, py-1.4.31, pluggy-0.3.1
rootdir: /vagrant/test/manual, inifile: pytest.ini
plugins: flask-0.10.0
collected 4 items / 1 errors
=============================================== ERRORS ================================================
______________ ERROR collecting programmatic_interface/file_upload/pytest_svm_session.py ______________
import file mismatch:
imported module 'pytest_svm_session' has this __file__ attribute:
/vagrant/test/manual/programmatic_interface/dataset_url/pytest_svm_session.py
which is not the same as the test file we want to collect:
/vagrant/test/manual/programmatic_interface/file_upload/pytest_svm_session.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================= 1 error in 0.30 seconds =======================================from machine-learning.
jeff1evesque
commented on May 12, 2024
Since our manual unit tests required us to change the session_id, and model_id values within various sample programmatic json files, we need to make adjustments to the live_server unit tests. Specifically, the adjustments means that we need to perform #2711 in this issue. The reason for this, is because both the manual, and live_server unit tests, implement the same programmatic json files. So, placing a fixed assumption(s) on the json files for the manual unit tests, would mean the same assumption(s) for the live_server unit tests.
from machine-learning.
jeff1evesque
commented on May 12, 2024
We've truncated our mariadb tables, then FLUSHALL our redis structures, and performed our manual unit tests (without a vagrant destroy, and vagrant build):
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ mysql -u root -p db_machine_learning
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 833
Server version: 5.5.49-MariaDB-1ubuntu0.14.04.1 (Ubuntu)
Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [db_machine_learning]> show tables;
+-------------------------------+
| Tables_in_db_machine_learning |
+-------------------------------+
| tbl_dataset_entity |
| tbl_feature_count |
| tbl_feature_value |
| tbl_model_type |
| tbl_observation_label |
+-------------------------------+
5 rows in set (0.00 sec)
MariaDB [db_machine_learning]> truncate tbl_dataset_entity;
Query OK, 0 rows affected (0.00 sec)
MariaDB [db_machine_learning]> truncate tbl_feature_count;
Query OK, 0 rows affected (0.01 sec)
MariaDB [db_machine_learning]> truncate tbl_feature_value;
Query OK, 0 rows affected (0.00 sec)
MariaDB [db_machine_learning]> truncate tbl_observation_label;
Query OK, 0 rows affected (0.01 sec)
MariaDB [db_machine_learning]> exit;
Bye
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ redis-cli
127.0.0.1:6379> flushall
OK
127.0.0.1:6379> exit
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ (cd test && pytest manual)
========================================= test session starts =========================================
platform linux2 -- Python 2.7.6, pytest-3.0.2, py-1.4.31, pluggy-0.3.1
rootdir: /vagrant/test/manual, inifile: pytest.ini
plugins: flask-0.10.0
collected 16 items
manual/programmatic_interface/dataset_url/pytest_svm_session.py ....
manual/programmatic_interface/dataset_url/pytest_svr_session.py ..FF
manual/programmatic_interface/file_upload/pytest_svm_session.py ....
manual/programmatic_interface/file_upload/pytest_svr_session.py ....
============================================== FAILURES ===============================================
_________________________________________ test_model_generate _________________________________________
def test_model_generate():
'''@test_model_generate
This method tests the 'model_generate' session.
'''
> assert requests.post(
endpoint_url,
headers=headers,
data=get_sample_json('svr-model-generate.json', 'svr')
)
E assert <Response [500]>
E + where <Response [500]> = <function post at 0x7fdef9b6b1b8>('http://localhost:5000/load-data/', headers={'Content-Type': 'application/json'}, data='{"properties": {"model_type": "svr", "sv_kernel_type": "rbf", "session_id": "2", "session_type": "model_generate"}}')
E + where <function post at 0x7fdef9b6b1b8> = requests.post
E + and '{"properties": {"model_type": "svr", "sv_kernel_type": "rbf", "session_id": "2", "session_type": "model_generate"}}' = get_sample_json('svr-model-generate.json', 'svr')
manual/programmatic_interface/dataset_url/pytest_svr_session.py:99: AssertionError
_________________________________________ test_model_predict __________________________________________
def test_model_predict():
'''@test_model_predict
This method tests the 'model_predict' session.
'''
> assert requests.post(
endpoint_url,
headers=headers,
data=get_sample_json('svr-model-predict.json', 'svr')
)
E assert <Response [500]>
E + where <Response [500]> = <function post at 0x7fdef9b6b1b8>('http://localhost:5000/load-data/', headers={'Content-Type': 'application/json'}, data='{"properties": {"model_id": "2", "prediction_input[]": ["22.22", "96.24", "338", "72.55", "0.001", "28", "0.678"], "session_type": "model_predict"}}')
E + where <function post at 0x7fdef9b6b1b8> = requests.post
E + and '{"properties": {"model_id": "2", "prediction_input[]": ["22.22", "96.24", "338", "72.55", "0.001", "28", "0.678"], "session_type": "model_predict"}}' = get_sample_json('svr-model-predict.json', 'svr')
manual/programmatic_interface/dataset_url/pytest_svr_session.py:113: AssertionError
================================ 2 failed, 14 passed in 52.94 seconds =================================from machine-learning.
Related Issues (20)
- Update 'base.dockerfile' with updated git
- Ensure 'install_rancher' runs on trusty64 HOT 1
- Implement bash error testing in 'install_rancher'
- Fix links in 'README.md'
- Change nginx ports + adjust documentation HOT 2
- Rancher AWS instance unhealthy HOT 3
- Fix css alignment on various pages
- Add sample association rules mining datasets
- Create pytest unit tests for associative rule mining
- Create documentation for association rule mining
- Determine flask routes for association rule mining
- Create component to present support, confidence, lift
- Create component to list association rules
- Create association rules input component HOT 1
- Replace all 'client.post' doc with 'requests.post'
- External 'requests.post' not working HOT 8
- Fix jest errors
- Update puppet5 + python3 HOT 16
- Segregate rancher into another repository
- Fix login menu item
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from machine-learning.