bahnzumberg / zuugle-api Goto Github PK
View Code? Open in Web Editor NEWPublic repositority for backend components of www.zuugle.at
Home Page: https://www.zuugle.at
License: GNU General Public License v3.0
Public repositority for backend components of www.zuugle.at
Home Page: https://www.zuugle.at
License: GNU General Public License v3.0
Tracking issue for:
Tracking issue for:
Remove table city_favourites from db (database.sql) and code (cities.js)
Abdul just showed me with this link, taht Autosuggestion is not working from the API side (or is wrongly calöled): https://www2.zuugle.at/api/searchPhrases?search=aben&city=amstetten&language=en
Enhance the daily load by adding a daily import of the data in the table "interface_duplicate_tours_to_search". Import a full load of this table. It contains the tours, which are so similar to another, that we can look at them as duplicates.
Prerequisite: https://github.com/orgs/bahnzumberg/projects/4/views/1?pane=issue&itemId=31487720
Import column "allow_gpx_download" from vw_provider_to_search to enable https://github.com/orgs/bahnzumberg/projects/2?pane=issue&itemId=34295492
What is the PostgreSQL table city_favourites used for?
move the file up one level
Tracking issue for:
uncomment code in authenticate.js and solve any open issues
The column month_order is always 1 in table tour.
SELECT
DISTINCT month_order
FROM tour
Change the load so, that the GPX coordinates are stored in a table in the PostgreSQL database. From there the GPX files should be created.
in the API repo: Translate the templates/tour-details.html into the other languages (sl, en, fr, it)
On my local machine, if i activate authentication in file authentiate.js i get an axios error (comes from suchseite of course) and white page. (potential fix : axios configuarion ?) .
Main point in this issue:
my local environment works normally without this code , but this file should be checked for the UAT installation, if it is necessary there and if so make sure it is activated in that environment.
The function mergeToursWithFahrplan() is doing nothing other than updating the columns country_at, country_de, country_ch, country_it, country_fr and country_si. Is my analysis correct?
If yes, then we should deactivate it completely and remove the columns from the table as well.
What do the columns country_at, country_de, country_de, ... do? They are calculated in the loading process in sync.js. They have been used in the search. Are they used in the frontend?
If not used, please remove columns & code.
If necessary, please document the function in the sync.js file.
Tracking issue for:
The call in the client side does not return the expected response (returns empty), while on the BE side , it is producing a whole list of all available tours in the local DB.
In sync.js there is the function syncGPXdata. The purpose of the function is to fetch datachunks from the remote "vw_gpx_to_search" and store these into the local table "gpx".
The amount of data is huge. A delta load mechanism is worth thinking about. As a first step the while loop should run in a limited way in parallel. 5 to 10 instances can read and write in parallel without a problem.
export async function syncGPXdata(){
try {
await knex.raw(`TRUNCATE gpx;`);
} catch(err){
console.log('error: ', err)
}
let limit = 5000;
const query_count = await knexTourenDb('vw_gpx_to_search').count('* as anzahl');
let count_gpx = query_count[0]["anzahl"];
let count_chunks = round(count_gpx / limit, 0);
let counter = 0;
console.log('GPX data count_chunks:', count_chunks)
console.log('Info: Handling ', count_gpx.toLocaleString("de-de"), ' rows with gpx datapoints.');
/* The following loop has to be parallised */
while(counter < count_chunks){
const result_query = knexTourenDb('vw_gpx_to_search').select('provider', 'hashed_url', 'typ', 'waypoint', 'lat', 'lon', 'ele').whereRaw(`ROUND(lat*lon*10000) % ${count_chunks} = ${counter}`);
const result = await result_query;
try {
await knex('gpx').insert([...result]);
} catch(err){
console.log('error: ', err)
}
counter++;
}
}
Remove the column from api , review related notes in issue #256
Depending on https://github.com/orgs/bahnzumberg/projects/2/views/1?pane=issue&itemId=31349270 the columns "internal_status" and "connection_url", "return_url" in table fahrplan should be removed.
In table tours the column "children" and "internal_status" should be removed. If "publishing_date" is not used, remove it too.
The table "Fahrplan" has been altered. The folowing columns have been added:
'connection_description_json',
'connection_lastregular_arrival_stop',
'connection_lastregular_arrival_stop_lon',
'connection_lastregular_arrival_stop_lat',
'connection_lastregular_arrival_datetime',
'return_description_json',
'return_firstregular_departure_stop',
'return_firstregular_departure_stop_lon',
'return_firstregular_departure_stop_lat',
'return_firstregular_departure_datetime'
The functionality of the two JSON columns are descibed here: https://github.com/bahnzumberg/hermes/wiki/Connection-and-Return-Description
The JSONs allow us to translate all elements like "Zug" or "Bus", etc.
The others are needed for the itinerary as well.
Please enhance the api towards the surface (suchseite), so we can implement the additional functionality.
Issue:
conditional rendering using Redux state works only for one of the conditions and not for the other.
Detailed description:
From our Postgres database we pull a value named totalTours through an axios call (see loadTotalTours()), this value we save in a state inside redux store (same name), this works fine but when we do access it (inside Start component) and attempt to conditionally render the Header -depending on the value of totalTours- we get the expected behaviour for a value of >0 but not when the value == 0, for that we get an error (“Rendered fewer hooks than expected. This may be caused by an accidental early return statement.”).
Related Files: (frontend "zuugle_suchseite" repo)
Start/Start.js, Start/Header.js, reducers/tours.js, actions/tourActions.js, actions/types.js, actions/crudActions.js
Union of three SQLs:
Over this union, there has to be a query, which is doing the ORDER BY and the LIMIt & OFFSET
On https://www.zuugle.at/suche?city=wien the hike "Kennenlernen am Wasserleitungsweg" (https://www.bahn-zum-berg.at/wienerwald/kennenlernen-am-wasserleitungsweg/) is shown (at least) two times - see screenshot:
I was unable to run npm run import-data-full under 20.3.0 or 19.8. The only solution which worked was to downgrade p-limit to 3.1, which doesn't require ESM.
The UAT website https://www2.zuugle.si/suche?sort=relevanz is not showing any results.
We have a problem with loading the tours, but there are two tours in UAT database. Still they are not shown.
Error handling, when no Tour is found for given ID: zuugle-api cannot handle calls with an unknown ID and crashes
Tracking issue for:
Bitte neue API "minimal" erstellen. Dieser muss man die folgenden Werte übermitteln, um eine Antwort zu erhalten:
Das Resultat ist eine einzelne Zeile, mit den notwendigen Informationen.
Siehe dazu die neu angelegte View: SELECT * FROM zuugle_api.minimal;
Tracking issue for:
Prevent SQL injection at the search. Maybe this helps: https://www.stackhawk.com/blog/react-command-injection-examples-and-prevention/
In Tours.js in line 794 "const query_con = knex('fahrplan').select().where({hashed_url: tour.hashed_url, provider: tour.provider, city_slug: city});" the select should be restricted to the columns, which are really used.
Tracking issue for:
The command "npm run import-files" in the api directory is throwing errors: "Protocol error (Page.navigate): Cannot navigate to invalid URL"
Translation feature for PDF - analog to the suchseite feature.
Once the translation is working, the PDF button should be active for all languages (undo https://github.com/orgs/bahnzumberg/projects/2/views/1?pane=issue&itemId=30353558).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.