baratin-tools / baratinage Goto Github PK

1/ remplacer les ≤ par des < pour éviter les pbs d'encodage.

2/ indiquer les hmin/hmax de la grille de calcul de la CT comme valeurs extrêmes d'application du modèle

This option would be useful for old-school North-American / EDF hydrographers, and also for visualizing rating shifts.

Last time I checked (years ago!), this was not possible:
"At the moment, the localisation files are loaded statically according to the default locale. There's no option to configure on-the-fly the locale/language for a given chart or chart-panel. It would be a nice feature to have."

We should reconsider the use of tabs for Hydraulic configs, Gaugings, RCs etc.

• They are not necessary since using the explorer on the left serves the same purpose
• I'm not sure they are useful: I don't really see a situation where tabs are more convenient than the explorer (but I may be wrong)
• They may be confusing: it's not clear when you click on e.g. a RC tab which of the existing RC's is going to be shown.
• The implementation of how panels and plots are redrawn when clicking a tab might be buggy

So getting rid of tabs might simplify code and ergonomics with no loss of functionality as far as I can see. However maybe some users are used to tabs and are in fact using them regularly? Maybe we should ask them? (poll on baratin.users, for this and possibly other changes we're not sure of?)

Hi

I don't understand why I get this strange oscillation in this stage series. Do you reproduce the bug? Do you see the problem?

Aisne_Berry_limni_crue_2021.csv

If this folder is really useful, then we should explain why and how. But is it ?

export of flow series: year, month, day, etc. should be formatted as integers in the output file, not reals (i.e. "2017.0")

Original request was the ability to switch h and Q axis (in many country the figure is h=f(Q) rather than Q=f(h)).

But more generally, since with BaM we may have several inputs (e.g. Q=f(h,dh/dt)), the way figures are built should be completely rethought. We should probably start with a default choice but allow the user to choose axes and possibly other attributes (e.g. color and size of lines/points, multiple curves in the same panel, etc.)

This sounds like a potentially endless endeavor so we need to be realistic about what's reasonably feasible within BaRatinAGE - we don't want to re-implement ggplot...

Current behavior and problem:

The button "show legend" open an image file. This means that one image per language must be created (wasted spaces, time consuming, difficult to track). In addition, the images are stored in the img folder of the documentations which doesn't make much sense, I think.

Expected behavior

I expect:

• the legend should be created dynamically within BaRatinAGE and use translated text.
• when clicked, the button "show legend" should:
  • option 1: the legend to be added next to the plot (the "show legend" could become a "toggle legend" button)
  • option 2: a popup within BaRatinAGE containing the legend (e.g. similar to what happens when you click "external figure")...

Since the translations used within BaRatinAGE may change, could be useful to offer the possibility to the user to update the translation directly from BaRatinAGE itself.

A possible solution could be to add a button "Update translations" in the menu "Options".

This feature could be implemented using the fact that the address of the translation file on github doesn't change (https://raw.githubusercontent.com/BaRatin-tools/BaRatinAGE/main/lang/dico.txt) and can be used to automate downloading and replacing the dico.txt file within the /lang/ folder of BaRatinAGE.

I noticed that duplicating a hydraulic configuration is not very useful when you change the number of controls afterwards (which is a common thing to do I think) since all the parameter values are reset...

I would expect the following behavior:

• increasing the number of controls should not reset the existing controls
• decreasing the number of controls should trigger a warning (as it does today) and delete the controls that need to be deleted without reset the ones that don't need to be deleted.

@benRenard and @JeromeLeCoz, what do you think?

Enhancement to consider for v3?

There are probably some links with ticket #17 about out-of-date objects.

Examples of issues / suggestions:

Il y a quelque chose de pas intuitif dans la succession du bouton appliquer central, avec le bouton exécuter de la partie droite.
Je ne sais pas comment le dire, mais il n'y a rien d'évident visuellement dans le fait qu'appliquer ne suffit pas et qu'il faut aller lancer le calcul avec Exécuter.

Ce que j'imagine de plus élégant est simplement un petit encart qui apparaît quand tu fais "appliquer" et qui t’avertit que les paramètres ont changé et que l'affichage actuel n'est pas représentatif des changements réalisés et qu'il ne le sera qu'après avoir fait "exécuter", un truc comme ça.

Afficher un message d'erreur genre "some value(s) missing" quand Apply ne fonctionne pas parce que toutes les cases ne sont pas renseignées. Ca aiderait à voir que le paramètre c, caché car trop bas, n'a pas été renseigné...
Indiquer par un code (couleur?) que les valeurs ont été "applied" ou pas.
Faire "entrée" en fin de saisie d'un paramètre devrait faire "apply".

La difficulté est qu'il faudrait distinguer les infos qui une fois modifiées rendent le graphique caduque et réclament donc un "Exécuter" (par ex. pour le graphique de la CT: les a priori ou les jaugeages) et les infos qui ne réclament pas une mise à jour du graphique après modification (par ex. toutes les descriptions). Pas trivial à implémenter proprement, d'autant que ces infos sont dispatchées dans plusieurs objets et plusieurs onglets.

La solution la plus élégante que j'imagine consisterait en 2 petites loupiotes:

1. la loupiote 1 serait à côté du bouton "appliquer", rouge s'il y a des changements non sauvegardés dans l'onglet courant, vert sinon.
2. la loupiote 2 serait dans la partie graphique, rouge si le graphique n'est pas à jour par rapport aux infos actuellement sauvegardées dans BaRatinAGE, vert sinon. Encore une fois c'est pas trivial car par exemple un changement d'apriori va rendre caduque non seulement le graphique de la CT a priori, mais aussi celui de toutes les CT a posteriori qui utilisent cet a priori, ainsi que tous les hydrogrammes qui utilisent ces CTs a posteriori!

Dans l'Assistant Priors, ajouter un schéma géométrique type guide Onema pour chaque contrôle-type, en plus de l'équation, pour visualiser plus directement la signification des paramètres

nitpicks: at least on the flow series plot, I have the impression that the return to the initial view is done on the MaxPost curve but truncates the highest uncertainty envelopes (whereas when you open the graph for the first time, or re-open it, the framing seems wider and ok).

voici la version xls à jour

Version coréenne à faire apparaître (pb d'encodage?)

dico_BaRatin_V8.xlsx

salut

un détail un peu gênant sur l'affichage des "k" sur les plots de "RC", c'est que quand deux enveloppes se recouvrent, l'effet de transparence donne le même vert (violet) que celui de la valeur MaxPost (MaxPrior). Ca complique un peu l'interprétation, exemple :

Serait-il possible de rendre les enveloppes encore plus transparentes pour éviter cet effet, ou encore d'afficher les MaxPost/MaxPrior avec une autre couleur? (genre noir comme la CT MaxPost/MaxPrior? en pointillés peut-être pour ne pas détourner l'attention de la RC?)

Hello

I think I met a bug in modifying this bar.zip file (created with v2-1). I created a second hydraulic config, then computed the prior RC (OK), then failed to compute the posterior RC ("execution problem"). I can still compute the RC with the first config, but it corresponds to the second config! I fear something is mixed up in the files...

Ramon_Baratinage.bar.zip

original: Nom étude affichée en haut
ou ailleurs sur la fenêtre (ce serait pratique pour se repérer, avec le chemin du bar.zip aussi si possible)

J'ai noté plusieurs cas d'études créées avec la 2-1 qui posent pb à l'ouverture avec la 2-2, voici deux exemples ici : https://filesender.renater.fr/?s=download&token=ba9d9035-81c2-496e-a773-aec80668441e

Currently inactive gaugings are just removed.

Or alternatively, give the user the choice of what to do

On Linux, the coloring of the cells of the control matrix do not work as expected. In particular, the light green color does not appear where it should. No problem in Windows, so this may have something to do with Linux look-and-feel and the way it refreshes itself.

I tried for hours to fix this with no success... The main conclusion of this attempt is that BaRatinAGE code looks OK in the sense that the color assigned to each cell is the expected one. And then something happens at the level of Javaswing rendering, but I couldn't understand what...

For the linux version, the UI matches the OS theme. However, when the dark mode is the user's preference, the BaRatinAGE UI appears broken as illustrated in the screenshot: some of the UI elements have hard coded colors which only match a light theme.

It is probably better to avoid using hard coded colors in the UI except for specific UI elements such as charts.

BaRatin calcule un hydrogramme même quand la valeur du limni dépasse la gamme de définition de la courbe de tarage. C'est plutôt bien, mais il faudrait que ce soit spécifié sur le graphique de l'hydrogramme, genre un fond coloré qui précise qu'on est au-delà de la définition de la CT. Peut-être faudrait-il aussi le préciser lorsqu'on exporte l'hydrogramme, genre une colonne avec un code qui précise ce truc là.

Ou alors remplacer ces valeurs avec un code lacune

As of v2.2, BaRatinAGE still assumes Gaussian priors. This should be generalized in v3, since lognormal or uniform priors are in fact quite useful. Difficulties are:

• It makes the prior assistant more complex, but this can probably be handled by using Monte-Carlo propagation
• This added feature should not compromise simplicity or ergonomics

For instance: edit imported data within BaRatinAGE, sort/search tools, add new columns, etc. Keeping in mind, for the longer term, that in the context of BaM these data could be anything.

As for figures we need to be realistic, we don't want to reinvent excel!

Wouldn't it be safer to delete the results (RC prior, RC post, flow series) when changing one of the parameters or inputs used to calculate them? On the display, one can currently see results that do not correspond to the inputs/parameters (e.g. a RC post with a different set of gauges than the one used to calculate it).

I noticed that the Open dialog (and probably all the other file related dialog) are always in english regardless of the computer's default locale and the language picked by the user.

It seems to be a known issue with Java Swing File dialog. I think this is worth fixing (although I don't know yet what's the best approach). In any case, it will require additional entries in dico.txt:

• File name
• Look in
• View Menu
• Create New Folder
• Up One Level
• View
• Refresh
• New Folder
• List
• Details
• All Files

Au lancement des Runs Prior RC et Posterior RC, déclencher un warning (sans empêcher le calcul) si la condition de "largeur" des contrôles chenal rectangulaire et parabolique n'est pas respectée pour la plus haute hauteur H pour laquelle ils sont actifs, i.e. si :

H - b > 5 B (chenal rectangulaire)

H - b > 5 * 3/8 * Bp² / Hp (chenal parabolique)

Pas de condition de ce type pour chenal triangulaire et contrôles par section.

Voir ci-dessous pour des messages possibles en FR/EN

Attention, la condition de largeur du contrôle chenal n'est pas respectée!

Pour la plus haute hauteur d'eau H pour laquelle il est actif, on doit en effet avoir :

H - b > 5 B (pour un chenal rectangulaire)

H - b > 5 * 3/8 * Bp² / Hp (pour un chenal chenal parabolique)

Warning: the condition on the channel control width is not respected!

For the highest water level H for which it is active, the condition is:

H - b > 5 B (for a rectangular channel)

H - b > 5 * 3/8 * Bp² / Hp (for a parabolic channel)

c'est pas si simple...

On ne peut pas le faire a posteriori car on n'a pas accès à la largeur, qui est "noyée" dans le coefficient a.

On peut en théorie le faire a priori car on a bien accès à la largeur, mais malheureusement dans BaRatinAGE on n'a pas accès à l'offset "b" a priori (pas de la façon dont j'ai codé le truc en tout cas). Ce serait faisable mais il faudrait que je change aussi la façon dont c'est fait dans BaRatin (surement faisable dans BaM par contre puisque b est un paramètre derivé)

for instance: Q vs. h or h vs. Q in figures; Manning or Strickler; Imperial or SI units; etc.

Note that some of these issues may just disappear with the use of BaM (which is not aware of any unit), the implementation of user-defined flexible figures, etc.

Currently we have html help files, linking to word-created pdf files with equations/figures. In BaM, we started documenting each model with LateX-produced model sheets. In RBaM, vignettes are in R-markdown.

Should we change this and try to be more consistent?

I'm personally keen to get rid of all word-like documents, and instead move to either LateX or markdown.

In particular, could we do everything with markdown? An advantage is that we could render the markdown files as either html or pdf.

Also, should we go 100% online or should all these files be distributed along with the executable?

Une idée assez pertinente d'un stagiaire : ce serait chouette que pour une courbe de tarage, via un clic droit et via le menu, on puisse exporter une synthèse des informations dans un pdf, ou un format image, dans la perspective de l'insérer dans un rapport par exemple.
Config (matrice), paramètres a,k,c mais aussi les paramètres intermédiaires de K, B, C etc.

The idea would be to use one or several Q(h) curve from a hydraulic model to specify priors on k/a/c

Outline:

• subsample Q(h) to create pseudo-gaugings
• fit the desired RC -> gives a (k,a,c) parameter set
• If a single Q(h) curve is available (a pity!), use some order-of-magnitude relative uncertainty around k, a, c
• If several (ideally many) Q(h) curves are available, repeat and then use the many (k,a,c)'s to estimate the prior (e.g. fit some gaussian or lognormal distribution for each parameter and possibly some copula for their inter-dependence)

My feeling is that before deciding what to do, we should review what has been done so far and document the various ways of doing it.

I also feel that this could/should be done in some external app (a bit like the specification of Manning/Strickler discussed in #25). Especially since the use of BaM in v3 will make it easier to use RBaM within a Shiny app for instance

The "Manning" option was deactivated in v2.2 because it wasn't implemented properly

Implementing it properly is tricky: it would require rethinking the implementation of several key objects and changing the way studies are saved. Is it worth the candle?

Note that it's still possible to use the 'Manning prior assistant', but it's just an intermediate to compute a prior on Ks.

Currently the README.md provides a very general overview (OK but could be improved), but then the fact that there is also a README.txt is confusing: should we remove it and merge it into README.md? Or rename it e.g. VersionNotes.md? or use this file as a VersionNotes file?

In addition, renaming some compilation-related files (wind_dist.ps1 and DevMiscInfo.md) to make their purpose more explicit would be useful.

Val suggested two improvements to decrease computing times and storage requirements:

• Lossless compression of the stage time series: if 3 values are aligned, remove the middle one. Val tried it for a 25-year, 10-min stage series, and went down from 870 309 to 256 302 points - with no loss!
• Transformation into discharge by value rather than by time step. Because of rounding, the stage time series above containing 870 309 time steps only contains 4004 distinct stage values! It would therefore be much more efficient to transform only these 4004 values, and then put them back in the correct order to rebuild the time series. It's actually a bit more complicated than that because of stage errors, but it's certainly feasible to round the noisy stage values using the same resolution as the original stage data.

More generally, this raises the question to offer some pre-processing tools for input datasets, in relation with issue #26. Subsampling tools would be particularly valuable - but is it BaRatinAGE's job, or should rather be done by some external app?

otherwise ensuring reverse-compatibility will be impossible

Probably online, with a html page explaining the case study and offering to download the associated bar.zip file.

Markdown is probably the way to go for this, to be discussed in relation with #34

A very nice inspiration: https://mc-stan.org/users/documentation/case-studies

Il arrive que les utilisateurs ne réalisent pas que la courbe de tarage est calculée sur des pas de hauteur trop lâches. Une solution pourrait être de proposer une grille adaptative (à pas log? à pas ajustés sur la variation relative de Q Maxpost? à pas incluant les hauteurs des jaugeages?)

The main novelties of v2.2.1 are summarized below, and I was thinking using this text as version notes accompanying the release. Please have a look and correct, complete and reorder as you see fit. I'll do a French translation as well (unless you wanna go for it, in which case please help yourself!)
Also question: do we publish the list of persons who helped with translations, and if so where? I was thinking maybe as a big "thank you" at the end or the beginning of these version notes?

• BaRatinAGE and BaRatin become open-source (GPL3 license) and are available at https://github.com/BaRatin-tools
• BaRatinAGE is now available on Linux
• BaRatinAGE is now "stand-alone": it should work out-of-the-box on any (at least most!) Windows/Linux computers as it does not depend on a locally-installed version of Java anymore
• Rating curve can now be exported as an equation
• MCMC simulations can now be exported
• Unfeasible values are now removed before computing uncertainty envelops, unless they represent more than 75% of values
• Offset parameters "b" are now displayed in figures and tables (in addition to parameters k-a-c)
• A new prior vs. posterior consistency check is available (see More Plots...Remarks in the Rating Curve panel)
• Comma-separated .csv files are now supported for gaugings and stage time series (in addition to semicolon-separated ones)
• "SaveSpaghetti" option was not working and has been fixed
• New languages can now be added "on-the-fly", by just completing lang/dico.txt and providing help files in help/mylanguage (if not provided, fallback is English)
• This version ships with 10 languages, validated by native speakers (fr, en, es, de, it, br, ko, hb, cn, jp)
• New "purely proportional" remnant error model, with sigma = gamma*Q
• Color of prior activation stages has changed (to clarify that they are not the same as the posterior activation stages)
• Specifying non-increasing prior activation stages triggers an error message when attempting to compute the prior rating curve
• "useManning" preference (from menu Options...Preferences) was not handled properly and has been deactivated until a proper fix is implemented
• A warning message is issued before computing hydrographs if the (roughly) estimated disk space is above 10MB
• A warning message is issued before changing the number of controls
• Added a template .xls file to create a stage time series .csv file
• Various minor ergonomic improvements
• Updating of help files to reflect all these changes

Integrate the information contained in Strickler / Manning tables as an interactive frame within BaRatinAGE, rather than relying on external excel files?

Or alternatively and possibly better, how about developing a small online Shiny-like tool to derive Strickler / Manning with uncertainties from these tables, augmented with pictures of real-life sites, a catalogue of typical examples etc.?

Original demand below---------------------------------------------
Lien vers table de Chow et site web USGS proposant des valeurs de Strickler/Manning's n pour différents faciès de lit (cf. xls aide au calcul)
La table de Chow est visible ici en version originale :
http://www.fsl.orst.edu/geowater/FX3/help/8_Hydraulic_Reference/Mannings_n_Tables.htm
peut-être mieux de l'incorporer directement dans l'interface, avec une bonne traduction FR

and more generally, of any kind of export requiring some user input.

Original request:

Actuellement, lorsqu'on exporte une courbe au format bareme, on doit renseigner le code Hydro, le nom de la courbe, et les dates de validité.
Le problème, c'est que si on souhaite refaire cet export, il faut tout re-rentrer, rien n'est gardé en mémoire.
Ce serait bien qu'au moins le code Hydro et les dates de validité, une fois rentrés une fois, soient conservés.

Some help files probably need some updating to reflect changes made in v2-2. In particular, 'overview', 'going further' and 'FAQ' are partially out-of-date due to new licence / Linux version / MCMC export. Need to check other pages too.

L'iconique pipeau est irremplaçable mais quand on crée un raccourci Windows, la version en miroir ci-joint rend mieux : cf. capture écran ci-joint.

Je suggère de remplacer le fichier .ico avec le miroir.

BaRatinAGE_icon.zip

This minor change suggestion is mentioned in issue #10 .

Hi, something looks strange to me in the display of the k3 results in this example:
Chiers_Chauvency.bar.zip

For instance, for the "S+2C_sans juillet 2021" rating curve, k3 = 1.4 +/-0.4 in the prior/post table and in the RC plot, but in the prior/post plot, the results look distributed like k3 = 1.8 +/-0.4...

For the "S+2C_juillet 2021" rating curve, the shift of the cenral value look even more obvious, as the k3 results follow the prior distribution (k3 = 1.4 +/-0.4) whereas the display is k3 = 1.2 +/-0.4...

Perhaps, this is due to discarded realizations (when k3<k2)? But then the statistics should not include the discarded values. I'm puzzled...

It would be very convenient that stage series files from Hydroportail can be imported directly in BaRatinAGE. Example for Verrières:
https://hydro.eaufrance.fr/stationhydro/H602102001/series

I'm doing the conversion with this Excel spreadsheet manually:
limni_crue_2021_1h.xlsx

It does not look difficult to implement this specific format, which would be useful as BaRatinAGE is increasingly used in the French hydrological services.