Comments (33)
Until we get a toolchain for the rM2 and can actually start building applications for it, I think we should hold off on any decisions relating to it.
I think we likely should setup some sort of automation around promoting things to stable to help with when we aren't as active.
from toltec.
But how would you go about it then?
* Create a new up-to-date branch / nuke a branch to the up-to-date state * Just keep not updating until you get merge conflicts
What would you choose as an alternative or have I overlooked any? I'm personally working with git for a long time now, but I'm still a student and this is one of my first times, I have major code merging going on that highlights this problems for me.
I'd just merge the remote into my current branch and deal with the conflicts in the single commit. That way you don't have to merge every single commit.
The other more tedious option is to create a new branch and cherry-pick your commits over one at a time. That said, I'd only ever do that if a rebase showed a conflict on enough commits that it would take less time than just doing the rebase.
from toltec.
But how would you go about it then?
it's unfortunate, but github doesn't support stacked diffs properly (which people use in industry). people have generated their own workflows with github that emulate stacked diff (albeit somewhat tedious but better than no stacked diffs).
from my perspective: i do not want to see people's merge commits, nor do i want to see the history of feature branches, so i always squash and merge. if you want to be in sync, you can squash and merge locally, doing force pushes to your PR branch (so the PR only ever has one commit in it). i haven't caught up on the thread, but i'd like to propose that PRs always get squash merged and people don't use git merge on non-feature branches (stable/testing) if it isn't already part of the process.
around the overall process: looking good, i still need to catch up and write comments though
from toltec.
Though we should account for people not wanting to maintain certain packages anymore. People would then still wait on those reviewers and delay the process possibly a long time. Maybe set a time limit when a PR is then free for other reviewers to do.
I think if a maintainer is unresponsive about a PR, they will probably be unresponsive for subsequent ones, so we might as well allow maintainers to orphan their packages, i.e. allow packages to have no maintainer. In this case, any maintainer can review the PR, and this person will become the new maintainer of the package.
from toltec.
This is related to the merge window concept proposed by @Eeems above. As you say, it will be less work than creating separate PRs, so I think we’ll go with that unless there are clear advantages for creating separate PRs.
from toltec.
So do we also want an unstable repo where devs can manage their packages for testing deployments?
from toltec.
Can you clarify what would be the difference of aim between the testing branch and the unstable branch? To make things manageable, I’d say that it would be best to find a solution that minimizes the number of branches while still achieving the goal of actual stability for the stable branch.
from toltec.
From the Debian wiki:
Packages from Debian Unstable enter the next-stable testing distribution automatically, when a list of requirements is fulfilled:
- The package has been in "unstable" at least for 2-10 days (depending on the urgency of the upload).
- The package has been built for all the architectures which the present version in testing was built for.
- Installing the package into testing will not make the distribution more uninstallable.
- The package does not introduce new release critical bugs.
So basically unstable is being actively worked on by developers/maintainers and contains bleeding edge. Testing contains packages at versions that are deemed "stable" enough to move forward. After that they move from testing to stable.
If we don't want the overhead we could be like ArchLinux and just have testing and stable though.
from toltec.
I find this idea good, but I doubt, that we'll stay active all the time to maintain even more branches.
Although we also need to consider having some kind of separation between rM1 and rM2 software when they diverge. The simplest idea in my mind would be to have a check in a preinst file and abort the installation (if possible) when a user wants to install a software that makes not sense for his device (e.g. software that disables buttons on the rM2).
One check to find out whether it's a reMarkable 1 or 2 could be with:
if grep deviceid=RM100- /home/root/.config/remarkable/xochitl.conf >/dev/null; then
echo "This is a first gen"
else
echo "This is a second gen"
fi
My initial idea was to have a stable-rm1 and stable-rm2 branch, but it think the above idea is more maintainable.
What are your opinions on this?
from toltec.
I agree with @Eeems about holding off decisions on how to split packages between rM1 and rM2 until when we have a toolchain available & enough testers owning rM2.
What kind of automated criteria would you suggest @Eeems? It seems to me that fully automating the move from testing to stable kind of defeats the purpose of having separate branches. Ideally, we’d have to find enough trustworthy maintainers so that at least one is available at all times.
I propose a two-branches system (stable
and testing
) which would work as follows:
- All pull requests regarding new packages or package updates are based on
testing
. - A pull request can be merged into
testing
after review from a maintainer and if it builds successfully in the CI.- If the pull request is an update for an existing package, the preferred reviewer is the maintainer of that specific package.
- If it adds a new package, the pull request’s reviewer becomes the maintainer of that package.
- After merging a package update or a new package into
testing
, it can be moved tostable
if:- At least two full days have passed since the merge in
testing
. - At least one repository maintainer, different from the maintainer of that package, has successfully checked that the package works and does not break any other package.
- At least two full days have passed since the merge in
Here are the rationales:
- Having a designated maintainer for each package saves the time required for anyone to get acquainted with the specifics of each package, and ensures consistency between the different versions of each package.
- The time delay between the merge in
testing
and the move tostable
allows for the original submitter of the new package or package update to make sure they did not make a mistake in packaging (e.g. wrong version, commit, file permission, install path, …). - Requiring that the reviewer for merging a package into
testing
is different from the reviewer for moving it tostable
is intended to increase the level of scrutiny on the testing of the package. It is especially important that the package integrates well with other ones in the repository.
Under that system, the testing
branch offers no stability guarantee and is only intended for testing by maintainers, and the stable
branch is the only one that users need to use.
This is only a first draft, and any comment is welcome.
from toltec.
So how do we want to manage moving things from testing
to stable
in terms of the actual git workflow? We can't merge testing
itself into stable
since it might have other package updates that aren't ready yet.
This would influence what I would even recommend for automated moving.
from toltec.
I’m not very good at Git workflows. Would cherry-picking from testing
to stable
work? (Assuming that each commit only changes at most one package.)
from toltec.
So it would be a manual process where someone would have to create a PR against stable
with the cherry-picked commits?
from toltec.
Package maintainers would have write access to the repo. So they could simply push to stable
when the above conditions are met? Since moving a package from testing
to stable
does not require an external review, I’m not sure that PRs would be useful.
from toltec.
PRs help avoid mistakes. I'd prefer not to be directly pushing to stable
as an extra check to make sure I'm not doing something silly. That way the normal CI can alert you to errors before merging the branch so you can "stage" the push to stable as you sort through what you are adding in a "merge window" of sorts. You could have the migration to stable
started at the start of the week and start adding things to it as you identify they are stable. And when you are ready to merge all the CI checking has already been done.
from toltec.
These are good points. Having a merge window sure seems more efficient than moving packages one by one.
from toltec.
And for the merge window we could automate its creation, and potentially sort out a way to automate the cherry-picking. One thing to note on cherry-picking is that you'll want to merge stable
back into testing
every so often so that it doesn't start thinking there are merge conflicts.
from toltec.
Updated proposal:
- New packages or package updates are proposed through pull requests based on the
testing
branch.- A proposal can be merged into testing after review from a maintainer and if it builds successfully in the CI.
- If it is a proposal for a new package, the maintainer who reviews the pull request becomes the maintainer for that package.
- If it is a proposal for an update, the maintainer of that specific package should do the review.
- Package changes are moved to
stable
following this process:- Each Monday, a pull request is opened by a maintainer or a bot, which cherry-picks commits from
testing
tostable
. - This pull request can only contain package changes dating from the previous Friday or older.
- Each of these package changes must be tested by a maintainer different from the maintainer of the affected package.
- Each Monday, a pull request is opened by a maintainer or a bot, which cherry-picks commits from
- Things to check when testing a package:
- The package should work.
- It must not destroy user data from previous versions of the package.
- It must not break other packages.
from toltec.
One thing to note on cherry-picking is that you'll want to merge
stable
back intotesting
every so often so that it doesn't start thinking there are merge conflicts.
Why is that?
from toltec.
Because cherry-picked commits are not the same commit. So technically you've changed the same file. Usually git is pretty good at realizing there is no conflict, but every so often it gets confused.
from toltec.
Because cherry-picked commits are not the same commit. So technically you've changed the same file. Usually git is pretty good at realizing there is no conflict, but every so often it gets confused.
That's probably similar to how I had problems with adding multiple PRs. The rebase would find a lot of conflicts when I wanted to merge the upstream repo back into my fork. Where I had like 5 commits, the PR squashed it into one totally new one. My next PR would've then tried to also push my old commits that god squashed await in previous PRs. I usually then just created a new branch based on the current state of upstream/testing and did my changes there to avoid those problems.
@matteodelabre Your proposal looks really good. 👍
Though we should account for people not wanting to maintain certain packages anymore. People would then still wait on those reviewers and delay the process possibly a long time. Maybe set a time limit when a PR is then free for other reviewers to do.
from toltec.
Because cherry-picked commits are not the same commit. So technically you've changed the same file. Usually git is pretty good at realizing there is no conflict, but every so often it gets confused.
That's probably similar to how I had problems with adding multiple PRs. The rebase would find a lot of conflicts when I wanted to merge the upstream repo back into my fork. Where I had like 5 commits, the PR squashed it into one totally new one. My next PR would've then tried to also push my old commits that god squashed await in previous PRs. I usually then just created a new branch based on the current state of upstream/testing and did my changes there to avoid those problems.
One of the reasons I tend not to rebase :)
from toltec.
Because cherry-picked commits are not the same commit. So technically you've changed the same file. Usually git is pretty good at realizing there is no conflict, but every so often it gets confused.
That's probably similar to how I had problems with adding multiple PRs. The rebase would find a lot of conflicts when I wanted to merge the upstream repo back into my fork. Where I had like 5 commits, the PR squashed it into one totally new one. My next PR would've then tried to also push my old commits that god squashed await in previous PRs. I usually then just created a new branch based on the current state of upstream/testing and did my changes there to avoid those problems.
One of the reasons I tend not to rebase :)
But how would you go about it then?
- Create a new up-to-date branch / nuke a branch to the up-to-date state
- Just keep not updating until you get merge conflicts
What would you choose as an alternative or have I overlooked any? I'm personally working with git for a long time now, but I'm still a student and this is one of my first times, I have major code merging going on that highlights this problems for me.
from toltec.
looking at this, i like everything i see!
re: cherry picks: i don't particularly understand the rationale for using cherry picks here (and i see there's discussion about the problems it would cause). if we could, i think it's fine to just copy the packages/ directory over from testing -> stable and make a pull request for each subdir that changed. there's still a question of the remaining files in the repo (the infra and tooling) - i think those might need to be cherry-picked (but its also fine to just blanket copy them). the reason i think its fine is because i don't see a reason we would want to merge from stable -> testing except for resolving the potential merge conflicts.
misc questions:
- what if a developer of software opens a PR with a bugfix that they think is urgent, can they get that merged into stable more quickly?
- can a package be maintained by more than one person?
from toltec.
re: cherry picks: i don't particularly understand the rationale for using cherry picks here (and i see there's discussion about the problems it would cause). if we could, i think it's fine to just copy the packages/ directory over from testing -> stable and make a pull request for each subdir that changed. there's still a question of the remaining files in the repo (the infra and tooling) - i think those might need to be cherry-picked (but its also fine to just blanket copy them). the reason i think its fine is because i don't see a reason we would want to merge from stable -> testing except for resolving the potential merge conflicts.
Cherry-picks or a single commit where you just grab the apps you care about would basically be the same end result and have the same pitfalls.
misc questions:
- what if a developer of software opens a PR with a bugfix that they think is urgent, can they get that merged into stable more quickly?
I'd suggest we handle that on a case by case basis.
- can a package be maintained by more than one person?
You'd still want one person named as the primary maintainer.
from toltec.
from toltec.
i'm not suggesting picking commits, i'm suggesting just doing copy of files, then make a commit for each package dir. i don't think we will ever need to merge from stable -> testing, so this would be ok. but since we never need to merge stable -> testing, maybe cherry picks are fine too
…
Right, which has the same end result as a cherry-pick, and the same pitfalls. That said, if you are saying we are branching off of stable and copying the files in, that would negate the concern with merge conflicts.
from toltec.
from my perspective: i do not want to see people's merge commits, nor do i want to see the history of feature branches, so i always squash and merge. if you want to be in sync, you can squash and merge locally, doing force pushes to your PR branch (so the PR only ever has one commit in it). i haven't caught up on the thread, but i'd like to propose that PRs always get squash merged and people don't use git merge on non-feature branches (stable/testing) if it isn't already part of the process.
Seems sensible enough that PRs must be squash merged. Unfortunately, there doesn’t seem to be a GitHub option to enforce that. I added the following branch protection rules to testing
and stable
, which are better than nothing:
from toltec.
So, what emerges from the above discussion seems to be that the testing -> stable
PRs will contain one commit in which we copy the updated recipes that we want to move to stable, plus updated infra files if necessary. Does everyone agree on that?
from toltec.
Seems sensible enough that PRs must be squash merged. Unfortunately, there doesn’t seem to be a GitHub option to enforce that. I added the following branch protection rules to
testing
andstable
, which are better than nothing:
The only way to do that would be to enforce it repo wide. That said, it might not be a bad idea, and we just don't use PRs to do the testing -> stable move.
Edit:
So, what emerges from the above discussion seems to be that the testing -> stable PRs will contain one commit in which we copy the updated recipes that we want to move to stable, plus updated infra files if necessary. Does everyone agree on that?
Actually I guess enforcing squash merge everywhere would work with that.
from toltec.
Seems sensible enough that PRs must be squash merged. Unfortunately, there doesn’t seem to be a GitHub option to enforce that. I added the following branch protection rules to
testing
andstable
, which are better than nothing:The only way to do that would be to enforce it repo wide. That said, it might not be a bad idea, and we just don't use PRs to do the testing -> stable move.
Didn’t know about this option, thanks! I just changed it.
from toltec.
PRs will contain one commit in which we copy the updated recipes that we want to move to stable, plus updated infra files if necessary.
this sounds fine to me (and less work in total), but originally i imagined that we have one PR per package move (and the PR is just a blank copy of package/$PACKAGE/
from testing -> stable) and the owner can accept the PRs for their packages.
from toltec.
with separate pr, then each maintainer can choose what to migrate and when. with one pr it means coordinating across multiple people while they figure out if packages are safe to be migrated
i still think one is fine
from toltec.
Related Issues (20)
- Splashscreens don't replace properly
- Fingerterm icon incorrect
- rmfm crashes if there is a broken symlink HOT 1
- innernet package doesn't disable/stop services on uninstall
- Add hook to automatically systemctl daemon-reload
- Binary modification fills root partition until xochitl restart HOT 3
- Automated testing
- toltecctl generate-opkg-conf should have -y flag
- Add systemd generator for entware init scripts to toltec HOT 2
- rmfakecloudctl: set-upstream does not validate that URL is valid HOT 7
- rmfakecloudctl is missing from toltec-completion
- Bootstrap creates /opt when installing toltec-bootstrap HOT 1
- linux-stracciatella: add 'CONFIG_MODULE_UNLOAD=y' HOT 14
- Changing shell to anything but /bin/sh and /bin/bash breaks SSH HOT 1
- The 'strings' utilty is missing libbfd HOT 3
- 3.x support HOT 12
- Separate toltec and entware libraries
- Allow starting stock xochitl by accessing a different port
- how-to-install outputs multiple disable lines for xochitl
- Uninstalling toltec, ddvk package crashed and soft bricked RM2 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from toltec.