nix-community / infra Goto Github PK
View Code? Open in Web Editor NEWnix-community infrastructure [maintainer=@zowoq]
Home Page: https://nix-community.org
License: MIT License
nix-community infrastructure [maintainer=@zowoq]
Home Page: https://nix-community.org
License: MIT License
https://nixos.org/community/teams/security.html
Like the nixos org I suppose we should have a method and some sort of policy for reporting potential security issues with the infra and repos that don't already have their own security reporting or aren't responsive.
Easiest may be taking reports via github itself: Private vulnerability reporting (beta)
Guess an alternative could be encrypted email but we'd probably want a dedicated address that gets forwarded to everyone rather than it potentially being sent directly to one person who isn't responsive.
Do the same treatment as terraform cloud. Give the admins access to cloudflare.
I'm using this for the docker-nixpkgs repo. This should be added to the list of apps that we manage.
We have a workflow in ethereum.nix that needs to sign commits: nix-community/ethereum.nix#165
Ideally there would be a Github profile for a nix-community account against which gpg keys can be added. If a repository requires signing they would generate a gpg key and set it as a repo secret, then also create a PR against this project to have that GPG key added to the bot profile.
Relevant background:
I want to avoid hidden dependencies. Right now the domains are held by numtide. We need a credit card for this.
I was going to do a manual backup of these logs before reinstalling build02 but I suppose we should really be doing regular backups so they aren't all lost if build02 dies.
If we don't have somewhere external to nix-community to back them up to I guess we'd want backups on a couple of our machines?
cc @nix-community/admin
Any objections to adding the https://github.com/apps/settings app to the org?
It introduces a small attack window where users with push access can change the settings of their repo. But it also makes the config more transparent and self-serving. IMO the trade-off is worth it.
Using the new post build hook NixOS/nix#2995
Similar to how @ryantm used to do at https://discourse.nixos.org/t/nixpkgs-update-r-ryantm-logs/1464
cc @grahamc
Since the last nixpkgs upgrade hydra-send-stats.service fails like this:
โ hydra-send-stats.service
Loaded: loaded (/nix/store/ws5saprglyw83al17hwpzvsnllf78h39-unit-hydra-send-stats.service/hydra-send-stats.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sat 2021-03-20 05:40:50 UTC; 1min 51s ago
Process: 28067 ExecStart=hydra-send-stats (code=exited, status=25)
Main PID: 28067 (code=exited, status=25)
IP: 0B in, 0B out
CPU: 92ms
Mar 20 05:40:50 nix-community-build01 systemd[1]: Started hydra-send-stats.service.
Mar 20 05:40:50 nix-community-build01 hydra-send-stats[28067]: Undefined subroutine &main::getHydraConfig called at /nix/store/m3mz9nv3wad8qigf9mb88k8adk6pwacm-hydra-2021-03-10/bin/.hydra-se>
Mar 20 05:40:50 nix-community-build01 systemd[1]: hydra-send-stats.service: Main process exited, code=exited, status=25/n/a
Mar 20 05:40:50 nix-community-build01 systemd[1]: hydra-send-stats.service: Failed with result 'exit-code'.
Basically type up a nicer version of #399 and anything else that might be needed.
I'll follow up on this in the next week or two.
Not really a problem at the moment but if we keep adding accounts to the top level secrets.yaml
we may want something easier to use?
1password has an open source program: https://github.com/1Password/1password-teams-open-source
Could try bitwarden as well but I think we'd need to host it ourselves or pay for it?
I'll leave the server hardware off this list as we have do have access via sops secrets and consolidating those accounts is more a finance issue.
Dear nix-community team,
I've noticed that you're running niv-updater-action without PR descriptions (the changelog). As I'm constantly looking to improve my software and help people, I'm wondering why you're doing it that way? Would you prefer to have something else in the PR description?
I've created an issue for your usecase: knl/niv-updater-action#51
Thanks,
Nikola
@makefu recommended go-neb: https://github.com/matrix-org/go-neb/blob/master/config.sample.yaml
This would allow for keeping the domains short. Eg: https://hydra.n11y.org
But maybe that's just my dislike of typing :)
I'd like to move the marvin-mk2 bot (repo) onto nix-community infrastructure before starting to test it on nixpkgs.
It should consume very little resources. Would that be possible? If so, how should I go about it?
We're using https://github.com/numtide/srvos/blob/master/roles/github-actions-runner.nix in a number of places now, maybe the community could also benefit from having faster CI by pushing the builds to permanent machines?
I was just thinking a bit about the deploy script a bit.
Would it be possible to extend it, so that it uses screen or tmux when executing the tasks on the remote machine.
The idea is to make it less problematic in case the connection breaks or nixos-rebuild
takes a long time and you can disconnect and reconnect at a later point.
I would like to move the funding from GitHub to Open Collective to be more transparent. Open Collective also allows acquiring virtual credit cards, so we could use that to pay for them. We also have an admin@ address that we can use to forward billing emails.
similar to the aarch64 community builder it would be nice to offer the same service for x86.
This is especially interesting for people with less capable hardware (but potentially more time)
to help pushing the project.
Needed by nix-community/docker-nixpkgs#46 for example.
Since nix-community now hosts marvin-mk2, it would be nice if I could deploy new versions myself. This is especially relevant on major changes where I'd like to control the timing and fixes which should be deployed quickly.
In addition to that, I'd like to transfer the bot instance itself (https://github.com/apps/marvin-mk2) to nix-community to reduce dependency on a single person, but to do that I while maintaining access to the admin interface I think I would need to be a member.
Previously discussed in #24 and privately with @adisbladis.
Until recently, the nixpkgs-swh script was executed every day by a buildkite agent. However, this agent no longer exists.
Would it be possible to run a daily systemd cron jobs?
Since it evaluates the whole nixpkgs repository, it takes a bit of time (about 20min on my laptop i guess).
Moreover, we would need to expose the generated JSON file. We would then have to spawn a nginx server to serve it. Note this file doesn't strictly need to be persisted since it is generated each day.
Note that using hydra.nix-community.org seems to be difficult because the script calls hydra.nixos.org to get latest evaluated nixpkgs commits.
WDYT?
Right now it seems to spawn a new cachix instance for every derivation
instead of passing multiple derivation at the time.
This is very slow for to big amount of emacs derivation in our hydra.
This could be used to pay for #396 and servers.
Right now servers are paid for by ryantm and Numtide.
After the last upgrade the machine did not come back.
cc @zimbatm
I think it might be worth to have a backup in case someone maliciously deletes repositories or github decides to ban us for whatever reason (we just received a DMCA takedown order).
I have some terraform code to automatically create mirrors from github to gitlab: https://github.com/Mic92/dotfiles/blob/main/terraform/gitlab/repo-mirror.tf
Originally posted by @Mic92 in #399 (comment)
It looks like the builds are intended to end up in Cachix:
infra/services/hydra/default.nix
Lines 115 to 120 in 1c9f920
However, I've tried to check it with Kittybox. The build output is present in the Hydra, but is not signed. Cachix doesn't contain the output.
Is this intentional?
Is there a free service we can use to multiple [email protected] to all our emails?
Could be useful for #392 as well
Could something like iptables/dns issues w.r.t. docker:
https://github.com/nix-community/templates
How would I go about becoming a maintainer?
I would like to keep the license the same.
nixpkgs-update has been stuck for a day trying to do nixpkgs-review for pythonPackages.fastapi
, a package with only 10 reverse dependencies. It seems like nix-build is not reliably running. I've left it in this state in case someone with more knowledge about the nix-daemon or nix-build can diagnose the problem more.
@zimbatm thinks it might be a problem with the nix-daemon having trouble with man concurrent builds
cc @adisbladis
It looks like there is a machine that could be used for building aarch64-linux
packages, build04
- however, it doesn't seem to be used and builds are instead rejected with "Unsupported system type". Is this intentional?
Example failing build: https://hydra.nix-community.org/build/14455702
https://github.com/organizations/nix-community/settings/secrets/actions
I think we had all agreed these needed to be removed, how to we want to do it? 60 days notice seems reasonable to me?
Projects that want to use the nix-community cachix need to move to hercules or hydra, alternatively they can create their own separate cachix.
This will mean that there is no nix-community cache for darwin as we currently don't have a builder for that platform.
It's come up in the poetry2nix matrix chat that a build box for macos would be useful for debugging, as macos builds are causing some blockers currently. Would both aarch64-darwin and x86_64-darwin be needed? I'm happy to help sponsor some of the costs of this.
5 days ago emacs package updates stopped in the nix-community/emacs-overlay repo probably due to f47b49e. See the log at https://gitlab.com/nix-community/emacs-overlay/-/jobs/967612808, from line 485 and on.
Related issue: nix-community/emacs-overlay#104
cc @ryantm
I'd like to host our own monitoring so we are self sufficient.
I guess we could switch to something else but easiest option is import the monitoring we're currently using from https://github.com/Mic92/dotfiles.
I'd focus on getting it working here first but eventually we could look at sharing the boilerplate config, maybe https://github.com/numtide/srvos or somewhere else.
Would also like set up a dashboard as well.
@adisbladis
@flokli
@grahamc
@Mic92
@nlewo
@ryantm
@zimbatm
@zowoq
I propose to write down a few rules for the administrators of this project for transparency reasons and to make it more democratic.
Our mission is to support the users of the org, and the Nix project in general.
This is a voluntary effort, on a best-effort basis. Things that are good to do are:
To reduce the attack surface on the project, I propose limiting the number of administrators to 5 people.
In the current configuration, I propose to keep Mic92 and zowoq because they are actively managing the infra, and ryantm so he can deploy his bot. That leaves 2 other people from the existing list.
In order to become an administrator, we ask that you already started contributing to this repo through PRs, and that you are trustworthy. Trust is built on top of personal relationships and past behaviour.
Right now we trust everybody on this list, so that makes it easy for you to come back.
Did I miss anything? Does that sound reasonable?
Result of smartctl --all /dev/sd{a,b}
:
Summary:
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: FAILED!
Drive failure expected in less than 24 hours. SAVE ALL DATA.
No failed Attributes found.
# ...
202 Percent_Lifetime_Used 0x0030 001 001 001 Old_age Offline FAILING_NOW 99
Apparently disks are EOL, no data error visible.
Should we ask Hetzner for new drives or ignore the error?
Last week I discovered that it's possible to create issue templates with form fields:
https://github.com/NixOS/foundation/blob/master/.github/ISSUE_TEMPLATE/funding_form.yml
We could create issue templates for common operations and help clarify our workflows.
For example:
What do you think?
This has been mentioned on #nix-community
matrix a few times now.
I want to move marvin-2k to build03. I saw that it currently uses webhooks as callbacks.
What domain did you configure as a callback, if it is hardcoded can we make a new subdomain just for marvin-2k?
Really I'd prefer to do it just for this repo but unfortunately that doesn't seem to be possible.
Only a few that don't already have 2fa set:
7 of the 16 have bot
or ci
in their name.
We have the Settings app for repository owners to self-configure their repos.
What's missing is to describe all of the teams and members as code to increase transparency even further.
That also helps create a membership list, potentially allow members to send PRs to add new repos, ...
let's see
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.