stackexchange / blackbox Goto Github PK

The mktemp command in GNU coreutils, when given a -t argument, expects it to contain at least three consecutive X characters

It looks like you added a fix for this in 56232aa then subsequently clobbered it in 02700b5

• can you restore it please? Sorry no patch as I don't understand the code well enough to know what the difference is between make_self_deleting_tempfile and create_self_deleting_tempfile

(NB: Consulting manual pages on both MacOS and Linux suggests that both OSes will accept -t _stacklib_.XXXXX so there should be no need here to switch on uname)

blackbox_register_new_file should add the file to .gitignore or .hgignore as appropriate.

I believe the syntax is different for each of those, so I'd appreciate pointers on what the format should be for each. In particular, we want to block path/path/foo but not path/path/foo.gpg (and not other/path/foo)

I did a gem install fpm, then make packages-deb and I got an AMD64 deb.

$ ls ~/debbuild-stack_blackbox/
bin-packages.txt installroot stack-blackbox_1.0-1_amd64.deb
$ uname -m
x86_64.

Maybe this is a bug in fpm, but since you depend on it, it's your bug too!

Thanks.

Add and commit should check to see if a unencrypted file is being added/committed when it is listed in the registry.

I assumed it deletes all the files that were decrypted by post_deploy, but it seems it doesn't work that way? Or did I mess something up?

Here's the output when I try to run it (notice no files are listed that will be shredded):

$ blackbox_shred_all_files
========== ENCRYPTED FILES THAT WERE UNLOCKED:
    myproject/encrypted.py.gpg
    myproject/resources/secrets.json.gpg
========== FILES THAT WILL BE SHREDDED:
========== DONE.

Calling blackbox_register_new_file fails, if the repository path contains spaces, eg:
========== Encrypting:
========== Encrypting: DONE
File "", line 1
import os ; print(os.path.relpath("/Users//Library/Application
^
SyntaxError: EOL while scanning string literal
In this case, the path contains in "Application Support" which includes spaces. I guess the issue is in get_unencrypted_filename?

blackbox_edit_start pops up a similar error message, but somehow still manages to decrypt the file and save it properly.

This is more of a question than an issue but...

Can I host more than 1 blackbox in a repository?

From looking at the code (_blackbox_common.sh) it seems that if I set the env variable BLACKBOXDATA before invoking, I can create multiple blackboxes e.g.

export BLACKBOXDATA=keyrings/prod
(some blackbox commands)
export BLACKBOXDATA=keyrings/dev
(somre different blackbox commands)

etc. etc. Mostly, before I go using this feature, I want to insure that this functionality is intended vs. a happy accident that will be deprecated later.

Thanks!

"make confidence" uses gpg-agent so that the password doesn't have to be entered over and over.

However, it does ask for Alice's password once after switching Bob->Alice and vice-versa. It isn't clear to me what about become_alice and become_bob is broken that makes that happen.

It would be great if it only asked for Alice's password exactly once, and Bob's password exactly once.

When trying to deploy via TeamCity with blackbox_postdeply, above error is emitted.

Setting up bash -x to give debug info, REPOBASE is being set as /dev/null. There isn't a .git subdirectory in the project directory.

Using 6765988 - any attempt to get initialize a repository fails. Works fine using a much older version

$ blackbox_initialize
Enable blackbox for this git repo? (yes/no) yes
ERROR: is not a blackbox Repo

Is that intended? If it's no longer part of the blackbox, it should show up as a file that isn't checked in.

It would be convenient to just be able to type "blackbox_edit FILE" and not have to think about the entire process:

Is the file encrypted already?
No: Ask "are you sure?" and then blackbox_register_new_file.
Yes: decrypt.
Call $EDITOR
Was "-d" option specified?"
Yes: Re-encrypt and delete the plaintext version.
No: Re-encrypt and leave the plaintext version alone. Output instructions on how to safely shred it.

This is a great project and I would love to be able to install it and keep it up to date via a package manager. It is great that you have it set up for antigen, but not everyone uses zsh.

Please consider supporting installation via:

• https://github.com/bpkg/bpkg/
• https://github.com/basherpm/basher
• https://github.com/Homebrew/homebrew
• apt
• yum
• something else - these are just a few ideas.

I am more than happy to help with this work, just let me know which direction(s) you prefer or which ones should be prioritized.

I initialized blackbox on a branch of my project a couple of weeks ago, and now I want to finish setting it up.

Currently, in my .gitignore file, there are these 2 lines, which were added automatically (though not sure if it was when I initialized or just recently):

pubring.gpg~
secring.gpg

Now I'm confused since the contents of keyrings/live/ directory are:

blackbox-admins.txt
blackbox-files.txt
pubring.gpg
secring.gpg
trustdb.gpg

Should I update my .gitignore to ignore the other .gpg files in keyrings/live/?

gpg has a --use-agent flag which uses the gpg agent if available. If it's not it does nothing, so users that don't care are not affected.

@snowpong noticed that blackbox_decrypt_all_files does not work on Git for Windows. It fails with:

../blackbox/bin/blackbox_decrypt_all_files: line 19: gpg-agent: command not found
../blackbox/bin/blackbox_decrypt_all_files: line 22: exec: blackbox_postdeploy: not found

The README file mentions that the way to remove a file from the system is to run blackbox_deregister_file but this script isn't available in the latest release .tar.

Perhaps it's time to have a new release?

"make confidence" pauses to ask for passwords. It shouldn't.

GPG makes it very difficult to supply passphrase automatically... which is a very smart decision in general. However it makes automated testing a pain.

Maybe someone can make a very stupid pinentry clone that gives "a" as the password for alice, and "b" as the password for bob.

This makes it difficult/impossible to use blackbox_cat in a pipeline (for which I have a use case), and violates the expectation of 'cat'. Sending such diagnostics to stderr seems like a better plan - if people agree I'll try to produce an appropriate pull request.

Do we have any guidelines or best-practices on how to best protect secrets in releases, such as in a .war file? one approach that comes to mind is encrypting the releases as well, such as creating an encrypted war file, however, it will involve the release management personnel be added as blackbox admins.

I am using Blackbox's Production release (343b85a) on Cygwin on Windows 8 and going through the first time setup.

When running the addadmin function, I get an "ERROR: Unknown OS. Exiting.".

u0044975@U0044975-TPL-C /cygdrive/c/code/meetingtimesuggester
$ blackbox_addadmin [email protected]
ERROR: Unknown OS. Exiting.

u0044975@U0044975-TPL-C /cygdrive/c/code/meetingtimesuggester
$ uname -s
CYGWIN_NT-6.2

u0044975@U0044975-TPL-C /cygdrive/c/code/meetingtimesuggester

Currently, whenever a file is added to the registry, filename.txt.gpg will be created and saved with 644 permissions. When it's decrypted, same story, 644. I'd like to be able to have whatever the permissions were at time of encryption persist through the process, so if I have something set to 600, it gets decrypted to 600.

blackbox_edit foo.txt bar.txt should iterate over all the files.

Currently there is a single line depending on python. It would be nice to remove this unnecessary dependency entirely or make it optional.

Currently blackbox_postdeploy prompts for the passphrase once per file. Would be good to have an option to prompt it only once.

Forgetting quote variables in shell script lead to many security implications. I took a look into some blackbox scripts and found some issue.

Example in blackbox_addadmin:

KEYNAME="$1"
: ${KEYNAME:?ERROR: First argument must be a keyname (email address)} ;

It looks no harm but if someone call blackbox_addadmin like:

blackbox_addadmin '/*/*/*/*/../../../../*/*/*/*/../../../../*/*/*/*'

then it push server to heavy load, even if mightiest server. Imaging some command like this ran and what happen with server.

I have made a pull request to fix this, you can look here

Is possible to tag a release version or a stable version? that would help creating an acceptable brew (http://brew.sh/) formula.

I ran blackbox_addadmin key.asc thinking I could do that to insert a public key. This failed, but, it still put key.asc in the blackbox-admins.txt file. This causes all further operations to fail because there is no key in the keyring with the name key.asc.

People want to be able to do:

export EDITOR="subl -nw"

GnuPG 2.1 uses by default a new keyring format. When a new admin is added to the system, its key is imported. But if no pubring.gpg is found, gpg defaults to the new keybox format and creates a pubring.kbx keybox file. When blackbox_addadmin is called, expects a file named pubring.gpg, see:

https://github.com/StackExchange/blackbox/blob/master/bin/blackbox_addadmin#L48

But if the repo was initialized with gpg 2.1, it fails to add the file to the vcs. With git, for example, you'll get this error:

fatal: pathspec '/home/user/project/keyrings/live/pubring.gpg' did not match any files

For more info about the new keyring format see:

https://www.gnupg.org/faq/whats-new-in-2.1.html#keybox

Hi i'm looking for a way to use git with encrypt files without losing diff tools. Is the blackbox my solution? after i made git pull and decrypt i will be able to see merge conflict in case of existing ones?

Please consider adding functionality along the lines of "blackbox_encrypt_all_files". We currently have:

• "blackbox_update_all_files" which decrypts and re-encrypts all files while overwriting any local changes previously decrypted files.
• "blackbox_shreds_all_files" which securely deletes decrypted files.

However, there seems to be no function that would allow to encrypt all "blackboxed" files in the repository en masse. I understand that this is only a convenience functionality, as one can simply call blackbox_edit_end for each file before commiting. It would however great simply the workflow.

People have asked for per-file or per-subfolder access controls. I'm opening this ticket to ask for suggestions on how they'd like to see it implemented. What should it look like from the user's perspective?

We run into this rare enough we've been able to do it by hand, but it very quickly confuses anyone who hasn't done it properly before

The procedure is something like:

• blackbox_merge "mergetoolcmd" base local remote
  • decrypts base, local, remote to temp files
  • runs mergetoolcmd with same argument set (ala kdiff3 base local remote)
  • re-encrypts result if success

The idea is that one could use this as a git-mergetool strategy, wrapping their 'favorite' merge program.

There are probably some failure cases, etc, I'm forgetting to cover.

Currently blackbox is not aware of blackbox enabled git submodules. The end result is a behavior that blackbox_postdeploy does not attempt to unpack files in submodules of a repository.

The work around for this currently is to have a deploy agent call blackbox on each submodule, or to register files a second time in the main repo of a given project; Both of which are less than ideal.

A nice feature would be to have blackbox_postdeploy use the .gitmodule file to check for registered modules and attempt to unpack any files registered to that submodules project.

The use of gpg-agent seems to be broken on Mac OS X.

become_alice and become_bob don't seem to work. The "a" and "b" passwords have to be entered over and over again.

Every time it asks for a password, it first prints:

can't connect to `/tmp/gpg-xhWU7m/S.gpg-agent:5356:1': No such file or directory
gpg: can't connect to `/tmp/gpg-xhWU7m/S.gpg-agent:5356:1': connect failed

gpg (GnuPG) 1.4.19
gpg-agent (GnuPG) 2.0.28
pinentry-curses (pinentry) 0.9.4

A non-admin can gain access to the encrypted data as follows:

1. Run blackbox_addadmin as normal but do not ask someone to approve their membership.
2. Wait until someone runs blackbox_update_all_files

This can happen if a newbie runs blackbox_addadmin by mistake. This situation can be fixed by moving the change to blackbox-admins.txt out of blackbox_addadmin, and to the "step 2" part (which should be automated via a new command such as blackbox_acceptadmin).

This can happen maliciously but someone simply editing blackbox-admins.txt. However this is generally prevented by repo access controls. However, this assumes anyone with write access to the repo is trusted.

NOTE: This is safe as long as anyone with write access to the repo is also an admin (or soon to become an admit). This is the use case currently at Stack Exchange, but it should be fixed so other use-cases are safe.

Suggested solution: GPG sign the blackbox-admins.txt and only permit encryption/re-encryption to happen if the signature matches. Alternatively, I think keeping the blackbox-admins.txt file encrypted might be sufficient. I'll have to think about it.

Still trying to wrap my head around this problem.

After doing a git pull, running blackbox_register_new_file shows this error:

% echo nevermind > whatever.txt
% blackbox_register_new_file whatever.txt
gpg: whatever.txt: encryption failed: No public key

I double-checked that all public keys had subkeys which were not expired.

When I step back from the latest release to eab0418 the command works.

Adding "set -x" to the top of blackbox_register_new_file reveals this output:

+ gpg --use-agent --yes --trust-model=always --encrypt -o whatever.txt.gpg '-rRECIPIENT1 -rRECIPIENT2 -rRECIPIENT3' whatever.txt gpg: RECIPIENT1 -rRECIPIENT2 -rRECIPIENT3: skipped: No public key gpg: whatever.txt: encryption failed: No public key

In release eab0418 and before, output looks like this:

+ gpg --yes --trust-model=always --encrypt -o whatever.txt.gpg -rRECIPIENT1 -rRECIPIENT2 -rRECIPIENT3 whatever.txt

All the recipients are on the same line.

What baffles me is that the code in _blackbox_common.sh is the same between the two versions:

gpg --yes --trust-model=always --encrypt -o "$encrypted" $(awk '{ print "-r" $1 }' < "$BB_ADMINS") "$unencrypted"

Anybody else seeing this error?

it should distinguish between the puppet directory and the blackbox install directory
when deploying independent environments, the current BASEDIR-definition assumes blackbox to be installed into every environment's bin directory
(alternatively assume blackbox/bin is in the path, ie. simply delete line 11)

Very few people use MacPorts anymore. It would be nice to see support for installing blackbox via HomeBrew.

You'll have to refactor the blacbox_home variable to be externally set, or be able to not put the underscore scripts in the bin directory, that's a deal breaker for HomeBrew, they don't want any scripts going into bin that aren't called directly.

Last release was more than a month ago, and there are quite a few changes included.

I know I can clone the repository and use the bin folder, or even store the artifacts somewhere else, but I need some of the bug fixes and having a new release in https://github.com/StackExchange/blackbox/releases would be awesome.

Is there any idea when that will happen?

Hello,

We have a workflow requirement that consists of having all of the encrypted files unlocked, and then relocked (but not necessarily with changes).

It appears that blackbox_postdeploy is intended to provide the "unlock all" requirement, however I don't see a mirroring "relock all" command.

Is this something supported, or how would you recommend accomplishing this?

We've had a number of git commits from Alice or Bob due to "make confidence" being overzealous about becoming Alice or Bob for the tests.

Surely this can be fixed.

blackbox_edit and blackbox_cat use other blackbox scripts internally, but assume that they will be found on $PATH. It's sometimes useful to be able to run these scripts by pathname without first putting them on $PATH, and all the other scripts work just fine in these circumstances.

I've created PR #56 to fix this.

I run blackbox_register_new_file foo, and after it does the dance and the file is encrypted and added to the repo with a commit automatically, git-repo-root-directory/.gitignore is updated as expected, but it isn't actually committed in the same automatic commit that foo.gpg was.

I work around it by git commit --amend .gitignore, but it would be better if the .gitignore change was included in the automatic * registered in blackbox: path/to/foo commit

If a key for a user listed in blackbox-admins.txt is not found then the re-encrypt fails leaving the unencrypted file on disk without any working.

Fixes that I can think of (not necessarily the only ones)

• check for keys for all users prior to decryption.
• perform the decrypt and re-encrypt in memory w/o a temp file
• remove temp file on failure
• warn the user that an unencrypted file was left on the drive

How to reproduce:

• echo '[email protected]' >> keyrings/live/blackbox-admins.txt
• blackbox_update_all_files

'make confidence' unconditionally cd's to ~/gitwork/blackbox which may not be where the source is installed.

It also sets (rather then modifies) $PATH which may be a portability problem (though it seems to work on MacOS+Homebrew).

========== Encrypting: DONE
========== COMMITING TO HG:
abort: no repository found in '/current/repo/path' (.hg not found)!

There's manual steps for removing a file from blackbox but it would be far better to have a tool to do it so I know that I won't screw up. Is there a possibility of adding this feature?