Encrypt dotenv file inplace more than once should be failsafe. about sops HOT 11 CLOSED

I have tested @r10r issue and confirm with SOPS v3.8.1

Error Response

$ sops -i -e testing.enc.sops.yaml
The file you have provided contains a top-level entry called 'sops'. This
is generally due to the file already being encrypted. SOPS uses a top-level
entry called 'sops' to store the metadata required to decrypt the file. For
this reason, SOPS can not encrypt files that already contain such an entry.

If this is an unencrypted file, rename the 'sops' entry.

If this is an encrypted file and you want to edit it, use the editor mode,
for example: `sops my_file.yaml`

I would suspect this is a current expected output.

You are interacting with the file humanistically so you're getting a humanistic message back. If you just are editing a SOPS encrypted file, just use sops file, it will open in an editor and when you exit the editor, it'll save it.

You could also update your command: sops -i -d testing.enc.sops.yaml; sops -i -e testing.enc.sops.yaml
To something more safe: sops -i -d testing.enc.sops.yaml; if [[ $(cat testing.enc.sops.yaml | grep -c 'sops:') -gt 0 ]]; then echo 'file already encrypted'; else sops -i -e testing.enc.sops.yaml;fi

When interacting with a SOPS encrypted/decrypted file programmatically, you can build this kind of logic into the program, and it is a noop at that point.

Hope this helps ever so slightly while you keep working while waiting on the maintainers to decide how they want to proceed and get back to you.

TTYL,
Peter

from sops.

@r10r which version of sops are you using? Sops 3.8.1 does not seem to show the behavior you claim.

from sops.

@silentpete Thanks for testing this. I'll consider to create a script with the proposed update command.

@r10r which version of sops are you using? Sops 3.8.1 does not seem to show the behavior you claim.

sops v3.8.1-49-g418ea23e3

I use a development version to be able set yaml indent to 2 to avoid re-indentation for our yaml files.
You should be able to reproduce the issue with the following command sequence:

echo HELLO=world > foo.env
echo "stores:" > .sops.yaml
sops -i -e foo.env 
sops -i -e foo.env 
sops -i -d foo.env 

error message:

MAC mismatch. File has 712A6E486181E64F8949DAF476EAD3FF0924F26D69A28DADCA03B52BA9DF573C0D2E704FC17985EF0763DBF98C0FA8C3C81A20CF322FBB8AD57E2E600070E3C6, computed 1ADDF28E3BCC682983D0FD44D7C1DD81C15D24B22904B9858E6E7CE8D3C1F30C5FDE2F06A37041C0102E40F201B61EB4264C941F427F2A75C051A710727BB853

NOTE echo stores: > .sops.yaml is required or I get the following exception

panic: runtime error: invalid memory address or nil pointer dereference                                                                                                                        
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xdfa520]                                                                                                                         
                                                                                                                                                                                               
goroutine 1 [running]:                                                                                                                                                                         
github.com/getsops/sops/v3/cmd/sops/common.newDotenvStore(0xf145e0?)                                                                                                                           
        /workdir/cmd/sops/common/common.go:46                                                                                                                                                  
github.com/getsops/sops/v3/cmd/sops/common.StoreForFormat(...)                                                                                                                                 
        /workdir/cmd/sops/common/common.go:162                                                                                                                                                 
github.com/getsops/sops/v3/cmd/sops/common.DefaultStoreForPathOrFormat(0x10bb462?, {0xc000451520, 0xd}, {0x0?, 0x10c35db?})                                                                    
        /workdir/cmd/sops/common/common.go:177 +0x9d                                                                                                                                           
main.inputStore(0xc000187a20, {0xc000451520, 0xd})                                                                                                                                             
        /workdir/cmd/sops/main.go:1085 +0x65                                                                                                                                                   
main.main.func8(0xc000187a20)                                                                                                                                                                  
        /workdir/cmd/sops/main.go:796 +0x770                                                                                                                                                   
github.com/urfave/cli.HandleAction({0xec9b80?, 0x1125d88?}, 0xc00046d6c0?)                                                                                                                     
        /go/pkg/mod/github.com/urfave/[email protected]/app.go:524 +0x50
github.com/urfave/cli.(*App).Run(0xc00046d6c0, {0xc0000400c0, 0x4, 0x4})
        /go/pkg/mod/github.com/urfave/[email protected]/app.go:286 +0x766
main.main()
        /workdir/cmd/sops/main.go:1018 +0x35be

I can try to reproduce the issue with the current HEAD if required.

from sops.

Ok, I've tested it with a yaml file and then I'll get the expected error response. So it seems to be limited to dotenv files.

from sops.

My guess is that it happens both with INI and dotenv files, since both of them employ flattening of metadata.

from sops.

(wrong tab)

from sops.

NOTE echo stores: > .sops.yaml is required or I get the following exception

panic: runtime error: invalid memory address or nil pointer dereference                                                                                                                        
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xdfa520]                                                                                                                         
                                                                                                                                                                                               
goroutine 1 [running]:                                                                                                                                                                         
github.com/getsops/sops/v3/cmd/sops/common.newDotenvStore(0xf145e0?)                                                                                                                           
        /workdir/cmd/sops/common/common.go:46                                                                                                                                                  
github.com/getsops/sops/v3/cmd/sops/common.StoreForFormat(...)                                                                                                                                 
        /workdir/cmd/sops/common/common.go:162                                                                                                                                                 
github.com/getsops/sops/v3/cmd/sops/common.DefaultStoreForPathOrFormat(0x10bb462?, {0xc000451520, 0xd}, {0x0?, 0x10c35db?})                                                                    
        /workdir/cmd/sops/common/common.go:177 +0x9d                                                                                                                                           
main.inputStore(0xc000187a20, {0xc000451520, 0xd})                                                                                                                                             
        /workdir/cmd/sops/main.go:1085 +0x65                                                                                                                                                   
main.main.func8(0xc000187a20)                                                                                                                                                                  
        /workdir/cmd/sops/main.go:796 +0x770                                                                                                                                                   
github.com/urfave/cli.HandleAction({0xec9b80?, 0x1125d88?}, 0xc00046d6c0?)                                                                                                                     
        /go/pkg/mod/github.com/urfave/[email protected]/app.go:524 +0x50
github.com/urfave/cli.(*App).Run(0xc00046d6c0, {0xc0000400c0, 0x4, 0x4})
        /go/pkg/mod/github.com/urfave/[email protected]/app.go:286 +0x766
main.main()
        /workdir/cmd/sops/main.go:1018 +0x35be

I can try to reproduce the issue with the current HEAD if required.

I cannot reproduce this part. Can you provide instructions on how to reproduce it, including the exact git commit you have been using?

from sops.

#1393 should fix the main issue from this thread (not detecting that DotEnv files are already encrypted).

Once I can reproduce the stores config problem I can also fix that one, but right now I cannot reproduce it...

from sops.

Thanks @felixfontein for fixing this so fast. I'll double check for the nil pointer issue and open a separate issue if required. Happy new Year!

from sops.

@r10r thanks for checking it! And a Happy New Year to you as well :)

from sops.

@felixfontein

I've upgraded sops to sops v3.8.1-109-g28ecd70c7 on my development machine and the dotenv file encoding now works as expected.
The runtime error does not happen anymore. Thanks again for your work 👍

from sops.