ulif / diceware Goto Github PK

Joseph Bonneau has published several new wordlists under the auspices of the EFF, which offer improved usability over the original list. These lists should be added as alternate sources.

Hey,
the last tag is v0.9.6 and does not contain the german wordlist:

kmille@linbox:diceware git diff v0.9.6..HEAD --stat
 .github/workflows/tests.yml                   |   44 +
 .gitignore                                    |    1 +
 .travis.yml                                   |   16 -
 CHANGES.rst                                   |   22 +-
 COPYRIGHT                                     |   12 +-
 LICENSE                                       |   17 +-
 README.rst                                    |   72 +-
 diceware.1                                    |   21 +-
 diceware/__init__.py                          |   23 +-
 diceware/config.py                            |   17 +-
 diceware/logger.py                            |    2 +-
 diceware/random_sources.py                    |   23 +-
 diceware/wordlist.py                          |    2 +-
 diceware/wordlists/wordlist_de.txt            | 7776 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 diceware/wordlists/wordlist_de_8k.txt         | 8192 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 diceware/wordlists/wordlist_en.txt            | 8192 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 diceware/wordlists/wordlist_en_adjectives.txt | 1296 +++++++++++++++++++++++++++++
 diceware/wordlists/wordlist_en_nouns.txt      | 7776 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 diceware/wordlists/wordlist_en_orig.asc       | 7789 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 diceware/wordlists/wordlist_pt-br.txt         | 7776 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 docs/conf.py                                  |    2 +-
 docs/index.rst                                |   10 +-
 docs/manpage.rst                              |   25 +-
 setup.py                                      |    5 +-
 tests/test_config.py                          |   17 +-
 tests/test_diceware.py                        |    6 +-
 tests/test_random_sources.py                  |   44 +-
 tests/test_wordlist.py                        |   29 -
 tox.ini                                       |   18 +-
 29 files changed, 33101 insertions(+), 16124 deletions(-)
kmille@linbox:diceware 

I've spoken with the maintainer of the Arch Linux package [0] [1]. The german wordlist will be part of the package if it's part of an official release. Can you please create a new release/tag with the german wordlist? Thanks in advance!

[0] https://aur.archlinux.org/packages/diceware
[1] https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=diceware

Right now wordlists are placed inside the package. This is straight and there were little problems yet with this approach.

It might, however, be more useful to place these files in a system directory (via setuptools, etc.). For instance for sysadmins. This should be possible at least if people can still use virtualenv and friends during install and are not bound to the good-will of root.

TODO: collect pros and cons.

Numbered wordlists denote dice throws with a sequence of dice-numbers like this:

123456  some-word

This is not a problem, as long as you expect ordinary six-sided dice to be used. But some dice have 10 or more sides. For instance 11111 with 12-sided dice could mean 11 followed by 11 followed by 1 or it could mean 1 followed by 11 followed by... you get the idea.

In short: 11111 might be ambigous.

To overcome this problem, we should additionally allow dashes in numbered wordlists:

1-2-3-4-5-6  some-word

such, allowing clear separation of single numbers in the sequence.

This change would not be a big thing for diceware as we do not interpret numbers in wordlist. We just have to detect them.

Current input of dice rolls with the realdice randomness-source expects whitespace characters to separate numbers. With reguiar expressions we could allow also colons, dots, or other (non-digit) chars.

For instance, instead of 1 2 3<CR>2 3 1<CR> we could accept 1, 2, 3<CR>2-3-1<CR> and similar input.

This change was brought up by @alanhoyle in #58 based on recent changes from @adinklotz .

By default the single phrase words are capitalized, i.e. the first char of each
word is made uppercase. This does not neccessarily give better security
(1 bit at most), but it helps reading a phrase.

This is not true, most diceware wordlists used is not prefix-free. Using --no-caps reduces entropy by indeterminate amount since the system using the generated password cannot distinguish between:

AirPortable
AirportAble

when using --no-caps, both are lower-cased to:

airportable

Unless you have a prefix-free wordlist (among the built-in wordlist, only the EFF wordlist is prefix-free), the amount of entropy lost when there is neither capitalisation or word separator is greater than 1 bit.

Both the LICENSE file and http://world.std.com/~reinhold/diceware.html state that "diceware" is a trademark of Arnold G. Reinhold. Did he grant permission for the name to be used in this project? If so, under what conditions?

while it's nice that you can add an arbitrary delimiter to generated passwords, it would be even better if this software would have the (optional) feature to inject random symbols and digits as separators.

by default, the search space of generated passwords is quite large, but that's mostly due to the length of the password. while one can't assume that the password to be cracked will be all letters in the general case, that means the password generation routine becomes yet another "something you know" and allows the attacker to reduce the search space significantly once they know you use this software.

allowing for random passwords would force attackers to not limit themselves to 52 characters. adding numbers and symbols to that space would add about 20 bits of entropy to every password, at little cost to the human memory.

[1022]anarcat@curie:~$ diceware | wc
      1       1      29
[1023]anarcat@curie:~$ diceware | wc
      1       1      28
[1023]anarcat@curie:~$ diceware | wc
      1       1      36
[1023]anarcat@curie:~$ diceware | wc
      1       1      26
[1023]anarcat@curie:~$ qalc
> log2((26+26)^26)

  log2((26 + 26)^26) = approx. 148.21143

> log2((26+26+10+10)^26)

  log2((26 + 26 + 10 + 10)^26) = approx. 160.41805

> log2((26+26)^36)

  log2((26 + 26)^36) = approx. 205.21583

> log2((26+26+10+10)^36)

  log2((26 + 26 + 10 + 10)^36) = approx. 222.1173

thanks for your consideration!

See list of items provided in https://github.com/grempe/diceware/tree/master/lists and https://github.com/grempe/diceware/issues

Hi!

I am the packager of Diceware for Debian, and because Python 2 is approaching end of life I have already ported the package to Python 3.6. But Debian will soon (2 to 3 weeks, actually) remove support for Python 3.6, so I will need to port the package to Python 3.7.

I have not tried to run Diceware with Python 3.7 yet, but thought I should report here first to check if there's already a movement in that direction or if I should/could help in some way.

Cheers!

I've been working on https://github.com/simonft/phrasebook and have been talking with @drebs about using diceware as the backend for it. One thing I want to add to phrasebook is the ability to choose from wordlists in multiple languages. Is adding wordlists in other languages something you'd consider adding (or allowing someone else to add) to diceware as well?

Hi, I'm trying to create a dicewre package for NixOS, and it would be great if I could simply pull the latest release. At the moment, there are bugs/problems in the tests on the latest release.

Thanks!

I think it would be nice to provide a machine-readable copyright file with diceware. It looks like Debian has a policy for that:

https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/

We have a mix (not to say: mess) of copyright holders and license types due to the different wordlists. To ease additions of further lists, we should implement the Debian copyright policy a.s.a.p., I think.

@drebs As you are more into Debian packaging: could you help with that?

Two related issues here:

1. Some password systems consider digits and symbols to be a separate category and require at least one character from each.
2. Some password systems require that your symbols come from their defined list of symbols.

Quoting my work's password rules (in part):

• It must contain at least one digit
• It must contain at least one of these characters: !@#$%&*+={}?<>"'

Add infile=None to OPTIONS_DEFAULTS in diceware/config.py to permit configuring custom word lists that are not installed in diceware/wordlists.

@heartsucker: what about a name like securedrop for your wordlist? Or is there another project you would like to promote a bit? If I understood correctly, securedrop was the place where the list was born in the first place, right?

Just tell, what name you would like and I be glad to care for everything.

Currently there is no documentation for using this package as an import to other python applications, the documentation covers command-line use only.

Logging handlers should be cleaned up after runs. At least they can bite in tests. A respective fixture should help.

Hey Uli. Just discovered your great software. Thanks so much for it.

One function I'm unable to find is the possibility to generate many passphrases at the same time. Is it there? If not, would it be a good idea to add it?

Currently we test with py.test whether headers and docs reflect the current year/month.

When a year/month marker in docs or source headers outdates, then this is a problem for packagers and release maintainers. It is not a problem for people running the code.

We might therefore do this kind of checks with tox, but in a different manner.

This isn't really a bug, but I want to mention it anyway, just to make sure.

If my diceware.ini file contains the following:

[diceware]
delimiter = " "
caps = off

an example output from the diceware -n 2 command is as follows:

rowdy" "aubrey

That is, the delimiter is being read literally as " " from the .ini file. Thus, there is no way (that I am aware of) to set space as your default delimiter using the ini file.

One way to fix this (if it is indeed an issue) is to add a strip('"') to line 85 in the config.py file in the get_config_dict function, as mentioned in this answer. That will strip out all instances of " from the ini file. This may or may not be a good thing.

The default wordlist_en.txt is not included during installation via pip. Steps to reproduce:

vagrant@vagrant-ubuntu-trusty-64:~$ sudo aptitude install python-pip
vagrant@vagrant-ubuntu-trusty-64:~$ sudo pip install diceware

Then:

vagrant@vagrant-ubuntu-trusty-64:~$ diceware
Traceback (most recent call last):
  File "/usr/local/bin/diceware", line 9, in <module>
    load_entry_point('diceware==0.2', 'console_scripts', 'diceware')()
  File "/usr/local/lib/python2.7/dist-packages/diceware.py", line 137, in main
    fd=options.infile
  File "/usr/local/lib/python2.7/dist-packages/diceware.py", line 110, in get_passphrase
    fd = open(get_wordlist_path(lang), 'r')
IOError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/wordlists/wordlist_en.txt'

This can be resolved via the data_files directive in setup.py (docs):

    data_files=[
        ('wordlists', ['wordlists/wordlist_en.txt']),
        ],

Doing so will ensure that wordlist_en.txt is accessible by the module. Pull request on its way. Added a few Travis checks as well.

The (Shannon-)entropy of a generated passphrase, given in bits, could help to compare passphrase complexity and "secureness". It is at least more reliable than the red-yellow-green indicators shown with some password managers.

travis-ci seems not to really support open source projects any more.
Let's switch to github actions.

Wordlist objects might maintain own file descriptors. These are not closed, when Namelists are destroyed.

We can read words from cryptographically signed wordlists. But yet we do not check these lists.

More wordlists

We would like to provide more wordlists.

Although users can use any wordlist they want::

  $ diceware mywordlist.txt
  $ cat /usr/share/dict/words | diceware -

we might offer some of them for convenience out-of-the-box. Typical candidates could be wordlists in other languages, the original 7776-terms wordlist from diceware.com, or others.

Register wordlists by name, not by language

As one can see from that list, it might not be convenient to offer/select wordlists by language as it is the case currently (yeah there is only one list built-in but we have a language selector at start).

Instead we could name each wordlist with arbitrary strings (something like "8k", "original", ...) and process these wordlist names in user interface.

The --help option should display all lists available (or we offer a special switch for that).

An uncomplete collection of possible wordlist types:

• Language lists
• Business lists:
  Lists that provide only terms suitable for little children. These lists could be used by services that
  generate passphrases for others and must make sure no one is offended by the words picked.
• Other lists used for good reasons (like the original 7776 list)

Support GPG-signed wordlists

Something else regarding wordlists one could consider: the wordlist format expected currently, is a file (or file-like object) with lines. All non-empty lines are treated as a word. This makes it hard for GPG-signatures or similar data sources to be processed. Especially GPG-signed files could be interesting, because we could check authenticity of a wordlist when we use it.

Support dynamic wordlists

Static files are one possibility to create a wordlist. It should be possible to create wordlists also from dynamic sources on the fly (this might even improve entropy). In that case we would need a mechanism to hide the real format of a wordlist from the core components.

Instead we would need some piece of API that allows something like mygenerator = get_wordlist_generator(name) and then retrieves words from this generator piece by piece.

Collect wordlists from other packages by entry_points

A last thing we could support: third-party packages could provide their own wordlists for use with diceware. They might register their lists with some diceware component which, when called returns the lists. The registration and list lookup could again be done via entry points.

This last point is far from being urgent.

As drebs suggested in #4 we should offer real dice as a source of randomness.

The procedure should come close to the original procedure described on http://world.std.com/~reinhold/diceware.html.

This plugin could then go into the next festure release, v0.5.

at first glance, it sure looks like the EFF word list is superior to the other alternatives. is there any reason why it's not the default list?

The path to wordlist files is hardcoded in the generated sphinx
documentation and because it may differ depending on how the package is
installed this creates different problems:

• the path hardcoded in the online api documentation may differ from
  the one in each installation, leading to incorrect information
  provided to the user of the api.

• packaging diceware documentation to specific operating systems may
  generate unreproducible packages.

One possible solution is to replace the WORDLISTS_DIR with a function called get_wordlists_dir() and have the path be constructed during execution of the function rather than during module load time.

The following command results in an error:

diceware -n 1 -s 1 -r realdice
#2
#2
#2
#2
#2

The word that should be chosen in this case is Cumin. But it results in an error since I requested special characters, and the word Cumin is only 5 characters long. The program isn't setup to choose from a sequence that is shorter than the number of sides on the die.

When trying to use the realdice source of randomness, diceware fails with an AttributeError caused by an invalid access to an argparse.Namespace object.

This error was introduced in version 0.9, discovered, and fixed by @LogosOfJ. Many thanks for that.

Is it possible to release a new version?

ArchLinux refers to the 0.9.6 version in it's user package repository. Having a new version would allow for a 'clean' update and the commit log suggests that there are enough changes to do so.

IMHO

flake8 revealed that we still use Python 2.7 code regarding raising exceptions. Culprit is in random_sources.py, where raise is used as a function, not as a keyword in line 141.

The disitribution of values picked from a sequence in diceware.RealDiceRandomSource.choice() is not equal, if the sequence length is not a power of the number of dice_sides.

For instance for a sequemce [1, 2, 3] and four-sided dice, the distribution will be:
(6/16, 5/16, 5/16), i.e. the first item has a higher probabilty to be picked than the other elements.

This is because we do a modulo, where instead we might should discard rolls.

We use pkg_resources for retrieving package version infos. While this is not exactly bad (and even convenient from a developers point of view), it is also the only external dependency we carry with us. It would be nice, to depend on standard lib packages only, which woulg also mean less trouble for maintainers.

Hey,
it's cool to have config file support at ~/.diceware.ini. But it's uncool to have the home directory full of .dotfiles.
Can you please honor the XDG_CONFIG_HOME envrionment variable? It's documented here:
https://wiki.archlinux.org/title/XDG_Base_Directory
https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

The idea is: if XDG_CONFIG_HOME is set, use it as base dir to look for the config file. If there is no config file, use the default location at ~.

There's plenty of code out there that caps the maximum password length. Rather than requiring me to manually crop the generated passwords, it would be great to be able to specify -m NUM, --max NUM to make diceware select words accordingly so that they fit.

Alternatively, diceware could just crop the generated password.

Yes, I am aware that both will decrease the entropy of the generated password. But I didn't write the braindead code limiting the length of passwords users can use.

Is there some way to get this to accept

diceware -r realdice <<EOF
1
2
3
4
5
EOF

so we don't have to manually type every number?

When we have dice with n sides and use a wordlist of m words, then we would like to get an optimum entropy of log(m) per word. If, however, m is not a power of n, then, I think, we lose entropy when mapping dice throws to the words in the wordlist.

Consider n=3 and m=2:

1  →  A
2  →  B
3  →  ?

What to do when the user rolls a "3"?

Two possibilities come to mind:

a)
1 → A
2 → B
3 → A

b)
1 → A
2 → B
3 → deny, roll again, until one of (1, 2) is rolled

The second one (b) is, what we implement currenty. But is this mathematically the best?

It would be nice to have the option to check generated passphrases against the list of leaked passwords from sources like Pwned Passwords which contains list of more than half a billion passwords which have previously been exposed in data breaches.

Pwned Passwords also implements a k-Anonymity model that allows a password to be searched for by partial hash.

If the user opts for checking the passphrases against the Pwned Passwords and it is found in the list then a new password should be generated for the user. This issue is inspired by the a Computerphile video.

The --no-caps option turns off capitalizazion of words.

This is not the same as turning all words into lower-case. The behaviour is probably good, because upper- and lower-case letters could make a difference in some wordlists (say "pet" and "PET") and then we should leave words untouched. At least we should give a switch to turn capitalization off.

But the docs still say, that --no-caps means uncapitalization of words which is simply not true.

This must be fixed.

I have discovered a bug. If I have the following in my .diceware.ini file:

[diceware]
delimiter = SS
caps = off

Then the output of the command:

diceware -n 2 -r realdice
#2
#2
#2 ... etc
#

is

ChickentestChicken

Thus, the caps setting doesn't work.

I have located the error in the code. The init.py file calls the option capitalize while the config.py calls this option caps. All of the unit tests check whether the caps variable is successfully set from the ini file, but they don't test whether the capitalization is applied.

I think the best way to fix it would be to replace all the instances of capitalize with caps in the init.py file, unless you prefer otherwise. Happy to take care of this bug and submit a pull request myself.

Hello,! I think a GUI would increase usage of diceware among less tech savvy users, and could help increase the overall security of passphrases people have to remember.

This is a list of features I thought about (not necessarily implemented in this order):

• A text box that displays the generated passphrase.
• A button to generate a new passphrase.
• Fields for each configurable parameter (number of words, capitalize, number of special characters, choice of delimiter, word list to use, source of randomness plus a special field for inputting real dice values, if that is the case).
• A button to save defaults.
• An entry in the applications menu.

@ulif, do you think this could be added to this repository and be distributed in the same Python package? Or does it make sense to separate it somehow?

Another question I have is which graphical library to use. Some criteria I thought for choosing are the least amount of dependencies, availability in most distributions, and maintainability in the long run.

Any thoughts are appreciated, and I'd be willing to contribute code for this if you think it's a good idea.

I have 5 real dice that I use with real-world diceware. If I specify --randomsource realdice it would be much nicer if I could just type "12345" instead of having to type each number separately and with a carriage return.

I'm using this package currently, but I would like to switch to diceware (this project) because it is active.
AUR (en) - hsxkpasswd
https://aur.archlinux.org/packages/hsxkpasswd/

However, I don't see all the config options I use. I would like to request support for the different prefix and postfix options of hsxkpasswd, and ability to generaate a specified number of words on each invocation. You can see them in action here: https://xkpasswd.net/s/

Here's a config file I just created there.
{
"num_words": 6,
"word_length_min": 5,
"word_length_max": 9,
"case_transform": "RANDOM",
"separator_character": "-",
"padding_digits_before": 2,
"padding_digits_after": 2,
"padding_type": "FIXED",
"padding_character": "RANDOM",
"symbol_alphabet": [
"!",
"@",
"$",
"%",
"^",
"&",
"*",
"-",
"_",
"+",
"=",
":",
"|",
"~",
"?",
"/",
".",
";"
],
"padding_characters_before": 2,
"padding_characters_after": 2,
"random_increment": "AUTO"
}

Looking through the documentation here I don't see support for most of that.

$ diceware nonexistingfile
Traceback (most recent call last):
  File "/home/bgandhi/diceware/py36/bin/diceware", line 11, in <module>
    load_entry_point('diceware==0.9.4.dev0', 'console_scripts', 'diceware')()
  File "/home/bgandhi/diceware/diceware/__init__.py", line 211, in main
    print(get_passphrase(options))
  File "/home/bgandhi/diceware/diceware/__init__.py", line 184, in get_passphrase
    word_list = WordList(options.infile)
  File "/home/bgandhi/diceware/diceware/wordlist.py", line 126, in __init__
    self.fd = open(self.path, "r")
FileNotFoundError: [Errno 2] No such file or directory: 'nonexistingfile'
Exception ignored in: <bound method WordList.__del__ of <diceware.wordlist.WordList object at 0x7f94008f2dd8>>
Traceback (most recent call last):
  File "/home/bgandhi/diceware/diceware/wordlist.py", line 131, in __del__
    self.fd.close()
AttributeError: 'WordList' object has no attribute 'fd'

I'm working on a patch for this.

With mergin #13 the @heartsucker wordlist is the new default.

There are two minor things left I would love to improve: is there a shorter name available than "en_heartsucker"? Maybe a name that describes it better? Or one that is better memorizable? Like famous hackers or whistleblowers? Just an idea, but we should clarify that before the next release.

Then, @heartsucker, I wonder whether you could (gpg-)sign the new wordlist (just like the other list was signed by Mr. Reinhold. That might simply make the list more trustworthy and people should be able to check whether their local list was modified by someone other than you.

It would be nice to have automatic coverage detection with tox (and maybe with Travis CI).

Right now, when running tox, we get a flake8 check and tests are run for all supported platforms.
Test coverage is not run, although we can get the test coverage manually with something like:

$ py.test --cov=diceware

I do not like to register with coveralls, as coverage tests should also be run locally. But how can one make that happen?

Any hints are welcome.