Password guessers under a microscope: an in-depth analysis to inform deployments

Abstract

Password guessers are instrumental for assessing the strength of passwords. Despite their diversity and abundance, comparisons between password guessers are limited to simple success rates. Thus, little is known on how password guessers can best be combined with or complement each other. To extend analyses beyond success rates, we devise an analytical framework to compare the types of passwords that guessers generate. Using our framework, we show that different guessers often produce dissimilar passwords, even when trained on the same data. We leverage this result to show that combinations of computationally cheap guessers are as effective in guessing passwords as computationally intensive guessers, but more efficient. Our framework can be used to identify combinations of guessers that will best complement each other. To improve the success rate of any guesser, we also show how an effective training dataset can be identified for a given target password dataset, even when the target dataset is hashed. Our insights allow us to provide a concrete set of practical recommendations for password checking to effectively and efficiently measure password strength.

Appendix

1.1 Appendix A: preliminary analyses and visualization

Figure 5 captures the average success rates for various pairs of training and testing datasets. One can make two important observations: (1) some datasets (e.g., Twitter, Mate1) are more effective as training data than others (e.g., Webhost); (2) some pairs of datasets are effective for training and testing against each other, i.e., when one dataset can train guessers well against another dataset (e.g., RockYou-Mate1, ClixSense-Mate1, etc.). These two observations motivate us towards a deeper analysis of the characteristics of effective training datasets, discussed in Sect. 4.2.

1.2 Appendix B: proofs

Proof of Proposition 1

By Lemma 1 and Lemma 2, the generalized Jaccard index between the password list A and unhashed password list B (which is not accessible) can be computed by:

$$\begin{aligned} J(A,B)= \frac{\sum \limits _{{w \in {{\,\mathrm{supp}\,}}{A}}} \mathrm{min}\left( o(w,A), o(w, B)\right) }{|A| + |B|- \sum \limits _{{w \in {{\,\mathrm{supp}\,}}{A}}} \mathrm{min}\left( o(w,A), o(w, B)\right) }, \nonumber \\ \end{aligned}$$

(13)

Defining $g(w, B_h) = \sum _{y\in B_h}\mathbbm {1}[y = H(w+s_y)]$ for counting the number of occurrences of password w in the salted & hashed password list $B_h$, we note that $o(w,B)=g(w,B_h)$ and |B| = $|B_h|$. So Eq. 13 is equivalent to:

$$\begin{aligned} J(A,B_h)= \frac{\sum \limits _{{w \in {{\,\mathrm{supp}\,}}{A}}} \mathrm{min}\left( o(w,A), g(w, B_h)\right) }{|A| + |B_h|- \sum \limits _{{w \in \varOmega \left( A\right) }} \mathrm{min}\left( o(w,A), g(w, B_h)\right) }. \end{aligned}$$

Letting $F_{\!{\mathrm{min}}}(A,B_h) = \sum \limits _{{w \in {{\,\mathrm{supp}\,}}{A}}} \mathrm{min}\left( o(w,A), g(w, B_h)\right)$, we derive Eq. 12. $\square$

Lemma 1

Let o(w, A) and o(w, b) be the number of occurrences of password w in password lists A and B, respectively. We have

$$\begin{aligned}&\sum \limits _{\qquad \quad {w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) =|A| + |B| \\&\quad - \sum \limits _{{w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} \mathrm{max}\left( o(w,A), o(w, B)\right) . \end{aligned}$$

Here $|A|= \sum _{w \in \varOmega \left( A\right) } o(w,A)$ and $|B| = \sum _{w \in \varOmega \left( B\right) } o(w,B)$ are the number of passwords in A and B respectively. Also, $\varOmega \left( A\right)$ is the set of unique passwords in A.

Proof

One can observe that for any two numbers a and b: $min\left( a, b\right) +max\left( a, b\right) = a + b.$ Using this equality, we can derive

$$\begin{aligned}&\sum \limits _{{w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} \Big [\mathrm{min}\left( o(w,A), o(w,B)\right) \\&\quad + \mathrm{max}\left( o(w,A), o(w,B)\right) \Big ]\\&\quad =\sum \limits _{{w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} o(w,A) + o(w, B) \\&\quad =\sum \limits _{{w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }\quad } o(w,A)\\&\quad + \sum \limits _{\quad {w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} o(w, B)\\&\quad =\sum \limits _{{w \in \varOmega \left( A\right) }} o(w,A)\\&\quad + \sum \limits _{w \in \varOmega \left( B\right) } o(w, B). \end{aligned}$$

The last equality holds as $o(w,A)=0$ when $w \notin A$ and $o(w,B)=0$ when $w \notin B$. By decomposing the first summation, we have shown

$$\begin{aligned}&\sum \limits _{\qquad \quad {w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) \\&\quad + \sum \limits _{{w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} \mathrm{max}\left( o(w,A), o(w, B)\right) =|A| + |B|, \end{aligned}$$

where $|A|= \sum _{w \in \varOmega \left( A\right) } o(w,A)$ and $|B| = \sum _{w \in \varOmega \left( B\right) } o(w,B)$. By rearranging the terms of this equality, we derive

$$\begin{aligned}&\sum \limits _{\qquad \quad {w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) =|A| + |B| \\&\quad - \sum \limits _{{w \in \left( \varOmega \left( A\right) \cup \varOmega \left( B\right) \right) }} \mathrm{max}\left( o(w,A), o(w, B)\right) . \end{aligned}$$

$\square$

Lemma 2

Letting o(w, A) and o(w, b) be the number of occurrences of password w in password lists A and B, respectively,

$$\begin{aligned}&\sum _{\qquad \quad {w \in \varOmega \left( A\right) \cup \varOmega \left( B\right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) \nonumber \\&\quad =\sum \limits _{{w \in {{\,\mathrm{supp}\,}}{A}}} \mathrm{min}\left( o(w,A), o(w, B)\right) ,&\end{aligned}$$

(14)

where $\varOmega \left( A\right)$ is the set of unique passwords in A.

Proof

Partitioning $\varOmega \left( A\right) \cup \varOmega \left( B\right)$ to two disjoint sets of $\varOmega \left( A\right)$ and $\varOmega \left( B\right) -\varOmega \left( A\right)$, we have

$$\begin{aligned}&\sum _{{w \in \varOmega \left( A\right) \cup \varOmega \left( B\right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) \\&\quad =\sum \limits _{{w \in \varOmega \left( A\right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) \\&\qquad + \sum \limits _{{w \in \varOmega \left( B\right) -\varOmega \left( A\right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) . \end{aligned}$$

As $o(w,A)=0$ for $w \in \varOmega \left( B\right) -\varOmega \left( A\right)$, we have $min\left( o(w,A), o(w, B)\right) =0$ for all $w \in \varOmega \left( B\right) -\varOmega \left( A\right)$. So we have derived

$$\begin{aligned}&\sum _{{w \in \varOmega \left( A\right) \cup \varOmega \left( B\right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) \\&\quad =\sum \limits _{{w \in \varOmega \left( A\right) }} \mathrm{min}\left( o(w,A), o(w, B)\right) . \end{aligned}$$

$\square$