Optimistically tuning synchronous byzantine consensus: another win for null messages

“I am prepared for the worst, but hope for the best”

Benjamin Disraeli [9]

Abstract

Modular methods that transform Byzantine consensus protocols for the synchronous model into ones that are fast and communication efficient in failure-free executions are presented. Small and short protocol segments called layers are custom designed to act as a highly efficient preliminary stage that solves Consensus if no failures occur. When composed with a Byzantine consensus protocol of choice, they allow considerable control over the tradeoff in the combined protocol’s behavior in the presence of failures and its performance in their absence. In failure-free executions, they are more efficient than all existing Byzantine consensus protocols. In the presence of failures, they incur a small cost over the complexity of the original consensus protocol being transformed. A key ingredient underlying the efficiency of the new layers is the judicious use of null messages for broadcasting information in failure-free runs. In particular, the notion of a silent validation round, which implements such a broadcast, is defined and used in several ways.

Appendices

Multi-valued variants

In multi-valued Consensus, the decision value domain is commonly defined as $V\cup \{ \bot \}$, where $\bot$ is some default value. A key technical difference between the two Consensus problems is that in the multi-valued case it is sometimes impossible to guarantee that processes decide on a value proposed by a correct process. More precisely, if $\frac{n}{|V|}\le t$ then it is impossible to guarantee that in every execution, all processes decide on some correct proposal.

Many multivalued Byzantine protocols circumvent this issue by having a strong tendency to decide on the default value $\bot$. (These protocols decide $\bot$ in all runs, except when there is a value v that is proposed by at least $n-t$ processes.) In a practical sense, deciding on $\bot$ often means a “no-op” or a “blank” result. As stated in Theorem 4(3), our multivalued layers do not produce this effect. On the contrary, they allow a designer to improve her solution’s time and communication costs while also emulating a fair voting mechanism (plurality) in failure-free runs. Formally, the plurality function $PLUR (\cdot ): V^n\rightarrow V$ is defined as

$$\begin{aligned} PLUR (\mathbf{v} )\triangleq \text {most common}\ v\ \text {in}\ \mathbf{v} . \end{aligned}$$

If several values are tied for the most common value, $PLUR (\mathbf{v} )$ is the minimal such value.

While implementing plurality voting in is desirable in most cases, it may also be desirable not to decide on values that no correct process proposed and to decide on ‘$\bot$’ (no-op) in that case instead. If the designer wishes to guarantee this “correct decision” criteria, she can replace normal plurality with the t-thresholded plurality function $t PLUR (\cdot ): V^n\rightarrow V\cup \{\bot \}$, which we define as

$$\begin{aligned} t PLUR (\mathbf{v} )\triangleq \left\{ \begin{array}{l} \begin{array}{ll} \bot &{} \qquad \text {if no value in}\ \mathbf{v} \ \text {has more}\\ &{} \qquad \text {than}\ t\ \text {occurrences} \\ \end{array} \\ \text {most common}\ v\ \text {in}\ \mathbf{v} \quad \text {otherwise}\\ \end{array}\right\} . \end{aligned}$$

A Consensus protocol that guarantees the “correct decision” criteria in every execution, and emulates t-thresholded plurality in failure-free runs, conforms with plurality as much as possible in these runs.

Algorithms 4 and 3 present $\textsc {GtSn}^{ mv }$ and $\textsc {SlCl}^{ mv }$, which are slight modifications of the layers presented in Sects. 4.1 and 4.2 that handle multi-valued Consensus. As stated before, this is achievable due to the generality of the techniques from Sect. 3 in employing null messages. The new layers differ from the original ones in two minor ways. One is that the majority computation on line 11 of the original layers is replaced by a plurality computation. The other difference is even smaller. For ease of exposition, null messages are used in the first round to encode a single, fixed, proposed value (e.g., the most likely one). No further changes are needed. We remark that in the first round of the layers for binary Consensus we bounded the worst-case message complexity by sending null messages to half of the recipients both when proposing 0 and when proposing 1. It is clearly possible to use null messages selectively for different values in the multi-valued case. But as |V| grows, the advantage of doing so diminishes.

Correctness proofs for the protocols

The Proofs of Theorems 2 and 3 make use of the following lemma:

Lemma 1

Fix a run r of $\textsc {GtSn}\odot \mathtt {Base}$ (resp. $\textsc {SlCl}\odot \mathtt {Base}$). If a correct process i does not decide at time 2 (resp. 3), then all the correct processes participate in the $\mathtt {Base}$ phase from time 3 on (resp. 4 on).

Proof

Let r and i satisfy the assumptions, and let j be a correct process in r. Denote by $\texttt {all}\_\texttt {decided}$ the fact “all correct processes have decided”. Then, line 18 of $\textsc {GtSn}$ (resp. line 23 of $\textsc {SlCl}$) implements an svr(all_decided) in the third round (resp. in the fourth round). Suitably, line 25 (resp. line 26) dictates that j halts and does not participate in the base protocol only if j received no third round (resp. fourth round) messages whatsoever. By line 25 (resp. 26) and Theorem 1 we have that j participates in the base protocol unless it knows at time 3 (resp. time 4) that $\texttt {all}\_\texttt {decided}$ was true at time 2 (resp. 3). Since i does not decide at time 2 (resp. 3) in r, then $\texttt {all}\_\texttt {decided}$ is not true at time 2 (resp. 3). By the knowledge property, j does not know that $\texttt {all}\_\texttt {decided}$ was true at time 2 (resp. 3), because it is false. Consequently, no correct process j halts at time 3 (resp. 4) in r, and they all participate in the $\mathtt {Base.Protocol}$ phase from time 3 (resp. 4) on. $\square$

We now turn to prove the algorithm properties stated in Theorems 2–4. While, for ease of exposition, the Algorithms include messages sent from a process to itself, the analysis will assume that these are implemented without explicit messages being sent.

Theorem 2

Let $k\ge 3$ and let $\mathtt {Base}$ be a binary Consensus protocol for $n>kt$. Then $\textsc {GtSn}\odot \mathtt {Base}$ yields a binary Consensus protocol in which

• 1. In failure-free runs decisions occur after 2 rounds and at most $2n(t+1)$ bits are communicated, while

• 2. When failures cause $\mathtt {Base}$ to be invoked, at most $2n(t+1)+n^2$ bits are sent by correct processes, and 3 rounds elapse before control reverts to $\mathtt {Base}$.

• 3. In failure-free runs, the composed protocol decides on the majority value.

Proof

We now prove that $\textsc {GtSn}\odot \mathtt {Base}$ is a binary Consensus protocol. Fix a run r of $\textsc {GtSn}\odot \mathtt {Base}$. We show that r satisfies Decision, Validity and Agreement:

Decision Let i be a correct process in r. If i decides at time 2 we are done. If it doesn’t, then by Lemma 1 all correct processes participate in the $\mathtt {Base}$ phase. By the Decision property of the $\mathtt {Base}$ protocol, process i completes the execution of line 28 and decides on line 30.

Validity Let i be a correct process in r and assume that all correct processes propose the same value v. Recall that, since $n>3t$ by assumption, the correct processes consist of a strict majority. By the pigeonhole principle, at least $t+1$ Sanhedrin members are correct. These correct Sanhedrin members follow the protocol on lines 06–11 and compute the majority of votes reported to them, which is v, thus, they recommend v to all by lines 12–16. Thereafter, by time 2, every correct process receives at least those $t+1$ recommendations on v and sets its estimation to v either by line 18 (in case of a unanimous recommendation), or by line 21. If i decides at time 2, it decides on its estimation v by line 18, and we are done. Assume it didn’t, then by Lemma 1i and all other correct process participate in the base protocol on line 28. As we have shown, the estimation of all correct processes is set to v on lines 18 and 21. Thus, all correct processes enter the base protocol with a proposal of v. From Validity of the base protocol, this ensures that i performs $\textsf {dec}\leftarrow v$ on line 28 and decides on v in line 30.

Agreement Let i and j be correct processes in r. Assume w.l.o.g. that i decides no later than j. If i does not decide at time 2 then both it and j participate in the base protocol and decide according to it. In particular, their decisions satisfy Agreement. Let us assume that i decides at time 2 on v. Specifically, line 18 is the only line in which a correct process decides at time 2. A correct process (such as i) decides in line 18 iff it received a unanimous recommendation on v. Recall that every unanimous recommendation includes a report of at least $t+1$ correct processes. It follows that every correct process receives at least $t+1$ recommendations on v and therefore sets its estimation to v in lines 18 or 21. Moreover, since no unanimous recommendation on $u\ne v$ is possible, if j also decides at time 2, then it decides on v as well, and Agreement holds. If j does not decide at time 2, then, by Lemma 1, all correct processes participate in the $\mathtt {Base.Protocol}$ phase. And, since all correct processes fixed their estimations to v at time 2, they all enter the base protocol with $est =v$. The Validity of the base protocol ensures that j will decide v in line 30, upholding Agreement.

• 1. In a failure-free run at time 0 every process transmits its proposal to half of the Sanhedrin by silence and the other half by messages. Sanhedrin members have one less message to send in half the cases (to themselves), thus at most $n\lceil (2t+1)/2\rceil -\lfloor (2t+1)/2\rfloor \le n(t+1)$ bits are sent in total during the first round. Since no failures occur $MAJ (\texttt {values}_j)$ is the same for every Sanhedrin member $j\in \{0,\ldots ,2t\}$ and they all recommend the same value $v= MAJ (\texttt {values})$. At time 1, by lines 12–16, Sanhedrin members send their recommendations on v to at least half of the processes by silence and the other part by messages. Thus, the Sanhedrin sends at most a total of $(2t+1)\lceil (n-1)/2\rceil \le n(t+1)$ bits in the second round. The unanimous recommendation of the second round causes every $i\in {\mathbb {P}}$ to decide v at time 2 by line 18, remain silent in the third round and halt at time 3 by line 26.

• 2. Again, correct processes send their proposals to the Sanhedrin at a cost of at most $n(t+1)$ bits in the first round, and correct Sanhedrin members send their recommendations to processes with a total cost of at most $n(t+1)$ bits in the second round. The difference lays in the third round, when correct processes may not receive a unanimous recommendation and would therefore send !‘help! messages (that can be implemented using a single bit) by line 24. This costs in the worst case $n(n-1)$ bits. After this, all remaining communication is due to the base protocol.

• 3. In a failure-free run, all processes transmit their proposals according to protocol at time 0 and a Sanhedrin member calculates its majority at time 1 on lines 06–11. The majority value v is unique and therefore all Sanhedrin members send the same recommendations of v by lines 12–16 at time 1. All processes receive the unanimous recommendation on v by time 2 and therefore decide on it in line 18.

$\square$

Theorem 3

Let $k\ge 3$ and let $\mathtt {Base}$ be a binary Consensus protocol for $n>kt$. Then $\textsc {SlCl}\odot \mathtt {Base}$ yields a binary Consensus protocol in which

• 1. In failure-free runs, decisions occur after 3 rounds and at most $n(t+1.5)$ bits are communicated, while

• 2. When failures cause $\mathtt {Base}$ to be invoked, at most $n(t+1.5)+2n^2$ bits are sent by correct processes, and 4 rounds elapse before control reverts to $\mathtt {Base}$.

• 3. In failure-free runs, the composed protocol decides on the majority value.

Proof

We now prove that $\textsc {SlCl}\odot \mathtt {Base}$ is a binary Consensus protocol. Fix a run r of $\textsc {SlCl}\odot \mathtt {Base}$. We show that r satisfies Decision, Validity and Agreement:

Decision Let i be a correct process in r. If i decides at time 3 we are done. If it doesn’t, then by Lemma 1 all correct processes participate in the base protocol. By the Decision property of the base protocol, process i completes the execution of line 29 and decides on line 31.

Validity Let i be a correct process in r and assume that all correct processes propose the same value v. Recall that, since $n>3t$ by assumption, the correct processes consist of a strict majority. By the pigeonhole principle, at least one process $j_c\in \left\{ 0,1,2,\ldots ,t\right\}$ is correct. Process $j_c$ follows the protocol and at time 1 on lines 06–11 it computes the majority of votes as reported to it. This value is v, obviously. Consequently, by lines 12–16 $j_c$ recommends v in the second round. Thereafter, at time 2 every correct process receives $j_c$’s recommendation on v and therefore sets its estimation to v ($est \leftarrow v$), either in line 18 due to a unanimous recommendation, or in line 20 because its own initial proposal is v. If i decides at time 3, by line 23 it decides on its estimation, which we have established is v. The only other option for i to decide is on line 31 by using the base protocol. It remains to show that if i decides using the base protocol, then its decision is also v. Assume that i decides using the base protocol. Since i is a correct process that does not decide at time 3, by Lemma 1 all correct processes participate in the base protocol. As we have shown, the estimation of every correct process is set to v at time 2 by lines 18 and 20, and so all correct processes enter the base protocol on line 29 at time 4 with the proposal v. The Validity of the base protocol ensures that i sets $\textsf {dec}\leftarrow v$ on line 29 and that i decides v on line 31. Hence, we are done.

Agreement Let i and j be correct processes in r. Assume w.l.o.g. that i decides no later than j. If i does not decide at time 3 then both it and j participate in the base protocol and decide according to it. In particular, their decisions satisfy Agreement. Let’s assume that i decides at time 3 on a value v. The third round of layer $\textsc {SlCl}$ (line 21), implements a silent validation round for the global fact ${\bar{\varphi }_{c}^{}}\triangleq$“a unanimous recommendation was received by all correct processes.” The svr information transfer guarantees of Theorem 1 and line 22 at time 3, imply that i decides at time 3 only if ${\bar{\varphi }_{c}^{}}$ was true at time 2. In particular,if i decides in line 23, then it decides on its estimate value ($est _{i}=v$). Recall that every unanimous recommendation includes at least one correct process’ recommendation which it recommended to all. It follows that if two correct processes receive unanimous recommendations, then these recommendations are the same. Thus, the ${\textsf {svr}({\bar{\varphi }_{c}^{}})}$ in the third round informs i that all correct processes have their estimations set to v. If j also decides at time 3 (in line 23), then it decides on its estimation v, and Agreement holds. If j does not decide at time 3, then, by Lemma 1, all correct processes participate in base protocol. Moreover, since all correct processes have the same estimate v, they all propose v to the base protocol in line 29. Validity of the base protocol guarantees that $\textsf {dec}\leftarrow v$ in line 29 and j decides v by line 31, ensuring Agreement.

• 1. In a failure-free run at time 0 every process transmits its proposal to half of the Council by silence and to the other half by messages. Council members have one less message to send in half the cases (to themselves), thus at most a total of $n\lceil (t+1)/2\rceil -\lfloor (t+1)/2\rfloor \le n(t+2)/2 -t/2$ bits are sent during the first round. Since no failures occur $MAJ (\texttt {values}_j)$ is the same for every Council member $j\in \{0,\ldots ,t\}$ and they all recommend the same value $v= MAJ (\texttt {values})$. At time 1, lines 12–16, Council members send their recommendation on v to at least half of the processes by silence and to the other part by messages. Thus, sending at most a total of $(t+1)\lceil (n-1)/2\rceil \le n(t+1)/2$ bits in the second round. The unanimous recommendation on v of the second round causes every $i\in {\mathbb {P}}$ to set its estimate to $est _i\leftarrow v$ at time 2, and remain silent in the third round. Thus, in a failure-free run, no message is sent in the third round. At time 3, no message is received and in particular no ‘$\overline{{ err}}$’ message, therefore, every process decides on its estimate, remains quiet in the fourth round and halts at time 4. In conclusion, the total number of messages/bits sent in a failure-free run of $\textsc {SlCl}\odot \mathtt {Base}$ is at most $n(t+2)/2 -t/2 + n(t+1)/2<n(t+1.5)$.

• 2. Again, correct processes send their proposals to the Council at a total cost of at most $n(t+2)/2 -t/2$ bits in the first round, and correct Council members send their recommendations to processes with a total cost of at most $n(t+1)/2$ bits in the second round. The difference lies in the third and fourth rounds, when correct processes may not receive a unanimous recommendation and would therefore send ‘$\overline{{ err}}$’ messages by line 21 in the third round and !‘help! messages by line 25 in the fourth round (each message can be implemented using a single bit). In the worst case, this adds a total cost of $2n(n-1)$ bits in the third and fourth rounds. After this, starting at time 4, all remaining communication is due to the base protocol.

• 3. In a failure-free run, all processes transmit their proposals according to protocol at time 0. A Council member calculates the correct majority value v at time 1 on lines 06–11. The majority value v is unique and therefore all Council members send the same recommendation v at time 1 by lines 12–16. All processes receive the unanimous recommendation on v by time 2 and therefore set their estimation to v by line 18 and remain silent. In the third round, no messages are sent in a failure-free run, in particular no ‘$\overline{{ err}}$’ messages. Therefore, every process decides in line 23 on its estimate v which is the majority value.

$\square$

Theorem 4

Let $k\ge 3$, and let $\mathtt {Base}$ be a multi-valued Consensus protocol for $n>kt$. Then composing each of $\textsc {GtSn}^{ mv }$ and $\textsc {SlCl}^{ mv }$ with $\mathtt {Base}$ yields a multi-valued Consensus protocol. Moreover,

• 1. In failure-free runs of $\textsc {GtSn}^{ mv }$ (resp. $\textsc {SlCl}^{ mv }$) decisions occur after 2 (resp. 3) rounds, and at most $4n(t+1)\log _2|V|$ (resp. $2n(t+1)\log _2|V|$) bits are communicated, while

• 2. When failures cause $\mathtt {Base}$ to be invoked, at most $4n(t+1)\log _2|V|+n^2$ (resp. $2n(t+1)\log _2|V|+2n^2$) bits are sent in total by correct processes, and 3 (resp. 4) rounds elapse before control reverts to $\mathtt {Base}$.

• 3. In a failure-free run, both protocols are guaranteed to decide on a plurality value.

Proving $\textsc {GtSn}^{ mv }\odot \mathtt {Base}$ (resp. $\textsc {SlCl}^{ mv }\odot \mathtt {Base}$) is a Consensus protocol stems directly from the proof for $\textsc {GtSn}\odot \mathtt {Base}$ in Theorem 2 (resp. $\textsc {SlCl}\odot \mathtt {Base}$ in Theorem 3). The only minor modification is in Validity, replacing majority with plurality. However, since when all correct processes propose the same value v both plurality and majority have the same result, this modification is insignificant. Hence, Validity is maintained for the multi-valued as well. We are thus left only with proving the rest:

Proof

(for $\textsc {GreatSanhedrin}^{ mv }\odot \mathtt {Base}$)

• 1. In a failure-free run at time 0 every process transmits its proposal to all Sanhedrin members by messages or silence (a messages can encode any value by at most $\log _2|V|$ bits). Sanhedrin members have one less message to send (to themselves). A message encodes a value by $\log _2|V|$ bits. Thus a total of at most $(n-1)(2t+1)\log _2|V|$ bits are sent during the first round. Since no failures occur $PLUR (\texttt {values}_j)$ is the same for every Sanhedrin member $j\in \{0,\ldots ,2t\}$ and they all recommend the same value $v= PLUR (\texttt {values})$. At time 1, Sanhedrin members send their recommendations of v to all processes. Thus, sending at most $(2t+1)(n-1)\log _2|V|$ bits in the second round. The unanimous recommendation of the second round causes every $i\in {\mathbb {P}}$ to decide v at time 2 by line 17, remain silent during the third round and halt at time 3 by line 25.

• 2. Again, correct processes send their proposals to the Sanhedrin at a total cost of at most $(n-1)(2t+1)\log _2|V|$ bits in the first round, and correct Sanhedrin members send their recommendations to processes with a cost of at most $(2t+1)(n-1)\log _2|V|$ bits in the second round. The difference lies in the third round, when correct processes may not receive a unanimous recommendation and would therefore send !‘help! messages (that can be implemented using a single bit) by line 23. This costs in the worst case $n(n-1)$ bits. After this, all remaining communication is due to the base protocol.

• 3. In a failure-free run, all processes transmit their proposals according to protocol at time 0 and a Sanhedrin member calculates their plurality at time 1 on lines 05–10. The plurality value v is unique (a known tie-breaker exists), and therefore all Sanhedrin members send the same recommendations on v by lines 11–15 at time 1. All processes receive the unanimous recommendation on v by time 2 and therefore decide on it in line 17.

$\square$

Proof

(for $\textsc {SmallCouncil}^{ mv }\odot \mathtt {Base}$)

• 1. In a failure-free run at time 0 every process sends its proposal to the Council by an explicit or a null messages (an explicit message can encode any value by at most $\log _2|V|$ bits). Council members have one less message to send (to themselves), thus at most $(n-1)(t+1)\log _2|V|$ bits are sent during the first round. Since no failures occur $PLUR (\texttt {values}_j)$ is the same for every Council member $j\in \{0,\ldots ,t\}$ and they all recommend the same value $v= PLUR (\texttt {values})$. At time 1, lines 10–15, Council members send their recommendation of v to all the processes. Thus, the number of bits sent in the second round is at most $(t+1)(n-1)\log _2|V|$. The unanimous recommendation of v in the second round causes every $i\in {\mathbb {P}}$ to set its estimate to $est _i\leftarrow v$ at time 2, and remain silent during the third round. Thus, in a failure-free run, no message is sent in the third round. At time 3, no message is received and in particular no ‘$\overline{{ err}}$’ message, therefore, every process decides on its estimate, remains quiet in the fourth round and halts at time 4.

• 2. Again, correct processes send their proposals to the Council at a total cost of at most $(n-1)(t+1)\log _2|V|$ bits in the first round, and correct Council members send their recommendations to processes with a total cost of at most $(t+1)(n-1)\log _2|V|$ bits in the second round. The difference lies in the third and fourth rounds, when correct processes might not receive a unanimous recommendation and would therefore send ‘$\overline{{ err}}$’ messages by line 20 in the third round and !‘help! messages by line 24 in the fourth round (each of these messages can be implemented using a single bit). In the worst case, this adds a total cost of $2n(n-1)$ bits in the third and fourth rounds. After this, starting at time 4, all remaining communication is due to the base protocol.

• 3. In a failure-free run, all processes transmit their proposals according to protocol at time 0 and a Council member calculates their plurality at time 1 (on lines 05–10). The plurality value v is unique (a known tie-breaker exists), and therefore all Council members send the same recommendations of v by lines 11–15 at time 1. All processes receive the unanimous recommendation of v by time 2 and by line 17 set their estimate to v and remain silent. Thereafter, at time 3 by line 22 the processes decide on v.

$\square$