Multi-Class Cost-Constrained Random Coding for Correlated Sources over the Multiple-Access Channel

Abstract

This paper studies a generalized version of multi-class cost-constrained random-coding ensemble with multiple auxiliary costs for the transmission of N correlated sources over an N-user multiple-access channel. For each user, the set of messages is partitioned into classes and codebooks are generated according to a distribution depending on the class index of the source message and under the constraint that the codewords satisfy a set of cost functions. Proper choices of the cost functions recover different coding schemes including message-dependent and message-independent versions of independent and identically distributed, independent conditionally distributed, constant-composition and conditional constant composition ensembles. The transmissibility region of the scheme is related to the Cover-El Gamal-Salehi region. A related family of correlated-source Gallager source exponent functions is also studied. The achievable exponents are compared for correlated and independent sources, both numerically and analytically.

1. Introduction

In information theory, the fundamental problem of communication over a channel is studied from two complementary perspectives. First, one characterizes the transmissibility conditions, namely the circumstances under which the error probability asymptically vanishes as the blocklength goes to infinity. Second, one describes by means of error exponents the speed at which this error probability vanishes; the larger the exponent, the faster the error probability tends to zero. Since finding an exact expression for error probability is very difficult, a large body of work has investigated upper and lower bounds on the average error probability, or equivalently lower and upper bounds for the error exponent. In point-to-point, that is, single-user communication, using separate source-channel random coding [1,2], possibly with expurgation [1] (Eq. 5.7.10), yields lower bounds on the error exponent. In contrast, finding an upper bound to the error exponent satisfied by every code is more challenging. Generally, the hypothesis-testing method [3] is employed to derive upper bounds for the error exponent. Two well-known upper bounds to the error exponent are the sphere-packing exponent [4] and the minimum-distance exponent [5]. In fact, for rates greater than critical rate [1] (Sec. 5.6), the random-coding and sphere-packing bounds coincide with each other, while the expurgated and minimum-distance bounds coincide at rate zero.

For point-to-point communication, it was shown in ref. [1] (Prob. 5.16) that joint source-channel coding leads in general to a larger exponent than separate source-channel coding. Additionally, using codewords with a composition dependent on the source message leads to a larger exponent than the case where codewords are drawn according to a fixed product distribution [6,7]. Moreover, a scheme where source messages are assigned to disjoint classes and encoded by codes that depend on the class index, attains the sphere-packing exponent in those cases where it is tight [8].

Many works have been devoted to studying the transmissibility and the error exponent for a two-user multiple-access channel (MAC) [9,10,11]. Separate source-channel coding for the MAC with independent sources was studied in refs. [9,12]. In ref. [13], a universal exponent for the MAC was derived by considering separate source-channel coding. In ref. [14], a transmissible region is derived for the MAC under mismatched decoding, where the decoding rule is fixed and possibly suboptimal. In ref. [15], it was shown that using structure coding can improve the error exponent of the MAC. The maximum-error-probability criterion and the impact of feedback for the MAC were studied in ref. [16]. By considering separate source-channel coding, lower and upper bounds for the error exponent of the MAC were respectively obtained in refs. [17,18]. For the MAC with independent sources, the idea of exploting the dependency between messages and codewords was studied in ref. [19]. In ref. [20], an achievable exponent for the MAC with independent sources was given in the dual domain, that is, as a lower dimensional problem over parameters in terms of Gallager functions. For the MAC with correlated sources, it was shown in ref. [11] that considering statistical dependency between messages and codewords for the MAC with correlated sources leads to a larger transmissible region. However, an example presented in ref. [21] shows that one can reliably transmit information through the MAC without satisfying the reliable transmission obtained in ref. [11]. In another line of work, superposition coding with Gacs Körner Witsenhausen (GKW) common part is used in ref. [22] to to describe the sufficient conditions lossless recoverability.

In contrast to single-user communication, the problem of reliable transmission of two correlated sources has not been solved yet and just the sufficient conditions of a reliable transmission has been derived. In ref. [23], by applying coding techniques, a new set of sufficient conditions were proposed. Moreover, in ref. [24] new sufficient conditions for the three-user MAC with correlated sources were studied. In ref. [25], an achievable exponent derived was presented in the primal domain, that is, as a multi-dimensional optimization problem over distributions that is generally difficult to analyze.

In this paper, we examine how statistical dependency between the messages and codewords improves the exponent, as well as its impact on the transmissibility region. In view of refs. [1] (Ch. 7) and [26], we study a generalized message-dependent cost-constrained random-coding ensemble with multiple cost functions. By choosing the proper cost functions, the multi-class cost-constrained ensemble subsumes multiple ensembles previously considered in the literature and recovers the transmissibility region in ref. [11].

The paper is organized as follows—in Section 2, we present the problem of transmission of N correlated sources over an N-input discrete memoryless multiple-access channel and provide the key definitions of error probability, transmissibility, random-coding ensemble, and achievable exponent. In Section 3, we review the existing random-coding ensembles, define a novel generalized multi-class cost-constraint ensemble and characterize its achievable exponent. In the discussion Section 4, we characterize the transmissibility region for our error exponent, relate the exponent to standard Gallager source and channel functions, and provide numerical results and formulas that allow us to rank the exponents attained by the various standard random-coding ensembles.

2. Problem Formulation

We study the simultaneous transmission of N correlated, discrete, memoryless sources over a channel; users are indexed by $\nu \in \mathcal{N}=\{1,2,\dots ,N\}$. The source messages ${\mathit{u}}_{\nu }$ of user $\nu$ have n symbols drawn from the alphabet ${\mathcal{U}}_{\nu }$. We denote by ${\mathit{u}}_{\sigma }$ the ordered vector of source messages for all users in a set $\sigma \subset 2^{\mathcal{N}}$, i.e., a subset of the set of all user indices, and similarly by ${\mathcal{U}}_{\sigma }$ the Cartesian product of the source alphabets in the set $\sigma$. When $\sigma =\mathcal{N}$, ${\mathit{u}}_{\mathcal{N}}$ and ${\mathcal{U}}_{\mathcal{N}}$ denote the ordered vector of source messages for all users and the Cartesian product of the all source alphabets respectively. The sources are memoryless and are characterized by the joint probability distribution ${\mathrm{P}}_{\mathcal{N}}$

$$\begin{array}{c}\hfill {\mathrm{P}}_{\mathcal{N}}\left({\mathit{u}}_{\mathcal{N}}\right)=\prod _{t=1}^n{P}_{\mathcal{N}}\left(u_{\mathcal{N},t}\right),\end{array}$$

(1)

and by the symbol joint probability distribution $P_{\mathcal{N}}$. The source message and symbol marginal distributions of user $\nu \in \mathcal{N}$ are denoted by ${\mathrm{P}}_{\nu }$ and $P_{\nu }$ respectively. Assuming that the sources are independent, the marginal distributions induce new joint (mismatched) probability distributions of sets of users $\sigma \subset 2^{\mathcal{N}}$. The induced independent-message and -symbol probabilities, denoted by ${\mathrm{P}}_{\sigma }^{\mathrm{ind}}$ and $P_{\sigma }^{\mathrm{ind}}$, are given by

$$\begin{array}{c}\hfill {\mathrm{P}}_{\sigma }^{\mathrm{ind}}\left({\mathit{u}}_{\sigma }\right)=\prod _{\nu \in \sigma }{\mathrm{P}}_{\nu }\left({\mathit{u}}_{\nu }\right),\end{array}$$

(2)

and similarly for $P_{\sigma }^{\mathrm{ind}}$.

Each user $\nu$ has an encoder that maps, without cooperation with the other users, the source message ${\mathit{u}}_{\nu }$ onto a codeword ${\mathit{x}}_{\nu }\left({\mathit{u}}_{\nu }\right)$ also of length n and with symbols drawn from the alphabet ${\mathcal{X}}_{\nu }$. We denote the codebook of user $\nu$ by ${\mathcal{C}}_n^{\nu }$. We denote by ${\mathit{x}}_{\sigma }\in {\mathcal{X}}_{\sigma }^n$ the vector of codewords for all users in a set $\sigma \subset 2^{\mathcal{N}}$. Both terminals simultaneously send these codewords over a discrete memoryless multiple access channel with output alphabet $\mathcal{Y}$. The symbolwise transition probability is denoted by W, and the channel is characterized by a conditional probability distribution

$$\begin{array}{c}\hfill \mathrm{W}\left(\mathit{y}\right|{\mathit{x}}_{\mathcal{N}})=\prod _{t=1}^{n}W\left(y_t\right|x_{\mathcal{N},t}),\end{array}$$

(3)

where $\mathit{y}$ is the received sequence of length n.

Based on $\mathit{y}$, a joint decoder estimates all transmitted source messages ${\mathit{u}}_{\mathcal{N}}$ according to the maximum a posteriori criterion:

$$\begin{array}{c}\hfill {\widehat{\mathit{u}}}_{\mathcal{N}}=\underset{{\mathit{u}}_{\mathcal{N}}\in {\mathcal{U}}_{\mathcal{N}}^n}{arg\; max}{\mathrm{P}}_{\mathcal{N}}\left({\mathit{u}}_{\mathcal{N}}\right)\mathrm{W}\left(\mathit{y}|{\mathit{x}}_{\mathcal{N}}\left({\mathit{u}}_{\mathcal{N}}\right)\right),\end{array}$$

(4)

where ${\mathcal{U}}_{\mathcal{N}}^n$ denotes the set of all possible source messages ${\mathit{u}}_{\mathcal{N}}$. An error occurs if the decoded messages ${\widehat{\mathit{u}}}_{\mathcal{N}}$ differ from the transmitted ${\mathit{u}}_{\mathcal{N}}$; we refer to ${\widehat{\mathit{u}}}_{\mathcal{N}}\ne {\mathit{u}}_{\mathcal{N}}$ as an error event. The error probability for a given set of codebooks, $P_{\mathrm{e}}\left({\mathcal{C}}_n^{\mathcal{N}}\right)$, is thus given by

$$\begin{array}{c}\hfill P_{\mathrm{e}}\left({\mathcal{C}}_n^{\mathcal{N}}\right)\triangleq Pr\left\{{\widehat{\mathit{U}}}_{\mathcal{N}}\ne {\mathit{U}}_{\mathcal{N}}\right\}.\end{array}$$

(5)

In our analysis, it will prove convenient to split the error event into $2^N-1$ distinct types of error events indexed by the non-empty subsets in the power set of the user indices $2^{\mathcal{N}}\backslash \varnothing$, for example, $\tau \in \left\{\right\{1\},\{2\},\{1,2\left\}\right\}$ for $N=2$. More precisely, the error event of type $\tau$ corresponds to the conditions ${\widehat{\mathit{u}}}_{\nu }\ne {\mathit{u}}_{\nu }$ for all $\nu \in \tau$ and ${\widehat{\mathit{u}}}_{\nu }={\mathit{u}}_{\nu }$ for all $\nu \in {\tau }^c$, where ${\tau }^c$ is the complement of $\tau$ in the power set of the user indices.

We are interested in the asymptotics of the error probability for sufficiently large n, namely whether the error probability vanishes and how fast this probability tends to zero as it vanishes. The sources $U_{\mathcal{N}}$ are said to be transmissible over the channel if there exists a sequence of codebooks ${\mathcal{C}}_n^{\mathcal{N}}$ such that ${lim}_{n\to \infty }{P}_{\mathrm{e}}\left({\mathcal{C}}_n^{\mathcal{N}}\right)=0$. To characterize the speed at which the error probability vanishes, we use the notion of exponent. An exponent $\mathrm{E}$ is said to be achievable if there exists a sequence of codebooks such that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; inf}-{\displaystyle \frac{1}{n}}log{P}_{\mathrm{e}}\left({\mathcal{C}}_n^{\mathcal{N}}\right)\ge \mathrm{E}.\end{array}$$

(6)

Source transmissibility and error-exponent achievability are typically studied by means of random coding. With random coding, one generates and studies sequences of ensembles of codebooks whose codewords are randomly drawn from a distribution ${\mathrm{Q}}_{\nu }\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })$ independently for each user; as indicated by the notation, this distribution may possibly depend on the source message ${\mathit{u}}_{\nu }$. The random-coding probability distribution for the channel input ${\mathrm{Q}}_{\mathcal{N}}\left({\mathit{x}}_{\mathcal{N}}\right|{\mathit{u}}_{\mathcal{N}})$ combined for all users is given by

$$\begin{array}{c}\hfill {\mathrm{Q}}_{\mathcal{N}}\left({\mathit{x}}_{\mathcal{N}}\right|{\mathit{u}}_{\mathcal{N}})=\prod _{\nu \in \mathcal{N}}{\mathrm{Q}}_{\nu }\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu }).\end{array}$$

(7)

The use of random coding allows us to study how the error probability averaged over the ensemble, denoted by ${\overline{P}}_{\mathrm{e}}$ vanishes as n grows. More importantly, it shows the existence of good codes in the ensemble such that their error probability vanishes. For the point-to-point and the multiple-access channels, a number of such random-coding ensembles have been studied in the literature, as reviewed in the following section, where we also present a multi-class cost-constrained ensemble subsuming all these ensembles and characterize the achievable exponent and transmissibility region of this ensemble.

Summary of Notation Used in the Paper

Sets are usually denoted by calligraphic upper case letter, e.g., $\mathcal{X}$, and the n-Cartesian product set of $\mathcal{X}$ is denoted by ${\mathcal{X}}^n$. The cardinality of a set such as $\mathcal{X}$ is denoted by $\left|\mathcal{X}\right|$. The indicator function representing an error event or that an element x belongs to a set $\mathcal{X}$ is denoted by $\mathbf{1}\{x\in \mathcal{X}\}$.

The number of users is denoted by N and user indices are typically represented by $\nu$. The set of all users is denoted by $\mathcal{N}$. The power set of all subsets of $\mathcal{N}$ is denoted by $2^{\mathcal{N}}$ and the complement of a subset $\sigma \subset 2^{\mathcal{N}}$ is denoted by ${\sigma }^c$; sets in the power set of users are denoted that by Greek letters, for example, $\tau$ and $\sigma$. The number of source-message classes and of cost functions for user $\nu$ are respectively denoted by $K_{\nu }$ and $L_{\nu }$; the sets of such classes are functions are respectively denoted by ${\mathcal{K}}_{\nu }$ and ${\mathcal{L}}_{\nu }$. Indices for source classes and cost functions are typically denoted by $i_{\nu }$ and ${\ell }_{\nu }$ respectively.

Subscripts and superscripts in a quantity A may represent sets of user indices $\sigma$. Depending on the context, the quantity represents a list or a suitable product of variables for all elements in the set $\sigma$. For instance, for $\sigma =\{1,2\}$, $A^{\sigma }=(A^1,A^2)$ or $A_{\sigma }=(A_1,A_2)$. If the quantity is a probability distribution, its value for $\sigma$ represents the probability distribution of the sequence, for example, $Q_{\sigma }^{i_{\sigma }}\left(x_{\sigma }\right)={\prod }_{\nu \in \sigma }{Q}_{\nu }^{i_{\nu }}\left(x_{\nu }\right)$. If the quantity is a set, its value for $\sigma$ is the Cartesian product, for example, ${\mathcal{U}}_{\sigma }={\mathcal{U}}_1\times {\mathcal{U}}_2$ for $\sigma =\{1,2\}$. If $\sigma =\varnothing$, then $A_{\sigma }=A^{\sigma }=0$. If $\sigma$ is a singleton, for example, $\sigma =\left\{2\right\}$, we simply write $A_2$ or $A^2$. We denote the operation that merges and sorts two lists $A_{{\sigma }_1}$ and $A_{{\sigma }_2}$ with ${\sigma }_1\cap {\sigma }_2=\varnothing$ into an ordered list containing all users in the union ${\sigma }_1\cup {\sigma }_2$ by $[A_{{\sigma }_1},A_{{\sigma }_2}]$. For sets of user indices, we denote such merging operation by $[{\sigma }_1,{\sigma }_2]$ and we have $[\sigma ,{\sigma }^c]=\mathcal{N}$.

Scalar random variables are denoted by capital letters, for example, X and lowercase letters represent a particular realisation, for example, $x\in \mathcal{X}$. Capital bold letter denotes random vectors or sequences, for example, $\mathit{X}$, while small bold letter $\mathit{x}\in {\mathcal{X}}^n$ denote deterministic vectors or sequences. Probability distributions for vectors or sequences, typically of length n, (resp. for symbols) are represented by text-style letters, for example, $\mathrm{P}$, $\mathrm{Q}$, $\mathrm{W}$ (resp. math-style letters, for example, P, Q, W). Sequences symbols are usually affixed a subscript to indicate a user index; the t-th symbol in the sequence ${\mathit{x}}_{\nu }$ is denoted by $x_{\nu ,t}$.

The source-symbol distribution for user $\nu$ is denoted by $P_{\nu }\left(u_{\nu }\right)$. The joint distribution for users $\sigma$ is denoted by $P_{\sigma }\left(u_{\sigma }\right)$; the joint distribution, computed as if the sources were independent, is denoted by $P_{\sigma }^{\mathrm{ind}}\left(u_{\sigma }\right)$. The conditional source distribution for users ${\sigma }_1$ given another set ${\sigma }_2$ is denoted by $P_{{\sigma }_1|{\sigma }_2}\left(u_{{\sigma }_1}\right|u_{{\sigma }_2})$. Vector or sequence distributions are defined analogously with P replaced by $\mathrm{P}$. Channel input distributions are denoted by $Q_{\nu }\left(x_{\nu }\right)$, $Q_{\nu }^{i_{\nu }}\left(x_{\nu }\right)$, or $Q_{\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)$, where $i_{\nu }$ denotes the index of the class source message and $Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)$ is a shorthand for the conditional distribution $Q_{\nu }\left(x_{\nu }\right|u_{\nu })$. Cost functions are similarly denoted by $a_{\nu }\left(x_{\nu }\right)$, $a_{\nu }^{i_{\nu }}\left(x_{\nu }\right)$, or $a_{\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)$. Vector or sequence distributions are defined analogously with Q or a respectively replaced by $\mathrm{Q}$ or $\mathrm{a}$. The conditional distribution for the channel output symbol (resp. sequence) is denoted by $W\left(y\right|x_{\mathcal{N}})$ (resp. $\mathrm{W}\left(y\right|x_{\mathcal{N}})$).

3. Multi-Class Cost-Constrained Ensemble with Statistical Dependency

3.1. Review of Random-Coding Ensembles

The simplest and oldest random-coding ensemble is the independent, identically distributed (iid) [1,12,17,27], where the symbols $x_{\nu ,t}$ in all codewords ${\mathit{x}}_{\nu }$ of a given user $\nu$ are generated independently according to the same input distributions $Q_{\nu }\left(x_{\nu ,t}\right)$ for all source messages ${\mathit{u}}_{\nu }$. Throughout the paper, we shall identify ensembles by hyphenated acronyms, where the first part indicates the possible dependence of the codeword on the source message and the second part describes the generation of symbols in a codeword. This first ensemble is thus the message-independent iid (mi-iid) ensemble, since codewords have the same distribution for all source messages and symbols are independent of each other and independent of the source message symbols too. For the mi-iid ensemble, the random-coding distribution is given by

$$\begin{array}{c}\hfill {\mathrm{Q}}_{\nu }^{\mathit{mi}-\mathit{iid}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })=\prod _{t=1}^n{Q}_{\nu }\left(x_{\nu ,t}\right).\end{array}$$

(8)

In the message-independent, independent-conditionally-distributed (mi-icd) ensemble, the codewords ${\mathit{x}}_{\nu }$ of user $\nu$ are generated identically for all source messages ${\mathit{u}}_{\nu }$, independently of the full message ${\mathit{u}}_{\nu }$, and with symbols according to a set of $|{\mathcal{U}}_{\nu }|$ conditional probability distributions $Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)\triangleq Q_{\nu }\left(x_{\nu }\right|u_{\nu })$. To this end, let ${\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)$ denote the set of positions where the symbol $u_{\nu }\in \mathcal{U}$ appears in the sequence ${\mathit{u}}_{\nu }$, namely

$$\begin{array}{cc}\hfill {\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)& =\left\{t\in \{1,2,\dots ,n\}:{\mathit{u}}_{\nu ,t}=u_{\nu }\right\}.\hfill \end{array}$$

(9)

Within each subsequence of ${\mathit{u}}_{\nu }$ where ${\mathit{u}}_{\nu ,t}=u_{\nu }$, represented by ${\mathit{u}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$, symbols are drawn independently according to $Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)$. For this mi-icd ensemble, codewords are generated according to

$$\begin{array}{cc}\hfill {\mathrm{Q}}_{\nu }^{\mathit{mi}-\mathit{icd}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })& =\prod _{u_{\nu }\in {\mathcal{U}}_{\nu }}\prod _{t\in {\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)}{Q}_{\nu ,u_{\nu }}\left(x_{\nu ,t}\right)\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill & =\prod _{t=1}^n{Q}_{\nu ,u_{\nu ,t}}\left(x_{\nu ,t}\right).\hfill \end{array}$$

(11)

Compared to the mi-iid ensemble, the mi-icd ensemble can lead to a larger transmissible region for the multiple-access channel with correlated sources [11,21]. An example of generation of three codewords ${\mathit{x}}_{\nu }^{\left(1\right)}$, ${\mathit{x}}_{\nu }^{\left(2\right)}$ and ${\mathit{x}}_{\nu }^{\left(3\right)}$ in the mi-icd ensemble is shown in Figure 1, for a given source sequence ${\mathit{u}}_{\nu }=(\alpha ,\beta ,\beta ,\gamma ,\beta ,\gamma ,\gamma ,\alpha ,\beta ,\alpha )$ with source alphabet $\mathcal{U}=\{\alpha ,\beta ,\gamma \}$. To generate each codeword ${\mathit{x}}_{\nu }$ with alphabet $\mathcal{X}=\{a,c,e\}$, three subcodewords ${\mathit{x}}_{\nu }({\mathcal{I}}_{\alpha }\left({\mathit{u}}_{\nu }\right)$, ${\mathit{x}}_{\nu }({\mathcal{I}}_{\beta }\left({\mathit{u}}_{\nu }\right)$ and ${\mathit{x}}_{\nu }({\mathcal{I}}_{\gamma }\left({\mathit{u}}_{\nu }\right)$ are pairwise-independently generated with i. i. d. distributions $Q_{\nu ,\alpha }=(1\backslash 3,1\backslash 3,1\backslash 3)$, $Q_{\nu ,\beta }=(1\backslash 2,1\backslash 4,1\backslash 4)$ and $Q_{\nu ,\gamma }=(1\backslash 3,2\backslash 3,0)$, respectively. Symbols generated according to $Q_{\nu ,\alpha }$, $Q_{\nu ,\beta }$ and $Q_{\nu ,\gamma }$ are respectively represented as green circles, blue boxes and red diamonds in the figure. In the example, ${\mathcal{I}}_{\alpha }\left({\mathit{u}}_{\nu }\right)=\{1,8,10\}$, ${\mathcal{I}}_{\beta }\left({\mathit{u}}_{\nu }\right)=\{2,3,5,9\}$ and ${\mathcal{I}}_{\gamma }\left({\mathit{u}}_{\nu }\right)=\{4,6,7\}$. For instance, the subcodeword ${\mathit{x}}_{\nu }^{\left(1\right)}({\mathcal{I}}_{\gamma }\left({\mathit{u}}_{\nu }\right)$ has three symbols, each generated independently from $Q_{\nu ,\gamma }$, leading to the red-diamond symbols ${\mathit{x}}_{\nu }^{\left(1\right)}({\mathcal{I}}_{\gamma }\left({\mathit{u}}_{\nu }\right)=(a,a,a)$.

Next, we have the message-dependent iid (md-iid) ensemble [6,8,19,25,28], where codewords for each user are generated with i. i. d. symbols according to different distributions $Q_{\nu }^{i_{\nu }}\left(x_{\nu }\right)$ that depend on the full source message through the class index $i_{\nu }$ of the class the source message belongs to. More precisely, for user $\nu =1,2$ with source marginal distribution $P_{\nu }$, the $i_{\nu }$-th class ${\mathcal{A}}_{\nu }^{i_{\nu }}$, where $i_{\nu }\in {\mathcal{K}}_{\nu }=\{1,\dots ,K_{\nu }\}$, is defined as the set of all source messages whose probability ${\mathrm{P}}_{\nu }\left({\mathit{u}}_{\nu }\right)$ is within a given interval, that is,

$$\begin{array}{c}\hfill {\mathcal{A}}_{\nu }^{i_{\nu }}=\left\{{\mathit{u}}_{\nu }\in {\mathcal{U}}_{\nu }^n:{\gamma }_{\nu ,i_{\nu }}^n<{\mathrm{P}}_{\nu }\left({\mathit{u}}_{\nu }\right)\le {\gamma }_{\nu ,i_{\nu }-1}^n\right\},\end{array}$$

(12)

where the thresholds ${\gamma }_{\nu ,j}$ are $K_{\nu }+1$ non-negative numbers, ordered from higher to lower, such that $0={\gamma }_{\nu ,K_{\nu }}\le {\gamma }_{\nu ,K_{\nu }-1}\le \dots \le {\gamma }_{\nu ,1}<{\gamma }_{\nu ,0}=1$, and ${min}_{u_{\nu }}{P}_{\nu }\left(u_{\nu }\right)<{\gamma }_{\nu ,K_{\nu }-1}$ and ${\gamma }_{\nu ,1}\le {max}_{u_{\nu }}{P}_{\nu }\left(u_{\nu }\right)$. The md-iid random-coding distribution is given by

$$\begin{array}{c}\hfill {\mathrm{Q}}_{\nu }^{\mathit{md}-\mathit{iid}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })=\prod _{t=1}^n{Q}_{\nu }^{i_{\nu }\left({\mathit{u}}_{\nu }\right)}\left(x_{\nu ,t}\right).\end{array}$$

(13)

The exponent of this md-iid ensemble can be larger than that of the mi-iid ensemble for joint source-channel coding [8,20,28].

In the message-dependent, independent conditional symbol distributions (md-icd) ensemble, messages in the class $i_{\nu }$ for user $\nu$ are encoded with codewords whose symbols are generated independently according to the conditional input distribution $Q_{\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)$. The random-coding distribution of the md-icd ensemble is thus given by

$$\begin{array}{c}\hfill {\mathrm{Q}}_{\nu }^{\mathit{md}-\mathit{icd}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })=\prod _{u_{\nu }\in {\mathcal{U}}_{\nu }}\prod _{t\in {\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)}{Q}_{\nu ,u_{\nu }}^{i_{\nu }\left({\mathit{u}}_{\nu }\right)}\left(x_{\nu ,t}\right).\end{array}$$

(14)

In the message-independent, constant-composition (mi-cc) ensemble [29,30], codewords ${\mathit{x}}_{\nu }$ are drawn independently with an empirical distribution ${\widehat{Q}}_{\nu }\left(x_{\nu }\right)$ close to a given $Q_{\nu }\left(x_{\nu }\right)$, independently of the source message ${\mathit{u}}_{\nu }$. For each user, codewords ${\mathit{x}}_{\nu }$ are randomly picked from ${\mathcal{T}}_{\nu }^n\left(Q_{\nu }\right)$, the set of all sequences whose empirical distribution has a variational distance to $Q_{\nu }$ of at most $1\backslash n$, that is

$$\begin{array}{c}\hfill {\mathcal{T}}_{\nu }^n\left(Q_{\nu }\right)=\left\{{\mathit{x}}_{\nu }\in {\mathcal{X}}_{\nu }^n:\underset{x_{\nu }}{max}\left|{\widehat{Q}}_{\nu }\left(x_{\nu }\right)-Q_{\nu }\left(x_{\nu }\right)\right|<{\displaystyle \frac{1}{n}}\right\}.\end{array}$$

(15)

For this mi-cc ensemble, the random-coding distribution is given by

$$\begin{array}{c}\hfill {\mathrm{Q}}_{\nu }^{\mathit{mi}-\mathit{cc}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })=\frac{1}{|{\mathcal{T}}_{\nu }^n\left(Q_{\nu }\right)|}\mathbf{1}\left\{{\mathit{x}}_{\nu }\in {\mathcal{T}}_{\nu }^n\left(Q_{\nu }\right)\right\}.\end{array}$$

(16)

While the mi-cc and mi-iid ensembles lead to identical transmissibility conditions, the former may achieve strictly larger exponents for suboptimal input distributions already in single-user settings [29].

The message-independent, conditional constant-composition (mi-ccc) ensemble combines features of the mi-icd and mi-cc ensembles. For each subsequence ${\mathit{u}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$, the corresponding subcodewords ${\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$ are drawn independently from the set ${\mathcal{T}}_{\nu }^{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}\left(Q_{\nu ,u_{\nu }}\right)$ of subsequences with empirical distribution close to $Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)$, namely

$$\begin{array}{c}\hfill {\mathcal{T}}_{\nu }^{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}\left(Q_{\nu ,u_{\nu }}\right)=\left\{{\mathit{x}}_{\nu }\in {\mathcal{X}}_{\nu }^{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}:\underset{x_{\nu }\in {\mathit{x}}_{\nu }}{max}\left|{\widehat{Q}}_{\nu }\left(x_{\nu }\right)-Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)\right|<{\displaystyle \frac{1}{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}}\right\}.\end{array}$$

(17)

The random-coding distribution of the mi-ccc ensemble is given by

$$\begin{array}{c}\hfill {\mathrm{Q}}_{\nu }^{\mathit{mi}-\mathit{ccc}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })=\prod _{u_{\nu }\in {\mathcal{U}}_{\nu }}\frac{1}{\left|{\mathcal{T}}_{\nu }^{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}\left(Q_{\nu ,u_{\nu }}\right)\right|}\mathbf{1}\left\{{\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)\in {\mathcal{T}}_{\nu }^{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}\left(Q_{\nu ,u_{\nu }}\right)\right\}.\end{array}$$

(18)

An example of the generation of three codewords ${\mathit{x}}_{\nu }^{\left(4\right)}$, ${\mathit{x}}_{\nu }^{\left(5\right)}$ and ${\mathit{x}}_{\nu }^{\left(6\right)}$ in the mi-ccc ensemble is also shown in Figure 1 as a comparison to the md-iid ensemble, for the same source sequence ${\mathit{u}}_{\nu }$, source alphabet $\mathcal{U}=\{\alpha ,\beta ,\gamma \}$ and input alphabet $\mathcal{X}=\{a,c,e\}$. Now, to generate each codeword ${\mathit{x}}_{\nu }$, three subcodewords ${\mathit{x}}_{\nu }\left({\mathcal{I}}_{\alpha }\left({\mathit{u}}_{\nu }\right)\right)$, ${\mathit{x}}_{\nu }\left({\mathcal{I}}_{\beta }\left({\mathit{u}}_{\nu }\right)\right)$ and ${\mathit{x}}_{\nu }\left({\mathcal{I}}_{\gamma }\left({\mathit{u}}_{\nu }\right)\right)$ are pairwise-independently, uniformly drawn in the type classes with empirical distributions ${\widehat{Q}}_{\nu ,\alpha }$, ${\widehat{Q}}_{\nu ,\beta }$ and ${\widehat{Q}}_{\nu ,\gamma }$ that are closest to $Q_{\nu ,\alpha }$, $Q_{\nu ,\beta }$ and $Q_{\nu ,\gamma }$, respectively. Since in the example $|{\mathcal{I}}_{\alpha }\left({\mathit{u}}_{\nu }\right)|=3$, $|{\mathcal{I}}_{\beta }\left({\mathit{u}}_{\nu }\right)|=4$ and $|{\mathcal{I}}_{\gamma }\left({\mathit{u}}_{\nu }\right)|=3$, it follows that ${\widehat{Q}}_{\nu ,\alpha }=(1\backslash 3,1\backslash 3,1\backslash 3)$, ${\widehat{Q}}_{\nu ,\beta }=(1\backslash 2,1\backslash 4,1\backslash 4)$ and ${\widehat{Q}}_{\nu ,\gamma }=(1\backslash 3,2\backslash 3,0)$. Symbols generated according to ${\widehat{Q}}_{\nu ,\alpha }$, ${\widehat{Q}}_{\nu ,\beta }$ and ${\widehat{Q}}_{\nu ,\gamma }$ are respectively represented as green doubled circles, blue doubled boxes and red doubled diamonds in the figure. For instance, all subcodewords ${\mathit{x}}_{\nu }^{\left(j\right)}({\mathcal{I}}_{\gamma }\left({\mathit{u}}_{\nu }\right))$, for $j=4,5,6$, have three symbols jointly generated from the constant-composition type ${\widehat{Q}}_{\nu ,\gamma }$, that is, exactly one a and two cs.

The message-dependent, constant-composition (md-cc) ensemble combines the features of having different distributions for different messages with constant-composition random coding. For messages in the class $i_{\nu }\in \{1,\dots ,K_{\nu }\}$ for user $\nu$, codewords are drawn from the set of sequences with empirical distribution close to $Q_{\nu }^{i_{\nu }}\left(x_{\nu }\right)$. For this ensemble, the random-coding distribution is given by

$$\begin{array}{c}\hfill {\mathrm{Q}}_{\nu }^{\mathit{md}-\mathit{cc}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })=\frac{1}{\left|{\mathcal{T}}_{\nu }^n\left(Q_{\nu }^{i_{\nu }\left({\mathit{u}}_{\nu }\right)}\right)\right|}\mathbf{1}\left\{{\mathit{x}}_{\nu }\in {\mathcal{T}}_{\nu }^n\right(Q_{\nu }^{i_{\nu }\left({\mathit{u}}_{\nu }\right)})\}.\end{array}$$

(19)

Finally, the message-dependent, conditional constant-composition (md-ccc) ensemble combines several of the ensembles listed above. For a given message ${\mathit{u}}_{\nu }=(u_{\nu ,1},\dots ,u_{\nu ,n})$ in the $i_{\nu }$-th class, that is, ${\mathit{u}}_{\nu }\in {\mathcal{A}}_{\nu }^{i_{\nu }}$, the subsequence of ${\mathit{u}}_{\nu }$ having the same symbol $u_{\nu }$, that is, ${\mathit{u}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$, is encoded with pairwise-independent codewords generated from the set of codewords with empirical distribution very close to $Q_{\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)$. The random-coding distribution of the md-ccc ensemble is thus given by

$$\begin{array}{c}\hfill {\mathrm{Q}}_{\nu }^{\mathit{md}-\mathit{ccc}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })=\prod _{u_{\nu }\in {\mathcal{U}}_{\nu }}\frac{1}{\left|{\mathcal{T}}_{\nu }^{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}\left(Q_{\nu ,u_{\nu }}^{i_{\nu }\left({\mathit{u}}_{\nu }\right)}\right)\right|}\mathbf{1}\left\{{\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)\in {\mathcal{T}}_{\nu }^{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}\left(Q_{\nu ,u_{\nu }}^{i_{\nu }\left({\mathit{u}}_{\nu }\right)}\right)\right\}.\end{array}$$

(20)

3.2. Generalized Multi-Class Cost-Constrained Ensemble

Motivated by the ensembles listed in the previous section, and inspired by refs. [1] (Ch. 7) and [26] (Sec. II), we study a generalized message-dependent multi-class cost-constrained random-coding ensemble with multiple auxiliary costs.

For each user, we partition the set of source messages into $K_{\nu }$ disjoint classes with thresholds on the message probabilities as in Equation (12). Let the source message be in the $i_{\nu }$-th class, that is, $i_{\nu }\left({\mathit{u}}_{\nu }\right)=i_{\nu }$. Given the source message ${\mathit{u}}_{\nu }$ and the source symbol $u_{\nu }$, we consider the subsequence ${\mathit{u}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$, where ${\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)$ is defined in Equation (9), and we denote the corresponding source subsequence and subcodeword by ${\mathit{u}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$ and ${\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$ respectively. For each user $\nu$, class index $i_{\nu }$, and source message symbol $u_{\nu }$, the subcodeword ${\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$ is drawn according to a symbolwise i. i. d. distribution $Q_{\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)$ conditioned on a set of cost constraints being satisfied. We consider $L_{\nu }$ additive cost functions $a_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu }\right)$, ${\ell }_{\nu }\in {\mathcal{L}}_{\nu }=\{1,\dots ,L_{\nu }\}$. The total cost ${\mathrm{a}}_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left({\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)\right)$ of the subcodeword ${\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$ is given by the sum of the symbol costs $a_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}$, namely

$$\begin{array}{c}\hfill {\mathrm{a}}_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left({\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)\right)=\sum _{j\in {\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)}{a}_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu ,j}\right).\end{array}$$

(21)

We assume that the average cost ${\varphi }_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}$ under the conditional distribution $Q_{\nu ,u_{\nu }}^{i_{\nu }}$ is zero:

$$\begin{array}{c}\hfill {\varphi }_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}=\sum _{x_{\nu }\in {\mathcal{X}}_{\nu }}{Q}_{\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)a_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu }\right)=0.\end{array}$$

(22)

Finally, fix some parameters ${\delta }_{\nu }>0$ and let ${\mathcal{D}}_{\nu }^{i_{\nu }}$ be the set of codewords for which the average empirical cost of its constituent subcodewords $\frac{1}{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}{\mathrm{a}}_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left({\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)\right)$ is close to the statistical mean ${\varphi }_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}=0$ for all cost functions and source symbols, i.e.,

$$\begin{array}{c}\hfill {\mathcal{D}}_{\nu ,{\mathit{u}}_{\nu }}^{i_{\nu }}\triangleq \left\{{\mathit{x}}_{\nu }\in {\mathcal{X}}_{\nu }^n:\left|\frac{1}{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}{\mathrm{a}}_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left({\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)\right)\right|\le \frac{{\delta }_{\nu }}{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|},u_{\nu }\in {\mathcal{U}}_{\nu },{\ell }_{\nu }\in {\mathcal{L}}_{\nu }\right\}.\end{array}$$

(23)

Codewords ${\mathit{x}}_{\nu }$ are the combination of subcodewords ${\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)$ with respective positions in ${\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)$. For this multi-class cost-constrained ensemble, the random-coding distribution is thus given by

$$\begin{array}{cc}\hfill {\mathrm{Q}}_{\nu }^{\mathit{cost}}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })& ={\displaystyle \frac{1}{{\Xi }_{\nu }}}\prod _{u_{\nu }\in {\mathcal{U}}_{\nu }}\prod _{t\in {\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)}{Q}_{\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu ,t}\right)\mathbf{1}\left\{{\mathit{x}}_{\nu }\in {\mathcal{D}}_{\nu ,{\mathit{u}}_{\nu }}^{i_{\nu }}\right\}\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill & ={\displaystyle \frac{1}{{\Xi }_{\nu }}}\prod _{t=1}^n{Q}_{\nu ,u_{\nu ,t}}^{i_{\nu }}\left(x_{\nu ,t}\right)\mathbf{1}\left\{{\mathit{x}}_{\nu }\in {\mathcal{D}}_{\nu ,{\mathit{u}}_{\nu }}^{i_{\nu }}\right\},\hfill \end{array}$$

(25)

where ${\Xi }_{\nu }$ is a normalizing constant and the class index is determined by the source message, $i_{\nu }=i_{\nu }\left({\mathit{u}}_{\nu }\right)$.

The multi-class cost-constrained ensemble subsumes all the ensembles described in Section 3.1. First of all, the iid and icd ensembles are recovered by setting $L_{\nu }=0$ and choosing the appropriate number of classes $K_{\nu }$ and random-coding distributions $Q_{\nu }$, $Q_{\nu ,u_{\nu }}$, $Q_{\nu }^{i_{\nu }}$ and $Q_{\nu ,u_{\nu }}^{i_{\nu }}$. For all these cases, the set ${\mathcal{D}}_{\nu ,{\mathit{u}}_{\nu }}^{i_{\nu }}$ includes all generated codewords and the normalizing constant is ${\Xi }_{\nu }=1$.

To recover the constant-composition ensembles, for which constraints force the subcodewords to belong to some set ${\mathcal{T}}_{\nu }^n\left(Q_{\nu }\right)$ or ${\mathcal{T}}_{\nu }^{|{\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)|}\left(Q_{\nu ,u_{\nu }}^{i_{\nu }}\right)$, for each of the $K_{\nu }$ classes for user $\nu$ we set ${\delta }_{\nu }<1$, $L_{\nu }=\left|{\mathcal{X}}_{\nu }\right|$ and bijectively map the channel input symbols to cost function indices ${\ell }_{\nu }\left(x_{\nu }\right)$ so that

$$\begin{array}{c}\hfill a_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu }\right)=\mathbf{1}\left\{x_{\nu }={\ell }_{\nu }\right\}-Q_{\nu ,u_{\nu }}^{i_{\nu }}\left({\ell }_{\nu }\right).\end{array}$$

(26)

In case the ensemble does not depend on either $i_{\nu }$ or $u_{\nu }$, these symbols are dropped from Equation (26). For example, for the md-cc ensemble, we have $a_{\nu ,{\ell }_{\nu }}^{i_{\nu }}\left({\ell }_{\nu }\right)=\mathbf{1}\left\{x_{\nu }={\ell }_{\nu }\right\}-Q_{\nu }^{i_{\nu }}\left(x_{\nu }\right)$. In addition, the codeword set ${\mathcal{D}}_{\nu ,{\mathit{u}}_{\nu }}^{i_{\nu }}$ in Equation (23) is simplified as

$$\begin{array}{c}\hfill {\mathcal{D}}_{\nu ,{\mathit{u}}_{\nu }}^{i_{\nu }}=\left\{{\mathit{x}}_{\nu }\in {\mathcal{X}}_{\nu }^n:\left|\frac{1}{n}\sum _{t=1}^n\mathbf{1}\left\{x_{\nu ,t}=x\right\}-Q_{\nu }^{i_{\nu }}\left(x\right)\right|\le \frac{1}{n},x\in {\mathcal{X}}_{\nu }\right\},\end{array}$$

(27)

which is the same as ${\mathcal{T}}^n\left(Q_{\nu }^{i_{\nu }}\right)$ given a version of Equation (15) where $Q_{\nu }$ may depend on $i_{\nu }$.

Again, choosing the right number of classes $K_{\nu }$ and random-coding distributions $Q_{\nu }$, $Q_{\nu ,u_{\nu }}$, $Q_{\nu }^{i_{\nu }}$, and $Q_{\nu ,u_{\nu }}^{i_{\nu }}$ recovers the various constant-composition ensembles. By construction, the set ${\mathcal{D}}_{\nu ,{\mathit{u}}_{\nu }}^{i_{\nu }}$ includes only the (sub)codewords with empirical distribution close to respectively $Q_{\nu }$, $Q_{\nu ,u_{\nu }}$, $Q_{\nu }^{i_{\nu }}$, and $Q_{\nu ,u_{\nu }}^{i_{\nu }}$, and the normalizing constant ${\Xi }_{\nu }$ is the probability of the corresponding type set (or product thereof). As an example, for the md-ccc ensemble, choosing the cost functions in Equation (26) as follows

$$\begin{array}{c}\hfill {\mathrm{a}}_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left({\mathit{x}}_{\nu }\left({\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)\right)\right)=\sum _{j\in {\mathcal{I}}_{u_{\nu }}\left({\mathit{u}}_{\nu }\right)}\mathbf{1}\left\{x_{\nu ,j}={\ell }_{\nu }\right\}-Q_{\nu ,u_{\nu }}^{i_{\nu }}\left({\ell }_{\nu }\right)\end{array}$$

(28)

yields the following cost-constraint set, which is equivalent to Equation (17),

$$\begin{array}{c}\hfill {\mathcal{D}}_{\nu ,{\mathit{u}}_{\nu }}^{i_{\nu }}=\left\{{\mathit{x}}_{\nu }\in {\mathcal{X}}_{\nu }^n:|\frac{{\sum }_{j\in {\mathcal{I}}_u\left({\mathit{u}}_{\nu }\right)}\mathbf{1}\left\{x_{\nu ,j}=x\right\}}{\left|{\mathcal{I}}_u\left({\mathit{u}}_{\nu }\right)\right|}-Q_{\nu ,u_{\nu }}^{i_{\nu }}\left(x\right)|\le \frac{1}{\left|{\mathcal{I}}_u\left({\mathit{u}}_{\nu }\right)\right|},u\in {\mathcal{U}}_{\nu },x\in {\mathcal{X}}_{\nu }\right\}.\end{array}$$

(29)

3.3. Exponent for the Generalized Multi-Class Cost-Constrained Ensemble

Theorem 1.

For the transmission of N correlated memorlyess sources with joint distribution $P_{\mathcal{N}}$, where $\mathcal{N}=\{1,2,\dots ,N\}$, over a channel with input $x_{\mathcal{N}}$ over a memoryless channel with transition probabilitiy $W\left(y\right|x_{\mathcal{N}})$, consider a random-coding multi-class cost-constrained ensemble where source messages for each user $\nu \in \mathcal{N}$ are allocated, depending on their probabilities, into $K_{\nu }$ classes with thresholds $\{{\gamma }_{\nu ,0},{\gamma }_{\nu ,1},\dots ,{\gamma }_{\nu ,K_{\nu }}\}$, as in Equation (12), and encoded onto codewords randomly generated with a distribution ${\mathrm{Q}}_{\nu }^{i_{\nu }}\left({\mathit{x}}_{\nu }\right|{\mathit{u}}_{\nu })$ that depends on the source message according to Equation (24) through symbol distributions $Q_{\nu ,u_{\nu }}^{i_{\nu }}$ that possibly depend on the source-message class index $i_{\nu }$ and source symbol $u_{\nu }$ and $L_{\nu }$ cost functions $a_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}$, ${\ell }_{\nu }\in \{1,2,\dots ,L_{\nu }\}$. This random-coding ensemble attains the following exponent ${\mathrm{E}}^{\mathit{cos}\mathit{t}}$

$$\begin{array}{cc}\hfill {\mathrm{E}}^{\mathit{cost}}& =\underset{\begin{array}{c}\tau \in 2^{\mathcal{N}}\backslash \varnothing ,i_{\mathcal{N}}\in {\mathcal{K}}_{\mathcal{N}}\end{array}}{min}\underset{0\le \rho \le 1}{max}\underset{\begin{array}{c}{\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}}\ge 0,r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}\in \mathbf{R}\end{array}}{max}{E}_{\tau }^{i_{\mathcal{N}}}\left(\rho ,{\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}},r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}\right),\hfill \end{array}$$

(30)

where the Gallager function $E_{\tau }^{i_{\mathcal{N}}}\left(\rho ,{\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}},r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}\right)$ is given by

$$\begin{array}{cc}\hfill & E_{\tau }^{i_{\mathcal{N}}}\left(\rho ,{\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}},r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}\right)=\hfill \\ \hfill & -log\sum _{u_{{\tau }^c},x_{{\tau }^c},y}{\left(\sum _{u_{\tau },x_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{\Lambda }_{\mathcal{N}}^{i_{\mathcal{N}}}\left(u_{\mathcal{N}}\right)Q_{\tau ,u_{\tau }}^{i_{\tau }}\left(x_{\tau }\right)R_{\mathcal{N},u_{\mathcal{N}}}^{i_{\mathcal{N}}}\left(x_{\mathcal{N}}\right){\left(Q_{{\tau }^c,u_{{\tau }^c}}^{i_{{\tau }^c}}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho },\hfill \end{array}$$

(31)

and the functions ${\Lambda }_{\sigma }^{i_{\sigma }}\left(u_{\sigma }\right)$ and $R_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)$ are respectively given by

$$\begin{array}{c}{\Lambda }_{\sigma }^{i_{\sigma }}\left(u_{\sigma }\right)=\prod _{\nu \in \sigma }{\left(\frac{{\mathrm{P}}_{\nu }\left(u_{\nu }\right)}{{\gamma }_{\nu ,i_{\nu }}}\right)}^{{\lambda }_{\nu }^{\mathrm{L}}}{\left(\frac{{\gamma }_{\nu ,i_{\nu }-1}}{{\mathrm{P}}_{\nu }\left(u_{\nu }\right)}\right)}^{{\lambda }_{\nu }^{\mathrm{U}}},\end{array}$$

(32)

$$\begin{array}{c}{R}_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)=\prod _{\nu \in \sigma }\prod _{{\ell }_{\nu }\in {\mathcal{L}}_{\nu }}{e}^{r_{\nu u_{\nu }}^{{\ell }_{\nu }}{a}_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu }\right)},\end{array}$$

(33)

and implicitly depende on the set of optimization parameters $\left({\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}},r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}\right)$.

Proof. 

This result is proved in Appendix A. □

The random-coding exponent in Equation (30) depends on the partitioning of the source-message set into classes, the channel input distributions, and the codeword cost-constraint functions. The best possible generalized cost-constraint exponent is obtained by optimizing over the multi-class partitioning, the cost constraints and the input distributions. We briefly discuss the optimization w. r. t. the thresholds of the source messages partitioning in Appendix B. In the next section, we provide some numerical examples where we compute the optimal exponents for either independent or correlated sources, and find that the optimal number of classes is two. In ref. [31] (Sec. 3.2.1.1), we provide some indications of why this optimality of only two classes is harder to establish in multi-user scenarios, compared to the single-user case. In the next section, we use Equations (31) and (30) to respectively obtain the source and channel Gallager functions of the various ensembles in Section 3.1 and rank their achievable exponents and transmissibility regions.

4. Discussion

4.1. Gallager Functions for Correlated Sources

In this section, we evaluate the generalized Gallager function $E_{\tau }^{i_{\mathcal{N}}}\left(\rho ,{\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}},r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}\right)$ of the multi-class cost-constrained ensemble in Equation (31) for the various ensembles described in Section 3.1. In the cases where it is possible, we relate this Gallager function to the well-known [1] correlated-source and channel Gallager functions, respectively given by:

$$\begin{array}{c}{E}_{\mathrm{s},\sigma }(\rho ,P_{\mathcal{N}})=log\sum _{u_{{\sigma }^c}}{\left(\sum _{u_{\sigma }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho },\end{array}$$

(34)

$$\begin{array}{c}{E}_0(\rho ,Q,W)=-log\sum _y{\left(\sum _{x}Q\left(x\right)W{\left(y\right|x)}^{\frac{1}{1+\rho }}\right)}^{1+\rho },\end{array}$$

(35)

where $\sigma \in 2^{\mathcal{N}}$. Using that $[u_{\sigma },u_{{\sigma }^c}]=u_{\mathcal{N}}$, the standard Gallager source function is given by $E_{\mathrm{s}}(\rho ,P_{\mathcal{N}})=E_{\mathrm{s},\mathcal{N}}(\rho ,P_{\mathcal{N}})$, with $\mathcal{N}=\{1,\dots ,N\}$ the set of user indices.

For the simple mi-iid ensemble, with only one source class and no cost constraints, $K_{\nu }=1$ and $L_{\nu }=0$ for all $\nu \in \mathcal{N}$, and ${\Lambda }_{\sigma }^{i_{\sigma }}\left(u_{\sigma }\right)=R_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)=1$ for all $\sigma \in 2^{\mathcal{N}}$. With no statistical dependency between messages and codewords, $Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)=Q_{\nu }\left(x_{\nu }\right)$. Setting $i_{\mathcal{N}}=1$ and ${\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}}=r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}=0$ in Equation (31) gives the Gallager function $E_{\tau }^{\mathit{mi}-\mathit{iid}}(\rho ,P_{\mathcal{N}},Q_{\mathcal{N}},W)$,

$$\begin{array}{cc}\hfill E_{\tau }^{\mathit{mi}-\mathit{iid}}& (\rho ,P_{\mathcal{N}},Q_{\mathcal{N}},W)\hfill \\ \hfill & =-log\sum _{u_{{\tau }^c},x_{{\tau }^c},y}{\left(\sum _{u_{\tau },x_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{Q}_{\tau }\left(x_{\tau }\right){\left(Q_{{\tau }^c}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho }.\hfill \end{array}$$

(36)

Isolating the summations over $u_{{\tau }^c}$ and $u_{\tau }$, we can split the Gallager function as

$$\begin{array}{cc}\hfill E_{\tau }^{\mathit{mi}-\mathit{iid}}(\rho ,P_{\mathcal{N}},Q_{\mathcal{N}},W)& =E_0(\rho ,Q_{\tau },Q_{{\tau }^c}W)-E_{\mathrm{s},\tau }(\rho ,P_{\mathcal{N}}),\hfill \end{array}$$

(37)

where $Q_{{\tau }^c}W$ is a shorthand for $Q_{{\tau }^c}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})$, the transition probability of a channel with input $x_{\tau }$ and output $(x_{{\tau }^c},y)$.

For the mi-icd ensemble, we have a similar set-up as for the mi-iid ensemble, where $Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)$ now may depend on $u_{\nu }$. In this case, the Gallager function $E_{\tau }^{\mathit{mi}-\mathit{icd}}(·)$ is given by Equation (36) with $Q_{\sigma }\left(x_{\sigma }\right)$ replaced by $Q_{\sigma ,u_{\sigma }}\left(x_{\sigma }\right)$, for $\sigma \in \{\tau ,{\tau }^c\}$:

$$\begin{array}{cc}\hfill E_{\tau }^{\mathit{mi}-\mathit{icd}}& (\rho ,P_{\mathcal{N}},Q_{\mathcal{N},\mathcal{U}},W)\hfill \\ \hfill & =-log\sum _{u_{{\tau }^c},x_{{\tau }^c},y}{\left(\sum _{u_{\tau },x_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{Q}_{\tau ,u_{\tau }}\left(x_{\tau }\right){\left(Q_{{\tau }^c,u_{{\tau }^c}}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho }.\hfill \end{array}$$

(38)

As the summations over $u_{{\tau }^c}$ and $u_{\tau }$ are not independent from the rest, the Gallager function does not split into source and channel functions unless the sources are independent, in which case one can find an mi-iid ensemble with a tilted unconditional input distribution and identical exponent. To this end, and for a given conditional input distribution $Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)$, let us define a tilted distribution $Q_{\nu }^{\rho }\left(x_{\nu }\right)$ as

$$\begin{array}{c}\hfill Q_{\nu }^{\rho }\left(x_{\nu }\right)=\sum _{u_{\nu }}\frac{P_{\nu }{\left(u_{\nu }\right)}^{\frac{1}{1+\rho }}}{{\sum }_{{\overline{u}}_{\nu }}{P}_{\nu }{\left({\overline{u}}_{\nu }\right)}^{\frac{1}{1+\rho }}}{Q}_{\nu ,u_{\nu }}\left(x_{\nu }\right).\end{array}$$

(39)

From this equation, we have the following equality:

$$\begin{array}{c}\hfill Q_{\nu }^{\rho }\left(x_{\nu }\right)\sum _{{\overline{u}}_{\nu }}{P}_{\nu }{\left({\overline{u}}_{\nu }\right)}^{\frac{1}{1+\rho }}=\sum _{u_{\nu }}{P}_{\nu }{\left(u_{\nu }\right)}^{\frac{1}{1+\rho }}{Q}_{\nu ,u_{\nu }}\left(x_{\nu }\right).\end{array}$$

(40)

Substituting this identity together with $P_{\mathcal{N}}\left(u_{\mathcal{N}}\right)=P_{\tau }\left(u_{\tau }\right)P_{{\tau }^c}\left(u_{{\tau }^c}\right)$ in Equation (38) and rearranging the result, we obtain the following Gallager function for independent sources:

$$\begin{array}{cc}\hfill E_{\tau }^{\mathit{mi}-\mathit{icd}}(\rho ,P_{\mathcal{N}},Q_{\mathcal{N},\mathcal{U}},W)& =E_0(\rho ,Q_{\tau }^{\rho },W{Q}_{{\tau }^c})-E_{\mathrm{s}}(\rho ,P_{\tau })\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill & =E_{\tau }^{\mathit{mi}-\mathit{iid}}(\rho ,P_{\mathcal{N}},[Q_{\tau }^{\rho },Q_{{\tau }^c}],W).\hfill \end{array}$$

(42)

For the md-iid and md-icd ensembles, there are $K_{\nu }$ source classes per user and no cost constraints, i.e., $L_{\nu }=0$ and $R_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)=1$ for $\nu \in \mathcal{N}$ and $\sigma \in 2^{\mathcal{N}}$. Settting $r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}=0$ in Equation (31) gives the Gallager function $E_{\tau ,i_{\mathcal{N}}}^{\mathit{md}-\mathit{icd}}(·)$ for generic $i_{\mathcal{N}}$ [31] (Eq. (4.36)),

$$\begin{array}{cc}\hfill E& {}_{\tau ,i_{\mathcal{N}}}^{\mathit{md}-\mathit{icd}}(\rho ,P_{\mathcal{N}},Q_{\mathcal{N},\mathcal{U}}^{i_{\mathcal{N}}},W)\hfill \\ \hfill & =-log\sum _{u_{{\tau }^c},x_{{\tau }^c},y}{\left(\sum _{u_{\tau },x_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{\Lambda }_{\mathcal{N}}^{i_{\mathcal{N}}}\left(u_{{}_{\mathcal{N}}}\right)Q_{\tau ,u_{\tau }}^{i_{\tau }}\left(x_{\tau }\right){\left(Q_{{\tau }^c,u_{{\tau }^c}}^{i_{\tau c}}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho }.\hfill \end{array}$$

(43)

The Gallager function $E_{\tau ,i_{\mathcal{N}}}^{\mathit{md}-\mathit{iid}}(·)$ for the md-iid ensemble is obtained by setting $Q_{\sigma ,u_{\sigma }}\left(x_{\sigma }\right)=Q_{\sigma }\left(x_{\sigma }\right)$, independent of $u_{\nu }$, for $\sigma \in \{\tau ,{\tau }^c\}$ in Equation (43). As the summations over $u_{{\tau }^c}$ and $u_{\tau }$ are now independent from the rest, the Gallager function splits as

$$\begin{array}{cc}\hfill & E_{\tau ,i_{\mathcal{N}}}^{\mathit{md}-\mathit{iid}}(\rho ,P_{\mathcal{N}},Q_{\mathcal{N}}^{i_{\mathcal{N}}},W)=E_0(\rho ,Q_{\tau }^{i_{\tau }},Q_{{\tau }^c}^{i_{{\tau }^c}}W)-E_{\mathrm{s},\tau }^{i_{\mathcal{N}}}(\rho ,P_{\mathcal{N}}),\hfill \end{array}$$

(44)

where we defined $E_{\mathrm{s},\tau }^{i_{\mathcal{N}}}(\rho ,P_{\mathcal{N}})$, a modified Gallager $E_{\mathrm{s}}$-function, as

$$\begin{array}{c}\hfill E_{\mathrm{s},\tau }^{i_{\mathcal{N}}}(\rho ,P_{\mathcal{N}})=log\sum _{u_{{\tau }^c}}{\left(\sum _{u_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{\Lambda }_{\mathcal{N}}^{i_{\mathcal{N}}}\left(u_{\mathcal{N}}\right)\right)}^{1+\rho }.\end{array}$$

(45)

The maximization w.r.t. ${\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}}$ in Equation (30) only affects the second term in the r. h. s. of Equation (44), since the function ${\Lambda }_{\mathcal{N}}^{i_{\mathcal{N}}}$ only appears in the source part of the exponent. In Appendix C, we discuss the properties of Equation (45) after the maximization w.r.t. ${\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}}$ as a function of $\rho$, and establish some connections to the Gallager source function (34) and to the source functions for the single-user md-iid ensemble in ref. [8].

The Gallager functions for the constant-composition ensembles differ from the ones considered so far in the presence of $L_{\nu }=\left|{\mathcal{X}}_{\nu }\right|$ cost functions $a_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu }\right)$, given in Equation (26), for each input distribution $Q_{\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)$. These cost functions appear in the Gallager functions through the factors $R_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)$, for $\sigma \in \{\tau ,{\tau }^c\}$ that multiply each appearance of $Q_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)$ in the function, and through their associated optimization parameters $r_{\mathcal{N}{u}_{\mathcal{N}}}^{{\ell }_{\mathcal{N}}}$. The expressions of the Gallager functions for these constant-composition ensembles can be easily inferred from this obversation, so we focus on the factor $R_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)$ itself.

For the mi-cc and md-cc ensembles, the cost functions $a_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu }\right)$, factor $R_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)$, and associated optimization parameter $r_{\nu u_{\nu }}^{{\ell }_{\nu }}$ are independent of $u_{\nu }$, we thus write $a_{\nu }^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu }\right)$, $R_{\sigma }^{i_{\sigma }}\left(x_{\sigma }\right)$, and $r_{\nu }^{{\ell }_{\nu }}$. The expressions in Equations (26) and (33) for ${\mathcal{L}}_{\nu }={\mathcal{X}}_{\nu }$ give

$$\begin{array}{c}{R}_{\nu }^{i_{\nu }}\left(x_{\nu }\right)=e^{{\sum }_{{\ell }_{\nu }\in {\mathcal{X}}_{\nu }}{r}_{\nu }^{{\ell }_{\nu }}\left(\mathbf{1}\{x_{\nu }={\ell }_{\nu }\}-Q_{\nu }^{i_{\nu }}\left({\ell }_{\nu }\right)\right)}.\end{array}$$

(46)

The exponent in Equation (46) can be evaluated as

$$\begin{array}{cc}\hfill \sum _{{\ell }_{\nu }\in {\mathcal{X}}_{\nu }}{r}_{\nu }^{{\ell }_{\nu }}\left(\mathbf{1}\left\{x_{\nu }={\ell }_{\nu }\right\}-Q_{\nu }^{i_{\nu }}\left({\ell }_{\nu }\right)\right)& =r_{\nu }^{x_{\nu }}-\sum _{{\ell }_{\nu }\in {\mathcal{X}}_{\nu }}{r}_{\nu }^{{\ell }_{\nu }}{Q}_{\nu }^{i_{\nu }}\left({\ell }_{\nu }\right)\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill & ={\alpha }_{\tau ,\nu }^{i_{\nu }}\left(x_{\nu }\right),\hfill \end{array}$$

(48)

where we have defined a function ${\alpha }_{\tau ,\nu }^{i_{\nu }}\left(x_{\nu }\right)$ that depends on $\tau$ and $i_{\nu }$ through the optimization parameters $r_{\nu }^{{\ell }_{\nu }}$. We can be easily verify that ${\alpha }_{\tau ,\nu }^{i_{\nu }}$ has zero mean, in other words, ${\sum }_{x_{\nu }}{\alpha }_{\tau ,\nu }^{i_{\nu }}\left(x_{\nu }\right)Q_{\nu }^{i_{\nu }}\left(x_{\nu }\right)=0$. At this point, the parameters $r_{\nu }^{{\ell }_{\nu }}$ may be replaced by the equivalent real-valued functions ${\alpha }_{\tau ,\nu }^{i_{\nu }}\left(x_{\nu }\right)$. We obtain the mi-cc Gallager function $E_{\tau }^{\mathit{mi}-\mathit{cc}}(·)$ by setting $i_{\mathcal{N}}=1$ and ${\lambda }_{\mathcal{N}}^{\mathrm{L},\mathrm{U}}=0$ in Equation (31),

$$\begin{array}{l}E{}_{\tau }^{\mathit{mi}-\mathit{cc}}(\rho ,{\alpha }_{\tau ,\mathcal{N}},P_{\mathcal{N}},Q_{\mathcal{N}},W)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & =-log\sum _{u_{{\tau }^c},x_{{\tau }^c},y}{\left(\sum _{u_{\tau },x_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{Q}_{\tau }\left(x_{\tau }\right)e^{{\alpha }_{\tau ,\mathcal{N}}\left(x_{\mathcal{N}}\right)}{\left(Q_{{\tau }^c}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho }\hfill \end{array}$$

(49)

$$\begin{array}{cc}\hfill & =-log\sum _{x_{{\tau }^c},y}{\left(\sum _{x_{\tau }}{Q}_{\tau }\left(x_{\tau }\right)e^{{\alpha }_{\tau ,\mathcal{N}}\left(x_{\mathcal{N}}\right)}{\left(Q_{{\tau }^c}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho }-E_{\mathrm{s},\tau }(\rho ,P_{\mathcal{N}}),\hfill \end{array}$$

(50)

where we split the Gallager function into channel and source terms in analogy to Equation (37).

In ref. [31] (Eq. (4.49)), the md-cc ensemble was studied for $N=2$ users in both the primal and dual domains. The md-cc Gallager function $E_{\tau }^{\mathit{md}-\mathit{cc}}(·)$ for N users is obtained by combining the derivation of Equation (50) with that of Equation (44) to yield

$$\begin{array}{cc}\hfill E_{\tau }^{\mathit{md}-\mathit{cc}}(\rho ,{\alpha }_{\tau ,\mathcal{N}}^{i_{\mathcal{N}}},P_{\mathcal{N}},Q_{\mathcal{N}}^{i_{\mathcal{N}}},W)=-log& \sum _{x_{{\tau }^c},y}{\left(\sum _{x_{\tau }}{Q}_{\tau }\left(x_{\tau }\right)e^{{\alpha }_{\tau ,\mathcal{N}}^{i_{\mathcal{N}}}\left(x_{\mathcal{N}}\right)}{\left(Q_{{\tau }^c}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho }\hfill \\ & -log\sum _{u_{{\tau }^c}}{\left(\sum _{u_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{\Lambda }_{\mathcal{N}}^{i_{\mathcal{N}}}\left(u_{{}_{\mathcal{N}}}\right)\right)}^{1+\rho }.\hfill \end{array}$$

(51)

As in previous cases, the exponent is obtained after maximization over ${\alpha }_{\tau ,\mathcal{N}}^{i_{\mathcal{N}}}$.

Concluding our list, the cost functions $a_{\nu ,u_{\nu }}^{i_{\nu },{\ell }_{\nu }}\left(x_{\nu }\right)$, factors $R_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)$, and parameters $r_{\nu u_{\nu }}^{{\ell }_{\nu }}$ for the mi-ccc and md-ccc ensembles do depend on $u_{\nu }$. In analogy to Equation (48), we define a zero-mean function ${\beta }_{\tau ,\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)$ as

$$\begin{array}{c}\hfill {\beta }_{\tau ,\nu ,u_{\nu }}^{i_{\nu }}\left(x_{\nu }\right)=r_{\nu u_{\nu }}^{x_{\nu }}-\sum _{{\ell }_{\nu }\in {\mathcal{X}}_{\nu }}{r}_{\nu u_{\nu }}^{{\ell }_{\nu }}{Q}_{\nu ,u_{\nu }}^{i_{\nu }}\left({\ell }_{\nu }\right),\end{array}$$

(52)

and similarly for ${\beta }_{\tau ,\nu ,u_{\nu }}\left(x_{\nu }\right)$ for the mi-ccc ensemble. The Gallager function for the mi-ccc ensemble $E_{\tau }^{\mathit{mi}-\mathit{ccc}}(·)$ is obtained by combining the derivations of Equation (50) and of Equation (38),

$$\begin{array}{c}{E}_{\tau }^{\mathit{mi}-\mathit{ccc}}(\rho ,{\beta }_{\tau ,\mathcal{N},u_{\mathcal{N}}},P_{\mathcal{N}},Q_{\mathcal{N},\mathcal{U}},W)\hfill \\ =-log\sum _{u_{{\tau }^c},x_{{\tau }^c},y}{\left(\sum _{u_{\tau },x_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{Q}_{\tau ,u_{\tau }}\left(x_{\tau }\right)e^{{\beta }_{\tau ,\mathcal{N},u_{\mathcal{N}}}\left(x_{\mathcal{N}}\right)}{\left(Q_{{\tau }^c,u_{{\tau }^c}}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho }.\hfill \end{array}$$

(53)

Similarly, for the md-ccc ensemble, and in agreeement with the 2-user case studied in ref. [31] (Eq. (4.45)), combining the derivations of Equations (50) and (43), yields

$$\begin{array}{cc}\hfill & E_{\tau ,i_{\mathcal{N}}}^{\mathit{md}-\mathit{ccc}}(\rho ,{\beta }_{\tau ,\mathcal{N},u_{\mathcal{N}}}^{i_{\mathcal{N}}},P_{\mathcal{N}},Q_{\mathcal{N},\mathcal{U}}^{i_{\mathcal{N}}},W)\hfill \\ \hfill & =-log\sum _{u_{{\tau }^c},x_{{\tau }^c},y}{\left(\sum _{u_{\tau },x_{\tau }}{P}_{\mathcal{N}}{\left(u_{\mathcal{N}}\right)}^{\frac{1}{1+\rho }}{\Lambda }_{\mathcal{N}}^{i_{\mathcal{N}}}\left(u_{{}_{\mathcal{N}}}\right)Q_{\tau ,u_{\tau }}^{i_{\tau }}\left(x_{\tau }\right)e^{{\beta }_{\tau ,\mathcal{N},u_{\mathcal{N}}}^{i_{\mathcal{N}}}\left(x_{\mathcal{N}}\right)}{\left(Q_{{\tau }^c,u_{{\tau }^c}}^{i_{\tau c}}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\right)}^{\frac{1}{1+\rho }}\right)}^{1+\rho }.\hfill \end{array}$$

(54)

4.2. Transmissibility

We may obtain the transmissibility conditions from the achievable exponents derived in Section 4.1, following the random-coding method described in ref. [1] (Th. 5.6.4). The analysis extends the transmissibility condition for joint source-channel coding in ref. [1] (Prob. 5.16), to account for statistical dependency of the codeword on the source message in the multiuser set-up. As mentioned above, the source $U_{\mathcal{N}}$ is transmissible over the channel W if there exists a sequence of codes with vanishing error probability, or equivalently, with strictly positive achievable error exponent ${\mathrm{E}}^{cost}$ in Equation (30). As an example, we present the derivation for the mi-icd ensemble where the class and cost functions in Equations (32) and (33) are inactive, namely ${\Lambda }_{\sigma }^{i_{\sigma }}\left(u_{\sigma }\right)=R_{\sigma ,u_{\sigma }}^{i_{\sigma }}\left(x_{\sigma }\right)=1$ for all $\sigma \in 2^{\mathcal{N}}$, and leave the general case of $K_{\nu }>1$ classes and cost-constrained codewords as an open problem.

For the mi-icd case, and similarly to Gallager’s $E_0$-function [1] (Th. 5.6.3), the Gallager function $E_{\tau }^{\mathit{mi}-\mathit{icd}}(·)$ in Equation (38) is concave (∩) with respect to $\rho$ and satisfies $E_{\tau }^{\mathit{mi}-\mathit{icd}}(\rho =0,·)=0$. For every $\tau \subset 2^{\mathcal{N}}\backslash \varnothing$, let ${\widehat{\rho }}_{\tau }$ be the optimizer given by

$${\widehat{\rho }}_{\tau }=\underset{0\le \rho \le 1}{arg\; max}{E}_{\tau }^{\mathit{mi}-\mathit{icd}}(\rho ,P_{\mathcal{N}},Q_{\mathcal{N},\mathcal{U}},W).$$

(55)

Therefore, the achievable exponent is strictly positive, namely $E_{\tau }^{\mathit{mi}-\mathit{icd}}({\widehat{\rho }}_{\tau },·)>0$, as far as the slope of the $E_{\tau }^{\mathit{mi}-\mathit{icd}}(\rho ,·)$ function is strictly positive at $\rho =0$, that is

$$\frac{\partial }{\partial \rho }{\left.E_{\tau }^{\mathit{mi}-\mathit{icd}}(\rho ,P_{\mathcal{N}},Q_{\mathcal{N},\mathcal{U}},W)\right|}_{\rho =0}>0.$$

(56)

Taking the derivative with respect to $\rho$ at both sides of Equation (38), after some algebraic manipulations, we find that (56) is equivalent to

$$\begin{array}{cc}\hfill \sum _{u_{{\tau }^c}}{P}_{{\tau }^c}\left(u_{{\tau }^c}\right)\sum _{x_{{\tau }^c},y}\sum _{u_{\tau },x_{\tau }}& P_{\tau |{\tau }^c}\left(u_{\tau }\right|u_{{\tau }^c})Q_{\tau ,u_{\tau }}\left(x_{\tau }\right)Q_{{\tau }^c,u_{{\tau }^c}}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})\times \hfill \\ \hfill & \times log\frac{P_{\tau |{\tau }^c}\left(u_{\tau }\right|u_{{\tau }^c})Q_{{\tau }^c,u_{{\tau }^c}}\left(x_{{\tau }^c}\right)W\left(y\right|x_{\mathcal{N}})}{{\sum }_{{\overline{u}}_{\tau },{\overline{x}}_{\tau }}{P}_{\tau |{\tau }^c}\left({\overline{u}}_{\tau }\right|u_{{\tau }^c})Q_{\tau ,{\overline{u}}_{\tau }}\left({\overline{x}}_{\tau }\right)Q_{{\tau }^c,u_{{\tau }^c}}\left(x_{{\tau }^c}\right)W\left(y\right|[{\overline{x}}_{\tau },x_{{\tau }^c}])}>0.\hfill \end{array}$$

(57)

We next write the expression in the left hand-side of the inequality (57) in terms of entropy and mutual information. We denote as $H\left(P\right)$ the entropy of a source with distribution P [32] (Eq. (2.1)) and by $I(Q,W)$ the mutual information of a channel W with input distribution Q [32] (Eq. (2.28)). For $\sigma \subset 2^{\mathcal{N}}$, we define a channel input distribution $Q_{\tau |\sigma }$, that is conditioned to the source messages $u_{\sigma }$, as

$$Q_{\tau |\sigma }\left(x_{\tau }\right|u_{\sigma })=\sum _{u_{\tau }\in {\mathcal{U}}_{\tau }}{P}_{\tau |\sigma }\left(u_{\tau }\right|u_{\sigma })Q_{\tau ,u_{\tau }}\left(x_{\tau }\right).$$

(58)

Therefore, the transmissibility condition (57) can be compactly expressed as

$$\begin{array}{c}H\left(P_{\tau |{\tau }^c}\right)<I\left(Q_{\tau |{\tau }^c},W|P_{{\tau }^c}{Q}_{{\tau }^c|{\tau }^c}\right),\tau \subset 2^{\mathcal{N}}\backslash \varnothing .\end{array}$$

(59)

As it is, $Q_{{\tau }^c|{\tau }^c}$ is “transparent”, as it cancels inside the fraction, and the channel law may also be written as $Q_{{\tau }^c|{\tau }^c}W$, removing the conditioning in the mutual information. With $\mathcal{N}=\{1,2\}$ in Equation (59), we recover the achievable Cover-El Gamal-Salehi region [11] (Eq. (3)).

4.3. Numerical Examples

In this section, we present two simple examples showing that the exponent of the md-iid ensemble can be larger than that of the mi-iid ensemble with only two classes (and associated input distributions) for each user. First, we consider two correlated discrete memoryless sources, $N=2$ and $\mathcal{N}=\{1,2\}$, with alphabet ${\mathcal{U}}_{\nu }=\{0,1\}$ for both users $\nu \in \mathcal{N}$, and probability distribution $P_{\mathcal{N}}(u_1,u_2)$ given in matrix form as

$$P_{\mathcal{N}}=\left(\begin{array}{cc}0.0005& 0.0095\\ 0.0005& 0.9895\end{array}\right).$$

(60)

The sources are sent over a discrete memoryless multiple-access channel with input alphabets ${\mathcal{X}}_1={\mathcal{X}}_2=\{1,2,3,4,5,6\}$ and output alphabet $\mathcal{Y}=\{1,2,3,4\}$. The channel transition probabilites are given by a 36 × 4 matrix W, such that $W\left(y\right|x_1,x_2)$ is the row $x_1+6(x_2-1)$. The transition matrix W is given by

$$\begin{array}{c}\hfill W=\left(\begin{array}{c}{W}_1\\ W_2\\ W_3\\ W_4\\ W_5\\ W_6\end{array}\right),\end{array}$$

(61)

where the 6 × 4 submatrices $W_{\ell }$, $\ell =1,\dots ,6$ are given as follows. First, the submatrix $W_1$ corresponds to the point-to-point channel discussed in ref. [8] (Sec. IV.C), given by

$$\begin{array}{c}\hfill W_1=\left(\begin{array}{cccc}1-3k_1& k_1& k_1& k_1\\ k_1& 1-3k_1& k_1& k_1\\ k_1& k_1& 1-3k_1& k_1\\ k_1& k_1& k_1& 1-3k_1\\ 0.5-k_2& 0.5-k_2& k_2& k_2\\ k_2& k_2& 0.5-k_2& 0.5-k_2\end{array}\right),\end{array}$$

(62)

for $k_1=0.045$ and $k_2=0.01$. Let the m-th row of matrix $W_1$ is denoted by $W_1\left(m\right)$. The matrix $W_2$ (resp. $W_3$) is a $6\times 4$ matrix whose rows are all $W_1\left(5\right)$ (resp. $W_1\left(6\right)$). The matrices $W_4$, $W_5$ and $W_6$ are respectively given by

$$\begin{array}{c}\hfill W_4=\left(\begin{array}{c}{W}_1\left(2\right)\\ W_1\left(3\right)\\ W_1\left(4\right)\\ W_1\left(1\right)\\ W_1\left(6\right)\\ W_1\left(5\right)\end{array}\right),W_5=\left(\begin{array}{c}{W}_1\left(3\right)\\ W_1\left(4\right)\\ W_1\left(1\right)\\ W_1\left(2\right)\\ W_1\left(5\right)\\ W_1\left(6\right)\end{array}\right),W_6=\left(\begin{array}{c}{W}_1\left(4\right)\\ W_1\left(1\right)\\ W_1\left(2\right)\\ W_1\left(3\right)\\ W_1\left(6\right)\\ W_1\left(5\right)\end{array}\right).\end{array}$$

(63)

The optimal achievable exponent [8] (Sec. IV.C) for the single-user channel $W_1$ in Equation (62) is related to two different distributions $Q^{☆}$ and $Q^{†}$, given in vector form by

$$\begin{array}{c}{Q}^{☆}=\left(0,0,0,0,1\backslash 2,1\backslash 2\right),\end{array}$$

(64)

$$\begin{array}{c}{Q}^{†}=\left(1\backslash 4,1\backslash 4,1\backslash 4,1\backslash 4,0,0\right).\end{array}$$

(65)

We let each user employ these distributions in the md-iid ensemble with input distribution in Equation (13) according to the source message partitioning in Equation (12) with $K_{\nu }=2$ classes per user and thresholds ${\gamma }_{\mathcal{N}}=({\gamma }_1,{\gamma }_2)$. Since we consider two input distributions for each user, the channel Gallager function ${max}_{\rho \in [0,1]}{E}_0(\rho ,Q_{\tau }^{i_{\tau }},W{Q}_{{\tau }^c}^{i_{{\tau }^c}})$ is not concave in $\rho$ [8]. To find the md-iid exponent $E^{\mathit{md}-\mathit{iid}}$, we optimize over the class thresholds following the method in Appendix B with the Gallager function in Equation (44), exploit the properties of the source function in Equation (45) in Appendix C, and also find the optimal input distribution assignment of $Q_{\nu }^{i_{\nu }}$ for each $\nu \in \{1,2\}$. In our setting, we have four possible assignments, namely

$$\begin{array}{c}{\Omega }_1:Q_1^1=Q_2^1=Q^{☆},Q_1^2=Q_2^2=Q^{†},\end{array}$$

(66)

$$\begin{array}{c}{\Omega }_2:Q_1^1=Q_2^2=Q^{☆},Q_1^2=Q_2^1=Q^{†},\end{array}$$

(67)

$$\begin{array}{c}{\Omega }_3:Q_1^2=Q_2^1=Q^{☆},Q_1^1=Q_2^2=Q^{†},\end{array}$$

(68)

$$\begin{array}{c}{\Omega }_4:Q_1^2=Q_2^2=Q^{☆},Q_1^1=Q_2^1=Q^{†}.\end{array}$$

(69)

We start our numerical discussion by assessing which of the possible four assignments in Equations (66)–(69) leads to a higher error exponent. For each possible pair of thresholds $({\gamma }_1,{\gamma }_2)$, we numerically calculate the optimal assignment ${\Omega }^{☆}\left({\gamma }_{\mathcal{N}}\right)$ given by

$${\Omega }^{☆}\left({\gamma }_{\mathcal{N}}\right)=\underset{{\Omega }_j}{arg\; max}\underset{i_{\mathcal{N}}}{min}\underset{\tau }{min}{E}_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}\right),$$

(70)

and the corresponding achievable error exponent $E^{md-iid}\left({\gamma }_{\mathcal{N}}\right)$ as

$$E^{cost}\left({\gamma }_{\mathcal{N}}\right)=\underset{{\Omega }_j}{max}\underset{i_{\mathcal{N}}}{min}\underset{\tau }{min}{E}_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}\right),$$

(71)

where the exponent function $E_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}\right)$ is given in Equation (A55). Figure 2 and Figure 3 respectively show ${\Omega }^{☆}\left({\gamma }_{\mathcal{N}}\right)$ and $E^{cost}\left({\gamma }_{\mathcal{N}}\right)$ for the valid range of ${\gamma }_{\mathcal{N}}$. For most pair of thresholds $({\gamma }_1,{\gamma }_2)$, assignments ${\Omega }_1$ and ${\Omega }_3$ lead to the highest exponent among the possible assignments, while assignments ${\Omega }_2$ and ${\Omega }_4$ are optimal only for a marginal region. Using this information, and combined with the values of the achievable exponents in Figure 3, we determine the message-dependent exponent

$$E^{md-iid}=\underset{{\gamma }_{\mathcal{N}}}{max}{E}^{cost}\left({\gamma }_{\mathcal{N}}\right).$$

(72)

In this example, we obtained the achievable exponent $E^{\mathit{md}-\mathit{iid}}=0.2611$, corresponding to the input distribution assignment ${\Omega }_1$ in Equation (66) and optimal source message partitioning ${\gamma }_1^{☆}=0.8469$ and ${\gamma }_2^{☆}=0.6581$. The optimal point ${\gamma }_{\mathcal{N}}^{☆}$ is shown by a white (black) bullet in Figure 2 (Figure 3).

Alternatively, we may first optimize over ${\gamma }_{\mathcal{N}}$ and then over the assignments ${\Omega }_j$. To do so, we solve the system of Equation (A58) in Appendix B to numerically determine the optimal thresholds ${\gamma }_{\mathcal{N}}^{☆}$, and compute the exponent $E^{\mathit{cost}}\left({\Omega }_j\right)$ as

$$E^{\mathit{cost}}\left({\Omega }_j\right)=\underset{i_{\mathcal{N}}}{min}\underset{\tau }{min}{E}_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}^{☆}\right),$$

(73)

where the exponent function $E_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}\right)$ is given in Equation (A55). We provide in

Table 1. Correlated-sources optimal thresholds ${\gamma }_{\mathcal{N}}^{☆}$ and exponents $E_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}^{☆}\right)$ in Equation (73) for assignments ${\Omega }_j$ in Equations (66)–(69). For each assignment, the minimum over $i_{\mathcal{N}}$ and $\tau$ is highlighted in gray.

	Assignment ${\mathsf{\Omega }}_1$					Assignment ${\mathsf{\Omega }}_2$
	${\mathit{\gamma }}_{\mathbf{1}}^{☆}=\mathbf{0.8469}$${\mathit{\gamma }}_{\mathbf{2}}^{☆}=\mathbf{0.6581}$					${\mathit{\gamma }}_{\mathbf{1}}^{☆}=\mathbf{1}$${\mathit{\gamma }}_{\mathbf{2}}^{☆}=\mathbf{1}$
$({\mathit{i}}_{\mathbf{1}},{\mathit{i}}_{\mathbf{2}})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{1},\mathbf{2})$	$(\mathbf{2},\mathbf{1})$	$(\mathbf{2},\mathbf{2})$		$(\mathbf{1},\mathbf{1})$	$(\mathbf{1},\mathbf{2})$	$(\mathbf{2},\mathbf{1})$	$(\mathbf{2},\mathbf{2})$
$\tau =\left\{1\right\}$	0.3131	0.2735	0.3120	0.2611		0.0642	0.3268	0.1005	0.3604
$\tau =\left\{2\right\}$	0.3986	0.4369	0.2611	0.4119		0.3959	0.3986	0.4323	0.3110
$\tau =\{1,2\}$	0.2611	0.2972	0.2630	0.2883		0.2108	0.2108	0.2360	0.2637
	Assignment${\mathsf{\Omega }}_{\mathbf{3}}$					Assignment${\mathsf{\Omega }}_{\mathbf{4}}$
	${\mathit{\gamma }}_{\mathbf{1}}^{☆}=\mathbf{0.5605}$${\mathit{\gamma }}_{\mathbf{2}}^{☆}=\mathbf{0.6709}$					${\mathit{\gamma }}_{\mathbf{1}}^{☆}=\mathbf{0.6985}$${\mathit{\gamma }}_{\mathbf{2}}^{☆}=\mathbf{0.9033}$
$({\mathit{i}}_{\mathbf{1}},{\mathit{i}}_{\mathbf{2}})$	$(\mathbf{1},\mathbf{1})$	$(\mathbf{1},\mathbf{2})$	$(\mathbf{2},\mathbf{1})$	$(\mathbf{2},\mathbf{2})$		$(\mathbf{1},\mathbf{1})$	$(\mathbf{1},\mathbf{2})$	$(\mathbf{2},\mathbf{1})$	$(\mathbf{2},\mathbf{2})$
$\tau =\left\{1\right\}$	0.3120	0.2503	0.2763	0.2897		0.0879	0.3605	0.0879	0.3112
$\tau =\left\{2\right\}$	0.2503	0.3898	0.5675	0.5731		0.3664	0.2503	0.4720	0.4684
$\tau =\{1,2\}$	0.2630	0.2816	0.2503	0.3012		0.2360	0.2632	0.2097	0.2097

the values of the optimal thresholds ${\gamma }_{\mathcal{N}}^{☆}$ and exponents $E_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}^{☆}\right)$ under the different assignment ${\Omega }_j$, for the three types of error $\tau$ and the four possible user classes $i_{\mathcal{N}}$. For each assignment, the minimum over $i_{\mathcal{N}}$ and $\tau$ as in Equation (73) is highlighted in gray, leading to the exponent $E^{\mathit{cos}\mathit{t}}\left({\Omega }_j\right)$. The message-dependent exponent is then

$$\begin{array}{c}\hfill E^{\mathit{md}-\mathit{iid}}=\underset{j}{max}{E}^{\mathit{cos}\mathit{t}}\left({\Omega }_j\right),\end{array}$$

(74)

recovering the error exponent $E^{\mathit{md}-\mathit{iid}}=0.2611$ for input distribution assignment ${\Omega }_1$ obtained using the previous method in Equation (71).

In the second example, we consider the transmission of two independent discrete memoryless sources with identical source alphabets ${\mathcal{U}}_{\nu }=\{0,1\}$ with distributions induced by the marginals of Equation (60), given by $P_1\left(0\right)=0.01$ and $P_2\left(0\right)=0.001$. These sources are transmitted over the multiple-access channel with transition probability given by Equation (61), and are encoded using the md-iid ensemble with the input distribution assignments ${\Omega }_j$ in Equations (66)–(69). Following the footsteps of the correlated sources case, in

Table 2. Independent-sources md-iid optimal thresholds ${\gamma }_{\mathcal{N}}^{☆}$ and exponents $E_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}^{☆}\right)$ in Equation (73) for assignments ${\Omega }_j$ in Equations (66)–(69). For each assignment, the minimum over $i_{\mathcal{N}}$ and $\tau$ is highlighted in gray.

	Assignment ${\mathsf{\Omega }}_1$					Assignment ${\mathsf{\Omega }}_2$
	${\mathit{\gamma }}_{\mathbf{1}}^{☆}=\mathbf{0.8779}$${\mathit{\gamma }}_{\mathbf{2}}^{☆}=\mathbf{0.6933}$					${\mathit{\gamma }}_{\mathbf{1}}^{☆}=\mathbf{0.8776}$${\mathit{\gamma }}_{\mathbf{2}}^{☆}=\mathbf{1}$
$(i_1,i_2)$	$(1,1)$	$(1,2)$	$(2,1)$	$(2,2)$		$(1,1)$	$(1,2)$	$(2,1)$	$(2,2)$
$\tau =\left\{1\right\}$	0.3343	0.2458	0.3089	0.2458		0.0913	0.3341	0.0913	0.3089
$\tau =\left\{2\right\}$	0.3850	0.3987	0.2458	0.3788		0.4555	0.3850	0.4357	0.2459
$\tau =\{1,2\}$	0.2730	0.2870	0.2685	0.2863		0.3430	0.2728	0.2956	0.2685
	Assignment${\mathsf{\Omega }}_{\mathbf{3}}$					Assignment${\mathsf{\Omega }}_{\mathbf{4}}$
	${\mathit{\gamma }}_{\mathbf{1}}^{☆}=\mathbf{0.61}$${\mathit{\gamma }}_{\mathbf{2}}^{☆}=\mathbf{0.7043}$					${\mathit{\gamma }}_{\mathbf{1}}^{☆}=\mathbf{0.7092}$${\mathit{\gamma }}_{\mathbf{2}}^{☆}=\mathbf{1}$
$(i_1,i_2)$	$(1,1)$	$(1,2)$	$(2,1)$	$(2,2)$		$(1,1)$	$(1,2)$	$(2,1)$	$(2,2)$
$\tau =\left\{1\right\}$	0.3089	0.2367	0.2681	0.3078		0.0913	0.3117	0.0913	0.2648
$\tau =\left\{2\right\}$	0.2367	0.3672	0.5425	0.5538		0.4269	0.2367	0.5393	0.4683
$\tau =\{1,2\}$	0.2685	0.2811	0.2367	0.3133		0.3006	0.2685	0.2740	0.2164

we calculate optimal thresholds ${\gamma }_{\mathcal{N}}^{☆}$ and exponents $E_{\tau }^{i_{\mathcal{N}}}\left({\gamma }_{\mathcal{N}}^{☆}\right)$ for the possible input distribution assignments and determine the exponent of the md-iid ensemble using Equations (73) and (74). In this case, the optimal assignment is again ${\Omega }_1$, with optimal source message partitioning specified by the thresholds ${\gamma }_1^{☆}=0.8779$ and ${\gamma }_2^{☆}=0.6933$, achieving an exponent of $E^{\mathit{md}-\mathit{iid}}=0.2458$, slightly smaller than that of correlated sources.

For the sake of completeness and purpose of comparison, we also calculate the exponent for the mi-iid ensemble described in Equation (8). In the absence of message dependence, for a given assignment ${\Omega }_j$, the mi-iid exponent is given by

$$\begin{array}{c}\hfill E^{\mathit{no}-\mathit{cost}}\left({\Omega }_j\right)=\underset{\tau }{min}{E}_{\tau },\end{array}$$

(75)

where the exponent function $E_{\tau }$ is given by $E_{\tau }={max}_{\rho }{E}_{\tau }^{\mathit{mi}-\mathit{iid}}(\rho ,P_{\mathcal{N}},Q_{\mathcal{N}},W)$ and $E_{\tau }^{\mathit{mi}-\mathit{iid}}$ is the Gallager function in Equation (37), described in the previous subsection. For both the correlated and independent sources described above,

Table 3. Mi-iid exponents $E_{\tau }$ in Equation (75) for two correlated and two independent sources vs several input distribution assigments $(Q_1,Q_2)$. For each assignment, the minimum over $\tau$ is highlighted in gray.

	Correlated Sources
$({\mathit{Q}}_{\mathbf{1}},{\mathit{Q}}_{\mathbf{2}})$	$({\mathit{Q}}^{☆},{\mathit{Q}}^{☆})$	$({\mathit{Q}}^{☆},{\mathit{Q}}^{†})$	$({\mathit{Q}}^{†},{\mathit{Q}}^{☆})$	$({\mathit{Q}}^{†},{\mathit{Q}}^{†})$
$\tau =\left\{1\right\}$	0.2682	0.0642	0.3120	0.0879
$\tau =\left\{2\right\}$	0.3986	0.3986	0.2503	0.3696
$\tau =\{1,2\}$	0.2097	0.2097	0.2630	0.2360
	Independent Sources
$({\mathit{Q}}_{\mathbf{1}},{\mathit{Q}}_{\mathbf{2}})$	$({\mathit{Q}}^{☆},{\mathit{Q}}^{☆})$	$({\mathit{Q}}^{☆},{\mathit{Q}}^{†})$	$({\mathit{Q}}^{†},{\mathit{Q}}^{☆})$	$({\mathit{Q}}^{†},{\mathit{Q}}^{†})$
$\tau =\left\{1\right\}$	0.2648	0.3089	0.0627	0.0865
$\tau =\left\{2\right\}$	0.3850	0.2367	0.3850	0.3559
$\tau =\{1,2\}$	0.2164	0.2685	0.2164	0.2421

presents the achievable exponents $E_{\tau }$ for each type of error $\tau$ and input distribution assignment $(Q_1,Q_2)$, where $Q_1$ and $Q_2$ are either of $Q^{☆}$ and $Q^{†}$ in Equations (64) and (65). In our numerical example for correlated sources, the assignment with highest exponent is $(Q_1,Q_2)=(Q^{†},Q^{☆})$, giving an exponent of $E^{\mathit{mi}-\mathit{iid}}=0.2503$, slightly smaller than that of the md-iid ensemble. In contrast, the mi-iid exponent for independent sources, according to the second part of Table 3 is found to be ${\mathrm{E}}^{\mathit{mi}-\mathit{iid}}=0.2367$ with input distribution $(Q_1,Q_2)=(Q^{☆},Q^{†})$. In this case, the md-iid exponent ${\mathrm{E}}^{\mathit{md}-\mathit{iid}}$ is around $4\%$ larger that the mi-iid; this situation is in contrast with to-point communication, where the gain in exponent achieved by an ensemble with two distributions is typically smaller, for example, $1\%$ in ref. [8]. Hence, message-dependent random coding with two class distributions, compared to iid random coding, may lead to a higher error exponent gain in the MAC than in point-to-point communication.

4.4. Comparison of the Random-Coding Achievable Error Exponents

From the numerical results presented in Section 4.3, as well as from refs. [8,20,28,31], the message-dependent ensembles attain in general a larger exponent than their message-independent counterparts. We now compare the random-coding exponents for the ensembles presented in Section 3.1, whose Gallager functions were obtained in Section 4.1.

For independent sources, we found in Equation (42) that for a given conditional input distribution $Q_{\nu ,u_{\nu }}\left(x_{\nu }\right)$ and $\rho$, there exists an iiid distribution $Q_{\nu ,\rho }$ given by Equation (39) with identical Gallager function. Thus, the mi-iid and mi-icd ensembles attains the same exponent, after maximization over the input distributions. Similarly, we conclude that md-iid and md-icd-ensembles attain the same exponent.

In ref. [31] (Prop. 2.9), it was proved that for point-to-point communication, the exponent of the mi-ccc ensemble may be lower than that of the mi-cc ensemble. The same steps actually prove the same result for the MAC with independent sources. Thus, for the MAC with independent sources we have

$$\begin{array}{c}\hfill {\mathrm{E}}^{mi-ccc}\le {\mathrm{E}}^{mi-cc}\le {\mathrm{E}}^{md-cc},{\mathrm{E}}^{md-ccc}\le {\mathrm{E}}^{md-cc},\end{array}$$

(76)

$$\begin{array}{c}{\mathrm{E}}^{mi-iid}\le {\mathrm{E}}^{mi-cc}\le {\mathrm{E}}^{md-cc},{\mathrm{E}}^{md-iid}\le {\mathrm{E}}^{md-cc},\end{array}$$

(77)

and ${\mathrm{E}}^{md-cc}$ is thus largest among the ensembles in Section 3.1 for an arbitrary input distribution. As discussed in ref. [29] (Th. 4), for optimal input distributions both ${\mathrm{E}}^{md-cc}$ and ${\mathrm{E}}^{md-iid}$ may coincide.

Concerning the optimal partitioning into message classes, for point-to-point communication it is known that partitioning the source-message set into two classes is sufficient to attain the optimal error exponent [8,31] (Prop. 2.7). However, the proof of ref. [31] (Prop. 2.7) cannot be easily generalized to the MAC with independent sources. At the same time, we could not find an example showing that assigning more than two input distributions leads to a larger exponent. Hence, finding the sufficient number of input distributions is for the message-dependent exponent is an open problem.

The comparisons in Equations (76) and (77) for correlated sources require, in general, a more sophisticated machinery and we consider here two simple cases. For the message-dependent md-icd and md-ccc ensembles, we observe that compared to ${{\displaystyle E}}_{\tau ,i_{\mathcal{N}}}^{md-icd}$ in Equation (43) the ${{\displaystyle E}}_{\tau ,i_{\mathcal{N}}}^{md-ccc}$ exponent in Equation (54) contains an additional term ${\beta }_{\tau ,\mathcal{N},u_{\mathcal{N}}}^{i_{\mathcal{N}}}\left(x_{\mathcal{N}}\right)$ to guarantee the constant-composition distribution as in Equation (52). This allows to recover ${{\displaystyle E}}_{\tau ,i_{\mathcal{N}}}^{md-ccc}$ by setting ${\beta }_{\tau ,\mathcal{N},u_{\mathcal{N}}}^{i_{\mathcal{N}}}\left(x_{\mathcal{N}}\right)=0$ in ${{\displaystyle E}}_{\tau ,i_{\mathcal{N}}}^{md-icd}$ and to prove that ${\mathrm{E}}^{md-icd}\le {\mathrm{E}}^{md-ccc}$ after maximizing w. r. t. ${\beta }_{\tau ,\mathcal{N},u_{\mathcal{N}}}^{i_{\mathcal{N}}}(x_{\mathcal{N}})$. Similarly for the ensembles with statistical independence between messages and codewords, we observe that the constant-composition exponent ${{\displaystyle E}}_{\tau ,i_{\mathcal{N}}}^{md-cc}$ in Equation (51) also contains the additional term ${\alpha }_{\tau ,\mathcal{N}}^{i_{\mathcal{N}}}\left(x_{\mathcal{N}}\right)$ compared to its iid counterpart ${{\displaystyle E}}_{\tau ,i_{\mathcal{N}}}^{md-iid}$ in Equation (44), yielding ${\mathrm{E}}^{md-iid}\le {\mathrm{E}}^{md-cc}$. Put together, for correlated sources it holds that

$$\begin{array}{c}\hfill {\mathrm{E}}^{md-icd}\le {\mathrm{E}}^{md-ccc},{\mathrm{E}}^{md-iid}\le {\mathrm{E}}^{md-cc},\end{array}$$

(78)

suggesting that, as in the case of single-user communication, the use of constant-composition input distributions may lead to higher exponents than the symbol-wise independent distributions when transmitting correlated sources over the MAC.

Summarizing, proper choices of the cost functions recover the different coding schemes considered in Section 3.1, including message-dependent and message-independent versions of iid, independent conditionally distributed, constant-composition, and conditional constant composition ensembles. Thanks to the flexibility of the generalized cost-constraint random-coding ensemble, the achievable exponents of the various ensembles can be compared and ranked, both numerically and analytically.