• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-12-23

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-12-23

Provides a listing of current staff, committee members and society officers.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-12-23

The recipients of the 2019 IEEE Information Theory Society Paper Award are Emmanuel J. Candès, Xiaodong Li, and Mahdi Soltanolkotabi for the article “Phase Retrieval via Wirtinger Flow: Theory and Algorithms,” which appeared in the IEEE Transactions on Information Theory, vol. 61, no. 4, pp. 1985–2007, April 2015.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-12-23

The recipients of the 2019 IEEE Communications Society and Information Theory Society Joint Paper Award are Yuyi Mao, Jun Zhang, and Khaled B. Letaief for the article “Dynamic computation offloading for mobile-edge computing with energy harvesting devices” which appeared in the IEEE Journal on Selected Areas in Communications, vol. 34, no. 12, pp. 3590–3605, December 2016.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-10
Madhu Sudan; Himanshu Tyagi; Shun Watanabe

The task of manipulating correlated random variables in a distributed setting has received attention in the fields of both Information Theory and Computer Science. Often shared correlations can be converted, using a little amount of communication, into perfectly shared uniform random variables. Such perfect shared randomness, in turn, enables the solutions of many tasks. Even the reverse conversion of perfectly shared uniform randomness into variables with a desired form of correlation turns out to be insightful and technically useful. In this article, we describe progress-to-date on such problems and lay out pertinent measures, achievability results, limits of performance, and point to new directions.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-23
Meryem Benammar; Pablo Piantanida; Shlomo Shamai

This work investigates the general two-user compound Broadcast Channel (BC) in which an encoder wishes to transmit two private messages W 1 and W 2 to two receivers while being oblivious to the actual channel realizations controlling the communication. The focus is on the characterization of the largest achievable rate region by resorting to more involved encoding and decoding techniques than the usual coding schemes of the standard BC. Involved decoding schemes are first explored, and an achievable rate region is derived based on the principle of Interference Decoding (ID), in which each receiver decodes its intended message and chooses to (non-uniquely) decode, or not, the interfering non-itended message. This decoding scheme is shown to be capacity achieving for a class of non-trivial compound BEC/BSC broadcast channels while the worst-case of Marton’s inner bound—based on No Interference Decoding (NID)—fails to achieve the capacity region. Involved encoding schemes are later investigated, and an achievable rate region is derived based on Multiple Description (MD) coding wherin the encoder transmits a common description as well as multiple dedicated private descriptions to the many possible channel realizations of the users. It turns out that MD coding yields larger inner bounds than the single description scheme—Common Description (CD) coding—for a class of compound Multiple Input Single Output Broadcast Channels (MISO BC).

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-08-19
Mohamed Salman; Mahesh K. Varanasi

The $K$ -user discrete memoryless (DM) broadcast channel (BC) with two nested multicast messages is studied in which one common message is to be multicast to all receivers and the second private message to a subset of receivers. The receivers that must decode both messages are referred to as private receivers and the others that must decode only the common message as common receivers. For two nested multicast messages, we establish the capacity region for several classes of partially ordered DM BCs characterized by the respective associated sets of pair-wise relationships between and among the common and private receivers, each described by the well-known pair-wise more capable or less noisy condition. For three classes of partially ordered DM BCs, the capacity region is shown to be simply achieved by two-level superposition coding and the proofs of the converses rely on a recently found information inequality. The rate region achievable by two-level superposition coding is then enhanced through a multi-level superposition coding scheme after splitting the private message into as many parts as there are common receivers and indirect decoding. A closed-form two-dimensional polyhedral (polygonal) description is obtained for it for a given coding distribution in spite of the indeterminate number of split rates via a structured form of Fourier-Motzkin elimination. Through a converse result that relies on the Csiszar sum lemma and that information inequality, a specialization of this region that corresponds to splitting the private message into just two sub-messages is proved to be the capacity region for several classes of partially ordered DM BCs beyond those for which two-level superposition coding is capacity optimal, thereby underscoring the benefit of rate-splitting. All previously known capacity results for partially ordered DM BCs with two nested multicast messages for the two and three-receiver DM BCs as well as DM BCs with one private or one common receiver are subsumed in the general results obtained in this work.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-22
Minh Thanh Vu; Tobias J. Oechtering; Mikael Skoglund

We study a two-stage identification problem with pre-processing to enable efficient data retrieval and reconstruction. In the enrollment phase, users’ data are stored into the database in two layers. In the identification phase an observer obtains an observation, which originates from an unknown user in the enrolled database through a memoryless channel. The observation is sent for processing in two stages. In the first stage, the observation is pre-processed, and the result is then used in combination with the stored first layer information in the database to output a list of compatible users to the second stage. Then the second step uses the information of users contained in the list from both layers and the original observation sequence to return the exact user identity and a corresponding reconstruction sequence. The rate-distortion regions are characterized for both discrete and Gaussian scenarios. Specifically, for a fixed list size and distortion level, the compression-identification trade-off in the Gaussian scenario results in three different operating cases characterized by three auxiliary functions. While the choice of the auxiliary random variable for the first layer information is essentially unchanged when the identification rate is varied, the second one is selected based on the dominant function within those three. Due to the presence of a mixture of discrete and continuous random variables, the proof for the Gaussian case is highly non-trivial, which makes a careful measure theoretic analysis necessary. In addition, we study a connection of the previous setting to a two observer identification and a related problem with a lower bound for the list size, where the latter is motivated from privacy concerns.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-06-03
Neri Merhav; Asaf Cohen

Consider the problem of guessing the realization of a random vector ${X}$ by repeatedly submitting queries (guesses) of the form “Is ${X}$ equal to ${x}$ ?” until an affirmative answer is obtained. In this setup, a key figure of merit is the number of queries required until the right vector is identified, a number that is termed the guesswork . Typically, one wishes to devise a guessing strategy which minimizes a certain guesswork moment. In this work, we study a universal, decentralized scenario where the guesser does not know the distribution of ${X}$ , and is not allowed to use a strategy which prepares a list of words to be guessed in advance, or even remember which words were already used. Such a scenario is useful, for example, if bots within a Botnet carry out a brute–force attack in order to guess a password or decrypt a message, yet cannot coordinate the guesses between them or even know how many bots actually participate in the attack. We devise universal decentralized guessing strategies, first, for memoryless sources, and then generalize them for finite–state sources. In each case, we derive the guessing exponent, and then prove its asymptotic optimality by deriving a compatible converse bound. The strategies are based on randomized guessing using a universal distribution. We also extend the results to guessing with side information. Finally, for all above scenarios, we design efficient algorithms in order to sample from the universal distributions, resulting in strategies which do not depend on the source distribution, are efficient to implement, and can be used asynchronously by multiple agents.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-02
Joschka Roffe; Stefan Zohren; Dominic Horsman; Nicholas Chancellor

We introduce a new graphical framework for designing quantum error correction codes based on classical principles. A key feature of this graphical language, over previous approaches, is that it is closely related to that of factor graphs or graphical models in classical information theory and machine learning. It enables us to formulate the description of the recently-introduced ‘coherent parity check’ quantum error correction codes entirely within the language of classical information theory. This makes our construction accessible without requiring background in quantum error correction or even quantum mechanics in general. More importantly, this allows for a collaborative interplay where one can design new quantum error correction codes derived from classical codes.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-07
Mizanur Rahaman

A new bound on quantum version of Wielandt inequality for positive (not necessarily completely positive) maps has been established. Also bounds for entanglement breaking and PPT channels are put forward which are better bound than the previous bounds known. We prove that a primitive positive map $\mathcal {E}$ acting on $\mathcal {M}_{d}$ that satisfies the Schwarz inequality becomes strictly positive after at most $2(d-1)^{2}$ iterations. This is to say that after $2(d-1)^{2}$ iterations, such a map sends every positive semidefinite matrix to a positive definite one. This finding does not depend on the number of Kraus operators as the map may not admit any Kraus decomposition. The motivation of this work is to provide an answer to a question raised by Sanz-García-Wolf and Cirac in their work on quantum Wielandt bound.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-07-16
Asger Kjærulff Jensen; Péter Vrana

We study the exact, non-deterministic conversion of multipartite pure quantum states into one-another via local operations and classical communication (LOCC) and asymptotic entanglement transformation under such channels. In particular, we consider the maximal number of copies of any given target state that can be extracted exactly from many copies of any given initial state as a function of the exponential decay in the success probability, known as the converse error exponent. We give a formula for the optimal rate presented as an infimum over the asymptotic spectrum of LOCC conversion. A full understanding of exact asymptotic extraction rates between pure states in the converse regime thus depends on a full understanding of this spectrum. We present a characterization of spectral points and use it to describe the spectrum in the bipartite case. This leads to a full description of the spectrum and thus an explicit formula for the asymptotic extraction rate between pure bipartite states, given a converse error exponent. This extends the result on entanglement concentration in [1] , where the target state is fixed as the Bell state. In the limit of vanishing converse error exponent, the rate formula provides an upper bound on the exact asymptotic extraction rate between two states, when the probability of success goes to 1. In the bipartite case, we prove that this bound holds with equality.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-04
Giacomo Micheli

Let $q$ be a prime power and $\mathbb {F}_{q}$ be the finite field of size $q$ . In this paper we provide a Galois theoretical framework that allows to produce good polynomials for the Tamo and Barg construction of optimal locally recoverable codes (LRC). Using our approach we construct new good polynomials and therefore optimal LRCs with new parameters. The existing theory of good polynomials fits entirely in our new framework. The key advantage of our method is that we do not need to rely on arithmetic properties of the pair $(q,r)$ , where $r$ is the locality of the code.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-19
Fangwei Ye; Shiqiu Liu; Kenneth W. Shum; Raymond W. Yeung

The problem of exact-repair regenerating codes against eavesdropping attack is studied. The eavesdropping model we consider is that the eavesdropper has the capability to observe the data involved in the repair of a subset of $\ell$ nodes. An $(n,k,d,\ell )$ secure exact-repair regenerating code is an $(n,k,d)$ exact-repair regenerating code that is secure under this eavesdropping model. It has been shown that for some parameters $(n,k,d,\ell )$ , the associated optimal storage-bandwidth tradeoff curve, which has one corner point, can be determined. The focus of this paper is on characterizing such parameters. We establish a lower bound $\hat {\ell }$ on the number of wiretap nodes, and show that this bound is tight for the case $k = d = n-1$ .

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-11
Lingfei Jin; Haibin Kan; Yu Zhang

Locally repairable codes with more than one recovering set are demanded in the application to distributed storage. For each failure node (or disk), it is desired to have as many recovering sets as possible. In this paper, we make use of automorphisms of rational function fields to construct locally repairable codes with multiple recovering sets. Although we focus on two recovering sets, our construction can be easily generalized to the case of multiple recovering sets. In particular, we obtain a class of locally repairable codes with minimum distance only 1 less than the upper bound.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-11
Lingfei Jin; Liming Ma; Chaoping Xing

Locally repairable codes, or locally recoverable codes (LRC for short), are designed for applications in distributed and cloud storage systems. Similar to classical block codes, there is an important bound called the Singleton-type bound for locally repairable codes. In this paper, an optimal locally repairable code refers to a block code achieving this Singleton-type bound. Like classical MDS codes, optimal locally repairable codes carry some very nice combinatorial structures. Since the introduction of the Singleton-type bound for locally repairable codes, people have put tremendous effort into construction of optimal locally repairable codes. There are a few constructions of optimal locally repairable codes in the literature. Most of these constructions are realized via either combinatorial or algebraic structures. In this paper, we apply automorphism group of the rational function field to construct optimal locally repairable codes by considering the group action on projective lines over finite fields. Due to various subgroups of the projective general linear group, we are able to construct optimal locally repairable codes with flexible locality as well as smaller alphabet size comparable to the code length. In particular, we produce new families of $q$ -ary locally repairable codes, including codes of length $q+1$ via cyclic groups.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-18
Ehsan Yavari; Morteza Esmaeili

Locally repairable codes (LRC) have been studied from two approaches to locally repair multiple failed nodes: 1) parallel approach, in which a coordinate $i$ of an $[n,k,d]$ linear code is said to have locality $r$ and availability $t$ if there exist $t$ disjoint repair sets each of which contains at most $r$ other coordinates that can recover the value of the $i$ -th coordinate; 2) sequential approach, in which the erased symbols (failed nodes) are repaired, one by one, and any previously repaired node can be used to repair the remaining failed nodes. In this paper, we first consider LRC aiming at joint sequential-parallel repairing multiple failed nodes, and study the $(n,k,r,t,u)$ -ELRCs (Exact locally repairable codes) which are $[n,k]$ linear codes with the property that any set of failed nodes of size at most $t$ can be simultaneously repaired in parallel mode, and each element of a set $E$ of failed nodes of size at most $u$ can be sequentially repaired by $r$ ( $r< k$ ) other coordinates. We present a method by which with a given parity-check matrix of an $(n,k,r,t,u)$ -ELRC with minimum Hamming distance $d$ , a new ELRC with minimum Hamming distance $2d$ and availability $t+1$ is constructed that can repair each set of failed nodes $E$ of size at most $2u+1$ in sequential mode and this repair is done in at most $u-t+2$ steps. We construct a big family of LRCs by making use of orthogonal Latin rectangles and permutation cubes and some other combinatorial designs; the constructed codes contain the family of direct product codes; we also use $m$ -dimensional permutation cubes to construct LRCs with short block length for each $r$ .

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-19
Jianfa Qian; Lina Zhang

Locally recoverable codes are very important due to their applications in distributed storage systems. In this paper, by using cyclic codes, we construct two classes of optimal q-ary cyclic $(\text {r}, \delta _{1})$ locally recoverable codes with parameters $[2(\text {q}+1), 2(\text {q}+1)-2\delta _{1}, \delta _{1}+2]_{\text {q}}$ , where $q$ is an odd prime power and $\text {r}+\delta _{1}-1=\text {q}+1$ , and $(\text {r}, \delta _{2})$ locally recoverable codes with parameters $[2(\text {q}+1), 2(\text {q}+1)-4\delta _{2}+2, \delta _{2}+2]_{q}$ , where $\text {q}\geq 7$ is an odd prime power, $\text {q}\equiv 3~ (mod~ 4)$ and $r+\delta _{2}-1=\frac {\text {q}+1}{2}$ . Compared with the known cyclic and constacyclic $(\text {r}, \delta)$ locally recoverable codes, our construction yields new optimal cyclic $(\text {r}, \delta)$ locally recoverable codes.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-03
Hai Q. Dinh; Bac T. Nguyen; Songsak Sriboonchitta

Let p be an odd prime, s and m be positive integers. Cyclic codes of length $2p^{{s}}$ over $\mathbb {F}_{{p}^{{m}}}$ are the ideals $\langle ( {x}-1)^{i}({x}+1)^{j} \rangle$ , where $0 \le {i}, {j} \le {p}^{ {s}}$ , of the principal ideal ring $\mathbb {F}_{{p}^{{m}}}[{x}]/\langle {x}^{2\textit {p}^{\textit {s}}}-1 \rangle$ . Using this structure, the symbol-pair distances of all cyclic codes of length $2\textit {p}^{\textit {s}}$ over $\mathbb F_{{p}^{{m}}}$ are completely determined. In addition, we establish all MDS symbol-pair cyclic codes of length $2\textit {p}^{\textit {s}}$ over $\mathbb F_{{p}^{{m}}}$ . Some MDS symbol-pair cyclic codes are better than all the known ones. Among others, we discuss possible applications to construct quantum MDS symbol-pair codes.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-03
Ronald Cramer; Chaoping Xing; Chen Yuan

Reed-Muller codes are among the most important classes of locally correctable codes. Currently local decoding of Reed-Muller codes is based on decoding on lines or quadratic curves to recover one single coordinate. To recover multiple coordinates simultaneously, the naive way is to repeat the local decoding for recovery of a single coordinate. This decoding algorithm might be more expensive, i.e., require higher query complexity. In this paper, we focus on Reed-Muller codes with usual parameter regime, namely, the total degree of evaluation polynomials is $d=\Theta ({q})$ , where $q$ is the code alphabet size (in fact, $d$ can be as big as $q/4$ in our setting). By introducing a novel variation of codex, i.e., interleaved codex (the concept of codex has been used for arithmetic secret sharing), we are able to locally recover arbitrarily large number $k$ of coordinates of a Reed-Muller code simultaneously with error probability $\exp (-\Omega (k))$ at the cost of querying merely $O(q^{2}k)$ coordinates. It turns out that our local decoding of Reed-Muller codes shows ( perhaps surprisingly ) that accessing $k$ locations is in fact cheaper than repeating the procedure for accessing a single location for $k$ times. Precisely speaking, to get the same success probability by repeating the local decoding algorithm of a single coordinate, one has to query $\Omega (qk^{2})$ coordinates. Thus, the query complexity of our local decoding is smaller for $k=\Omega (q)$ . If we impose the same query complexity constraint on both algorithm, our local decoding algorithm yields smaller error probability when $k=\Omega (q^{q})$ . In addition, our local decoding is efficient, i.e., the decoding complexity is ${\mathrm{ Poly}}(k,q)$ . Construction of an interleaved codex is based on concatenation of a codex with a multiplication friendly pair, while the main tool to realize codex is based on algebraic function fields (or more precisely, algebraic geometry codes).

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-11
Jürgen Bierbrauer; Stefano Marcugini; Fernanda Pambianco

We study additive quaternary codes whose parameters are close to those of the extended cyclic $[{12,6,6}]_{4}$ -code or to the quaternary linear codes generated by the elliptic quadric in $PG(3,4)$ or its dual. In particular we characterize those codes in the category of additive codes and construct some additive codes whose parameters are better than those of any linear quaternary code. Our new code parameters are $[{22,17.5,4}]_{4}$ .

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-07-17

We provide novel coded computation strategies for distributed matrix–matrix products that outperform the recent “Polynomial code” constructions in recovery threshold, i.e., the required number of successful workers. When a fixed $1/m$ fraction of each matrix can be stored at each worker node, Polynomial codes require $m^{2}$ successful workers, while our MatDot codes only require $2m-1$ successful workers. However, MatDot codes have higher computation cost per worker and higher communication cost from each worker to the fusion node. We also provide a systematic construction of MatDot codes. Furthermore, we propose “PolyDot” coding that interpolates between Polynomial codes and MatDot codes to trade off computation/communication costs and recovery thresholds. Finally, we demonstrate a novel coding technique for multiplying $n$ matrices ( $n \geq 3$ ) using ideas from MatDot and PolyDot codes.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-07
Daniel Russo; James Zou

Modern data is messy and high-dimensional, and it is often not clear a priori what are the right questions to ask. Instead, the analyst typically needs to use the data to search for interesting analyses to perform and hypotheses to test. This is an adaptive process, where the choice of analysis to be performed next depends on the results of the previous analyses on the same data. Ultimately, which results are reported can be heavily influenced by the data. It is widely recognized that this process, even if well-intentioned, can lead to biases and false discoveries, contributing to the crisis of reproducibility in science. But while any data-exploration renders standard statistical theory invalid, experience suggests that different types of exploratory analysis can lead to disparate levels of bias, and the degree of bias also depends on the particulars of the data set. In this paper, we propose a general information usage framework to quantify and provably bound the bias and other error metrics of an arbitrary exploratory analysis. We prove that our mutual information based bound is tight in natural settings, and then use it to give rigorous insights into when commonly used procedures do or do not lead to substantially biased estimation. Through the lens of information usage, we analyze the bias of specific exploration procedures such as filtering, rank selection and clustering. Our general framework also naturally motivates randomization techniques that provably reduce exploration bias while preserving the utility of the data analysis. We discuss the connections between our approach and related ideas from differential privacy and blinded data analysis, and supplement our results with illustrative simulations.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-08-09
Liangjun Su; Wuyi Wang; Yichong Zhang

In this paper we prove the strong consistency of several methods based on the spectral clustering techniques that are widely used to study the community detection problem in stochastic block models (SBMs). We show that under some weak conditions on the minimal degree, the number of communities, and the eigenvalues of the probability block matrix, the K-means algorithm applied to the eigenvectors of the graph Laplacian associated with its first few largest eigenvalues can classify all individuals into the true community uniformly correctly almost surely. Extensions to both regularized spectral clustering and degree-corrected SBMs are also considered. We illustrate the performance of different methods on simulated networks.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-08-16
Aditya Kela; Kai Von Prillwitz; Johan Åberg; Rafael Chaves; David Gross

Testing whether a probability distribution is compatible with a given Bayesian network is a fundamental task in the field of causal inference, where Bayesian networks model causal relations. Here we consider the class of causal structures where all correlations between observed quantities are solely due to the influence from latent variables. We show that each model of this type imposes a certain signature on the observable covariance matrix in terms of a particular decomposition into positive semidefinite components. This signature, and thus the underlying hypothetical latent structure, can be tested in a computationally efficient manner via semidefinite programming. This stands in stark contrast with the algebraic geometric tools required if the full observable probability distribution is taken into account. The semidefinite test is compared with tests based on entropic inequalities.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-15
Shang Li; Xiaodong Wang

This work considers the cooperative sequential hypothesis testing problem in a distributed network with quantized communication channels. The sensors observe independent sequences of samples and in the meantime, exchange their local information in the form of quantized statistics at every sampling interval. The communication links are represented as an undirected graph. In this distributed setup, every sensor performs its own sequential test based on the local samples and the messages from the neighbour sensors. Our goal is to devise the distributed sequential test that comprises the quantization scheme, the message-exchange protocol and the test procedure such that every sensor in the network fully exploits the network diversity and achieves the (asymptotically) optimal performance in terms of the stopping time. In particular, two distributed sequential tests are proposed based on different quantization schemes and a quantized message-exchange protocol that satisfies certain conditions. The first quantization scheme uniformly quantizes the local statistic at each sensor and at every sampling interval; the second one hinges on a modified level-triggered quantization technique, and resembles the Lebesgue sampling of the running local statistic. Our analyses show that the uniform quantization based distributed sequential test yields sub-optimal performance, while the one based on level-triggered quantization achieves the order-2 asymptotically optimal performance at every sensor for any fixed quantization step-size. Furthermore, we generalize the proposed sequential tests to the cluster-based network. Numerical results are provided to corroborate our analyses and demonstrate the effectiveness of the proposed sequential tests.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-11
Keigo Takeuchi

Signal recovery from unitarily invariant measurements is investigated in this paper. A message-passing algorithm is formulated on the basis of expectation propagation (EP). A rigorous analysis is presented for the dynamics of the algorithm in the large system limit, where both input and output dimensions tend to infinity while the compression rate is kept constant. The main result is the justification of state evolution (SE) equations conjectured by Ma and Ping. This result implies that the EP-based algorithm achieves the Bayes-optimal performance that was originally derived via a non-rigorous tool in statistical physics and proved partially in a recent paper, when the compression rate is larger than a threshold. The proof is based on an extension of a conventional conditioning technique for the standard Gaussian matrix to the case of the Haar matrix.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-15
Shirin Jalali

Noiseless compressed sensing refers to the problem of recovering a (high-dimensional) signal from its under-determined linear measurements. For compressed sensing to be feasible, the signal needs to be structured. While the main focus of the field has been on simple structures such as sparsity, there has been a growing interest in moving beyond sparsity and having a comprehensive compressed sensing framework that covers general structures. Two recent approaches that aim at developing such a framework from different perspectives are i) Quantized maximum a posteriori (Q-MAP), a Bayesian method that assumes full knowledge of the source distribution, and ii) Lagrangian minimum entropy pursuit (L-MEP), a universal recovery method that requires no prior knowledge about the distribution of the source. In this paper, by establishing theoretical connections between L-MEP and Q-MAP, it is shown how the two methods are complementary to each other and lead to a theoretically-founded learning-based recovery method that applies to sources with general structures. Unlike a Bayesian or a universal method, a learning-based method is able to extract the source structure from training data. The effect of error in estimating the source structure on the performance of the learning-based compressed sensing recovery method is characterized.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-08-15

We examine the theoretical properties of enforcing priors provided by generative deep neural networks via empirical risk minimization. In particular we consider two models, one in which the task is to invert a generative neural network given access to its last layer and another in which the task is to invert a generative neural network given only compressive linear observations of its last layer. We establish that in both cases, in suitable regimes of network layer sizes and a randomness assumption on the network weights, that the non-convex objective function given by empirical risk minimization does not have any spurious stationary points. That is, we establish that with high probability, at any point away from small neighborhoods around two scalar multiples of the desired solution, there is a descent direction. Hence, there are no local minima, saddle points, or other stationary points outside these neighborhoods. These results constitute the first theoretical guarantees which establish the favorable global geometry of these non-convex optimization problems, and they bridge the gap between the empirical success of enforcing deep generative priors and a rigorous understanding of non-linear inverse problems.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-10
Yuqian Zhang; Han-Wen Kuo; John Wright

Blind deconvolution is a ubiquitous problem aiming to recover a convolution kernel $\boldsymbol a_{0}\in \mathbb R ^{k}$ and an activation signal $\boldsymbol x_{0}\in \mathbb R ^{m}$ from their convolution $\boldsymbol y\in \mathbb R ^{m}$ . Unfortunately, this is an ill-posed problem in general. This paper focuses on the short and sparse blind deconvolution problem, where the convolution kernel is short ( $k\ll m$ ) and the activation signal is sparsely and randomly supported ( $\left \|{ \boldsymbol x_{0} }\right \|_{0}\ll m$ ). This variant captures the structure of the convolutional signals in several important application scenarios. In this paper, we normalize the convolution kernel to have unit Frobenius norm and then cast the blind deconvolution problem as a nonconvex optimization problem over the kernel sphere. We demonstrate that (i) in a certain region of the sphere, every local optimum is close to some shift truncation of the ground truth, and (ii) for a generic unit kernel $\boldsymbol a_{0}$ , when the sparsity of activation signal satisfies $\theta \lesssim k^{-2/3}$ and number of measurements $m\gtrsim \mathop {\mathrm {poly}}\nolimits \left ({k }\right)$ , the proposed initialization method together with a descent algorithm which escapes strict saddle points recovers some shift truncation of the ground truth kernel.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-08-01
Martin Genzel; Peter Jung

This work deals with the problem of distributed data acquisition under non-linear communication constraints. More specifically, we consider a model setup where $M$ distributed nodes take individual measurements of an unknown structured source vector $\boldsymbol {x}_{0}\in \mathbb {R}^{n}$ , communicating their readings simultaneously to a central receiver . Since this procedure involves collisions and is usually imperfect, the receiver measures a superposition of non-linearly distorted signals . In a first step, we will show that an $s$ -sparse vector $\boldsymbol {x}_{0}$ can be successfully recovered from $O(s \cdot \log (2n/s)$ of such superimposed measurements, using a traditional Lasso estimator that does not rely on any knowledge about the non-linear corruptions. This direct method however fails to work for several “uncalibrated” system configurations. These blind reconstruction tasks can be easily handled with the $\ell ^{}$ -Group-Lasso, but coming along with an increased sampling rate of $O(s\cdot \max \{M, \log (2n/s) \}$ observations — in fact, the purpose of this lifting strategy is to extend a certain class of bilinear inverse problems to non-linear acquisition. Our two algorithmic approaches are a special instance of a more abstract framework which includes sub-Gaussian measurement designs as well as general (convex) structural constraints. These results are of independent interest for various recovery and learning tasks, as they apply to arbitrary non-linear observation models. Finally, to illustrate the practical scope of our theoretical findings, an application to wireless sensor networks is discussed, which actually serves as the prototypical example of our methodology.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-16
Mansoor Yousefi; Xianhe Yangzhang

Two signal multiplexing schemes for optical fiber communication are considered: Wavelength-division multiplexing (WDM) and nonlinear frequency-division multiplexing (NFDM), based on the nonlinear Fourier transform. Achievable information rates (AIRs) of NFDM and WDM are compared in a network scenario with an ideal lossless model of the optical fiber in the defocusing regime. It is shown that the NFDM AIR is greater than the WDM AIR subject to a bandwidth and average power constraint, in a representative system with one symbol per user. The improvement results from nonlinear signal multiplexing.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-07
Alejandro Lancho; Tobias Koch; Giuseppe Durisi

This article concerns the maximum coding rate at which data can be transmitted over a noncoherent, single-antenna, Rayleigh block-fading channel using an error-correcting code of a given blocklength with a block-error probability not exceeding a given value. A high-SNR normal approximation of the maximum coding rate is presented that becomes accurate as the signal-to-noise ratio (SNR) and the number of coherence intervals $L$ over which we code tend to infinity. Numerical analyses suggest that the approximation is accurate at SNR values above 15dB and when the number of coherence intervals is 10 or more.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-08-23
Xinping Yi; Hua Sun

We consider a $K$ -user interference network with $M$ states, where each transmitter has up to $M$ messages and over State $m$ , Receiver $k$ wishes to decode the first $\pi _{k}(m) \in \{ 1,2,\cdots ,M \}$ messages from its desired transmitter. This problem of channel with states models opportunistic communications, where more messages are decoded for better channel states. The first message from each transmitter has the highest priority as it is required to be decoded regardless of the state of the receiver; the second message is opportunistically decoded if the state allows a receiver to decode 2 messages; and the $M$ -th message has the lowest priority as it is decoded if and only if the receiver wishes to decode all $M$ messages. For this interference network with states, we show that if any possible combination of the channel states satisfies a condition under which power control and treating interference as noise (TIN) are sufficient to achieve the entire generalized degrees of freedom (GDoF) region of this channel state by itself, then a simple layered superposition encoding scheme with power control and a successive decoding scheme with TIN achieves the entire GDoF region of the network with $M$ states for all $KM$ messages.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-02
Ahmed Arafa; Jing Yang; Sennur Ulukus; H. Vincent Poor

An energy-harvesting sensor node that is sending status updates to a destination is considered. The sensor is equipped with a battery of finite size to save its incoming energy, and consumes one unit of energy per status update transmission, which is delivered to the destination instantly over an error-free channel. The setting is online in which the harvested energy is revealed to the sensor causally over time after it arrives, and the goal is to design status update transmission times (policy) such that the long term average age of information (AoI) is minimized. The AoI is defined as the time elapsed since the latest update has reached at the destination. Two energy arrival models are considered: a random battery recharge (RBR) model, and an incremental battery recharge (IBR) model. In both models, energy arrives according to a Poisson process with unit rate, with values that completely fill up the battery in the RBR model, and with values that fill up the battery incrementally in a unit-by-unit fashion in the IBR model. The key approach to characterizing the optimal status update policy for both models is showing the optimality of renewal policies , in which the inter-update times follow a renewal process in a certain manner that depends on the energy arrival model and the battery size. It is then shown that the optimal renewal policy has an energy-dependent threshold structure, in which the sensor sends a status update only if the AoI grows above a certain threshold that depends on the energy available in its battery. For both the random and the incremental battery recharge models, the optimal energy-dependent thresholds are characterized explicitly , i.e., in closed-form, in terms of the optimal long term average AoI. It is also shown that the optimal thresholds are monotonically decreasing in the energy available in the battery, and that the smallest threshold, which comes in effect when the battery is full, is equal to the optimal long term average AoI.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-19
Simon R. Blackburn; Tuvi Etzion; Maura B. Paterson

In the classical model for (information theoretically secure) Private Information Retrieval (PIR) due to Chor, Goldreich, Kushilevitz and Sudan, a user wishes to retrieve one bit of a database that is stored on a set of ${n}$ servers, in such a way that no individual server gains information about which bit the user is interested in. The aim is to design schemes that minimise the total communication between the user and the servers. More recently, there have been moves to consider more realistic models where the total storage of the set of servers, or the per server storage, should be minimised (possibly using techniques from distributed storage), and where the database is divided into ${R}$ -bit records with ${R}>1$ , and the user wishes to retrieve one record rather than one bit. When ${R}$ is large, downloads from the servers to the user dominate the communication complexity and so the aim is to minimise the total number of downloaded bits. Work of Shah, Rashmi and Ramchandran shows that at least ${R}+1$ bits must be downloaded from servers in the worst case, and provides PIR schemes meeting this bound. Sun and Jafar have considered the download cost of a scheme, defined as the ratio of the message length ${R}$ and the total number of bits downloaded. They determine the best asymptotic download cost of a PIR scheme (as ${R}\rightarrow \infty$ ) when a database of ${k}$ messages is stored by ${n}$ servers. This paper provides various bounds on the download complexity of a PIR scheme, generalising those of Shah et al. to the case when the number ${n}$ of servers is bounded, and providing links with classical techniques due to Chor et al. The paper also provides a range of constructions for PIR schemes that are either simpler or perform better than previously known schemes. These constructions include explicit schemes that achieve the best asymptotic download complexity of Sun and Jafar with significantly lower upload complexity, and general techniques for constructing a scheme with good worst case download complexity from a scheme with good download complexity on average.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-08
Rémi A. Chou; Aylin Yener

We consider authentication of messages sent from transmitters to a receiver over a multiple access channel, where each transmitter shares a secret key with the legitimate receiver. Additionally, there exists a computationally unbounded opponent who has access to noisy observations of the messages transmitted and can initiate impersonation or substitution attacks. We require that the legitimate receiver must be able to authenticate the messages he receives with respect to predetermined groups of transmitters, but at the same time must be kept ignorant of the transmitter’s identity of a given message in a given group. We propose an information-theoretic formulation of these anonymity constraints as well as an authentication coding scheme for which the asymptotic probability of successful attack is shown to optimally scale with the length of the secret keys shared between each transmitter and the legitimate receiver. Our results quantify the positive impact of the multiple access setting compared to the single-user setting on the probability of successful attack.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-02
Augusto Santos; Vincenzo Matta; Ali H. Sayed

This article studies the problem of reconstructing the topology of a network of interacting agents via observations of the state-evolution of the agents. We focus on the large-scale network setting with the additional constraint of partial observations, where only a small fraction of the agents can be feasibly observed. The goal is to infer the underlying subnetwork of interactions and we refer to this problem as local tomography . In order to study the large-scale setting, we adopt a proper stochastic formulation where the unobserved part of the network is modeled as an Erdős-Rényi random graph, while the observable subnetwork is left arbitrary. The main result of this work is to establish that, under this setting, local tomography is actually possible with high probability, provided that certain conditions on the network model are met (such as stability and symmetry of the network combination matrix). Remarkably, such conclusion is established under the low-observability regime , where the cardinality of the observable subnetwork is fixed, while the size of the overall network scales to infinity.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-11
Omer Bilgen; Aaron B. Wagner

Recent studies have suggested that the stability of peer-to-peer networks may rely on persistent peers , who dwell on the network after they obtain the entire file. In the absence of such peers, one piece becomes extremely rare in the network, which leads to instability. Technological developments, however, are poised to reduce the incidence of persistent peers, giving rise to a need for a protocol that guarantees stability with non-persistent peers. We propose a novel peer-to-peer protocol, the group suppression protocol , to ensure the stability of peer-to-peer networks under the scenario that all the peers adopt non-persistent behavior. Using a suitable Lyapunov potential function, the group suppression protocol is proven to be stable when the file is broken into two pieces, and detailed experiments demonstrate the stability of the protocol for arbitrary number of pieces. We define and simulate a decentralized version of this protocol for practical applications. Straightforward incorporation of the group suppression protocol into BitTorrent while retaining most of BitTorrent’s core mechanisms is also presented. Subsequent simulations show that under certain assumptions, BitTorrent with the official protocol cannot escape from the missing piece syndrome, but BitTorrent with group suppression does.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-10-09
Seyed Ali Saberali; Lutz Lampe; Ian F. Blake

We investigate the problem of coded caching for nonuniform demands when the structured clique cover algorithm proposed by Maddah-Ali and Niesen for decentralized caching is used for delivery. We apply this algorithm to all user demands regardless of their request probabilities. This allows for coding among the files that have different request probabilities but makes the allocation of memory to different files challenging during the content placement phase. As our main contribution, we analytically characterize the optimal placement strategy that minimizes the expected delivery rate under a storage capacity constraint. It is shown that the optimal placement follows either a two or a three group strategy, where a set of less popular files are not cached at all and the files within each of the other sets are allocated identical amounts of storage as if they had the same request probabilities. We show that for a finite set of storage capacities, that we call the base-cases of the problem, the two group strategy is always optimal. For other storage capacities, optimal placement is achieved by memory sharing between certain base-cases and the resulting placement either follows a two or a three group strategy depending on the corresponding base-cases used. We derive a polynomial time algorithm that determines the base-cases of the problem given the number of caches and popularity distribution of files. Given the base-cases of the problem, the optimal memory allocation parameters for any storage capacity are derived analytically.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-17
Jorge F. Silva; Pablo Piantanida

Motivated from the fact that universal source coding on countably infinite alphabets ( $\infty$ -alphabets) is not feasible, this work introduces the notion of “almost lossless source coding”. Analog to the weak variable-length source coding problem studied by Han ( IEEE Trans. Inf. Theory , vol. 46, no. 4, pp. 1217–1226, Jul. 2000), almost lossless source coding aims at relaxing the lossless block-wise assumption to allow an average per-letter distortion that vanishes asymptotically as the block-length tends to infinity. In this setup, we show on one hand that Shannon entropy characterizes the minimum achievable rate (similarly to the case of finite alphabet sources) while on the other that almost lossless universal source coding becomes feasible for the family of finite-entropy stationary memoryless sources with $\infty$ -alphabets. Furthermore, we study a stronger notion of almost lossless universality that demands uniform convergence of the average per-letter distortion to zero, where we establish a necessary and sufficient condition for the so-called family of “envelope distributions” to achieve it. Remarkably, this condition is the same necessary and sufficient condition needed for the existence of a strongly minimax (lossless) universal source code for the family of envelope distributions. Finally, we show that an almost lossless coding scheme offers faster rate of convergence for the (minimax) redundancy compared to the well-known information radius developed for the lossless case at the expense of tolerating a non-zero distortion that vanishes to zero as the block-length grows. This shows that even when lossless universality is feasible, an almost lossless scheme can offer different regimes on the rates of convergence of the (worst case) redundancy versus the (worst case) distortion.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-27
Avik Ranjan Adhikary; Sudhan Majhi; Zilong Liu; Yong Liang Guan

A pair of sequences is called a Z-complementary pair (ZCP) if it has zero aperiodic autocorrelation sums (AACSs) for time-shifts within a certain region, called zero correlation zone (ZCZ). Optimal odd-length binary ZCPs (OB-ZCPs) display closest correlation properties to Golay complementary pairs (GCPs) in that each OB-ZCP achieves maximum ZCZ of width $({N}+1)/2$ (where N is the sequence length) and every out-of-zone AACSs reaches the minimum magnitude value, i.e. 2. Till date, systematic constructions of optimal OB-ZCPs exist only for lengths $2^{\alpha } \pm 1$ , where $\alpha$ is a positive integer. In this paper, we construct optimal OB-ZCPs of generic lengths $2^\alpha 10^\beta 26^\gamma +1$ (where $\alpha,~\beta,~\gamma$ are non-negative integers and $\alpha \geq 1$ ) from inserted versions of binary GCPs. The key leading to the proposed constructions is several newly identified structure properties of binary GCPs obtained from Turyn’s method. This key also allows us to construct OB-ZCPs with possible ZCZ widths of $4 \times 10^{\beta -1} +1$ , $12 \times 26^{\gamma -1}+1$ and $12 \times 10^\beta 26^{\gamma -1}+1$ through proper insertions of GCPs of lengths $10^\beta,~26^\gamma, \text {and } 10^\beta 26^\gamma$ , respectively. Our proposed OB-ZCPs have applications in communications and radar (as an alternative to GCPs).

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-07-15
D. Gabric; J. Sawada; A. Williams; D. Wong

We present a simple framework for constructing $k$ -ary de Bruijn sequences, and more generally, universal cycles, via successor rules. The framework is based on the often used method of joining disjoint cycles. It generalizes several previously known de Bruijn sequence constructions based on the pure cycling register and is applied to derive a new construction that is perhaps the simplest of all successors. Furthermore, it generalizes an algorithm to construct binary de Bruijn sequences based on any arbitrary nonsingular feedback function. The framework is applied to derive and prove the correctness of successors to efficiently construct 1) universal cycles for $k$ -ary strings of length $n$ whose weight is bounded by some $w$ and 2) universal cycles for permutations. It has also been subsequently applied to find the first universal cycle constructions for weak orders.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-12-23

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-12-23

Provides instructions and guidelines to prospective authors who wish to submit manuscripts.

更新日期：2020-01-04
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2008-07-01
M Brandon Westover

We present a simple geometrical interpretation for the solution to the multiple hypothesis testing problem in the asymptotic limit. Under this interpretation, the optimal decision rule is a nearest neighbor classifier on the probability simplex.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2019-09-21
Abram Magner,Krzysztof Turowski,Wojciech Szpankowski

Compression schemes for advanced data structures have become a central modern challenge. Information theory has traditionally dealt with conventional data such as text, images, or video. In contrast, most data available today is multitype and context-dependent. To meet this challenge, we have recently initiated a systematic study of advanced data structures such as unlabeled graphs [8]. In this paper, we continue this program by considering trees with statistically correlated vertex names. Trees come in many forms, but here we deal with binary plane trees (where order of subtrees matters) and their non-plane version (where order of subtrees doesn't matter). Furthermore, we assume that each name is generated by a known memoryless source (horizontal independence), but a symbol of a vertex name depends in a Markovian sense on the corresponding symbol of the parent vertex name (vertical Markovian dependency). Such a model is closely connected to models of phylogenetic trees. While in general the problem of multimodal compression and associated analysis can be extremely complicated, we find that in this natural setting, both the entropy analysis and optimal compression are analytically tractable. We evaluate the entropy for both types of trees. For the plane case, with or without vertex names, we find that a simple two-stage compression scheme is both efficient and optimal. We then present efficient and optimal compression algorithms for the more complicated non-plane case.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2015-05-01
Jiantao Jiao,Kartik Venkat,Yanjun Han,Tsachy Weissman

We propose a general methodology for the construction and analysis of essentially minimax estimators for a wide class of functionals of finite dimensional parameters, and elaborate on the case of discrete distributions, where the support size S is unknown and may be comparable with or even much larger than the number of observations n. We treat the respective regions where the functional is nonsmooth and smooth separately. In the nonsmooth regime, we apply an unbiased estimator for the best polynomial approximation of the functional whereas, in the smooth regime, we apply a bias-corrected version of the maximum likelihood estimator (MLE). We illustrate the merit of this approach by thoroughly analyzing the performance of the resulting schemes for estimating two important information measures: 1) the entropy [Formula: see text] and 2) [Formula: see text], α > 0. We obtain the minimax L2 rates for estimating these functionals. In particular, we demonstrate that our estimator achieves the optimal sample complexity n ≍ S/ln S for entropy estimation. We also demonstrate that the sample complexity for estimating Fα (P), 0 < α < 1, is n ≍ S1/α /ln S, which can be achieved by our estimator but not the MLE. For 1 < α < 3/2, we show the minimax L2 rate for estimating Fα (P) is (n ln n)-2(α-1) for infinite support size, while the maximum L2 rate for the MLE is n-2(α-1). For all the above cases, the behavior of the minimax rate-optimal estimators with n samples is essentially that of the MLE (plug-in rule) with n ln n samples, which we term "effective sample size enlargement." We highlight the practical advantages of our schemes for the estimation of entropy and mutual information. We compare our performance with various existing approaches, and demonstrate that our approach reduces running time and boosts the accuracy. Moreover, we show that the minimax rate-optimal mutual information estimator yielded by our framework leads to significant performance boosts over the Chow-Liu algorithm in learning graphical models. The wide use of information measure estimation suggests that the insights and estimators obtained in this paper could be broadly applicable.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2015-05-01

The problem of performing similarity queries on compressed data is considered. We focus on the quadratic similarity measure, and study the fundamental tradeoff between compression rate, sequence length, and reliability of queries performed on the compressed data. For a Gaussian source, we show that the queries can be answered reliably if and only if the compression rate exceeds a given threshold-the identification rate- which we explicitly characterize. Moreover, when compression is performed at a rate greater than the identification rate, responses to queries on the compressed data can be made exponentially reliable. We give a complete characterization of this exponent, which is analogous to the error and excess-distortion exponents in channel and source coding, respectively. For a general source, we prove that, as with classical compression, the Gaussian source requires the largest compression rate among sources with a given variance. Moreover, a robust scheme is described that attains this maximal rate for any source distribution.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2010-02-01
Michael C Gastpar,Patrick R Gill,Alexander G Huth,Frédéric E Theunissen

Information theory has been used as an organizing principle in neuroscience for several decades. Estimates of the mutual information (MI) between signals acquired in neurophysiological experiments are believed to yield insights into the structure of the underlying information processing architectures. With the pervasive availability of recordings from many neurons, several information and redundancy measures have been proposed in the recent literature. A typical scenario is that only a small number of stimuli can be tested, while ample response data may be available for each of the tested stimuli. The resulting asymmetric information estimation problem is considered. It is shown that the direct plug-in information estimate has a negative bias. An anthropic correction is introduced that has a positive bias. These two complementary estimators and their combinations are natural candidates for information estimation in neuroscience. Tail and variance bounds are given for both estimates. The proposed information estimates are applied to the analysis of neural discrimination and redundancy in the avian auditory system.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2015-01-30
Tuo Zhao,Han Liu

We propose a semiparametric method for estimating a precision matrix of high-dimensional elliptical distributions. Unlike most existing methods, our method naturally handles heavy tailness and conducts parameter estimation under a calibration framework, thus achieves improved theoretical rates of convergence and finite sample performance on heavy-tail applications. We further demonstrate the performance of the proposed method using thorough numerical experiments.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2008-01-01
Matthew T Harrison

Suppose a string X1n=(X1,X2,…,Xn) generated by a memoryless source (X(n))(n≥1) with distribution P is to be compressed with distortion no greater than D ≥ 0, using a memoryless random codebook with distribution Q. The compression performance is determined by the "generalized asymptotic equipartition property" (AEP), which states that the probability of finding a D-close match between X1n and any given codeword Y1n, is approximately 2(-nR(P, Q, D)), where the rate function R(P, Q, D) can be expressed as an infimum of relative entropies. The main purpose here is to remove various restrictive assumptions on the validity of this result that have appeared in the recent literature. Necessary and sufficient conditions for the generalized AEP are provided in the general setting of abstract alphabets and unbounded distortion measures. All possible distortion levels D ≥ 0 are considered; the source (X(n))(n≥1) can be stationary and ergodic; and the codebook distribution can have memory. Moreover, the behavior of the matching probability is precisely characterized, even when the generalized AEP is not valid. Natural characterizations of the rate function R(P, Q, D) are established under equally general conditions.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2010-02-01
Yaniv Erlich,Assaf Gordon,Michael Brand,Gregory J Hannon,Partha P Mitra

Over the past three decades we have steadily increased our knowledge on the genetic basis of many severe disorders. Nevertheless, there are still great challenges in applying this knowledge routinely in the clinic, mainly due to the relatively tedious and expensive process of genotyping. Since the genetic variations that underlie the disorders are relatively rare in the population, they can be thought of as a sparse signal. Using methods and ideas from compressed sensing and group testing, we have developed a cost-effective genotyping protocol to detect carriers for severe genetic disorders. In particular, we have adapted our scheme to a recently developed class of high throughput DNA sequencing technologies. The mathematical framework presented here has some important distinctions from the 'traditional' compressed sensing and group testing frameworks in order to address biological and technical constraints of our setting.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2010-04-09

An essential step towards understanding how the brain orchestrates information processing at the cellular and population levels is to simultaneously observe the spiking activity of cortical neurons that mediate perception, learning, and motor processing. In this paper, we formulate an information theoretic approach to determine whether cooperation among neurons may constitute a governing mechanism of information processing when encoding external covariates. Specifically, we show that conditional independence between neuronal outputs may not provide an optimal encoding strategy when the firing probability of a neuron depends on the history of firing of other neurons connected to it. Rather, cooperation among neurons can provide a "message-passing" mechanism that preserves most of the information in the covariates under specific constraints governing their connectivity structure. Using a biologically plausible statistical learning model, we demonstrate the performance of the proposed approach in synergistically encoding a motor task using a subset of neurons drawn randomly from a large population. We demonstrate its superiority in approximating the joint density of the population from limited data compared to a statistically independent model and a maximum entropy (MaxEnt) model.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2008-12-17
Xiaodong Lin,Jennifer Pittman,Bertrand Clarke

Consider the relative entropy between a posterior density for a parameter given a sample and a second posterior density for the same parameter, based on a different model and a different data set. Then the relative entropy can be minimized over the second sample to get a virtual sample that would make the second posterior as close as possible to the first in an informational sense. If the first posterior is based on a dependent dataset and the second posterior uses an independence model, the effective inferential power of the dependent sample is transferred into the independent sample by the optimization. Examples of this optimization are presented for models with nuisance parameters, finite mixture models, and models for correlated data. Our approach is also used to choose the effective parameter size in a Bayesian hierarchical model.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2010-02-01
Aurel A Lazar

The recovery of (weak) stimuli encoded with a population of Hodgkin-Huxley neurons is investigated. In the absence of a stimulus, the Hodgkin-Huxley neurons are assumed to be tonically spiking. The methodology employed calls for 1) finding an input-output (I/O) equivalent description of the Hodgkin-Huxley neuron and 2) devising a recovery algorithm for stimuli encoded with the I/O equivalent neuron(s). A Hodgkin-Huxley neuron with multiplicative coupling is I/O equivalent with an Integrate-and-Fire neuron with a variable threshold sequence. For bandlimited stimuli a perfect recovery of the stimulus can be achieved provided that a Nyquist-type rate condition is satisfied. A Hodgkin-Huxley neuron with additive coupling and deterministic conductances is first-order I/O equivalent with a Project-Integrate-and-Fire neuron that integrates a projection of the stimulus on the phase response curve. The stimulus recovery is formulated as a spline interpolation problem in the space of finite length bounded energy signals. A Hodgkin-Huxley neuron with additive coupling and stochastic conductances is shown to be first-order I/O equivalent with a Project-Integrate-and-Fire neuron with random thresholds. For stimuli modeled as elements of Sobolev spaces the reconstruction algorithm minimizes a regularized quadratic optimality criterion. Finally, all previous recovery results of stimuli encoded with Hodgkin-Huxley neurons with multiplicative and additive coupling, and deterministic and stochastic conductances are extended to stimuli encoded with a population of Hodgkin-Huxley neurons.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2010-05-01
Shu Yang,Eric D Kolaczyk

A method of 'network filtering' has been proposed recently to detect the effects of certain external perturbations on the interacting members in a network. However, with large networks, the goal of detection seems a priori difficult to achieve, especially since the number of observations available often is much smaller than the number of variables describing the effects of the underlying network. Under the assumption that the network possesses a certain sparsity property, we provide a formal characterization of the accuracy with which the external effects can be detected, using a network filtering system that combines Lasso regression in a sparse simultaneous equation model with simple residual analysis. We explore the implications of the technical conditions underlying our characterization, in the context of various network topologies, and we illustrate our method using simulated data.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2012-01-31
Jianqing Fan,Jinchi Lv

Penalized likelihood methods are fundamental to ultra-high dimensional variable selection. How high dimensionality such methods can handle remains largely unknown. In this paper, we show that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of Non-Polynomial (NP) order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions. This fills a long-standing gap in the literature where the dimensionality is allowed to grow slowly with the sample size. Our results are also applicable to penalized likelihood with the L(1)-penalty, which is a convex function at the boundary of the class of folded-concave penalty functions under consideration. The coordinate optimization is implemented for finding the solution paths, whose performance is evaluated by a few simulation examples and the real data analysis.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2016-05-01
Fabian Steiner,Steffen Dempfle,Amir Ingber,Tsachy Weissman

We study the problem of compression for the purpose of similarity identification, where similarity is measured by the mean square Euclidean distance between vectors. While the asymptotical fundamental limits of the problem - the minimal compression rate and the error exponent - were found in a previous work, in this paper we focus on the nonasymptotic domain and on practical, implementable schemes. We first present a finite blocklength achievability bound based on shape-gain quantization: The gain (amplitude) of the vector is compressed via scalar quantization and the shape (the projection on the unit sphere) is quantized using a spherical code. The results are numerically evaluated and they converge to the asymptotic values as predicted by the error exponent. We then give a nonasymptotic lower bound on the performance of any compression scheme, and compare to the upper (achievability) bound. For a practical implementation of such a scheme, we use wrapped spherical codes, studied by Hamkins and Zeger, and use the Leech lattice as an example for an underlying lattice. As a side result, we obtain a bound on the covering angle of any wrapped spherical code, as a function of the covering radius of the underlying lattice.

更新日期：2019-11-01
• IEEE Trans. Inform. Theory (IF 3.215) Pub Date : 2013-01-01
Kittipong Kittichokechai,Yeow-Khiang Chia,Tobias J Oechtering,Mikael Skoglund,Tsachy Weissman

We consider secure multi-terminal source coding problems in the presence of a public helper. Two main scenarios are studied: 1) source coding with a helper where the coded side information from the helper is eavesdropped by an external eavesdropper, 2) triangular source coding with a helper where the helper is considered as a public terminal. We are interested in how the helper can support the source transmission subject to a constraint on the amount of information leaked due to its public nature. We characterize the tradeoff between transmission rate, incurred distortion, and information leakage rate at the helper/eavesdropper in the form of a rate-distortion-leakage region for various classes of problems.

更新日期：2019-11-01
Contents have been reproduced by permission of the publishers.

down
wechat
bug