Skip to main content
SpringerLink
Log in

Polynomial Asymptotically Optimal Coding of Underdetermined Bernoulli Sources of the General Form

  • Source Coding
  • Published:
Problems of Information Transmission Aims and scope Submit manuscript

Abstract

An underdetermined Bernoulli source generates symbols of a given underdetermined alphabet independently with some probabilities. To each underdetermined symbol there corresponds a set of basic (fully defined) symbols such that it can be substituted (specified) by any of them. An underdetermined source is characterized by its entropy, which is implicitly introduced as a minimum of a certain function and plays a role similar to the Shannon entropy for fully defined sources. Coding of an underdetermined source must ensure, for any sequence generated by the source, recovering some its specification. Coding is asymptotically optimal if the average code length is asymptotically equal to the source entropy. Coding is universal if it does not depend on the probabilities of source symbols. We describe an asymptotically optimal universal coding method for underdetermined Bernoulli sources for which the encoding and decoding procedures can be realized by RAM-programs of almost linear complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Sholomov, L.A., Elements of Underdetermined Information Theory, Prikl. Diskr. Mat., 2009, Suppl. no. 2 (Talks given at the Int. Conf./Workshop for Young Researchers “Computer Security and Cryptography,” Omsk State Tech. Univ., Omsk, Russia, Sept. 7–12, 2009), pp. 18–42.

  2. Bongard, M.M., O ponyatii “polesnaya informatsiya” (On the Concept of “Useful Information”), Probl. Kibern., vol. 9, Moscow: Fizmatgiz, 1963.

  3. Shannon, C.E., Coding Theorems for a Discrete Source with a Fidelity Criterion, IRE Nat. Conv. Rec., 1959, vol. 7, no. 4, pp. 142–163 [Russian Transl. in Raboty po teorii informatsii i kibernetike (Papers in Information Theory and Cybernetics), Moscow: Inostr. Lit., 1963, pp. 587–621].

    Google Scholar 

  4. Gallager, R.G., Information Theory and Reliable Communication, New York: Wiley, 1968. Translated under the title Teoriya informatsii i nadezhnaya svyaz’, Moscow: Sov. Radio, 1974.

    MATH  Google Scholar 

  5. Veroytanost’ i mathematicheskaya statistika. Entsiklopedicheskii slovar’ (Probability and Mathematical Statistics: Encyclopedic Dictionary), Prokhorov, Yu.V., Ed., Moscow: Bol’shaya Rossiiskaya Entsiklopediya, 1999.

  6. Sholomov, L.A., On the Coding of Underdetermined Sequences with a Given Fidelity Criterion, Dokl. Akad. Nauk, 2009, vol. 429, no. 5, pp. 605–609 [Dokl. Math. (Engl. Transl), 2009, vol. 80, no. 3, pp. 882–886].

    MathSciNet  MATH  Google Scholar 

  7. Aho, A.V., Hopcroft, J.E., and Ullman, J.D., The Design and Analysis of Computer Algorithms, Reading: Addison-Wesley, 1976. Translated under the title Postroenie i analiz vychislitel’nykh algoritmov, Moscow: Mir, 1979.

  8. Sholomov, L.A., Compression of Partially Defined Information, Nelineinaya dinamika i upravlenie (Nonlinear Dynamics and Control), vol. 4, Emel’yanov, S.V. and Korovin, S.K., Eds., Moscow: Fizmatgiz, 2004, pp. 377–396 [Comput. Math. Model. (Engl. Transl.), 2008, vol. 19, no. 1, pp. 116–132].

  9. Potapov, V.N., Arithmetic Coding of Messages Using Random Sequences. Prikl. Diskr. Mat., 2008, no. 2 (2), pp. 131–133.

    MATH  Google Scholar 

  10. Sholomov, L.A., Theoretically Effective Asymptotically Optimal Universal Coding of Partially Defined Sources, Prikl. Diskr. Mat., 2020, no. 47, pp. 30–56.

  11. Sholomov, L.A., Informational Properties of Complexity Functionals for Systems of Underdetermined Boolean Functions, Probl. Kibern., 1978, no. 34, pp. 133–150.

    Google Scholar 

  12. Potapov, V.N., Vvedenie v teoriyu informatsii (Introduction to Information Theory), Izhevsk, Russia: Regulyarnaya i khaoticheskaya dinamika, 2014.

  13. Kolmogorov, A.N., Three Approaches to the Quantitative Definition of Information, Probl. Peredachi Inf., 1965, vol. 1, no. 1, pp. 3–11 [Probl. Inf. Transm. (Engl. Transl.), 1965, vol. 1, no. 1, pp. 1–7].

    MathSciNet  MATH  Google Scholar 

  14. Lupanov, O.B., On a Certain Approach to the Synthesis of Control Systems—The Principle of Local Coding, Probl. Kibern., 1965, no. 14, pp. 31–110.

    MathSciNet  Google Scholar 

  15. Chashkin, A.V., Methods for Computation of Partial Boolean Functions, in Proc. VII Int. Conf. on Discrete Models in Control-System Theory, Pokrovskoe, Moscow Distr., Mar. 4–6, 2006, Moscow: MAKS Press, 2006, pp. 390–404.

  16. Shannon, C.E., The Synthesis of Two-Terminal Switching Circuits, Bell Syst. Tech. J., 1949, vol. 98, no. 1, pp. 59–98 [Russian Transl. in Raboty po teorii informatsii i kibernetike (Papers in Information Theory and Cybernetics), Moscow: Inostr. Lit., 1963, pp. 59–101].

    Article  MathSciNet  Google Scholar 

  17. Nechiporuk, E.I., Complexity of Gating Circuits Realized by Boolean Matrices with Undetermined Elements, Dokl. Akad. Nauk SSSR, 1965, vol. 163, no. 1, pp. 40–42.

    MATH  Google Scholar 

  18. Nechiporuk, E.I., The Topological Principles of Self-correction, Probl. Kibern., 1969, no. 21, pp. 5–102.

    MathSciNet  MATH  Google Scholar 

  19. Krichevsky, R.E., Occam’s Razor, Partially Specified Boolean Functions, String Matching, and Independent Sets, Inform. and Comput., 1994, vol. 108, no. 1, pp. 158–174.

    Article  MathSciNet  Google Scholar 

  20. Berger, T., Rate Distortion Theory: A Mathematical Basis for Data Compression, Englewood Cliffs, NJ: Prentice-Hall, 1971.

    MATH  Google Scholar 

  21. Krichevsky, R., Universal Compression and Retrieval, Dordrecht: Kluwer, 1994.

    Book  Google Scholar 

  22. Andreev, A.E., Clementi, A.E.F., and Rolim, J.D.P., Hitting Sets Derandomize BPP, Proc. 23rd Int. Colloq. on Automata, Languages and Programming (ICALP’96), Paderborn, Germany, July 8–12, 1996, Meyer auf der Heide, F. and Monien, B., Eds., Lect. Notes Comp. Sci., vol. 1099, Berlin: Springer, 1996, pp. 357–368.

  23. Goldreich, O. and Wigderson, A., Improved Derandomization of BPP Using a Hitting Set Generator, Randomization, Approximation, and Combinatorial Optimization: Algorithms and Techniques (Proc. 3rd Int. Workshop on Randomization and Approximation Techniques in Computer Science, and 2nd Int. Workshop on Approximation Algorithms for Combinatorial Optimization Problems [RANDOM-APPROX’99], Berkeley, CA, USA, Aug. 8–11, 1999), Hochbaum, D.S., Jansen, K., Rolim, J.D.P., and Sinclair, A., Eds., Lect. Notes Comp. Sci., vol. 1671, Berlin: Springer, 1999, pp. 131–137.

  24. Nigmatulin, R.G., Steepest Descent Method in Covering Problems, in Proc. Symp. on Problems of Fidelity and Efficiency of Computational Algorithms, Kiev: Inst. Kibern. Akad. Nauk Ukr. SSR, 1969, vol. 5, pp. 116–126.

  25. Nigmatulin, R.G., Slozhnost’ bulevykh funktsii (Complexity of Boolean Functions), Moscow: Nauka, 1991.

  26. Cramér, H., Mathematical Methods of Statistics, Princeton: Princeton Univ. Press, 1946. Translated under the title Matematicheskie metody statistiki, Moscow: Mir, 1975, 2nd ed.

Download references

Author information

Authors and Affiliations

  1. Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences, Moscow, Russia

    L. A. Sholomov

  2. Institute of System Analysis, Russian Academy of Sciences, Moscow, Russia

    L. A. Sholomov

Authors
  1. L. A. Sholomov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L. A. Sholomov.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sholomov, L. Polynomial Asymptotically Optimal Coding of Underdetermined Bernoulli Sources of the General Form. Probl Inf Transm 56, 373–387 (2020). https://doi.org/10.1134/S0032946020040079

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0032946020040079

Keywords

Search

Navigation