Estimates of entropy numbers in probabilistic setting

Yongjie Han; Hanyue Xiao; Guanggui Chen

doi:10.1515/math-2020-0121

Open Access Published by De Gruyter Open Access December 31, 2020

Estimates of entropy numbers in probabilistic setting

Yongjie Han , Hanyue Xiao and Guanggui Chen

From the journal Open Mathematics

https://doi.org/10.1515/math-2020-0121

Abstract

In this paper, we define the entropy number in probabilistic setting and determine the exact order of entropy number of finite-dimensional space in probabilistic setting. Moreover, we also estimate the sharp order of entropy number of univariate Sobolev space in probabilistic setting by discretization method.

Keywords: finite-dimensional; probabilistic setting; entropy numbers; Gaussian measure

MSC 2010: 41A46; 42A61; 46C99

1 Introduction

Entropy numbers are closely related to Kolmogorov’s concept of metric entropy, which dates back to the 1930s. Basic properties of entropy numbers may be found in the monographs by Pietsch [1], Carl and Stephani [2], and Lorentz et al. [3]. Schütt [4], Edmunds and Triebel [5], and Kühn [6] have determined the asymptotic behavior of the entropy numbers of identity operators and diagonal operators under mild regularity and decay conditions on the generating sequence. Dung [7] investigated optimal entropy numbers of multivariate periodic functions with mixed smoothness. Belinsky [8] obtained estimates for the entropy numbers of classes of functions with conditions on the mixed derivative in the uniform and integral metrics.

In [9], an approximation criterion called the probabilistic criterion was considered that permits one to construct the distribution function with respect to a given measure for the best approximation functional. In [10], Maiorov introduced the concept of Kolmogorov ( n , δ ) -widths, i.e., Kolmogorov-widths in probabilistic setting, and studied the Kolmogorov ( n , δ ) -widths of univariate Sobolev space W 2 r ( T ) with the Gaussian measure μ . In particular, he also gave some beautiful results of Kolmogorov ( n , δ ) -widths of the finite-dimensional space R m equipped with the standard Gaussian measure. In [11], Chen determined the asymptotic order of the Kolmogorov ( n , δ ) -widths of the multivariate Sobolev space with mixed derivative M W 2 r ( T d ) in the space L q ( T d ) , 1 < q < ∞ .

Motivated by the aforementioned studies, in this paper, we introduce the concept of ( n , δ ) -entropy numbers which is entropy numbers in probabilistic setting. In the probabilistic setting, ( n , δ ) -entropy number is defined as in the worst case setting, but disregarding a set of measures at most δ , where δ ∈ [ 0 , 1 ] . If disregarding a null set, then ( n , δ ) -entropy number equals entropy number. This concept is an analogue to the Kolmogorov ( n , δ ) -widths and it generalizes the concept of entropy numbers. Then we give the asymptotic order of the entropy numbers in probabilistic setting of the finite-dimensional space ℝ m equipped with the standard Gaussian measure in l q m -metric, 1 ≤ q ≤ 2 . In fact, this asymptotic quantity has the same order with Kolmogorov ( n , δ ) -widths of the finite-dimensional space. Moreover, using the discretization method we study the entropy numbers in probabilistic setting of Sobolev space W 2 r ( T ) , r > 1 / 2 , equipped with a Gaussian measure μ in L q ( T ) , 1 < q ≤ 2 .

2 Preliminaries

We first give some notions. We use Z to denote the set of integer numbers. Let Z 0 = { k ∈ Z : k ≠ 0 } , Z + = { k ∈ Z : k > 0 } , ℕ = { k ∈ Z : k ≥ 0 } . Assume that c, c i , i = 0 , 1 , … , are positive constants depending only on the parameters p, q, r, ρ . For two positive functions a ( y ) and b ( y ) , y ∈ D , we write a ( y ) ≍ b ( y ) or a ( y ) ≪ b ( y ) if there exists constants c, c 1 , and c 2 such that c 1 ≤ a ( y ) / b ( y ) ≤ c 2 or a ( y ) ≤ c b ( y ) for any y ∈ D .

Next, we recall some definitions. Let W and M be subsets of a normed linear space X, the quantity

e ( W , M , X ) ≔ sup x ∈ W inf y ∈ M ∥ x − y ∥ X

is the deviation of W from M. The number

ε n ( W , X ) ≔ inf M ∈ ℳ e ( W , M , X )

is called the entropy number of W in X, where ℳ is the family of all subsets M of X such that log | M | ≔ log 2 | M | ≤ n , and | M | denotes the cardinality of M.

Detailed information about the usual entropy numbers may be found in [2,5].

Assume that W contains a Borel field B consisting of open subsets of W and is equipped with a probability measure μ defined on B , i.e., μ is a σ -additive non-negative function on B , and μ(W) = 1.

Let δ ∈ 0 , 1 be an arbitrary number. The ( n , δ ) -entropy number of a set W with a measure μ in the space X is defined by

ε n , δ ( W , μ , X ) = inf G ε n ( W \ G , X ) ,

where G runs through all possible subsets in B with measure μ ( G ) ≤ δ .

Let l p m be the m-dimensional normed space of vectors x = ( x 1 , x 2 , … , x m ) ∈ ℝ m , with the norm

∥ x ∥ p = ∑ i = 1 m | x i | p 1 / p , 1 ≤ p < ∞ , max 1 ≤ i ≤ m | x i | , p = ∞ .

Denote by B p m ( r 0 ) = { x ∈ l p m : ∥ x ∥ p ≤ r 0 } the ball of radius r 0 in l p m . Let B p m ≔ B p m ( 1 ) .

Denote by V ( K ) the Euclidean volume of a set K. From [10], we know that the ball B p m , 1 ≤ p ≤ ∞ , satisfy the inequalities

(1) ( c p ′ m ) − m / p < V ( B p m ) < ( c p ″ m ) − m / p ,

where c p ′ and c p ″ depend only on p. This follows from the relation

V ( B p m ) = [ 2 Γ ( 1 / p + 1 ) ] m / [ Γ ( m / p + 1 ) ] ,

where Γ is the Euler Γ -function.

For diagonal operator D : l p m → l p m , 1 ≤ p ≤ ∞ , the estimate of entropy number ε n ( D ( B p m ) ) is a case of Proposition 1.3.2 in [2].

Proposition 1

[2] Let D 1 ≥ D 2 ≥ ⋯ D m ≥ 0 be a non-increasing sequence of non-negative numbers and let

D x = ( D 1 x 1 , D 2 x 2 , … , D m x m )

for x = ( x 1 , x 2 , … , x m ) ∈ l p m be the diagonal operator from l p m with ≤ p ≤ ∞ into itself, generated by the sequence ( D i ) . Then

ε n ( D ( B p m ) ) ≤ 6 ⋅ sup 1 ≤ k ≤ m 2 − n / k ( D 1 D 2 ⋯ D k ) 1 / k .

We consider in ℝ m the standard Gaussian measure ν = ν m , which is defined on Borel subsets G ⊆ R m by

ν ( G ) = ( 2 π ) − m / 2 ∫ G exp − 1 2 ∥ x ∥ 2 2 d x ,

and satisfies ν ( ℝ m ) = 1 .

Let n ∈ N , 1 ≤ q ≤ ∞ , and δ ∈ ( 0 , 1 ] be arbitrary, we define ( n , δ ) -entropy numbers of the space ℝ m equipped with the Gaussian measure v in the space l q m :

ε n , δ ( ℝ m , ν , l q m ) = inf G ε n ( ℝ m \ G , l q m ) ,

where the infimum is taken over all possible Borel subsets G ⊂ ℝ m with measure ν ( G ) ≤ δ and ε n ( ℝ m \ G , l q m ) is the entropy number of the set ℝ m \ G in the l q m -metric.

The rest of the paper is organized as follows. In Section 3, we determine the asymptotic order of the ( n , δ ) -entropy numbers of finite-dimensional set. In Section 4, we calculate the asymptotic order of the ( n , δ ) -entropy numbers of the Sobolev space W 2 r ( T ) in the space L q ( T ) , 1 < q ≤ 2 .

3 ( n , δ ) -entropy numbers of finite-dimensional set

Now we state our main result.

Theorem 1

Let 1 ≤ q ≤ 2 . If δ ∈ ( 0 , 1 / 2 ] , then

ε n , δ ( ℝ m , ν , l q m ) ≍ 2 − n / m m 1/ q − 1/2 m + ln (1/ δ ) .

Remark 1

The estimate of Kolmogorov ( n , δ ) -widths of finite-dimensional set was obtained by Maiorov [10]. If n ≤ m , we can see that these two numbers are asymptotically equivalent when 1 ≤ q ≤ 2 .

To prove Theorem 1, we need some auxiliary assertions.

Lemma 1

If 1 ≤ p ≤ ∞ , then

ε n ( B p m , l p m ) ≤ c 2 − n / m ,

where c is an absolute constant.

Lemma 1 follows immediately from Proposition 1, here we omit the proof.

We first find some simple Borel subsets G ⊂ ℝ m with measure ν ( G ) ≤ δ such that ε n ( ℝ m \ G , l q m ) can be easily obtained. Obviously, this ε n ( ℝ m \ G , l q m ) is an upper bound of ε n , δ ( ℝ m , ν , l q m ) . The following lemma provides such subsets G.

Lemma 2

[12] There exists an absolute positive constant c 0 such that for any δ ∈ (0, 1/2] , we have

ν x ∈ ℝ m : ∥ x ∥ 2 ≥ c 0 m + ln ( 1 / δ ) ≤ δ .

Proof of Theorem 1

We first establish the upper estimate of ε n , δ ( ℝ m , ν , l q m ) .

Let G t = { x ∈ ℝ m : ∥ x ∥ 2 ≥ t } , t ∈ [ 0 , + ∞ ) , and t 0 = c 0 m + ln 1 / δ with the same c 0 as in Lemma 2. From Lemma 2 and the definition of ε n , δ ( ℝ m , ν , l q m ) , we get

ε n , δ ( ℝ m , ν , l q m ) ≤ ε n ( ℝ m \ G t 0 , l q m ) = ε n ( B 2 m ( t 0 ) , l q m ) = t 0 ε n ( B 2 m , l q m ) .

It follows from B 2 m ⊂ m 1 / q − 1 / 2 B q m , 1 ≤ q ≤ 2 , and Lemma 1 that

(2) ε n , δ ( ℝ m , ν , l q m ) ≤ t 0 m 1 / q − 1 / 2 ε n ( B q m , l q m ) ≪ 2 − n / m m 1 / q − 1 / 2 m + ln ( 1 / δ ) .

Now we estimate the lower bound of ε n , δ ( ℝ m , ν , l q m ) .

For any G ⊆ ℝ m , it holds that ν ( G ) = δ ∈ ( 0 , 1 / 2 ] . We consider the following two cases:

ℝ m \ G is unbounded. In this case, it is trivial that
(3) ε n ( ℝ m \ G , l q m ) ≥ 2 − n / m m 1 / q − 1 / 2 m + ln ( 1 / δ ) .
ℝ m \ G is bounded. We can find t 1 such that ν ( G t 1 ) = δ . Indeed, we consider

f ( t ) = ( 2 π ) − m / 2 ∫ G t exp − 1 2 ∥ x ∥ 2 2 d x , t ∈ [ 0 , + ∞ ) .

Obviously, f is continuous monotone decreasing, f ( 0 ) = 1 and f ( t 0 ) ≤ δ . From this t 1 must exist. Maiorov gives the following result in [10]:

ν x ∈ ℝ m : ∥ x ∥ 2 > max m , ln ( 1 / δ ) > δ .

It follows from this and ν ( G t 1 ) = δ that

(4) t 1 ≥ max m , ln ( 1 / δ ) ≥ m + ln ( 1 / δ ) 2 .

Let D = G ∩ G t 1 , D 1 = G \ D , and D 2 = G t 1 \ D . Then

(5) ∥ x ∥ 2 ≤ t 1 if x ∈ D 1 , ∥ x ∥ 2 ≥ t 1 if x ∈ D 2 .

It is easy to verify that ν ( D 1 ) = ν ( D 2 ) , i.e.,

( 2 π ) − m / 2 ∫ D 1 exp − 1 2 ∥ x ∥ 2 2 d x = ( 2 π ) − m / 2 ∫ D 2 exp − 1 2 ∥ x ∥ 2 2 d x ,

which together with (5) gives

exp − ( t 1 ) 2 2 ∫ D 1 d x ≤ ∫ D 1 exp − 1 2 ∥ x ∥ 2 2 d x = ∫ D 2 exp − 1 2 ∥ x ∥ 2 2 d x ≤ exp − ( t 1 ) 2 2 ∫ D 2 d x .

Therefore,

V ( D 1 ) ≤ V ( D 2 ) .

By the definition of G t 1 , D 1 , and D 2 , we have

V ( ℝ m \ G ) = V ( D 2 ) + V ( B 2 m ( t 1 ) ) − V ( D 1 )

and

V ( ℝ m \ G t 1 ) = V ( B 2 m ( t 1 ) ) .

Hence,

(6) V ( ℝ m \ G ) ≥ V ( R m \ G t 1 ) .

According to (1.2.1) (see [2, p. 10]), we get

(7) V ( ℝ m \ G ) ≤ 2 n ( ε n ( ℝ m \ G , l q m ) ) m V ( B q m ) .

It follows from (6) and (7) that

(8) V ( ℝ m \ G t 1 ) V ( B q m ) 1 / m = t 1 V ( B 2 m ) V ( B q m ) 1 / m ≤ V ( ℝ m \ G ) V ( B q m ) 1 / m ≤ 2 n / m ε n ( ℝ m \ G , l q m ) .

Combining the relations (1), (4), and (8), we have

(9) ε n ( ℝ m \ G , l q m ) ≥ 2 − n / m t 1 V ( B 2 m ) V ( B q m ) 1 / m ≥ c 2 − n / m m 1 / q − 1 / 2 m + ln ( 1 / δ ) ,

where c depends only on q. Using (3) and (9), we have

ε n , δ ( ℝ m , ν , l q m ) ≫ 2 − n / m m 1 / q − 1 / 2 m + ln ( 1 / δ ) .

The theorem follows from this and (2).□

4 ( N , δ ) -entropy number of univariate Sobolev space

In this section, we estimate the sharp order of ( N , δ ) -entropy numbers of univariate Sobolev space. First, we recall some definitions.

Denote by L q ( T ) , 1 < q < ∞ , the classical q-integral Lebesgue space of 2 π -periodic functions with the usual norm ∥ ⋅ ∥ L q ≔ ∥ ⋅ ∥ L q ( T ) . We consider the Hilbert space L 2 ( T ) consisting of all functions x defined on T ≔ [ 0 , 2 π ) with the Fourier series

x ( t ) = ∑ k ∈ Z c k exp ( i k t ) ) ,

and inner product

〈 x , y 〉 = 1 (2 π ) ∫ T x ( t ) y ( t ) ¯ d t , x , y ∈ L 2 ( T ) .

For any r ∈ ℝ , we define the rth order derivative of x in the sense of Weyl by

x ( r ) ( t ) ≔ ( D r x ) ( t ) = ∑ k ∈ Z ( i k ) r c k exp ( i k t ) ,

where ( i k ) r = | k | r exp ( ( π i / 2 ) sgn r ) .

The Sobolev space W 2 r ( T ) ( r > 0 ) consists of all functions x ∈ L 2 ( T ) satisfying x ( r ) ∈ L 2 ( T ) and the additional condition c 0 = 0 . The space W 2 r ( T ) is a Hilbert space with the inner product

〈 x , y 〉 r ≔ 〈 x ( r ) , y ( r ) 〉

and the norm

∥ x ∥ W 2 r ( T ) 2 = 〈 x ( r ) , x ( r ) 〉 .

It is well known that if r > max ( 0 , 1 / 2 − 1 / q ) , then W 2 r ( T ) can be imbedded in L q ( T ) , 1 ≤ q ≤ ∞ , continuously. In this paper, we suppose r > 1 / 2 .

Let the space W 2 r ( T ) be equipped with the Gaussian measure μ with zero mean and correlation operator C μ having eigenfunctions

e k = exp ( i k t )

and eigenvalues

λ k = | k | − ρ , ρ > 1 ,

i.e.,

C μ e k = λ k e k , k ∈ Z 0 .

Let y 1 , … , y n be any orthogonal system of functions in L 2 ( T ) , σ j = 〈 C μ y j , y j 〉 , j = 1 , … , n , and B be an arbitrary Borel subset of ℝ n . Then the Gaussian measure μ on the cylindrical subsets in the space W 2 r ( T )

G = { x ∈ W 2 r ( T ) : ( 〈 x , y 1 ( − r ) 〉 r , … , 〈 x , y n ( − r ) 〉 r ) ∈ B }

is defined by

μ ( G ) = ∏ j = 1 n ( 2 π σ j ) − 1 / 2 ∫ B exp − ∑ j = 1 n | u j | 2 2 σ j d u 1 ⋯ d u n .

More detailed information about the Gaussian measure in Banach space is contained in the books of [13,14].

Now we state our main result of this section.

Theorem 2

Let r > 1 / 2 , 1 < q ≤ 2 , ρ > 1 , and δ ∈ ( 0 , 1 / 2 ] . Then the ( N , δ ) -entropy numbers of W 2 r ( T ) with the Gaussian measure μ in the space L q ( T ) satisfy asymptotic relation

ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) ≍ N − ( r + ( ρ − 1)/2) 1 + (1/ N ) ln (1/ δ ) .

Remark 2

The estimate of Kolmogorov ( N , δ ) -widths of infinite-dimensional set was obtained by Maiorov [10]. We can see that these two numbers are asymptotically equivalent when 1 < q ≤ 2 .

To prove Theorem 2, we need two auxiliary discretization theorems (Theorems 3 and 4) that reduce the computation of ( N , δ ) -entropy numbers ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) of the class W 2 r ( T ) to the corresponding finite-dimensional problem for ( N , δ ) -entropy numbers ε N , δ ( ℝ m , ν m , l q m ) . We introduce some notations and lemmas.

For k ∈ Z + , we take Δ k = { n ∈ Z 0 : 2 k − 1 ≤ | n | < 2 k } . Then m k ≔ | Δ k | = 2 k .

If x ( t ) = ∑ n ∈ Z c n e int , then we set

Λ k x ( t ) ≔ ∑ n ∈ Δ k c n e int .

The following two known lemmas are crucial for establishing discretization theorems (Theorems 3 and 4).

Lemma 3

[15] Let k ∈ Z + , α ∈ ℝ , and 1 < q < ∞ . Then we have

2 k α ∥ Λ k x ∥ L q ≍ ∥ ( Λ k x ) ( α ) ∥ L q .

Lemma 4

[15] Let k ∈ Z + . Then the space of trigonometric polynomial F m k = span { e int : n ∈ Δ k } is isomorphic to the space ℝ 2 k via mapping:

x ( t ) ↦ { x ( t j ) } j , t j = π 2 2 − k j , j = 1 , … , 2 k .

Moreover, the following relation is true:

(10) ∥ x ∥ L q ≍ 2 − k / q ∥ { x ( t j ) } j ∥ l q m k , 1 < q < ∞ ,

where the constant in the equivalence does not depend on k.

Theorem 3

Let 1 < q < ∞ , r > 1 / 2 , N ∈ ℕ , δ ∈ ( 0 , 1 / 2 ] , and let the sequence { N k } and { δ k } ( k ∈ Z + ) of numbers be such that N k ∈ N , ∑ k ∈ Z + N k ≤ N , and δ k ∈ [ 0 , 1/2 ] , ∑ k ∈ Z + δ k ≤ δ . Then

ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ≪ ∑ k = 1 ∞ 2 − ( r + ( ρ − 1 ) / 2 ) k − k / q ε N k , δ k ( ℝ m k , ν , l q m k ) .

Proof

We consider in the space F m k the polynomials

φ j ( t ) = ∑ n ∈ Δ k exp ( i n ( t − t j ) ) , j = 1 , … , m k .

Obviously, these polynomials are orthogonal in L 2 ( T ) , and for any x ∈ F m k

( D r x ) ( t j ) = 〈 D r x , φ j 〉 , j = 1 , … , m k .

Lemma 3 and x ∈ F m k lead to

2 k r ∥ x ∥ L q ≍ ∥ ( x ) ( r ) ∥ L q .

From the aforementioned inequality and (10), we have

(11) ∥ x ∥ L q ≍ 2 − k r ∥ ( x ) ( r ) ∥ L q ≍ 2 − k r − k / q ∥ { ( D r x ) ( t j ) } j ∥ l q m k = 2 − k r − k / q ∥ { 〈 D r x , φ j 〉 } j ∥ l q m k .

For any k ∈ Z + , we consider a mapping

I k : F m k → ℝ m k , x ↦ { 〈 D r x , φ j 〉 } j .

It follows from Lemma 4 that I k is linear isomorphic from the space F m k to the space ℝ m k .

Take σ j ≔ 〈 C μ φ j , φ j 〉 . Then all σ j are equal and

σ j = ∑ n ∈ Z 0 | n | − ρ | 〈 φ j , e int 〉 | 2 = ∑ n ∈ Δ k | n | − ρ ≍ 2 − k ( ρ − 1) .

Hence, there exists a constant c 1 such that

σ ≔ σ j = c 1 2 2 − k ( ρ − 1 ) .

Let ε N k , δ k ≔ ε N k , δ k ( ℝ m k , ν , l q m k ) . Denote by M k a subset of ℝ m k such that log | M k | ≤ N k and

(12) ν { y ∈ ℝ m k : e ( y , σ − 1 / 2 M k , l q m k ) > ε N k , δ k } ≤ δ k ,

where σ − 1 / 2 M k = { σ − 1 / 2 x : x ∈ M k } .

Let x ∈ W 2 r ( T ) . Then by virtue of (11) there exists constant c 2 independent of k such that

(13) e ( Λ k x , D − r I k − 1 M k , L q ( T ) ) ≤ c 2 2 − k r − k / q e ( { 〈 D r x , φ j 〉 } j , M k , l q m k ) .

Consider the set of W 2 r ( T )

G k = { x ∈ W 2 r ( T ) : e ( Λ k x ; D − r I k − 1 M k , L q ( T ) ) > c 2 2 − k r − k / q σ 1 / 2 ε N k , δ k } .

From (13), the definitions of the measure μ and the standard Gaussian measure ν in the space of R m k , and (12) it follows that

μ ( G k ) ≤ μ { x ∈ W 2 r ( T ) : e ( { 〈 D r x , φ j 〉 } j , M k , l q m k ) > σ 1 / 2 ε N k , δ k } = ν { y ∈ ℝ m k : e ( y σ 1 / 2 , M k , l q m k ) > σ 1 / 2 ε N k , δ k } = ν { y ∈ ℝ m k : e ( y , σ − 1 / 2 M k , l q m k ) > ε N k , δ k } ≤ δ k .

Let us consider the set G = ⋃ k ∈ Z + G k and the subset M = ∑ k ∈ Z + D − r I k − 1 M k , which is the sum of the subset D − r I k − 1 M k , where ∑ k ∈ Z + D − r I k − 1 M k = { ∑ k ∈ Z + x k : x k ∈ D − r I k − 1 M k } . From the hypothesis of the theorem, we get

μ ( G ) ≤ δ , and log | M | ≤ N .

Consequently, by the definitions of ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) and ε N ( W 2 r ( T ) \ G , L q ( T ) )

(14) ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) ≤ e ( W 2 r ( T ) \ G , M , L q ( T ) ) = sup x ∈ W 2 r ( T ) \ G e ( x , M , L q ( T ) ) .

By the definitions of M and { M k }

(15) e ( x , M , L q ( T ) ) ≤ ∑ k = 1 ∞ e ( Λ k x , D − r I k − 1 M k , L q ( T ) ) .

For any x ∈ W 2 r ( T ) \ G , by the definition G and { G k }

(16) e ( Λ k x ; D − r I k − 1 M k , L q ( T ) ) ≪ 2 − k r − k / q σ 1 / 2 ε N k , δ k .

Combining the aforementioned three inequalities (14), (15), and (16), we obtain the desired result.□

For any N ∈ Z + , let k = [ log N ] . We consider the space of trigonometric polynomials F = span { e i n t : n ∈ Δ k } . It follows from (11) that there exist two positive constants c 3 and c 4 such that

(17) c 3 2 − ( r + 1 / q ) k ∥ { 〈 D r x , φ j 〉 } ∥ l q m k ≤ ∥ x ∥ L q ≤ c 4 2 − ( r + 1 / q ) k ∥ { 〈 D r x , φ j 〉 } ∥ l q k .

Let

I : F → l q m k , x ↦ { 〈 D r x , φ j 〉 } .

Then I is a linear isomorphic mapping from the space of trigonometric polynomials F to l q m k .

Theorem 4

Suppose that 1 < q < ∞ , r > 1 / 2 , N ∈ Z + , and δ ∈ ( 0 , 1 / 2 ] . Then it follows that

ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) ≫ 2 − ( r + 1 / q + ( ρ − 1 ) / 2 ) k ε N , δ ( ℝ m k , ν , l q m k ) ,

where k = [ log N ] .

Proof

Let M 1 be a subset of W 2 r ( T ) ∩ F such that log | M 1 | ≤ N and

(18) μ { x ∈ W 2 r ( T ) ∩ F : e ( x , M 1 , L q ( T ) ∩ F ) > ε N , δ } ≤ δ ,

where ε N , δ ≔ ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) . Set

G = { y ∈ ℝ m k : e ( y , σ − 1 / 2 I D r M 1 , l q m k ) > c 3 2 ( r + 1 / q ) k σ − 1 / 2 ε N , δ } ,

where c 3 is defined by (17). Then from (17) and (18), we get

(19) ν ( G ) = ν { y ∈ ℝ m k : e ( y σ 1 / 2 , I D r M 1 , ł q m k ) > c 3 2 ( r + 1 / q ) k ε N , δ } = μ { x ∈ W 2 r ( T ) ∩ F : e ( { 〈 D r x , φ j 〉 } , I D r M 1 , l q m k ) > c 3 2 ( r + 1 / q ) k ε N , δ } ≤ μ { x ∈ W 2 r ( T ) ∩ F : e ( x , M 1 , L q ( T ) ) > ε N , δ } ≤ δ .

Clearly, log | I D r M 1 | ≤ N . Therefore, by (19),

ε N , δ ( ℝ m k , ν , l q m k ) ≤ e ( ℝ m k \ G , σ − 1 / 2 I D r M 1 , l q m k ) = sup y ∈ ℝ m k \ G e ( y , σ − 1 / 2 I D r M 1 , l q m k ) ≪ 2 ( r + 1 / q + ( ρ − 1 ) / 2 ) k ε N , δ .

Theorem 4 follows from this.□

Proof of Theorem 2

We first estimate the upper bound of ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) . For a given N ∈ Z + , N k and δ k are defined by

N k = [2 k 2 (1 − β ) ( k ′ − k ) ] , k ≤ k ′ [2 k 2 (1 + β ) ( k ′ − k ) ] , k > k ′ , δ k = δ N k / N ,

where 0 < β < 1 and k ′ = log N . It is easy to verify that

∑ k = 1 ∞ N k ≪ N , ∑ k = 1 ∞ δ k ≪ δ .

By Theorem 3, we obtain

ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) ≤ ε N , δ ( W 2 r ( T ) , μ , L 2 ( T ) ) ≪ ∑ k = 1 ∞ 2 − ( r + ( ρ − 1 ) / 2 ) k − k / 2 ε N k , δ k ( R m k , ν , l 2 m k ) ≪ ∑ 1 ≤ k ≤ k ′ 2 − ( r + ( ρ − 1 ) / 2 ) k − k / 2 ε N k , δ k ( R m k , ν , l 2 m k ) + ∑ k > k ′ 2 − ( r + ( ρ − 1 ) / 2 ) k − k / 2 ε N k , δ k ( R m k , ν , l 2 m k ) ≔ I 1 + I 2 .

According to Theorem 1,

I 1 ≪ ∑ 1 ≤ k ≤ k ′ 2 − ( r + ( ρ − 1 ) / 2 ) k − k / 2 2 − 2 ( 1 − β ) ( k ′ − k ) ( 2 k / 2 + ln ( 2 k ′ / ( 2 k 2 ( 1 − β ) ( k ′ − k ) δ ) ) = ∑ 1 ≤ k ≤ k ′ 2 − ( r + ( ρ − 1 ) / 2 ) k 2 − 2 ( 1 − β ) ( k ′ − k ) + ∑ 1 ≤ k ≤ k ′ 2 − ( r + ρ / 2 ) k 2 − 2 ( 1 − β ) ( k ′ − k ) ln ( 2 ( k ′ − k ) β / δ ) ≪ 2 − ( r + ( ρ − 1 ) / 2 ) k ′ ∑ 1 ≤ k ≤ k ′ 2 ( r + ( ρ − 1 ) / 2 ) ( k ′ − k ) − 2 ( 1 − β ) ( k ′ − k ) + 2 − ( r + ρ / 2 ) k ′ ∑ 1 ≤ k ≤ k ′ 2 ( r + ρ / 2 ) ( k ′ − k ) − 2 ( 1 − β ) ( k ′ − k ) β ( k ′ − k ) ln ( 1 / δ ) ≪ 2 − ( r + ( ρ − 1 ) / 2 ) k ′ + 2 − ( r + ρ / 2 ) k ′ ln ( 1 / δ ) ≍ N − ( r + ( ρ − 1 ) / 2 ) + N − ( r + ρ / 2 ) ln ( 1 / δ ) .

We continue to estimate I 2 :

I 2 ≪ ∑ k > k ′ 2 − ( r + ( ρ − 1 ) / 2 ) k − k / 2 ( 2 k / 2 + ln ( 2 k ′ / ( 2 k 2 ( 1 + β ) ( k ′ − k ) δ ) ) ≪ ∑ k > k ′ 2 − ( r + ( ρ − 1 ) / 2 ) k + ∑ k > k ′ 2 − ( r + ρ / 2 ) k β ( k − k ′ ) ln ( 1 / δ ) ≪ ∑ k > k ′ 2 − ( r + ( ρ − 1 ) / 2 ) k + 2 − ( r + ρ / 2 ) k ′ ∑ k > k ′ 2 − ( r + ρ / 2 ) ( k − k ′ ) β ( k − k ′ ) ln ( 1 / δ ) ≍ N − ( r + ( ρ − 1 ) / 2 ) + N − ( r + ρ / 2 ) ln ( 1 / δ ) .

Therefore, we get the upper bound estimate of Theorem 2.

We proceed to estimate the lower bound of ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) for 1 < q ≤ 2 . Let k = [ log N ] . Then 2 k ≍ 2 N . By Theorems 4 and 1, we have

ε N , δ ( W 2 r ( T ) , μ , L q ( T ) ) ≫ 2 − ( r + 1 / q + ( ρ − 1 ) / 2 ) k ε N , δ ( ℝ m k , ν , l q m k ) ≫ 2 − ( r + 1 / q + ( ρ + 1 ) / 2 ) k 2 − N / m k ( m k 1 / q + m k 1 / q − 1 / 2 ) ln ( 1 / δ ) ≫ 2 − ( r + ( ρ − 1 ) / 2 ) k + 2 − ( r + ρ / 2 ) k ln ( 1 / δ ) ≍ N − ( r + ( ρ − 1 ) / 2 ) 1 + ( 1 / N ) ln ( 1 / δ ) ,

which is the required lower estimate of Theorem 2. The proof of Theorem 2 is complete.□

5 Conclusion

Entropy number is a geometric concept. It shows how well one can approximate a set by finite sets, and the Kolmogorov widths characterize the error of approximation of a set by finite dimensional sets. These two numbers describe approximation properties of sets from different viewpoints. In this paper, we see that ( n , δ ) -entropy numbers and Kolmogorov ( n , δ ) -widths have essentially the same asymptotic behavior.

Acknowledgments

This work was supported by the Key scientific research fund of Xihua University (Grant No. z1412621), the Natural Science Fund of Sichuan Education Department (Grant No. 15233448), and the National Natural Science Foundation of China (Grant No. 15233593). The authors would like to thank the anonymous referees for their valuable comments which help us to improve the content of this paper.

References

[1] Albrecht Pietsch, Operator Ideals, North-Holland Publishing Company, Amsterdam, 1980.Search in Google Scholar

[2] Bernd Carl and Irmtraud Stephani, Entropy, Compactness and the Approximation of Operators, Cambridge University Press, Cambridge, 1990.10.1017/CBO9780511897467Search in Google Scholar

[3] George G. Lorentz, Manfred V. Golitschek and Yuly Makovoz, Constructive Approximation, Springer, Berlin, 1996.10.1007/978-3-642-60932-9Search in Google Scholar

[4] Carsten Schütt, Entropy numbers of diagonal operators between symmetric Banach spaces, J. Approx. Theory 40 (1984), no. 2, 121–128. https://doi.org/10.1016/0021-9045(84)90021-2.10.1016/0021-9045(84)90021-2Search in Google Scholar

[5] David E. Edmunds and Hans Triebel, Function Spaces, Entropy Numbers, Differential Operators, Cambridge University Press, Cambridge, 1996.10.1017/CBO9780511662201Search in Google Scholar

[6] Thomas Kühn, Entropy numbers of general diagonal operators, Rev. Mat. Complut. 18 (2005), no. 2, 479–491. http://dx.doi.org/10.5209/rev_REMA.2005.v18.n2.16697.10.5209/rev_REMA.2005.v18.n2.16697Search in Google Scholar

[7] Dinh Dung, Non-linear approximations using sets of finite cardinality or finite pseudo-dimension, Constr. Approx. 17 (2001), no. 2, 467–492. https://doi.org/10.1006/jcom.2001.0579.10.1006/jcom.2001.0579Search in Google Scholar

[8] Eduard S. Belinsky, Estimates of entropy numbers and Gaussian measures for classes of functions with bounded mixed derivative, J. Approx. Theory 93 (1998), no. 1, 114–127. https://doi.org/10.1006/jath.1997.3157.10.1006/jath.1997.3157Search in Google Scholar

[9] Joseph F. Traub, Grzegorz W. Wasilkowski and Henryk Woźniakowski, Information-Based Complexity, Academic Press, New York, 1988.Search in Google Scholar

[10] Vitaly Maiorov, Kolmogorov’s (n,δ)-widths of spaces of smooth functions, Russian Acad Math. 79 (1994), no. 2, 265–279. https://doi.org/10.1070/SM1994v079n02ABEH003499.10.1070/SM1994v079n02ABEH003499Search in Google Scholar

[11] Guanggui Chen and Gensun Fang, Probabilistic and average widths of multivariate Sobolev spaces with mixed derivative equipped with the Gaussian measure, J. Complexity 20 (2004), no. 6, 858–875. https://doi.org/10.1016/j.jco.2004.04.001.10.1016/j.jco.2004.04.001Search in Google Scholar

[12] Vitaly Maiorov, Linear widths of function spaces equipped with the Gaussian measures, J. Approx. Theory 77 (1994), no. 1, 74–88. https://doi.org/10.1006/jath.1994.1035.10.1006/jath.1994.1035Search in Google Scholar

[13] Hui-Hsiung Kuo, Gaussian Measure in Banach Space, Springer, Berlin, 1975.10.1007/BFb0082007Search in Google Scholar

[14] Michel Ledoux and Michel Talagrand, Probability in Banach Space, Springer, Berlin, 1991.10.1007/978-3-642-20212-4Search in Google Scholar

[15] Èl′fat Mikhailovich Galeev, Kolmogorov widths in the space L˜q of the classes W˜pα¯ and H˜pα˜ of periodic functions of several variables, Math. USSR Izv. 27 (1986), no. 2, 219–237. https://doi.org/10.1070/IM1986v027n02ABEH001175.10.1070/IM1986v027n02ABEH001175Search in Google Scholar

Received: 2020-06-04

Revised: 2020-10-12

Accepted: 2020-11-20

Published Online: 2020-12-31

This work is licensed under the Creative Commons Attribution 4.0 International License.

Estimates of entropy numbers in probabilistic setting

Abstract

1 Introduction

2 Preliminaries

Proposition 1

3 ( n , δ ) -entropy numbers of finite-dimensional set

Theorem 1

Remark 1

Lemma 1

Lemma 2

Proof of Theorem 1

4 ( N , δ ) -entropy number of univariate Sobolev space

Theorem 2

Remark 2

Lemma 3

Lemma 4

Theorem 3

Proof

Theorem 4

Proof

Proof of Theorem 2

5 Conclusion

Acknowledgments

References

Journal and Issue

Articles in the same Issue