1 Introduction

In the work, we are concerned with the existence of infinitely many high energy solutions for the following fractional Schrödinger systems:

$$ \left \{ \textstyle\begin{array}{l@{\quad}l}(-\triangle)^{s} u +V(x)u=F_{u}(x,u,v),& x\in \mathbb{R}^{N}, \\(-\triangle)^{s} v +V(x)v=F_{v}(x,u,v),& x\in\mathbb{R}^{N}, \end{array}\displaystyle \right . $$
(1.1)

where \(s\in(0,1)\), \(N>2\) and \(F_{u}(x,u, v), F_{v}(x,u, v)\in C(\mathbb {R}^{N}\times\mathbb{R}\times\mathbb{R},\mathbb{R})\). We assume that there exists \(F(x,u, v)\in C(\mathbb{R}^{N}\times\mathbb{R}\times\mathbb {R},\mathbb{R})\) such that \(\nabla F=(F_{u},F_{v})\), where ∇F denotes the gradient of F in \((u, v)\in\mathbb{R}^{2}\). The operator \((-\Delta)^{s}\) is the fractional Laplacian of order s, which can be defined by the Fourier transform \((-\Delta)^{s}u =\mathscr{F}^{-1}(|\xi |^{2s}\mathscr{F}u)\). On the calculation and application of classical fractional differential equations and other aspects in mathematics, we refer the reader to [15] and the references therein.

Over the past years, the fractional Laplacian \((-\Delta)^{s}\) (\(0< s<1\)), as one of the fundamental nonlocal operators, has increasingly had impact on a number of important fields in science, technology and other fields. As a result, much attention has been focused on the problem of fractional Laplacians. For instance, Teng [6] studied the following fractional Schrödinger equation:

$$ (-\triangle)^{s} u +V(x)u=f(x,u),\quad x\in \mathbb{R}^{N}, $$
(1.2)

and proved the existence of infinitely many nontrivial high or small energy solutions by variant fountain theorems. Du and Mao [7] obtained a sufficient condition for the existence of infinitely many nontrivial high energy solutions by variant fountain theorems for (1.2). Some interesting results can be found in [823] and the references therein.

Recently, Di Nezza et al. [16] have proved that \((-\Delta)^{s}\) can be reduced to the standard Laplacian −Δ as \(s\rightarrow1\). If \(s=1\), Eq. (1.2) reduces to the classical Schrödinger equation

$$ -\triangle u +V(x)u=f(x,u),\quad x\in\mathbb{R}^{N}. $$
(1.3)

With the aid of variational method and critical theorems, for the potential \(V(x)\) and nonlinearity \(f(x,u)\) under various conditions, the results of existence and multiplicity for Eq. (1.3) have been extensively investigated in the literature; see [2427] and the references therein.

In recent decades, extensive attention of researchers has been devoted to the existence of solutions to the elliptic systems. Zhang and Zhang [28] considered some nonlinear elliptic systems and obtained the existence of weak solutions by using variational methods. Cao and Tang [29] considered the superlinear elliptic system. They presented the existence of infinitely many solutions which were characterized by the number of nodes of each component under some conditions on the nonlinear term. Pomponio [30] discussed the asymptotically linear cooperative elliptic system at resonance. They proved the existence of a non-zero solution and the existence of \(N-1\) pairs of nontrivial solutions due to the difference between the Morse index at zero and the Morse index at infinity by a penalization technique. In recent years, many interesting results have been presented on the class of systems; see [3138] and the references therein. However, the above literature is concerned with the problem of integer order Laplacian and there is little literature which discusses the Schrödinger systems with fractional order Laplacian. Based on the situation, we consider fractional Schrödinger systems (1.1). In this work, we will show the existence of infinitely many nontrivial high energy solutions by variant fountain theorems.

For convenience, we firstly present the following hypotheses:

\((V_{1})\):

\(V\in C(\mathbb{R}^{N}) \) satisfies \(\inf V(x)>0\) and there exist \(r_{0}>0\) and \(M>0\) such that \(\lim_{|y|\rightarrow\infty}\operatorname{meas}\{x\in \mathbb{R}^{N}: |x-y|\leq r_{0}, V(x)\leq M\}=0\), where meas denotes the Lebesgue measure in \(\mathbb{R}^{N}\).

\((f_{1})\):

\(F\in C^{1}(\mathbb{R}^{N}\times\mathbb{R}\times\mathbb{R}, \mathbb{R})\), \(F_{u}(x, u, v)u+ F_{v}(x, u, v)v\geq0\) for all \((x, u, v)\in \mathbb{R}^{N}\times\mathbb{R}\times\mathbb{R}\), \(F_{u}(x, u, v)\leq c(1+|(u, v)|^{p-1})\) and \(F_{v}(x, u, v)\leq c(1+|(u, v)|^{q-1})\) for some \(2< p, q<2^{*}_{s}\), where c denote different positive constants and \(|(u, v)|=(u^{2}+v^{2})^{\frac{1}{2}}\).

\((f_{2})\):

\(\lim_{|(u, v)|\rightarrow0}\frac{F_{u}(x, u, v)}{|(u, v)|}=0\) and \(\lim_{|(u, v)|\rightarrow0}\frac{F_{v}(x, u, v)}{|(u, v)|}=0\) uniformly in \(x\in\mathbb{R}^{N}\).

\((f_{3})\):

There exists \(\sigma\in[1,\min\{p, q\})\) such that \(\liminf_{|(u, v)|\rightarrow\infty}\frac{F(x, u, v)}{|(u, v)|^{\sigma}}\geq d>0\) uniformly for \(x\in\mathbb{R}^{N}\).

\((f_{4})\):

\(\lim_{|(u, v)|\rightarrow\infty}\frac{F(x, u, v)}{|(u, v)|^{2}}=\infty\) uniformly in \(x\in\mathbb{R}^{N}\).

\((f_{5})\):

There exist \(\mu>2\) and \(c>0\) such that

$$F_{u}(x, u, v)u+F_{v}(x, u, v)v-\mu F(x, u, v)\geq c \bigl(1+ \bigl\vert (u, v) \bigr\vert ^{2}\bigr), \quad\text{for all } x\in\mathbb{R}^{N}. $$

The paper is arranged as follows. In Sect. 2, we introduce preliminaries for proof of main results and variational setting. In Sect. 3, we present our main results and their proofs.

2 Preliminaries

Let us address a Hilbert space

$$ H^{s}\bigl(\mathbb{R}^{N}\bigr)=\biggl\{ u\in L^{2}\bigl(\mathbb{R}^{N}\bigr):\frac { \vert u(x)-u(y) \vert }{ \vert x-y \vert ^{\frac{N}{2}+s}}\in L^{2}\bigl(\mathbb{R}^{N}\times\mathbb {R}^{N}\bigr)\biggr\} . $$
(2.1)

The space is endowed with the natural norm

$$ \Vert u \Vert _{H^{s}}=\biggl( \int_{\mathbb{R}^{N}} \bigl\vert u(x) \bigr\vert ^{2}\,dx+ \int_{\mathbb{R}^{N}} \int _{\mathbb{R}^{N}}\frac{ \vert u(x)-u(y) \vert ^{2}}{ \vert x-y \vert ^{N+2s}}\,dx\,dy\biggr)^{\frac{1}{2}} $$
(2.2)

and with the inner product

$$ \langle u, \varphi\rangle= \int_{\mathbb{R}^{N}}u(x)\varphi(x)\,dx+ \int _{\mathbb{R}^{N}} \int_{\mathbb{R}^{N}}\frac{(u(x)-u(y))(\varphi(x)-\varphi (y))}{ \vert x-y \vert ^{N+2s}}\,dx\,dy. $$
(2.3)

By means of the Fourier transform, the space \(H^{s}(\mathbb{R}^{N})\) can be defined by

$$ H^{s}\bigl(\mathbb{R}^{N}\bigr)=\biggl\{ u\in L^{2}\bigl(\mathbb{R}^{N}\bigr): \int_{\mathbb{R}^{N}}\bigl(1+ \vert \xi \vert ^{2} \bigr)^{s} \bigl\vert \mathscr{F}u(\xi) \bigr\vert ^{2}\,d\xi< +\infty\biggr\} . $$
(2.4)

For Eq. (1.2), the Hilbert space H is defined by

$$ H=\biggl\{ u\in H^{s}\bigl(\mathbb{R}^{N}\bigr): \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr {F}u(\xi) \bigr\vert ^{2}\,d\xi+ \int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{2}\,dx< +\infty \biggr\} , $$
(2.5)

with the following inner product and norm:

$$ \langle u, \varphi\rangle_{H}= \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr {F}u(\xi) \bigr\vert \bigl\vert \mathscr{F}\varphi(\xi) \bigr\vert \,d\xi+ \int_{\mathbb{R}^{N}}V(x)u\varphi \,dx $$
(2.6)

and

$$ \Vert u \Vert _{H}=\biggl( \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}u(\xi) \bigr\vert ^{2}\,d\xi+ \int _{\mathbb{R}^{N}}V(x) \vert u \vert ^{2}\,dx \biggr)^{\frac{1}{2}}. $$
(2.7)

Then \(H\times H\) is a Hilbert space with the following the inner product \(\langle\cdot, \cdot\rangle\) and norm for any \((u, v), (\varphi , \psi)\in H\times H\):

$$ \bigl\langle (u, v), (\varphi, \psi)\bigr\rangle =\langle u, \varphi\rangle+ \langle v, \psi\rangle $$

and

$$ \bigl\Vert (u, v) \bigr\Vert ^{2}=\bigl\langle (u, v),(u, v)\bigr\rangle = \Vert u \Vert _{H}^{2}+ \Vert v \Vert _{H}^{2}. $$

Under the hypothesis \((V_{1})\), we have the following lemma.

Lemma 2.1

The Hilbert space\(H\times H\)is compactly embedded in\(L^{t}(\mathbb {R}^{N})\times L^{t}(\mathbb{R}^{N})\), where\(t\in[2, 2^{*}_{s})\)and\(2^{*}_{s}=\frac {2N}{N-2s}\).

Proof

Let \(\{u_{n}, v_{n}\}\subset H\times H\) be a sequence such that \(u_{n}\rightharpoonup u\), \(v_{n}\rightharpoonup v\) in H. Then \(\{u_{n}, v_{n}\} \) is bounded in \(H\times H\) and \(u_{n}\rightarrow u\), \(v_{n}\rightarrow v\) in \(L_{\mathrm{loc}}^{t}(\mathbb{R}^{N})\) for \(t\in[2, 2^{*}_{s})\). Using the famous Gagliado–Nirenberg inequality, we obtain \(u_{n}\rightarrow u\), \(v_{n}\rightarrow v\) in \(L^{t}(\mathbb{R}^{N})\). Thus, the proof is completed □

An element \((u,v)\in H\times H\) is called a weak solution of the systems (1.1), if the equation

$$ \begin{aligned}[b] & \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}u(\xi) \bigr\vert \bigl\vert \mathscr{F}\varphi(\xi ) \bigr\vert \,d\xi+ \int_{\mathbb{R}^{N}}V(x)u\varphi \,dx+ \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}v(\xi) \bigr\vert \bigl\vert \mathscr{F}\psi(\xi) \bigr\vert \,d\xi \\ &\qquad+ \int_{\mathbb{R}^{N}}V(x)v\psi \,dx \\ &\quad= \int_{\mathbb{R}^{N}} F_{u}(x,u,v)\varphi \,dx+ \int_{\mathbb{R}^{N}} F_{v}(x,u,v)\psi \,dx \end{aligned} $$
(2.8)

holds for all \((\varphi, \psi)\in H\times H\). A weak solution of the systems (1.1) corresponds to a critical point of the energy functional

$$ \begin{aligned}[b] I(u, v)&=\frac{1}{2} \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}u(\xi ) \bigr\vert ^{2}\,d\xi+\frac{1}{2} \int_{\mathbb{R}^{N}}V(x)u^{2}\,dx \\ &\quad+\frac{1}{2} \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}v(\xi) \bigr\vert ^{2}\,d\xi +\frac{1}{2} \int_{\mathbb{R}^{N}}V(x)v^{2} \,dx- \int_{\mathbb{R}^{N}} F(x,u,v)\,dx \end{aligned} $$
(2.9)

that is well defined. Furthermore, I is \(C^{1}(H\times H, \mathbb{R})\) functional with derivative given by

$$ \begin{aligned}[b] \bigl\langle I'(u, v), (\varphi, \psi)\bigr\rangle &= \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}u(\xi) \bigr\vert \bigl\vert \mathscr{F}\varphi(\xi) \bigr\vert d \xi+ \int_{\mathbb {R}^{N}}V(x)u\varphi \,dx \\ &\quad+ \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}v(\xi) \bigr\vert \bigl\vert \mathscr{F}\psi(\xi ) \bigr\vert d \xi+ \int_{\mathbb{R}^{N}}V(x)v\psi \,dx \\ &\quad- \int_{\mathbb{R}^{N}} F_{u}(x,u,v)\varphi \,dx- \int_{\mathbb{R}^{N}} F_{v}(x,u,v)\psi \,dx. \end{aligned} $$
(2.10)

Let \(H\times H\) be Banach space with the norm \(\|(\cdot,\cdot)\|\) and let \(\{H_{j}\}\) be a sequence of subspace of H, \(\operatorname{dim}H_{j}\) is finite for \(j\in N\). Set \(Y_{k}:=\bigoplus^{k}_{j=0}H_{j}\) and \(\mathscr{Y}_{k}=Y_{k}\times Y_{k}\), \(Z_{k}=\overline{\bigoplus^{\infty}_{j=k+1}H_{j}}\) and \(\mathscr {Z}_{k}=Z_{k}\times Z_{k}\), then \(H=Y_{k}\oplus Z_{k}\) and \(H\times H=\mathscr {Y}_{k}\oplus\mathscr{Z}_{k}\).

Let

$$B_{k}=\bigl\{ (u,v)\in\mathscr{Y}_{k}: \bigl\Vert (u, v) \bigr\Vert \leq\rho_{k}\bigr\} $$

and

$$S_{k}=\bigl\{ (u, v)\in\mathscr{Z}_{k}: \bigl\Vert (u, v) \bigr\Vert =r_{k}\bigr\} , $$

for \(\rho_{k}>r_{k}>0\). Consider a classical \(C^{1}\)-functional \(\varPhi_{\lambda}(u, v):H\times H\rightarrow\mathbb{R}\) defined by

$$ \varPhi_{\lambda}(u, v)=A(u, v)-\lambda B(u,v),\quad\lambda\in[1, 2]. $$
(2.11)

Now, we state two variant fountain theorems which come from the idea of Zou in [39].

Theorem 2.2

Assume that the functional\(\varPhi_{\lambda}(u, v)\)satisfies

\((B_{1})\):

\(\varPhi_{\lambda}(u, v)\)maps bounded sets to bounded sets uniformly for\(\lambda\in[1, 2]\), and\(\varPhi_{\lambda}(-u, -v)=\varPhi _{\lambda}(u, v)\)for all\((\lambda, u, v)\in[1, 2]\times H\times H\);

\((B_{2})\):

\(B(u, v)\geq0\)for all\((u, v)\in H\times H\), and\(B(u, v)\rightarrow\infty\)as\(\|(u, v)\|\rightarrow+\infty\)on any finite dimensional subspace\(H\times H\);

\((B_{3})\):

there exists\(\rho_{k}>r_{k}>0\)such that

$$\begin{gathered} a_{k}(\lambda)=\inf_{(u, v)\in\mathscr{Z}_{k},\|(u, v)\|=\rho_{k}}\varPhi_{\lambda}(u, v)\geq0,\\ b_{k}(\lambda)=\max_{(u, v)\in\mathscr{Y}_{k}, \|(u, v)\|=r_{k}} \varPhi_{\lambda}(u, v)< 0, \quad\forall\lambda\in[1, 2], \end{gathered}$$

and

$$d_{k}(\lambda)=\inf_{(u, v)\in\mathscr{Z}_{k},\|(u, v)\|\leq\rho_{k}}\varPhi _{\lambda}(u, v)\rightarrow0 \quad\textit{as } k\rightarrow+\infty \textit { uniformly for } \lambda\in[1, 2]. $$

Then there exist\(\lambda_{n}\rightarrow1\), \((u_{n}(\lambda_{n}),v_{n}(\lambda_{n}))\in \mathscr{Y}_{n}\)such that

$$\varPhi'_{\lambda_{n}}\big|_{\mathscr{Y}_{n}}\bigl(u( \lambda_{n}),v(\lambda_{n})\bigr)=0 \quad\textit {and}\quad \varPhi_{\lambda_{n}}\bigl(u(\lambda_{n}),v(\lambda_{n}) \bigr)\rightarrow c_{k}, \quad\textit {as } n\rightarrow+\infty, $$

where\(c_{k}\in[d_{k}(2), d_{k}(1)]\). Especially, if\(\{(u(\lambda _{n}),v(\lambda_{n}))\}\)has a convergent subsequence for everyk, then\(\varPhi_{1}\)has infinitely many nontrivial critical points\(\{u_{k}, v_{k}\} \in H\times H\setminus\{0,0\}\)satisfying\(\varPhi_{1}(u_{k}, v_{k})\rightarrow 0^{-}\)as\(k\rightarrow+\infty\).

Theorem 2.3

Assume that the functional\(\varPhi_{\lambda}(u, v)\)satisfies

\((A_{1})\):

\(\varPhi_{\lambda}(u, v)\)maps bounded sets to bounded sets uniformly for\(\lambda\in[1, 2]\), and\(\varPhi_{\lambda}(-u, -v)=\varPhi _{\lambda}(u, v)\)for all\((\lambda, u, v)\in[1, 2]\times H\times H\);

\((A_{2})\):

\(B(u, v)\geq0\)for all\((u, v)\in H\times H\), \(A(u, v)\rightarrow+\infty\)or\(B(u, v)\rightarrow+\infty\)as\(\|(u, v)\| \rightarrow+\infty\);

or

\((A_{3})\):

\(B(u, v)\leq0\)for all\((u, v)\in H\times H\), \(B(u, v)\rightarrow-\infty\)as\(\|(u, v)\|\rightarrow+\infty\);

\((A_{4})\):

there exists\(\rho_{k}>r_{k}>0\)such that

$$b_{k}(\lambda)=\inf_{(u, v)\in\mathscr{Z}_{k},\|(u, v)\|=r_{k}}\varPhi_{\lambda}(u, v)>a_{k}(\lambda)=\max_{(u, v)\in\mathscr{Y}_{k}, \|(u, v)\|=\rho_{k}}\varPhi _{\lambda}(u, v),\quad \forall\lambda\in[1, 2]. $$

Then

$$b_{k}(\lambda)\leq c_{k}(\lambda)=\inf _{\gamma\in\varGamma_{k}}\max_{(u, v)\in B_{k}}\varPhi_{\lambda}\bigl(\gamma(u, v)\bigr), \quad\forall\lambda\in[1, 2], $$

where\(\varGamma_{k}=\{\gamma\in C(B_{k}, H\times H):\gamma\textit{ is odd},\gamma|_{\partial B_{k}}=\mathit{id}\}\)and\(k\geq2\). Furthermore, for almost every\(\lambda\in[1, 2]\), we have a sequence\(\{(u^{k}_{n}(\lambda), v^{k}_{n}(\lambda))\}\)such that

$$\sup_{n} \bigl\Vert \bigl(u^{k}_{n}( \lambda), v^{k}_{n}(\lambda)\bigr) \bigr\Vert < +\infty,\qquad \varPhi'_{\lambda}\bigl(u^{k}_{n}( \lambda), v^{k}_{n}(\lambda)\bigr)\rightarrow0, $$

and

$$\varPhi_{\lambda}\bigl(u^{k}_{n}(\lambda), v^{k}_{n}(\lambda)\bigr)\rightarrow c_{k}( \lambda)\quad \textit{as } n\rightarrow+\infty. $$

In order to present our main work by the above variant fountain theorems, we define the functional A, B and \(\varPhi_{\lambda}(u, v)\) on the space \(H\times H\) by

$$\begin{gathered} \begin{aligned} A(u, v)&=\frac{1}{2} \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}u(\xi ) \bigr\vert ^{2}\,d\xi+\frac{1}{2} \int_{\mathbb{R}^{N}}V(x)u^{2}\,dx+\frac{1}{2} \int _{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}v(\xi) \bigr\vert ^{2}\,d\xi \\ &\quad+\frac{1}{2} \int_{\mathbb{R}^{N}}V(x)v^{2} \,dx, \end{aligned} \\ B(u,v)= \int_{\mathbb{R}^{N}} F(x,u,v)\,dx,\end{gathered} $$

and

$$\varPhi_{\lambda}(u, v)=A(u, v)-\lambda B(u,v). $$

3 Proofs of the main results

In this section, we will present the main results and their proofs.

Lemma 3.1

For any finite dimensional subspaceEof\(H\times H\setminus\{(0,0)\} \), we claim that there exists a positive constant\(\varepsilon_{0}>0\)such that

$$ \operatorname{meas}\bigl\{ x\in\mathbb{R}^{N}: \bigl\vert (u,v) \bigr\vert \geq\varepsilon_{0} \bigl\Vert (u,v) \bigr\Vert \bigr\} \geq \varepsilon_{0}, \quad\textit{for any } (u, v)\in E. $$

Proof

We argue by contradiction. Assume \((u_{n},v_{n})\in E \) such that

$$ \text{meas}\biggl\{ x\in\mathbb{R}^{N}: \bigl\vert (u_{n}, v_{n}) \bigr\vert \geq\frac{1}{n} \bigl\Vert (u_{n}, v_{n}) \bigr\Vert \biggr\} < \frac{1}{n},\quad \text{for any } n\in N. $$
(3.1)

For any \(n\in N\), let \((\tau_{n}, \omega_{n})=\frac{(u_{n}, v_{n})}{\|(u_{n}, v_{n})\|}\), then \(\|(\tau_{n}, \omega_{n})\| =1\) and

$$ \text{meas}\biggl\{ x\in\mathbb{R}^{N}: \bigl\vert (\tau_{n}, \omega_{n}) \bigr\vert \geq \frac{1}{n}\biggr\} < \frac{1}{n}. $$
(3.2)

Using the boundedness of \((\tau_{n}, \omega_{n})\), up to a subsequence, assume that \((\tau_{n}, \omega_{n})\rightarrow(\tau, \omega)\) with \(\|(\tau , \omega)\|=1\) for \((\tau, \omega)\in E\). Since E is a finite dimension space, by Lemma 2.1 and the Hölder inequality, we have

$$ \begin{aligned}[b] \int_{\mathbb{R}^{N}} \bigl\vert (\tau_{n}, \omega_{n})-(\tau, \omega) \bigr\vert ^{2}\,dx&= \int _{\mathbb{R}^{N}} \bigl\vert (\tau_{n}-\tau, \omega_{n}-\omega) \bigr\vert ^{2}\,dx \\ &\leq\biggl( \int_{\mathbb{R}^{N}} \vert \tau_{n}-\tau \vert ^{4}\,dx\biggr)^{\frac{1}{2}}\biggl( \int _{\mathbb{R}^{N}} \vert \omega_{n}-\omega \vert ^{4}\,dx\biggr)^{\frac{1}{2}} \\ &\rightarrow0. \end{aligned} $$
(3.3)

On the other hand, because of \((\tau, \omega)\neq(0, 0)\), there exists a constant \(\rho_{0}>0\) such that

$$ \text{meas}\bigl\{ x\in\mathbb{R}^{N}: \bigl\vert (\tau, \omega) \bigr\vert \geq\rho_{0}\bigr\} \geq \rho_{0}. $$
(3.4)

We set

$$ \begin{aligned} &\varOmega_{n}=\biggl\{ x\in \mathbb{R}^{N}: \bigl\vert (\tau_{n}, \omega_{n}) \bigr\vert < \frac{1}{n}\biggr\} , \\ &\varOmega^{c}_{n}=\biggl\{ x\in\mathbb{R}^{N}: \bigl\vert (\tau_{n}, \omega_{n}) \bigr\vert \geq \frac{1}{n}\biggr\} , \\ &\varOmega_{0}=\bigl\{ x\in\mathbb{R}^{N}: \bigl\vert ( \tau, \omega) \bigr\vert \geq\rho_{0}\bigr\} . \end{aligned} $$

From (3.2) and (3.4), there exists \(N_{0}\) such that, for \(\forall n>N_{0}\), we have

$$ \operatorname{meas}(\varOmega_{n}\cap\varOmega_{0}) \geq\operatorname{meas}(\varOmega_{0})-\operatorname {meas}\bigl( \varOmega^{c}_{n}\bigr)\geq\frac{(N_{0}-1)\rho_{0}}{N_{0}}. $$

Consequently, as \(n\rightarrow+\infty\),

$$ \begin{aligned}[b] \int_{\mathbb{R}^{N}} \bigl\vert (\tau_{n}, \omega_{n})-(\tau, \omega) \bigr\vert ^{2}\,dx&\geq \int _{\varOmega_{n}\cap\varOmega_{0}} \bigl\vert (\tau_{n}, \omega_{n})-(\tau, \omega) \bigr\vert ^{2}\,dx \\ &\geq \int_{\varOmega_{n}\cap\varOmega_{0}}\bigl[ \bigl\vert (\tau_{n}, \omega_{n}) \bigr\vert ^{2}-2(\tau_{n}, \omega_{n}) (\tau, \omega)+ \bigl\vert (\tau, \omega) \bigr\vert ^{2}\bigr]\,dx \\ &\geq \int_{\varOmega_{n}\cap\varOmega_{0}}\bigl[ \bigl\vert (\tau, \omega) \bigr\vert ^{2}-2(\tau_{n}, \omega _{n}) (\tau, \omega)\bigr]\,dx \\ &\geq \int_{\varOmega_{n}\cap\varOmega_{0}}\bigl[ \bigl\vert (\tau, \omega) \bigr\vert ^{2}-2 \bigl\vert (\tau_{n}, \omega _{n}) \bigr\vert \bigl\vert (\tau, \omega) \bigr\vert \bigr]\,dx \\ &\geq\rho_{0}\biggl(\rho_{0}-\frac{2}{n} \biggr)\operatorname{meas}(\varOmega_{n}\cap\varOmega_{0}) \\ &\geq\rho_{0}\biggl(\rho_{0}-\frac{2}{n} \biggr)\frac{(N_{0}-1)\rho_{0}}{N_{0}} \\ &\geq\frac{(N_{0}-1)\rho^{3}_{0}}{N_{0}} \\ &>0, \end{aligned} $$
(3.5)

which leads to a contradiction. The proof is completed. □

Lemma 3.2

Assume that\((f_{1})\)and\((f_{3})\)hold. Then\(B(u, v)\geq0\)for all\((u, v)\in H\times H\)and\(B(u, v)\rightarrow+\infty\)as\(\|(u, v)\| \rightarrow+\infty\)on any finite dimensional subspace of\(H\times H\).

Proof

Obviously, for all \((u, v)\in H\times H\), \(B(u, v)\geq0\) by the hypothesis \((f_{1})\).

Next, for any finite dimensional subspace of \(H\times H\), we show that \(B(u, v)\rightarrow+\infty\) as \(\|(u, v)\|\rightarrow+\infty\). By the hypothesis \((f_{3})\), there exists \(R>0\) such that

$$ F(x, u, v)\geq d \bigl\vert (u,v) \bigr\vert ^{\sigma}, \quad\text{for } x\in\mathbb{R}^{N} \text{ and } \bigl\vert (u,v) \bigr\vert >R. $$
(3.6)

Let \(D_{(u,v)}:=\{x\in\mathbb{R}^{N}:|(u, v)|>\varepsilon_{0}\|(u, v)\|\}\) for \((u, v)\in H\times H\setminus\{(0,0)\}\). by Lemma 3.1, for any \((u, v)\in H\times H\) with \(\|(u, v)\|\geq\frac{R}{\varepsilon _{0}}\), we have \(|(u, v)|>R\), for all \(x\in D_{(u,v)}\). Consequently, for any \((u, v)\in H\times H\) with \(\|(u, v)\|\geq\frac{R}{\varepsilon_{0}}\), from \((f_{3})\) and (3.6), with the help of Lemma 3.1, we get

$$ \begin{aligned}[b] B(u,v)&= \int_{\mathbb{R}^{N}} F(x,u,v)\,dx \\ &\geq \int_{D_{(u, v)}} F(x,u,v)\,dx \\ &\geq \int_{D_{(u, v)}}d \bigl\vert (u,v) \bigr\vert ^{\sigma}\,dx \\ &\geq d\varepsilon_{0}^{\sigma}\bigl\Vert (u, v) \bigr\Vert ^{\sigma}\operatorname{meas}(D_{(u, v)}) \\ &\geq d\varepsilon_{0}^{\sigma+1} \bigl\Vert (u, v) \bigr\Vert ^{\sigma}. \end{aligned} $$
(3.7)

This implies \(B(u, v)\rightarrow+\infty\) as \(\|(u, v)\|\rightarrow +\infty\) on any finite dimensional subspace of \(H\times H\). The proof is completed. □

Lemma 3.3

Assume\((f_{1})\), \((f_{2})\)and\((f_{4})\)hold, then there exist two sequences\(\rho_{k}>r_{k}>0\)such that

$$b_{k}(\lambda)=\inf_{(u, v)\in\mathscr{Z}_{k},\|(u, v)\|=r_{k}}\varPhi_{\lambda}(u, v)>a_{k}(\lambda)=\max_{(u, v)\in\mathscr{Y}_{k}, \|(u, v)\|=\rho_{k}}\varPhi _{\lambda}(u, v), \quad\forall\lambda\in[1, 2]. $$

Proof

. For \(\forall \varepsilon>0\), by \((f_{1})\) and \((f_{2})\), there exists \(c_{\varepsilon}\) such that

$$ \begin{aligned} & \bigl\vert F_{u}(x, u, v ) \bigr\vert \leq\varepsilon \bigl\vert (u,v) \bigr\vert +c_{\varepsilon} \bigl\vert (u,v) \bigr\vert ^{p-1}, \\ & \bigl\vert F_{v}(x, u, v ) \bigr\vert \leq\varepsilon \bigl\vert (u,v) \bigr\vert +c_{\varepsilon} \bigl\vert (u,v) \bigr\vert ^{q-1}, \end{aligned} $$
(3.8)

and

$$ \begin{aligned}[b] \bigl\vert F(x, u, v ) \bigr\vert &= \bigl\vert F(x, u, v)-F(x, 0, 0) \bigr\vert \\ &\leq \int_{0}^{1} \bigl\vert F_{u}(x, tu, tv ) \bigr\vert \vert u \vert \,dt+ \int_{0}^{1} \bigl\vert F_{v}(x, tu, tv ) \bigr\vert \vert v \vert \,dt \\ &\leq\varepsilon\biggl[\frac{1}{2} \bigl\vert (u,v) \bigr\vert \vert u \vert +\frac {1}{2} \bigl\vert (u,v) \bigr\vert \vert v \vert \biggr]+c_{\varepsilon}\biggl[\frac{1}{p} \bigl\vert (u,v) \bigr\vert ^{p-1} \vert u \vert +\frac {1}{q} \bigl\vert (u,v) \bigr\vert ^{q-1} \vert v \vert \biggr],\hspace{-12pt} \end{aligned} $$
(3.9)

where \((x, u, v)\in\mathbb{R}^{N}\times H\times H\). Therefore, for \((u, v)\in\mathscr{Z}_{k}\) and ε small enough, by (3.9) and the Hölder inequality, one has

$$\begin{aligned} \varPhi_{\lambda}(u, v)&=\frac{1}{2} \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr {F}u(\xi) \bigr\vert ^{2}\,d\xi+\frac{1}{2} \int_{\mathbb{R}^{N}}V(x)u^{2}\,dx+\frac{1}{2} \int _{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}v(\xi) \bigr\vert ^{2}\,d\xi \\ &\quad+\frac{1}{2} \int_{\mathbb{R}^{N}}V(x)v^{2} \,dx-\lambda \int_{\mathbb{R}^{N}} F(x,u,v)\,dx \\ &\geq\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}- \lambda \int_{\mathbb{R}^{N}}\varepsilon \biggl[\frac{1}{2} \bigl\vert (u,v) \bigr\vert \vert u \vert +\frac{1}{2} \bigl\vert (u,v) \bigr\vert \vert v \vert \biggr] \\ &\quad+c_{\varepsilon}\biggl[ \frac {1}{p} \bigl\vert (u,v) \bigr\vert ^{p-1} \vert u \vert +\frac{1}{q} \bigl\vert (u,v) \bigr\vert ^{q-1} \vert v \vert \biggr]\,dx \\ &\geq\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}- \lambda\varepsilon\biggl(\frac{1}{2} \bigl\Vert (u,v) \bigr\Vert _{2} \Vert u \Vert _{2}+\frac{1}{2} \bigl\Vert (u,v) \bigr\Vert _{2} \Vert v \Vert _{2} \biggr) \\ &\quad-\lambda c_{\varepsilon}\biggl[\frac{1}{p} \bigl\Vert (u,v) \bigr\Vert _{p}^{p-1} \Vert u \Vert _{p}+ \frac {1}{q} \bigl\Vert (u,v) \bigr\Vert _{q}^{q-1} \Vert v \Vert _{q}\biggr] \\ &\geq\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}- \lambda\varepsilon\biggl[\frac{1}{2} \bigl\Vert (u,v) \bigr\Vert _{2}^{2}+\frac{1}{2} \bigl\Vert (u,v) \bigr\Vert _{2}^{2}\biggr] \\ &\quad-\lambda c_{\varepsilon}\biggl[ \frac{1}{p} \bigl\Vert (u,v) \bigr\Vert _{p}^{p}+\frac{1}{q} \bigl\Vert (u,v) \bigr\Vert _{q}^{q} \biggr], \end{aligned}$$

where \(\|\cdot\|_{t}\) denotes the usual norm of \(L^{t}(\mathbb{R}^{N})\). Let \(\beta_{k}(2):=\sup_{(u,v)\in\mathscr{Z}_{k}, \|(u,v)\|=1}\|(u,v)\|_{2}\), \(\beta_{k}(p):=\sup_{(u,v)\in\mathscr{Z}_{k}, \|(u,v)\|=1}\|(u,v)\|_{p}\), \(\beta_{k}(q):=\sup_{(u,v)\in\mathscr{Z}_{k}, \|(u,v)\|=1}\|(u,v)\|_{q}\), then \(\beta_{k}(2)\rightarrow0\), \(\beta_{k}(p)\rightarrow0\), \(\beta _{k}(q)\rightarrow0\) as \(k\rightarrow\infty\) (cf.[40]). Consequently,

$$ \begin{aligned}[b] \varPhi_{\lambda}(u, v)&\geq \frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}-\lambda \varepsilon\beta ^{2}_{k}(2) \bigl\Vert (u, v) \bigr\Vert ^{2}-\frac{1}{p}\lambda c_{\varepsilon} \beta^{p}_{k}(p) \bigl\Vert (u, v) \bigr\Vert ^{p} -\frac{1}{q}\lambda c_{\varepsilon} \beta^{q}_{k}(q) \bigl\Vert (u, v) \bigr\Vert ^{q}\hspace{-24pt} \\ &\geq\biggl(\frac{1}{2}-\lambda\varepsilon\beta^{2}_{k}(2) \biggr) \bigl\Vert (u, v) \bigr\Vert ^{2}-\frac {1}{p} \lambda c_{\varepsilon}\beta^{p}_{k}(p) \bigl\Vert (u, v) \bigr\Vert ^{p}\\&\quad-\frac {1}{q}\lambda c_{\varepsilon}\beta^{q}_{k}(q) \bigl\Vert (u, v) \bigr\Vert ^{q}, \end{aligned} $$
(3.10)

for all \((u, v)\in\mathscr{Z}_{k}\). We choose the appropriate \(\varepsilon >0\) and λ such that \(\frac{1}{2}-\lambda\varepsilon\beta ^{2}_{k}(2)\geq\frac{1}{4}\), and we have

$$ \varPhi_{\lambda}(u, v)\geq\frac{1}{4} \bigl\Vert (u, v) \bigr\Vert ^{2}-\frac{1}{p}\lambda c_{\varepsilon} \beta^{p}_{k}(p) \bigl\Vert (u, v) \bigr\Vert ^{p}-\frac{1}{q}\lambda c_{\varepsilon} \beta^{q}_{k}(q) \bigl\Vert (u, v) \bigr\Vert ^{q}. $$

Note that \(p, q>2\); without loss of generality, assume \(p< q\), then, for \(\|(u, v)\|:=r_{k}:=(\frac{8}{p}\lambda c_{\varepsilon}\beta^{p}_{k}(p)+\frac {8}{q}\lambda c_{\varepsilon}\beta^{q}_{k}(q))^{\frac{1}{2-p}}\) or \(\|(u, v)\|:=r_{k}:=(\frac{8}{p}\lambda c_{\varepsilon}\beta^{p}_{k}(p)+\frac {8}{q}\lambda c_{\varepsilon}\beta^{q}_{k}(q))^{\frac{1}{2-q}}\) for any \((u, v)\in\mathscr{Z}_{k}\), one has

$$ \varPhi_{\lambda}(u, v)\geq\frac{1}{8}r_{k}^{2}>0. $$
(3.11)

The above inequality implies that

$$b_{k}(\lambda)=\inf_{(u, v)\in\mathscr{Z}_{k},\|(u, v)\|=r_{k}}\varPhi_{\lambda}(u, v)>0. $$

Therefore, by Lemma 3.1, for any \(k\in N\), there is a constant \(\varepsilon_{k}>0\) such that

$$ \operatorname{meas}(D_{(u, v)})\geq\varepsilon_{k},\quad \text{for all } (u, v)\in Y_{k}\times Y_{k}\setminus \bigl\{ (0,0)\bigr\} , $$
(3.12)

where \(D_{(u, v)}=\{x\in\mathbb{R}^{N}:|(u, v)|\geq\varepsilon_{k}\|(u, v)\|\}\). By the hypothesis \((f_{4})\), for \(\forall k\in N\), there is a constant \(R_{k}>0\) such that

$$ F(x, u, v)\geq\frac{1}{\varepsilon^{3}_{k}} \bigl\vert (u, v) \bigr\vert ^{2}, \quad\text{for all } \bigl\vert (u, v) \bigr\vert \geq R_{k}. $$
(3.13)

By (3.12), we know that, for \((u, v)\in Y_{k}\times Y_{k}\setminus\{ (0,0)\}\) with \(\|(u, v)\|\geq\frac{R_{k}}{\varepsilon_{k}}\), we obtain \(|(u, v)|\geq R_{k}\) for \(x\in D_{(u, v)}\). Therefore, by (3.12) and (3.13) for \((u, v)\in Y_{k}\times Y_{k}\setminus\{(0,0)\}\) with \(\|(u, v)\|\geq\frac{R_{k}}{\varepsilon_{k}}\), one has

$$ \begin{aligned}[b] \varPhi_{\lambda}(u, v)&=\frac{1}{2} \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr {F}u(\xi) \bigr\vert ^{2}\,d\xi+\frac{1}{2} \int_{\mathbb{R}^{N}}V(x)u^{2}\,dx+\frac{1}{2} \int _{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}v(\xi) \bigr\vert ^{2}\,d\xi \\ &\quad+\frac{1}{2} \int_{\mathbb{R}^{N}}V(x)v^{2} \,dx-\lambda \int_{\mathbb{R}^{N}} F(x,u,v)\,dx \\ &=\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}-\lambda \int_{\mathbb{R}^{N}} F(x,u,v)\,dx \\ &\leq\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}- \lambda \int_{D_{(u, v)}} F(x,u,v)\,dx \\ &\leq\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}- \lambda \int_{D_{(u, v)}} \frac {1}{\varepsilon^{3}_{k}} \bigl\vert (u, v) \bigr\vert ^{2}\,dx \\ &\leq\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}- \lambda\varepsilon^{2}_{k} \bigl\Vert (u, v) \bigr\Vert ^{2}\frac {\operatorname{meas}(D_{(u, v)})}{\varepsilon^{3}_{k}} \\ &\leq\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}- \lambda \bigl\Vert (u, v) \bigr\Vert ^{2}. \end{aligned} $$
(3.14)

Because \(\lambda\in[1, 2]\),

$$ \varPhi_{\lambda}(u, v)\leq\frac{1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}- \bigl\Vert (u, v) \bigr\Vert ^{2}=- \frac {1}{2} \bigl\Vert (u, v) \bigr\Vert ^{2}. $$
(3.15)

Now, we only need to choose \(\rho_{k}>\max\{r_{k},\frac{R_{k}}{\varepsilon_{k}} \}\); one has

$$ a_{k}(\lambda)=\max_{(u, v)\in\mathscr{Y}_{k}, \|(u, v)\|=\rho_{k}}\varPhi _{\lambda}(u, v)=-\frac{\rho_{k}}{2}< 0, $$
(3.16)

where \(k\in N\) and \(\lambda\in[1, 2]\). The proof is completed. □

Theorem 3.4

Assume\((f_{1})\)\((f_{5})\)hold, \(F_{u}(x,-u,-v)=-F_{u}(x,u,v)\)and\(F_{v}(x,-u,-v)= -F_{v}(x,u,v)\)for\((x,u,v)\in\mathbb{R}^{N}\times H \times H\). Then the system (1.1) possesses infinitely many high energy solutions\((u^{k},v ^{k})\in H \times H\)for all\(k\geq K_{0}\)with\(K_{0} \in N\), i.e., as\(k\rightarrow+\infty\)

$$ \begin{aligned}[b] \varPhi_{\lambda}\bigl(u^{k}, v^{k}\bigr)&=\frac{1}{2} \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}u^{k}(\xi) \bigr\vert ^{2}\,d\xi+ \frac{1}{2} \int_{\mathbb{R}^{N}}V(x) \bigl\vert u^{k} \bigr\vert ^{2}\,dx +\frac{1}{2} \int_{\mathbb{R}^{N}} \vert \xi \vert ^{2s} \bigl\vert \mathscr{F}v^{k}(\xi) \bigr\vert ^{2}\,d\xi\hspace{-36pt} \\ &\quad+\frac{1}{2} \int_{\mathbb{R}^{N}}V(x) \bigl\vert v^{k} \bigr\vert ^{2} \,dx-\lambda \int_{\mathbb {R}^{N}} F\bigl(x,u^{k},v^{k} \bigr)\,dx\\&\rightarrow+\infty \text{.} \end{aligned} $$
(3.17)

Proof

By the hypothesis \((f_{1})\), we conclude that \(B(u, v)\geq0\) for all \((u,v)\in H\times H\) and \(A(u, v)\rightarrow+\infty\) as \(\|(u, v)\| \rightarrow+\infty\). Furthermore, \(\varPhi_{\lambda}(-u, -v)=\varPhi_{\lambda }(u, v)\) for \((u, v)\in H\times H\) and \(\lambda\in[1, 2]\). Considering \((f_{4})\), \((f_{5})\) and Lemma 2.1, \(\varPhi_{\lambda}(u, v)\) maps a bounded set into a bounded set uniformly for any \(\lambda\in[1, 2]\). By Lemma 3.3, we can verify \((A_{3})\), \((A_{4})\) of Theorem 2.3. Consequently, from Theorem 2.3, there exists a sequence \(\{(u_{n}^{k}(\lambda),v_{n}^{k}(\lambda))\} _{n=1}^{\infty}\) for \(\lambda\in[1, 2]\) such that

$$ \sup_{n} \bigl\Vert \bigl(u_{n}^{k}( \lambda),v_{n}^{k}(\lambda)\bigr) \bigr\Vert < \infty,\qquad \varPhi'_{\lambda }\bigl(u_{n}^{k}( \lambda),v_{n}^{k}(\lambda)\bigr)\rightarrow0 $$
(3.18)

and

$$ \varPhi_{\lambda}\bigl(u_{n}^{k}( \lambda),v_{n}^{k}(\lambda)\bigr)\rightarrow c_{k}(\lambda), $$
(3.19)

as \(n\rightarrow\infty\). By Theorem 2.2 and (3.11), for \(p,q>2\), we see

$$ c_{k}(\lambda)\geq b_{k}(\lambda)\geq \frac{1}{8}r_{k}^{2}=\bar{b}_{k} \rightarrow +\infty \quad\text{as } k\rightarrow+\infty. $$
(3.20)

In addition

$$ c_{k}(\lambda)=\inf_{\gamma\in\varGamma_{k}}\max _{(u, v)\in B_{k}}\varPhi_{\lambda}\bigl(\gamma(u, v)\bigr)\leq \max_{(u, v)\in B_{k}}\varPhi_{1}(u, v)=\bar{c}_{k}. $$

Therefore

$$ \bar{b}_{k}\leq c_{k}(\lambda)\leq\bar{c}_{k}, $$
(3.21)

where \(k>K_{0}\). By (3.18) and (3.19), we can choose a sequence \(\lambda_{m}\rightarrow1\), as \(m\rightarrow\infty\), then the sequence \(\{(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))\}_{n=1}^{\infty}\) is bounded. Obviously, the sequence \(\{(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda _{m}))\}_{n=1}^{\infty}\) has a strong convergent subsequence as \(n\rightarrow\infty\). Hence, for \(m\in N\) and \(k>K_{0}\), we suppose \(u_{n}^{k}(\lambda_{m})\rightarrow u^{k}(\lambda_{m})\), \(v_{n}^{k}(\lambda _{m})\rightarrow v^{k}(\lambda_{m})\) as \(n\rightarrow+\infty\). By (3.18)–(3.21), one has

$$ \bigl\langle \varPhi'_{\lambda_{m}}\bigl(u^{k}( \lambda_{m}), v^{k}(\lambda_{m})\bigr), \bigl(u^{k}(\lambda _{m}), v^{k}( \lambda_{m})\bigr)\bigr\rangle =0 $$
(3.22)

and

$$ \varPhi_{\lambda_{m}}\bigl(u^{k}(\lambda_{m}), v^{k}(\lambda_{m})\bigr)\in[\bar{b}_{k}, \bar{c}_{k}] $$
(3.23)

for \(k>K_{0}\). By Lemma 3.5, \(\{(u^{k}(\lambda_{m}), v^{k}(\lambda_{m}))\}_{m=1}^{\infty}\) has a strong convergent subsequence with \(u^{k}(\lambda_{m})\rightarrow u^{k}\), \(v^{k}(\lambda_{m})\rightarrow v^{k}\) for \(k>k_{0}\). Consequently, the \(( u^{k}, v^{k})\) is the critical point of \(\varPhi (u^{k}, v^{k})=\varPhi_{1}(u^{k}, v^{k}) \) with \(\varPhi(u^{k}, v^{k})\in[\bar{b}_{k}, \bar {c}_{k}]\). Since \(\bar{b}_{k}\rightarrow+\infty\) as \(k\rightarrow+\infty\), we get infinitely many nontrivial solutions with high energy for systems (1.1). The proof is completed. □

Lemma 3.5

\(\{(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))\}_{n=1}^{\infty}\)is bounded in\(H\times H\).

Proof

We argue by contradiction. Suppose that \(\|(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))\|\rightarrow\infty\) as \(n\rightarrow\infty\). We consider \((\tau_{n}, \omega_{n}):=\frac{(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda _{m}))}{\|(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))\|}\). Then, up to a subsequence, we get

$$ \begin{aligned} &\tau_{n}\rightharpoonup\tau,\qquad \omega_{n}\rightharpoonup\omega\quad\text{in } H\times H, \\ &\tau_{n}\rightarrow\tau,\qquad \omega_{n}\rightarrow \omega\quad\text{in } L^{t}\bigl(\mathbb{R}^{N}\bigr) \times L^{t}\bigl(\mathbb{R}^{N}\bigr), \\ &\tau_{n}(x)\rightarrow\tau(x),\qquad \omega_{n}(x) \rightarrow\omega(x)\quad \text{a.e. } x\in\mathbb{R}^{N}. \end{aligned} $$
(3.24)

We consider two cases:

Case 1: If \(|(\tau, \omega)|\neq0\) in \(H\times H\). Since \(\langle\varPhi '_{\lambda_{m}}(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})),(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))\rangle=0\), one has

$$ \begin{aligned} 0&=\bigl\langle \varPhi'_{\lambda_{m}} \bigl(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda _{m})\bigr), \bigl(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})\bigr)\bigr\rangle \\ &= \bigl\Vert \bigl(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})\bigr) \bigr\Vert ^{2} \\ &\quad-\lambda_{m} \int_{\mathbb {R}^{N}}\bigl[F_{u}(x,u_{n}^{k}( \lambda_{m}),v_{n}^{k}(\lambda_{m})u_{n}^{k}( \lambda_{m}) + F_{v}(x,u_{n}^{k}( \lambda_{m}),v_{n}^{k}(\lambda_{m})v_{n}^{k}( \lambda_{m})\bigr]\,dx. \end{aligned} $$

Thus, by Fatou’s lemma and conditions \((f_{3})\) and \((f_{4})\)

$$ \begin{aligned} 1&= \lambda_{m} \int_{\mathbb{R}^{N}} \frac{1}{ \Vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \Vert ^{2}}\bigl[F_{u}(x,u_{n}^{k}( \lambda_{m}),v_{n}^{k}(\lambda _{m})u_{n}^{k}(\lambda_{m}) \\ &\quad+ F_{v}(x,u_{n}^{k}(\lambda_{m}),v_{n}^{k}( \lambda_{m})v_{n}^{k}(\lambda_{m}) \bigr]\,dx \\ &\geq\lambda_{m}\mu \int_{\mathbb{R}^{N}} \frac{F(x,u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))}{ \Vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \Vert ^{2}}\,dx \\ &=\lambda_{m}\mu \int_{\mathbb{R}^{N}} \bigl\vert (\tau_{n}, \omega_{n}) \bigr\vert ^{2}\frac {F(x,u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))}{ \vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \vert ^{2}}\,dx \rightarrow+\infty\quad\text{as } n\rightarrow+\infty. \end{aligned} $$

This is a contradiction.

Case 2: If \(|(\tau, \omega)|= 0\) in \(H\times H\). By (3.22), (3.23) and \((f_{4})\), we obtain

$$ \begin{aligned} &\mu\varPhi_{\lambda_{m}}\bigl(u_{n}^{k}( \lambda_{m}), v_{n}^{k}( \lambda_{m})\bigr)-\bigl\langle \varPhi '_{\lambda_{m}} \bigl(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})\bigr), \bigl(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})\bigr)\bigr\rangle \\ &\quad=\biggl(\frac{\mu}{2}-1\biggr) \bigl\Vert \bigl(u_{n}^{k}( \lambda_{m}), v_{n}^{k}( \lambda_{m})\bigr) \bigr\Vert ^{2} \\ &\qquad+\lambda_{m} \int_{\mathbb{R}^{N}}\bigl[F_{u}(x,u_{n}^{k}( \lambda_{m}),v_{n}^{k}(\lambda _{m})u_{n}^{k}(\lambda_{m})+F_{v}(x,u_{n}^{k}( \lambda_{m}),v_{n}^{k}(\lambda_{m})v_{n}^{k}( \lambda_{m}) \\ &\qquad-\mu F(x,u_{n}^{k}(\lambda_{m}),v_{n}^{k}( \lambda_{m})\bigr]\,dx. \\ &\quad\geq\biggl(\frac{\mu}{2}-1\biggr) \bigl\Vert \bigl(u_{n}^{k}( \lambda_{m}), v_{n}^{k}( \lambda_{m})\bigr) \bigr\Vert ^{2}+ \lambda_{m} \int_{\mathbb{R}^{N}}c \bigl(1+ \bigl\vert \bigl(u_{n}^{k}( \lambda_{m}), v_{n}^{k}( \lambda_{m})\bigr) \bigr\vert ^{2} \bigr)\,dx. \end{aligned} $$

Therefore,

$$ \begin{aligned} &\frac{\mu\varPhi_{\lambda_{m}}(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))-\langle \varPhi'_{\lambda_{m}}(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})),(u_{n}^{k}(\lambda _{m}), v_{n}^{k}(\lambda_{m}))\rangle}{ \Vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \Vert ^{2}} \\ &\quad\geq\biggl(\frac{\mu}{2}-1\biggr)+\lambda_{m} \int_{\mathbb{R}^{N}}c \biggl(\frac {1+ \vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \vert ^{2}}{ \Vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \Vert ^{2}} \biggr)\,dx. \\ &\quad=\biggl(\frac{\mu}{2}-1\biggr)+\lambda_{m} \int_{\mathbb{R}^{N}}c \biggl(\frac{1}{ \Vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \Vert ^{2}}+\frac{ \vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \vert ^{2}}{ \Vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \Vert ^{2}} \biggr)\,dx \\ &\quad=\biggl(\frac{\mu}{2}-1\biggr)+\lambda_{m} \int_{\mathbb{R}^{N}}c \biggl(\frac{1}{ \Vert (u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m})) \Vert ^{2}}+ \bigl\vert ( \tau_{n}, \omega_{n}) \bigr\vert ^{2} \biggr)\,dx. \end{aligned} $$

Letting \(n\rightarrow+\infty\), we get \(0\geq\frac{\mu}{2}-1\), i.e. \(\mu \leq2\); this is a contradiction with the hypothesis \(\mu>2\). Therefore, \(\{(u_{n}^{k}(\lambda_{m}), v_{n}^{k}(\lambda_{m}))\}_{n=1}^{\infty}\) is bounded in \(H\times H\). □