For conventional topological phases, the boundary gapless modes are determined by bulk topological invariants. Based on developing an analytic method to solve higher-order boundary modes, we present $\mathcal{PT}$-invariant 2D topological insulators and 3D topological semimetals that go beyond this bulk-boundary correspondence framework. With unchanged bulk topological invariants, their first-order boundaries undergo transitions separating different phases with second-order boundary zero modes. For the 2D topological insulator, the helical edge modes appear at the transition point for two second-order topological insulator phases with diagonal and off-diagonal corner zero modes, respectively. Accordingly, for the 3D topological semimetal, the criticality corresponds to surface helical Fermi arcs of a Dirac semimetal phase. Interestingly, we find that the 3D system generically belongs to a novel second-order nodal-line semimetal phase, possessing gapped surfaces but a pair of diagonal or off-diagonal hinge Fermi arcs.

• Received 12 May 2020
• Accepted 6 August 2020

DOI:https://doi.org/10.1103/PhysRevLett.125.126403