Customized bus (CB) is an increasingly popular mode of transportation in many cities around the world. However, studies on CB network design have mostly overlooked three options that may further improve system performance: passenger-route assignment, passenger transfer, and modular vehicles. To bridge this gap, this paper proposes to design a transfer-based CB network with a modular fleet while simultaneously optimizing the passenger-route assignment. To solve the optimal network structure with this new design paradigm, we formulate the network design problem into a nonlinear mixed integer optimization model. A linearization approach and a particle swarm optimization (PSO) algorithm are proposed to solve the exact and near-optimal solution(s) to the model, respectively. Numerical experiments are conducted on the Sioux Falls network and a large-scale network in Chengdu, China. Results show that the customized PSO algorithm efficiently provides high quality near-optimal solutions compared with CPLEX, the genetic algorithm, and the simulated annealing algorithm. Results also show that incorporating passenger-route assignment optimization and the transfer operation produces a more cost-effective CB operational network with less operational costs and higher service quality. The benefit increases as the passenger demand grows.