Abstract
Particular types of plankton in aquatic ecosystems can coordinate their motion depending on the local flow environment to reach regions conducive to their growth or reproduction. Investigating their swimming strategies with regard to the local environment is important to obtain in-depth understanding of their behavior in the aquatic environment. In the present research, to examine an impact of the shape and gravity on a swimming strategy, plankton is considered as settling swimming particles of ellipsoidal shape. The Q-learning approach is adopted to obtain swimming strategies for smart particles with a goal of efficiently moving upwards in a two-dimensional steady flow. Strategies obtained from reinforcement learning are compared to those of naive gyrotactic particles that are modeled considering the behavior of realistic plankton. It is found that the elongation of particles improves the performance of upward swimming by facilitating particles resistance to the perturbation of vortex. In the case when the settling velocity is included, the strategy obtained by reinforcement learning has similar performance to that of the naive gyrotactic one, and they both align swimmers in upward direction. The similarity between the strategy obtained from machine learning and the biological gyrotactic strategy indicates the relationship between the aspherical shape and settling effect of realistic plankton and their gyrotactic feature.
Similar content being viewed by others
References
D. F. Blair, Annu. Rev. Microbiol. 49, 489 (1995).
K. Drescher, K. C. Leptos, Tuval. Tuval, T. Ishikawa, T. J. Pedley, and R. E. Goldstein, Phys. Rev. Lett. 102, 168101 (2009), arXiv: 0901.2087.
J. O. Kessler, J. Fluid Mech. 173, 191 (1986).
T. J. Pedley, and J. O. Kessler, Proc. R. Soc. Lond. B 231, 47 (1987).
R. N. Bearon, A. L. Hazel, and G. J. Thorn, J. Fluid Mech. 680, 602 (2011).
W. M. Durham, J. O. Kessler, and Stocker. Stocker, Science 323, 1067 (2009).
W. M. Durham, Climent. Climent, and Stocker. Stocker, Phys. Rev. Lett. 106, 238102 (2011).
J. O. Kessler, Nature 313, 218 (1985).
G. J. Thorn, and R. N. Bearon, Phys. Fluids 22, 041902 (2010).
O. A. Croze, Sardina. Sardina, M. Ahmed, M. A. Bees, and Brandt. Brandt, J. R. Soc. Interface 10, 20121041 (2013).
F. De Lillo, Cencini. Cencini, W. M. Durham, Barry. Barry, R. Stocker, Climent. Climent, and Boffetta. Boffetta, Phys. Rev. Lett. 112, 44502 (2014), arXiv: 1310.1270.
F. Santamaria, F. De Lillo, Cencini. Cencini, and Boffetta. Boffetta, Phys. Fluids 26, 111901 (2014), arXiv: 1410.1671.
C. Zhan, Sardina. Sardina, E. Lushi, and Brandt. Brandt, J. Fluid Mech. 739, 22 (2014), arXiv: 1311.3185.
K. Tawada, and Miyamoto. Miyamoto, J. Eukaryot. Microbiol. 20, 289 (1973).
K. L. Poff, and Skokut. Skokut, Proc. Natl. Acad. Sci. 74, 2007 (1977).
J.-O. Kessler, Prog. Phycol. Res. 4, 258 (1986).
X. Garcia, S. Rafaï, and Peyla. Peyla, Phys. Rev. Lett. 110, 138106 (2013), arXiv: 1301.2431.
H. L. Fuchs, E. J. Hunter, E. L. Schmitt, and R. A. Guazzo, J. Exp. Biol. 216, 1458 (2013).
H. L. Fuchs, G. P. Gerbi, E. J. Hunter, A. J. Christman, and F. J. Diez, J. Exp. Biol. 218, 1419 (2015).
A. Sengupta, Carrara. Carrara, and Stocker. Stocker, Nature 543, 555 (2017).
G. Novati, Verma. Verma, D. Alexeev, Rossinelli. Rossinelli, W. M. van Rees, and Koumoutsakos. Koumoutsakos, Bioinspir. Biomim. 12, 036001 (2017), arXiv: 1610.04248.
M. Gazzola, A. A. Tchieu, Alexeev. Alexeev, A. de Brauer, and P. Kou-moutsakos, J. Fluid Mech. 789, 726 (2016), arXiv: 1509.04605.
G. Reddy, Celani. Celani, T. J. Sejnowski, and Vergassola. Vergassola, Proc. Natl. Acad. Sci. 113, E4877 (2016).
S. Colabrese, Gustavsson. Gustavsson, A. Celani, and Biferale. Biferale, Phys. Rev. Lett. 118, 158004 (2017), arXiv: 1701.08848.
S. Colabrese, Gustavsson. Gustavsson, A. Celani, and Biferale. Biferale, Phys. Rev. Fluids 3, 84301 (2018), arXiv: 1711.05853.
K. Gustavsson, Biferale. Biferale, A. Celani, and Colabrese. Colabrese, Eur. Phys. J. E 40, 110 (2017).
N. R. Challabotla, Zhao. Zhao, and H. I. Andersson, Phys. Fluids 27, 061703 (2015).
D. Kamykowski, R.-E. Reed, and G.-J. Kirkpatrick, Mar. Biol. 113, 319 (1992).
W. Lampert, Funct. Ecol. 3, 21 (1989).
H. Yamazaki, and Squires. Squires, Mar. Ecol. Prog. Ser. 144, 299 (1996).
R. Schuech, and S. Menden-Deuer, Limnol. Oceanogr. 4, 1 (2014).
K. A. Miklasz, and M. W. Denny, Limnol. Oceanogr. 55, 2513 (2010).
M. N. Ardekani, Sardina. Sardina, L. Brandt, L. Karp-Boss, R. N. Bearon, and E. A. Variano, J. Fluid Mech. 831, 655 (2017).
J. Qiu, Marchioli. Marchioli, H. I. Andersson, and Zhao. Zhao, Int. J. Multiphas. Flow 118, 173 (2019).
C. Siewert, R. P. J. Kunnen, Meinke. Meinke, and W. Schröder, Atmos. Res. 142, 45 (2014).
C. J. C. H. Watkins, and Dayan. Dayan, Mach. Learn. 8, 279 (1992).
R. S. Sutton, and A. G. Barto, Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning) (MIT Press, Cambridge, 1998).
M. Tan, Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents, Machine Learning Proceedings 1993 (Morgan Kaufmann, San Francisco, 1993), pp. 330–337.
K. Gustavsson, Berglund. Berglund, P. R. Jonsson, and Mehlig. Mehlig, Phys. Rev. Lett. 116, 108104 (2016), arXiv: 1501.02386.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the National Natural Science Foundation of China (Grant Nos. 11911530141, 11772172, and 91752205). JingRan Qiu, WeiXi Huang and LiHao Zhao acknowledge the support from the Institute for Guo Qiang of Tsinghua University (Grant No. 2019GQG1012).
Rights and permissions
About this article
Cite this article
Qiu, J., Huang, W., Xu, C. et al. Swimming strategy of settling elongated micro-swimmers by reinforcement learning. Sci. China Phys. Mech. Astron. 63, 284711 (2020). https://doi.org/10.1007/s11433-019-1502-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11433-019-1502-2