Yushi Yang is a 4th year PhD student studying soft matter physics in the university of Bristol, under the supervision of Thomas Machon and Chrissy Hammond, and blessed with the friendship with Paddy Royall, John Russo and Francesco Turci. Yushi works on the collective behaviour of zebrafish, including the experimental observation, the statistical analysis, as well as the relevant computer simulation.
Here, you can find his experience from participating in the GEMSTONE Virtual Mobility project:
“I started a two weeks short research on 7th October 2021, with the help of virtual mobile (VM) grant. It’s about using deep learning to capture the 3D behaviour of zebrafish, to better understand their musculoskeletal development. It is a very interdisciplinary project. Let me prove the claim by enumerating my collaborators: Erika Kague, a biologist in university of Bristol, and Kathleen Curran, a computer scientists from the University College Dublin. I am a soft matter physicist in the university of Bristol. The virtual collaboration worked very well for us.
The actual work is building a deep neural network model to find the locations of a school of fish in a video. The end result can be impressive: we will be able to follow the 3D movement a group of fish. This technology was used to study the starlings in Rome, leading to one of the best papers in physics, with a beautiful name of “Interaction ruling animal collective behavior depends on topological rather than metric distance: Evidence from a field study.
If you actually searched the aforementioned paper on the Internet, and managed to bypass the scientific paywall, I wanted to draw a little bit of your attention to the 9th author, who was awarded with the Nobel prices of physics in the year of 2021, “for the discovery of the interplay of disorder and fluctuations in physical systems from atomic to planetary scales.” Giorgio Parisi’s discovery is very true, and the relevant physics is also suitable for the animals. In fact, there are a few physicists working on observing animals, including me. These leisurely research can be practically useful: together with Erika Kague, we find the behaviour of some mutant fish changed significantly, as a result of their compromised musculoskeletal development.
I wanted to do everything even faster: my current software takes about 2 days to process a 60 minutes video. Ambitiously, I wanted it to be so fast that the data processing can be carried out in real time. If this goal is achieved, we will have the (very likely) first realtime 3D tracking system for aquarium animals. Being “the first” would be very cool. And I plan to achieve it with the help of deep learning, by constructing a neural network that can carry out numerical calculations efficiently on a graphics processing unit (GPU).
My research was not affected terribly by the Covid-19 pandemic. It’s largely sitting down to a computer and crunching the keyboard. Kathleen provided some nice suggestions along the way, about the basics of machine learning. And Erika always provides insights into the algorithms and results. I also made friends with Kathleen’s student, Adil and Katie. I might actually travel to Dublin and have a pint of beer with them. The VM grant money very nicely helped (with my work): I bought some computing environment from google, and a lot of cloud storage spaces for my training data. They speeded up the my workflow a lot.
After 2 weeks of very intense keyboard-crunching, I got a neural network model, and it does the job. Given an image, the model gives me a probability map indicating the chance for every pixel to contain the centre of a fish. The calculation with the neural network is extremely fast, processing 50 images per second. This brings me one step closer to a realtime 3D tracking system. I am very grateful about the chance that the GEMSTONE provides, and I hope more students to be helped, and more collaborations to be nourished in the future.
