Camera calibration and reconstruction for fish experiments

The last few months I have been working hard on the sophisticated new experimental set-ups in the lab with which we will be able to get high spatial and temporal resolution tracking of large schools of fish, in tanks that are up to 3x3m in size!

To get highly accurate spatial data of the fish we need to correct for the distortion of the camera lens, which almost all lenses have to some extent. I just finished the script (in Python) that enables us to undistort the image from a camera using functions in opencv based on a video of a moving checkerboard.

Me calibrating a camera with a checkerboard pattern, with colours showing the output of my python script, with a school of 1000 moderlieschen in the background :)

Me calibrating a camera with a checkerboard pattern, with colours showing the output of my python script, with a school of 1000 moderlieschen in the background :)

It works pretty well already, even with non-optimal videos. Next step will be to stitch the videos of multiple linked camera’s.

Short visual on vectorized movements of small fish school

My research is currently centred around understanding the role of consistent behavioural differences in the collective movements and functioning of animal groups. In particular, I assay large numbers of stickleback fish on various personality traits and expose them in groups to different ecological scenario’s. I have written custom tracking software in Python using the OpenCV library to be able to accurately track the position of individual fish in the freely-moving schools.

Today I wanted to share a simple visual that highlights the detailed individual-based tracking of a small fish school over time. Each fish is represented by a different colour, with the arrow showing its vectorized movement, with larger arrows indicating a higher speed. The video is centred around the vector of the group as a whole to better visualize the structure of the group over time. Lines indicate the smallest polygon encompassing all individuals and Individual Centre Distances. The moving axes indicate the relative speed of the group in a large circular arena.

In this short section of a 30-min long experimental trial it is clear that the group speed, cohesion, and structure fluctuate over time. At the same time, individuals also maintain to some extent their positions relative to the group centre, such as the green and yellow individual clearly having a stronger pulling power on the movements of the group as a whole.

I used RaspberryPi computers to film the fish, custom Python tracking scripts to acquire individual X,Y coordinates for each individual in the group, R to process the tracking data and acquire movement characteristics, and R with ffmpeg to create the visual.