Behavioral teleporting helps study social interactions in real time

Teleporting is a science fiction trope often associated with Star Trek. But a different kind of teleporting is being explored at the NYU Tandon School of Engineering, one that could let researchers investigate the very basis of social behavior, study interactions between invasive and native species to preserve natural ecosystems, explore predator/prey relationship without posing a risk to the welfare of the animals, and even fine-tune human/robot interfaces.

The team, led by Maurizio Porfiri, Institute Professor at NYU Tandon, devised a novel approach to getting physically separated fish to interact with each other, leading to insights about what kinds of cues influence social behavior.

The innovative system, called "behavioral teleporting" -- the transfer of the complete inventory of behaviors and actions (ethogram) of a live zebrafish onto a remotely located robotic replica -- allowed the investigators to independently manipulate multiple factors underpinning social interactions in real-time.

The research, "Behavioral teleporting of individual ethograms onto inanimate robots: experiments on social interactions in live zebrafish," appears in the Cell Press journal iScience.

The team, including Mert Karakaya, a Ph.D. candidate in the Department of Mechanical and Aerospace Engineering at NYU Tandon, and Simone Macrì of the Centre for Behavioral Sciences and Mental Health, Istituto Superiore di Sanità, Rome, devised a setup consisting of two separate tanks, each containing one fish and one robotic replica.

Within each tank, the live fish of the pair swam with the zebrafish replica matching the morphology and locomotory pattern of the live fish located in the other tank.

An automated tracking system scored each of the live subjects' locomotory patterns, which were, in turn, used to control the robotic replica swimming in the other tank via an external manipulator.

Therefore, the system allowed the transfer of the complete ethogram of each fish across tanks within a fraction of a second, establishing a complex robotics-mediated interaction between two remotely-located live animals.

By independently controlling the morphology of these robots, the team explored the link between appearance and movements in social behavior.

The investigators found that the replica teleported the fish motion in almost all trials (85% of the total experimental time), with a 95% accuracy at a maximum time lag of less than two-tenths of a second. The high accuracy in the replication of fish trajectory was confirmed by equivalent analysis on speed, turn rate, and acceleration.

Porfiri explained that the behavioral teleporting system avoids the limits of typical modeling using robots.

Since existing approaches involve the use of a mathematical representation of social behavior for controlling the movements of the replica, they often lead to unnatural behavioral responses of live animals."

Maurizio Porfiri, Institute Professor, New York University Tandon

"But because behavioral teleporting 'copy/pastes' the behavior of a live fish onto robotic proxies, it confers a high degree of precision with respect to such factors as position, speed, turn rate, and acceleration."

Porfiri's previous research proving robots are viable as behavior models for zebrafish showed that schools of zebrafish could be made to follow the lead of their robotic counterparts.

"In humans, social behavior unfolds in actions, habits, and practices that ultimately define our individual life and our society," added Macrì.

"These depend on complex processes, mediated by individual traits --baldness, height, voice pitch, and outfit, for example -- and behavioral feedback, vectors that are often difficult to isolate. This new approach demonstrates that we can isolate influences on the quality of social interaction and determine which visual features really matter."

The research included experiments to understand the asymmetric relationship between large and small fish and identify leader/follower roles, in which a large fish swam with a small replica that mirrored the behavior of the small fish positioned in the other tank and vice-versa.

Karakaya said the team was surprised to find that the smaller -- not larger -- fish "led" the interactions.

"There are no strongly conclusive results on why that could be, but one reason might be due to the 'curious' nature of the smaller individuals to explore a novel space," he said. "In known environments, large fish tend to lead; however, in new environments larger and older animals can be cautious in their approach, whereas the smaller and younger ones could be 'bolder.'"

The method also led to the discovery that interaction between fish was not determined by locomotor patterns alone, but also by appearance.

"It is interesting to see that, as is the case with our own species, there is a relationship between appearance and social interaction," he added.

Karakaya added that this could serve as an important tool for human interactions in the near future, whereby, through the closed-loop teleporting, people could use robots as proxies of themselves.

"One example would be the colonies on Mars, where experts from Earth could use humanoid robots as an extension of themselves to interact with the environment and people there. This would provide easier and more accurate medical examination, improve human contact, and reduce isolation. Detailed studies on the behavioral and psychological effects of these proxies must be completed to better understand how these techniques can be implemented into daily life."