Literature Review and Inspiration

The proposed research project draws inspiration from two key papers in the field of neural engineering.

The first paper, by Kagan et al. (2022), demonstrated the connection of in vitro neurons generated from pluripotent stem cells to a simulated game environment, specifically the game of Pong, through a feedback loop.

The second paper, by Serb et al. (2020), showcased the connection of a biological neuron with an artificial neuron using a Spiking Neural Network.

Both papers present an opportunity to explore the integration of biological and artificial neurons and systems, paving the way for the development of hybrid neural systems with enhanced computational capabilities.

These papers, among others in the field, serve as the main sources of inspiration for the proposed research project.

Research Question

How does the performance of cultured biological neural networks (BNN) in a simulated video game depend on the presence or absence of a Spiking Neural Network (SNN)?

Preliminary Pilot Studies

Before embarking on the full-scale experiment, we intend to conduct small-scale preliminary pilot studies with an exploratory objective.

Given that the relationship between biological neural networks, simulated game environments, and artificial neural networks is a novel field of research, there is no well-established body of literature that we can rely on to anticipate potential effects or outcomes.

Consequently, conducting preliminary pilot studies is essential in this instance, as it would provide us with greater insight into whether the presence or absence of a Spiking Neural Network would have a consistent and sizable impact on the performance of the biological neural network in the simulated game environment. This would enable us to determine if the effects are large enough to be detectable using our experimental design.

Furthermore, pilot studies would be essential to assess the feasibility, duration, cost, adverse events, and improve upon the study design prior to the performance of our full-scale research project

In case the pilot studies yield promising results we could start to proceed with the actual experiment after careful planning and discussions. This would allow us to eliminate confounding factors that might influence the observed effects.

Operationalization

As mentioned earlier, the primary objective of this research is to examine the extent to which cultured biological neural networks (BNN) rely on a Spiking Neural Network (SNN) in their performance in a simulated game environment, while the flow of communication between the BNN and SNN is manipulated.

To measure the extent of this dependence, we would monitor changes in the performance of the BNN in the simulated game environment, while altering the connection between the SNN and the BNN under different conditions. To ensure that any observed differences in performance are due to the reliance on the SNN and not other factors, such as familiarity of the BNN with the SNN or order effects, we would use a rigorous experimental design with multiple stages and factors.

To investigate whether the observed effects are due to the BNN's familiarity or prior training with the SNN, we can implement two initial training conditions. In the first condition, the BNN would be trained in the video game environment with an active feedback loop between the BNN and SNN, while in the second condition, the BNN would be trained in the same environment without the SNN.

After the initial training phase, we would move on to the testing stage, which would have two phases for each condition: a start or baseline phase and a subsequent phase during the experiment. In these phases, the SNN could either be ON, meaning that an active feedback loop for communication is enabled between the SNN and BNN, as the BNN performs in the video game environment, or OFF, meaning that this feedback loop is disabled between the SNN and BNN during this time. This would result in four possible conditions for each initial training condition, leading to a total of eight conditions in the testing stage.

The purpose of these eight conditions is to eliminate any potential order effects and explore as many scenarios as possible to gain insight into the relationship between the BNN's performance in the video game and the manipulation of communication between the BNN and SNN.

Experimental Conditions & Hypotheses

When the biological neurons are trained or not trained with the SNN and the SNN is turned ON throughout the whole experiment, their performance in the simulated video game will not decrease compared to the baseline (Condition 1 and 5).

When the biological neurons are trained with the SNN and the SNN is turned ON at the start of the experiment, their performance in the simulated video game will decrease when the SNN is turned OFF compared to the baseline (Condition 2).

When the biological neurons are trained with the SNN and the SNN is turned OFF at the start of the experiment, their performance in the simulated video game will increase when the SNN is turned ON compared to the baseline (Condition 3).

When the biological neurons are trained with the SNN and the SNN is turned OFF throughout the whole experiment, their performance in the simulated video game will not increase compared to the baseline (Condition 4).

When the biological neurons are NOT trained with the SNN and the SNN is turned ON at the start of the experiment, their performance in the simulated video game will decrease when the SNN is turned OFF compared to the baseline (Condition 6).

When the biological neurons are NOT trained with the SNN and the SNN is turned OFF at the start of the experiment, their performance in the simulated video game will decrease when the SNN is turned ON compared to the baseline (Condition 7).

When the biological neurons are NOT trained with the SNN and the SNN is turned OFF throughout the whole experiment, their performance in the simulated video game will not increase compared to the baseline (Condition 8).

When the biological neurons are trained with SNN and the SNN is ON throughout the whole experiment (Condition 1), their performance is higher than the performance of biological neurons in all other conditions (Conditions 2-8).

When the biological neurons are trained with SNN and the SNN is ON throughout the whole experiment (Condition 1), their performance is higher than the performance of all other conditions where biological neurons were trained with SNN.

When the biological neurons are NOT trained with SNN and the SNN is ON throughout the whole experiment (Condition 5), their performance is higher than the performance of the biological neurons in Conditions 6 and 7.

Overall, biological neurons trained with SNN will perform better than biological neurons NOT trained with SNN.

Experimental Design

Spiking Neural Networks

Spiking neural networks (SNNs) can learn and adapt to their environment by using spike-timing-dependent plasticity (STDP), an unsupervised learning algorithm that strengthens or weakens synaptic connections based on the temporal relationships between presynaptic and postsynaptic spikes.

Upon being presented with a novel stimulus, an SNN formulates a prediction regarding its nature. In the event that this prediction aligns with the SNN's internal representation, no alterations to its connections are required. Nevertheless, if the prediction proves to be inaccurate, the SNN's free energy increases, prompting it to modify its connections in an effort to minimize this energy.

Our proposition posits that spiking neural networks (SNNs) acquire data representations in accordance with the free energy principle (FEP), which asserts that all physical systems strive to minimize their free energy (Friston et al., 2006). Within the realm of SNNs, this implies that these networks endeavor to learn data representations that have the lowest possible free energy. Since there exist various feasible representations of the same data, we can guide an SNN by introducing random noise when it fails to learn a suitable representation. This will elevate the free energy of the SNN, causing it to explore alternative representations and ultimately settle on a new representation that aligns with the desired behaviour.

Future Applications and Benefits

Improved Understanding of the Integration of Biological and Artificial Neural Networks

Advanced Spiking Neural Networks

Mapping the Neuronal Mechanisms of Adaptation and Learning

Investigating the Effects of Spiking Neural Networks on Cell Cultures with Pathology

Developing New Therapeutic Approaches and Enhancing Neural Prosthetics

Application in Brain-Computer Interfaces and AR

References

Alemi, A., Machens, C. K., Denève, S., & Slotine, J. E. (2018). Learning Nonlinear Dynamics in Efficient, Balanced Spiking Networks Using Local Plasticity Rules. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). https://doi.org/10.1609/aaai.v32i1.11320

Brendel, W., Bourdoukan, R., Vertechi, P., Machens, C. K., & Denève, S. (2020). Learning to represent signals spike by spike. PLOS Computational Biology, 16(3), e1007692. https://doi.org/10.1371/journal.pcbi.1007692

Cho, M. W. (2019). Simulations in a Spiking Neural Network Model Based on the Free Energy Principle. Journal of the Korean Physical Society, 75(3), 261–270. https://doi.org/10.3938/jkps.75.261

Fernandez-Leon, J. A., Arlego, M., & Acosta, G. G. (2022). Is Free Energy an Organizational Principle in Spiking Neural Networks? In From Animals to Animats 16: 16th International Conference on Simulation of Adaptive Behavior, SAB 2022, Cergy-Pontoise, France, September 20–23, 2022, Proceedings (pp. 79–90). Springer. https://doi.org/10.1007/978-3-031-16770-6_7

Friston, K. J., Kilner, J. M., & Harrison, L. H. (2006). A free energy principle for the brain. Journal of Physiology-Paris, 100(1–3), 70–87. https://doi.org/10.1016/j.jphysparis.2006.10.001

Isomura, T., Sakai, K., Kotani, K., & Jimbo, Y. (2016). Linking Neuromodulated Spike-Timing Dependent Plasticity with the Free-Energy Principle. Neural Computation, 28(9), 1859–1888. https://doi.org/10.1162/NECO_a_00862

Kagan, B. J., Kitchen, A. C., Tran, N. T., Habibollahi, F., Khajehnejad, M., Parker, B. J., Bhat, A., Rollo, B., Razi, A., & Friston, K. J. (2022). In vitro neurons learn and exhibit sentience when embodied in a simulated game-world. Neuron, 110(23), 3952-3969.e8. https://doi.org/10.1016/j.neuron.2022.09.001

Lobov, S. A., Mikhaylov, A. N., Shamshin, M., Makarov, V. A., & Kazantsev, V. B. (2020). Spatial Properties of STDP in a Self-Learning Spiking Neural Network Enable Controlling a Mobile Robot. Frontiers in Neuroscience, 14. https://www.frontiersin.org/articles/10.3389/fnins.2020.00088

Nicola, W., & Clopath, C. (2016). Supervised learning in spiking neural networks with FORCE training. Nature Communications, 8(1). https://doi.org/10.1038/s41467-017-01827-3

Serb, A., Corna, A., George, R., Khiat, A., Rocchi, F., Reato, M., Maschietto, M., Mayr, C., Indiveri, G., Vassanelli, S., & Prodromakis, T. (2020). Memristive synapses connect brain and silicon spiking neurons. Scientific Reports, 10(1), Article 1. https://doi.org/10.1038/s41598-020-58831-9

Thalmeier, D., Uhlmann, M., Kappen, H. J., & Memmesheimer, R. (2016). Learning Universal Computations with Spikes. PLOS Computational Biology, 12(6), e1004895. https://doi.org/10.1371/journal.pcbi.1004895

Zhang, T., Jia, S., Cheng, X., & Xu, B. (2021). Tuning Convolutional Spiking Neural Network With Biologically Plausible Reward Propagation. IEEE Transactions on Neural Networks and Learning Systems, 33(12), 7621–7631. https://doi.org/10.1109/tnnls.2021.3085966

WHERE BIOLOGICAL AND ARTIFICIAL NEURAL NETWORKS MEET