Biologically Plausible Learning of Text Representation with Spiking Neural Networks

Marcin Białas, Marcin Mirończuk, Jacek Mańdziuk

2020 W: Parallel Problem Solving from Nature – PPSN XVI / Thomas Bäck, Mike Preuss, André Deutz, Hao Wang, Carola Doerr, Michael Emmerich, Heike Trautmann; Cham: Springer, s. 433-447

International Conference on Parallel Problem Solving from Nature. Leiden, 2020-09-05 - 2020-09-09

This study proposes a novel biologically plausible mechanism for generating low-dimensional spike-based text representation. First, we demonstrate how to transform documents into series of spikes (spike trains) which are subsequently used as input in the training process of a spiking neural network (SNN). The network is composed of biologically plausible elements, and trained according to the unsupervised Hebbian learning rule, Spike-Timing-Dependent Plasticity (STDP). After training, the SNN can be used to generate low-dimensional spike-based text representation suitable for text/document classification. Empirical results demonstrate that the generated text representation may be effectively used in text classification leading to an accuracy of 80.19% on the bydate version of the 20 newsgroups data set, which is a leading result amongst approaches that rely on low-dimensional text representations.