Permutations as a means to encode order in word space

Sahlgren, Magnus and Holst, Anders and Kanerva, Pentti (2008) Permutations as a means to encode order in word space. In: The 30th Annual Meeting of the Cognitive Science Society (CogSci'08), 23-26 July 2008, Washington D.C., USA.



We show that sequence information can be encoded into high-dimensional fixed-width vectors using permutations of coordinates. Computational models of language often represent words with high-dimensional semantic vectors compiled from word-use statistics. A word's semantic vector usually encodes the contexts in which the word appears in a large body of text but ignores word order. However, word order often signals a word's grammatical role in a sentence and thus tells of the word's meaning. Jones and Mewhort (2007) show that word order can be included in the semantic vectors using holographic reduced representation and convolution. We show here that the order information can be captured also by permuting of vector coordinates, thus providing a general and computationally light alternative to convolution.

Item Type:Conference or Workshop Item (Paper)
ID Code:3436
Deposited By:Magnus Sahlgren
Deposited On:03 Feb 2009
Last Modified:18 Nov 2009 16:22

Repository Staff Only: item control page