My heart sank a little when I learned of Michael Dummett's death this week. I also learned, however, that he was a follower of the latter-day Wittgenstein credo that "meaning is use." In other words, to understand a word's meaning is precisely to be able to use it correctly, to comprehend its uses. I, too, favor this account of things, and indeed it is pretty commonly assumed in many computational applications. Word senses are often identified with usage patterns in one way or another.
When discussing the idea that "meaning is use" with a colleague a little while ago, however, he was taken aback. "How could any cognitive scientist," he asked, "seriously hold such a view?" His objection seems to be that, cognitively speaking, a word's meaning should be equated with some kind of conceptual space that can be cognized. The problem, I'm told, is that a "usage" is not a cognitively real object.
On the other hand, I think that a cognitive notion of "concept" is actually pretty weak. I'm not acquainted with much empirical literature on this, but it would be worth reading up on how "concepts" have been shown to be cognitively real. And what is the concept behind a word such as and? I'm told that cognitive scientists allow a distinction between function words and content words, wherein function words such as and are tacitly allowed to have "use-based" meanings but content words are supposed to have conceptual meanings. Hmm, so perhaps meaning is use sometimes, even in cognitive science?
Lastly, let's consider a more modern approach like embodied cognition. This, very briefly, is the stance that studying human cognition must be undertaken by recognizing the complete context of the human. Humans seeking to understand words actually learn them by doing, by using them. So even if there is a cognitive "concept" behind a word, this is attained mostly by learning the conventional usage of the word, which is not purely linguistic but interacts with the world as well. It seems that the usage of a word could actually be used to bootstrap the concept behind it.
Thursday, January 5, 2012
Thursday, November 17, 2011
The emergence of syntactic structure
The title of this post alludes to a very deep and interesting paper from Marcus Kracht, which was published in Linguistics and Philosophy (2007). It is sort of embarrassing that it took me this long to notice the paper, but I am not in the loop sometimes, and this is better late than not at all I suppose.
In this paper, Kracht applies his considerable intellect to fundamental problems of compositional semantic representations and the syntax-semantics interface. I dearly love fundamental problems papers, and they are so rare and hard to get published, so this one is really a treat. Kracht begins by defining what a compositional semantics should involve, and more importantly, what it should not involve. It should not involve indices, in the sense that variables in logic can have indices to tell them apart. I think it is more than reasonable from a cognitive perspective, to say that if human semantic representations use "variables" in any way, it is highly unlikely that any sort of named or indexed variables are used. Without this commonplace crutch, Kracht has to design a new kind of semantic representational system. He proposes further that, if this can be done properly, then semantic representations provide motivation for evident syntactic constituents in sentences. Basically, the idea is that the semantics is too weak to get the right meaning without some assistance from syntactic constituency, I think.
In the rest of the paper, Kracht dismantles modern syntactic theory and Montague grammar, and puts the pieces back together in a new and interesting way. As a kind of large aside, he proves an interesting result about Dutch context-freeness (or lack thereof). I can't bring myself to try to summarize the technical details, so just take a look at Kracht's paper if you want to see a very useful alternative perspective on many things that are frequently taken for granted without much worry.
In this paper, Kracht applies his considerable intellect to fundamental problems of compositional semantic representations and the syntax-semantics interface. I dearly love fundamental problems papers, and they are so rare and hard to get published, so this one is really a treat. Kracht begins by defining what a compositional semantics should involve, and more importantly, what it should not involve. It should not involve indices, in the sense that variables in logic can have indices to tell them apart. I think it is more than reasonable from a cognitive perspective, to say that if human semantic representations use "variables" in any way, it is highly unlikely that any sort of named or indexed variables are used. Without this commonplace crutch, Kracht has to design a new kind of semantic representational system. He proposes further that, if this can be done properly, then semantic representations provide motivation for evident syntactic constituents in sentences. Basically, the idea is that the semantics is too weak to get the right meaning without some assistance from syntactic constituency, I think.
In the rest of the paper, Kracht dismantles modern syntactic theory and Montague grammar, and puts the pieces back together in a new and interesting way. As a kind of large aside, he proves an interesting result about Dutch context-freeness (or lack thereof). I can't bring myself to try to summarize the technical details, so just take a look at Kracht's paper if you want to see a very useful alternative perspective on many things that are frequently taken for granted without much worry.
Thursday, October 20, 2011
Quantum semantics
In my previous post, I suggested that syntax researchers put forth too many new theories without good reason. One good reason for a new theory would be to improve the way in which the aspects of language are modeled, and also to improve the interfaces between syntax and other realms. In this connection, some recent work by Mehrnoosh Sadrzadeh, Bob Coeke, Anne Preller, and other collaborators seems quite interesting.
This research program is promoting the "pregroup grammars" framework proposed by Jim Lambek over ten years ago, which has been gaining popularity in mathematical linguistics and appears to have overtaken the momentum of the type-logical framework. In some earlier posts here I suggested that I did not understand the motivation for pregroup grammars and saw no reason to pursue them. Considering syntax per se, I stand by that position. The research program of Sadrzadeh et al., however, is getting me to reconsider.
One of the purported advantages of the type-logical approach over, say, generative syntax, is the simple interface with semantics coded into a typed lambda calculus or its intensional variants such as Montague logic. That said, Montague logic is hardly the perfect system for natural language semantics. A major issue is that word meanings themselves are often just treated as "primitives." As I've explained to bemused linguists during colloquia on occasion, the meaning of a word is represented by the word set in boldface! I've jokingly referred to this as "the boldface theory of lexical semantics."
Now, Sadrzadeh and collaborators take up the mantle of vector space semantics, known mostly from information retrieval, for representing word meanings in a data-driven, usage-based fashion. I am sympathetic to this prospect; it is cognitively plausible, and it is certainly an improvement on the boldface theory.
The real interest of the research program, however, is neither pregroup grammars nor vector space semantics, but the key insight that the two are tightly connected through deep mathematical commonalities. In this way, a pregroup grammar can essentially be provided with a lexicon that has a vector space semantics in a coherently connected way. What is more, you even get a vector space semantics of sentences, which takes the old scheme a step further. The specific mathematical connection between these frameworks is provided by category theory. Pregroups are examples of compact closed categories, as is the category of vector spaces with linear maps and a tensor product (Coecke et al. 2010).
The diagrammatic "calculus" of such categories can be used to simplify meaning computations, and have also been applied to expose the "flow of information" within quantum information protocols (Coecke et al. 2010). Other work by Sadrzadeh has also highlighted the connections with quantum probability logic. This is very interesting stuff which most linguists are sorely unprepared to follow. Theoretical linguistics is currently at risk of falling into irrelevance, while scientists with training in other fields pick up the slack and do the really interesting research without us.
References:
Coecke, B., M. Sadrzadeh and S. Clark "Mathematical foundations for a compositional distributional model of meaning," arXiv.org (2010).
Preller, A. and M. Sadrzadeh "Semantic vector models and functional models for pregroup grammars," J. Logic, Lang. Inf. (2011) 20:419-443.
This research program is promoting the "pregroup grammars" framework proposed by Jim Lambek over ten years ago, which has been gaining popularity in mathematical linguistics and appears to have overtaken the momentum of the type-logical framework. In some earlier posts here I suggested that I did not understand the motivation for pregroup grammars and saw no reason to pursue them. Considering syntax per se, I stand by that position. The research program of Sadrzadeh et al., however, is getting me to reconsider.
One of the purported advantages of the type-logical approach over, say, generative syntax, is the simple interface with semantics coded into a typed lambda calculus or its intensional variants such as Montague logic. That said, Montague logic is hardly the perfect system for natural language semantics. A major issue is that word meanings themselves are often just treated as "primitives." As I've explained to bemused linguists during colloquia on occasion, the meaning of a word is represented by the word set in boldface! I've jokingly referred to this as "the boldface theory of lexical semantics."
Now, Sadrzadeh and collaborators take up the mantle of vector space semantics, known mostly from information retrieval, for representing word meanings in a data-driven, usage-based fashion. I am sympathetic to this prospect; it is cognitively plausible, and it is certainly an improvement on the boldface theory.
The real interest of the research program, however, is neither pregroup grammars nor vector space semantics, but the key insight that the two are tightly connected through deep mathematical commonalities. In this way, a pregroup grammar can essentially be provided with a lexicon that has a vector space semantics in a coherently connected way. What is more, you even get a vector space semantics of sentences, which takes the old scheme a step further. The specific mathematical connection between these frameworks is provided by category theory. Pregroups are examples of compact closed categories, as is the category of vector spaces with linear maps and a tensor product (Coecke et al. 2010).
The diagrammatic "calculus" of such categories can be used to simplify meaning computations, and have also been applied to expose the "flow of information" within quantum information protocols (Coecke et al. 2010). Other work by Sadrzadeh has also highlighted the connections with quantum probability logic. This is very interesting stuff which most linguists are sorely unprepared to follow. Theoretical linguistics is currently at risk of falling into irrelevance, while scientists with training in other fields pick up the slack and do the really interesting research without us.
References:
Coecke, B., M. Sadrzadeh and S. Clark "Mathematical foundations for a compositional distributional model of meaning," arXiv.org (2010).
Preller, A. and M. Sadrzadeh "Semantic vector models and functional models for pregroup grammars," J. Logic, Lang. Inf. (2011) 20:419-443.
Wednesday, September 28, 2011
The hunt for new syntactic theories
It seems like syntacticians, of both mathematical and generative stripes, are constantly hunting for a new and improved syntactic theory. But I'm not really sure what we are supposed to be looking for. Surely now, it is established that any theory capable of generating languages of sufficient complexity is capable of "capturing the data" or describing it or whatever. Yet syntacticians still publish papers where they demonstrate that some new notion or theory is capable of deriving some fancy piece of data that somehow eludes the others. Is this really what they are doing? Because this seems like a moot argument.
It seems to me that, if there is to be any rationale for improving syntactic theory, it should be something like cognitive plausibility or computational effectiveness, or perhaps even theoretical elegance. I don't know what people are driving at, because the desiderata of a better syntactic theory are almost never discussed. Should we all know what we are seeking? Because I'm not so sure anymore, and thus I am not convinced we should continue to hunt around. For the moment I'm satisfied that type-logical grammar is capable of deriving everything that needs to be derived. Am I wrong?
It seems to me that, if there is to be any rationale for improving syntactic theory, it should be something like cognitive plausibility or computational effectiveness, or perhaps even theoretical elegance. I don't know what people are driving at, because the desiderata of a better syntactic theory are almost never discussed. Should we all know what we are seeking? Because I'm not so sure anymore, and thus I am not convinced we should continue to hunt around. For the moment I'm satisfied that type-logical grammar is capable of deriving everything that needs to be derived. Am I wrong?
Wednesday, August 31, 2011
Analog computation
This post is a quick precis of something that I hope works out a little longer. There has been, over the years, some amount of research on the complexity theory of analog computation (i.e., using analog devices with no quantization or digitization, which operate in continuous time over the real numbers). There is not, as yet, a good and complete theory of this, but some key recent papers that a person could start with are found in the book New Computational Paradigms (Springer 2008).
One question that continues to be debated is whether analog algorithms have the same constraints or lower bounds on complexity as digital algorithms computing the same results. In the most extreme view, it has been suggested that analog methods could be used to solve NP-hard problems quickly, perhaps in polynomial time. Some papers have analyzed this question, and the results so far appear to be negative. For example, a paper posted online by Warren Smith of NEC Corp. (1998) showed that in order for the above suggestion to hold of a particular kind of physical computer (a "plane mechanism"), some other implausible things would have to be the case as well.
OK, so maybe we cannot go all the way from NP-hard to P. But this doesn't in any sense prove that the lower complexity bounds are exactly equal. There are plenty of digital computations which, while not NP-hard, are still crappy in practice because they involve some large power of the input. What good is polynomial time when the algorithm's average case complexity is O(n^82) for instance?
Why is all this important? Well, cognitive science entertains a wide array of computational models, which are usually being proposed as "cognitively plausible" in some vague sense. So now there are debates about how tractable should a simulation need to be in order to seem cognitively plausible. But, all the simulations are digital. Meanwhile, the brain is not digital, it is some sort of analog system. I think that debaters need to take care over this mismatch, because there is no clear correspondence between the complexity of a digital computation and the complexity of an analog computation accomplishing the same task.
One question that continues to be debated is whether analog algorithms have the same constraints or lower bounds on complexity as digital algorithms computing the same results. In the most extreme view, it has been suggested that analog methods could be used to solve NP-hard problems quickly, perhaps in polynomial time. Some papers have analyzed this question, and the results so far appear to be negative. For example, a paper posted online by Warren Smith of NEC Corp. (1998) showed that in order for the above suggestion to hold of a particular kind of physical computer (a "plane mechanism"), some other implausible things would have to be the case as well.
OK, so maybe we cannot go all the way from NP-hard to P. But this doesn't in any sense prove that the lower complexity bounds are exactly equal. There are plenty of digital computations which, while not NP-hard, are still crappy in practice because they involve some large power of the input. What good is polynomial time when the algorithm's average case complexity is O(n^82) for instance?
Why is all this important? Well, cognitive science entertains a wide array of computational models, which are usually being proposed as "cognitively plausible" in some vague sense. So now there are debates about how tractable should a simulation need to be in order to seem cognitively plausible. But, all the simulations are digital. Meanwhile, the brain is not digital, it is some sort of analog system. I think that debaters need to take care over this mismatch, because there is no clear correspondence between the complexity of a digital computation and the complexity of an analog computation accomplishing the same task.
Wednesday, July 27, 2011
Neuroelectrodynamics
One of the million or so things that interest me is the question of how the brain actually accomplishes anything. This is relevant to linguistics because, well, that's sort of obvious. A prevalent model of neural computation tells us that the brain computes by passing information among various neurons, and that these neurons encode messages in sequences of action potential 'spikes' by a mechanism known as "spike timing."
A new book by Aur and Jog challenges this whole paradigm. It carries the strangely ungrammatical title Neuroelectrodynamics: Understanding the Brain Language and the grammar between the covers is no improvement, but it is very provocative and enticing to those of us who think that the current state of understanding in neuroscience is extremely poor.
The authors first demonstrate that neuron 'spikes' are not uniform or stereotypical, which itself goes against a main tenet of the spike timing model. They then outline a new scheme by which a new quantity of spike 'directivity' is encoded into the charges in movement during an action potential. Empirically, the details of dendritic arbors and axonal branches significantly modulate the extracellular action potential. Axons themselves cannot be approximated by linear cable models.
In the authors' charge movement model, different charges or groups under an electric field have distinct movements. To apply independent component analysis (ICA), the action potential is assumed to be the result of several independent sources, generated by charges that move. For recorded action potentials, blind source separation (a known signal processing technique) can be performed using ICA. The charge localization is obtained from triangulation and a point charge approximation. Singular value decomposition is performed for the matrix of charge coordinates. A 'spike directivity' can be described by a preferred direction of propagation of the electric signal during each action potential, and approximated with a vector. This is shown to be a much more effective means of encoding information and computation than spike timing. Details of the spike directivity are said to be published in other papers by the authors, which did appear in major neuroscience journals.
It is interesting to see how the current paradigm in neural computation could well be founded on nothing. The authors' ideas may not be proven, but they certainly have some interesting empirical findings which defy explanation otherwise, and it seems clear to me that the brain can't be doing all its work with spike timing.
A new book by Aur and Jog challenges this whole paradigm. It carries the strangely ungrammatical title Neuroelectrodynamics: Understanding the Brain Language and the grammar between the covers is no improvement, but it is very provocative and enticing to those of us who think that the current state of understanding in neuroscience is extremely poor.
The authors first demonstrate that neuron 'spikes' are not uniform or stereotypical, which itself goes against a main tenet of the spike timing model. They then outline a new scheme by which a new quantity of spike 'directivity' is encoded into the charges in movement during an action potential. Empirically, the details of dendritic arbors and axonal branches significantly modulate the extracellular action potential. Axons themselves cannot be approximated by linear cable models.
In the authors' charge movement model, different charges or groups under an electric field have distinct movements. To apply independent component analysis (ICA), the action potential is assumed to be the result of several independent sources, generated by charges that move. For recorded action potentials, blind source separation (a known signal processing technique) can be performed using ICA. The charge localization is obtained from triangulation and a point charge approximation. Singular value decomposition is performed for the matrix of charge coordinates. A 'spike directivity' can be described by a preferred direction of propagation of the electric signal during each action potential, and approximated with a vector. This is shown to be a much more effective means of encoding information and computation than spike timing. Details of the spike directivity are said to be published in other papers by the authors, which did appear in major neuroscience journals.
It is interesting to see how the current paradigm in neural computation could well be founded on nothing. The authors' ideas may not be proven, but they certainly have some interesting empirical findings which defy explanation otherwise, and it seems clear to me that the brain can't be doing all its work with spike timing.
Wednesday, July 20, 2011
Road to Reality
This summer I've been reading one of those few books that can change your life, The Road to Reality by Roger Penrose. This is basically all of modern theoretical physics, including the mathematical fundamentals starting from first principles. I have finally gotten through all the initial mathematical chapters, which include a quick rundown of Riemannian geometry and differential forms on manifolds. Why is it that mathematical physicists make the best math teachers? I also changed my life a few years back by reading Lectures on Mathematical Physics by Robert Geroch.
I suspect that this approach to geometry could be useful for a new approach to semantics and cognitive science (see previous post on the musings of Fenstad).
Geometry is so crucial to our understanding of reality (i.e. physics), it should not be surprising if it turns out to be crucial to our understanding of language and cognition as well.
I suspect that this approach to geometry could be useful for a new approach to semantics and cognitive science (see previous post on the musings of Fenstad).
Geometry is so crucial to our understanding of reality (i.e. physics), it should not be surprising if it turns out to be crucial to our understanding of language and cognition as well.
Subscribe to:
Posts (Atom)
