Spike Statistics: A Tutorial

Transcription

1 Spike Statistics: A Tutorial File: spike statistics4.tex JV Stone, Psychology Department, Sheffield University, England. j.v.stone@sheffield.ac.uk December 10, Introduction Why do we need to know about the stastistical properties of spikes? Because there is only a loose coupling between stimuli that appear on the retina (for example) and the outputs of visual neurons. This means that the output of a neuron does not tell us with certainty exactly caused it to produce that output. If you (or your brain) wants to use the outputs of neurons to infer which world events caused those neurons to fire then you need to understand the nature of this coupling. Whether we consider neuronal outputs in terms of firing rates or indiviudal spikes, understanding this coupling requires that we know about the stastistical properties of spikes. Figure 1: Tuning curve of a cell sensitive to motion direction s. The preferred direction of this cell is zero degrees (which could be straight up, for example). The mean response (firing rate) at a given stimulus direction s is given by the height r = f(s) of the tuning curve f(s) at s. However, the response r varies from trial to trial. The distribution p(r s) of firing rates for a given direction of s = 75 degrees is shown to the right of the tuning curve. 1

2 Figure 2: Left: Repeated presentations (trials) of the same stimulus moving through the receptive field (top) of a cell with preferred direction of zero degrees evokes a different firing rate r on each trial (bottom), with a mean of r = 5Hz or spikes/second. Right: If the same stimulus is presented many thousands of times then a histogram of responses can be constructed. Note that this is the same as the small rotated histogram shown in Figure (1), which shows the distribution of responses for a given stimulus value s. This distribution is a close approximation to the probability density function (pdf) p(r s) shown as a black curve, which is the probability (density) of observing the firing rate r given the stimulus value s. In this chapter we will show how the individual spikes are analagous to coin flips, and can be described by the same machinery used to count coin flips. This leads to an account of spikes in terms of the binomial distribution, which can be approximated by the Poisson distribution, which (in turn) can be approximated by the Gaussian distribution. These approximations are fairly general, and make analysis of neuronal outputs reasonably straightforward. 2 What s the problem? Given a neuron which has a preferred direction, every time the same moving line is presented to the retina it evokes a slightly different firing rate, as shown in Figures (1) and (2). So, if we want to use the firing rate to infer what the line direction is then we need to know how probable each direction is given that firing rate. In other words, we need to know the conditional probability distribution p(s r), where s is the stimulus direction and r is the firing rate evoked by s. Using Bayes rule, we know that p(s r) = p(r s)p(s), (1) p(r) where p(r s) is the likelihood function (the probability of r given s), p(s) is the prior probability distrubution of s, and p(r) is a constant that we can safely ignore. The term 2

3 Figure 3: A series of action potentials or spikes can be represented as a sequence of zeros and ones. If the time period T contains n spikes then the firing rate is r = n/t. If T is divided into M bins of width t, and if these bins are sufficiently small that only one spike can appear in each bin, then the probability of observing a spike in each bin is b = r t. If each bin is considered as the flip of a coin with bias b (where b is the probability of obtaining a head) then the probability of observing n spikes in T seconds is the same as the probability of observing n heads in M coin flips. p(s) depends only on the environment, and describes the relative probability of different direction values s, and we shall leave this to one side for now. This leaves the term p(r s), which is the probability of observing a firing rate r given that the stimulus direction is s. Clearly, in order to evaluate p(s r), we need to know p(r s). Rather than considering p(r s) in terms of firing rate, it will be easier for now if we consider the number n of spikes in a fixed period T, where the firing rate can be recovere at any point from r = n/t. So, we proceed by asking the apparently simple question: what is the probability of observing n spikes in a period T? 2.1 Spikes, bins and coin flips The answer can be obtained if we split the time period T into a large number M of very narrow bins of width t, as shown in Figure (3). The bins need to be narrow to ensure that no more than one spike can appear in each bin. This, in turn, ensures that the probability b of observing a spike in each bin is b = r t, where r is the firing rate r measured over time T. Our question now becomes: if we have M bins and the probability of observing a spike in each bin is b, then what is the probability that n bins contain a spike? We can begin to answer this question if we treat each bin as if it represents the flip of a coin. For example, if we have 1000 bins then we would expect half of them to contain a head (spike) if the coin is fair. Note that a fair coin has a bias of b = 0.5; that is, the probability of a head is b = 0.5. Additionally, note that if neurons were coins then they would have a bias b much less than 0.5, where the precise value of b depends on how narrow we choose to make our bins. We will return to this topic, but for now the precise value of b does not matter. 3

4 What does matter is that we effectively get to flip the coin M times. For each sequence of M flips, we obtain an M-sequence of heads and tails, and what we would like to know is the proportion of M-sequences that contain exactly n heads (and, by implication, M n tails). In other words, what is the probability p(n) that a randomly chosen M-sequence contains exactly n heads? Nomenclature: There is much potential for confusion in discussing the probability of obtaining a number n of heads given M coin flips. On the one hand, we could mean, the probability of observing exactly n heads in a single sequence of M coin flips. On the other hand, we could mean, the probability of observing exactly n heads in a randomly chosen sequence of M coin flips, where this choice is made from amongst all possible sequences. This latter interpretation could be put more succinctly as, the proportion of M-sequences that contain exactly n heads. Where the intended meaning is unclear, we will use somewhat elaborate sentence constructions to clarify matters. If the outcome of successive coin flips are independent then the probability of observing n heads in any one M-sequence is p h = b b... (2) n = b (3) = b n. (4) Given that b is the probability of observing a head, it follows that the probability of observing a tail is a = (1 b). Therefore, if there are M coin flips, and n of these yield heads, then it follows that there must be (M n) tails. The probability of observing (M n) tails in any one M-sequence is p t = a a... (5) = M n a (6) = a M n. (7) Putting these together implies that the probability that M coin flips yields exactly n heads and (M n) tails with any one sequence is p h p t = b n a M n. (8) This is true of each M-sequence. But, there are many M-sequences that contain exactly n heads and (M n) tails, and they all occur with the same probability b n a M n. 3 The Binomial Distribution So far, all we know is the probability that each M-sequence has exactly n heads. Recall that what we want is the probability p(n) that any randomly chosen M-sequence contains exactly n heads, or equivalently, the proportion p(n) of M-sequences that contain exactly n heads. 4

5 Figure 4: Left: Examples of the binomial distribution. Each distribution is constructed by counting the number of heads for each of 1000 samples. Each sample consisted of M = 20 trials of a coin, so there could be no more than 20 heads, as indicated on the horizontal axis. The number of heads in each of 1000 such samples was counted, and a histogram of these 1000 count values was plotted for a coin with bias b = 0.1 (top). This was repeated for coins with bias b = 0.2 (middle), and b = 0.6 (bottom). Right: Examples of the Poisson distribution. If M is large and b is small (e.g. if t is small) then the binomial distribution is approximated by the Poisson distribution. Notice that the mean number of heads bm for the binomial has the same value as λ for the Poisson distribution. We can find this by noting that if the number of M-sequences that contain n heads is C M,n, and if all such sequences occur with the same probability (which they do), then the proportion p(n) of M-sequences that contain exactly n heads (i.e. has n heads and (M n) tails) is just p(n) = C M,n b n a M n. (9) That s all very fine, but what is C M,n? This number is known as a binomial coefficient, and is the standard notation for counting how many ways n items can be arranged amongst M positions, or bins 1. For example, if we wanted to choose n = 5 children from M = 10 then there are a total of C M,n = 252 possible unordered sets of 5 different children that we could end up with. Why 252? Well, this number is obtained from the standard formula for a binomial coefficient (see section 3.2 for details) C M,n = M! n!(m n)!, (10) 1 It is pronounced M choose n because we choose n items from an total of M items 5

6 where the notation n! is pronounced n-factorial, and is defined as Substituting equation (10) in (12) n! = n (n 1) (n 2)... (11) p(n) = M! n!(m n)! bn a M n. (12) This is the the proportion of M-sequences that contain exactly n heads, and is therefore equal to the probability that a randomly chosen M-sequence contains exactly n heads. If we divide a period T into M bins then p(n) is also the probability of observing exactly n spikes. Note that we can interpret n in terms of a firing rate r becuase r = n/t ; so, p(n) is also the probability of observing a firing rate r = n/t within a given interval T. Equation (12) defines a binomial probability distribution, and some properties of the binomial are as follows. First, the binomial distribution is defined by two parameters, b and M. Second, its mean is given by Mb. This could be anticipated because if each trial (flip) has a probability b of being a one, and there are M trials, then the expected number of heads is Mb. Third, and less obviously, the variance of a binomial distribution is Mb(1 b), which has a maximum value at b = 0.5. Examples of binomial distributions for various values of b and M are given in Figure (4). The key thing to notice is that as M b increases, so the distribution looks increasingly like a Gaussian, which will be explored further below. 3.1 Normalising Histograms The observant reader will have noticed that the vertical axes of the histograms on the left of Figure (4) are counts that vary from zero to several hundred, whereas those on the right are probabilities that vary from zero to about 0.4. So even though the shapes of the distributions is similar, they are essentially scaled versions of each other. This is because the distributions on the left have been retained in their raw form. For example, if the bin for n = 12 has a height of 200 then this reflects the fact that 20 coin flips yielded 12 heads on 200 occasions (out of a total of 1000 separate sets of 20 coin flips). The vertical axis could be relabelled as p(n), the probability of observing n heads, by normalising each of these histograms to have unit area. This could be achieved by dividing each column by the total area of the hisotgram. As the total number of samples is known to be 1000, this is also the sum of all bin heights. The area of each bin is given by its height times its width (1 in this case). Thus the total area is Therefore if we divide all bin heights on the left of Figure (4) by 1000 then we should obtain the bin heights obtained on the right (which we do). Notice that this normalisation also ensures that the total area in each distribution is unity, which corresponds to the fact the the sum of all possible outcome probabilities must sum to unity. In other words, the number n of possible heads varies over some finite range (zero to 20 here), and the total probability that n will be one of those values is exactly unity. 6

7 Figure 5: From numbered bins to spike sequence. Top: Bins numbered from 1 to 5. If we choose n = 3 bins at random then we could obtain the permutation (1, 3, 5), which effectively places spikes in bins 1, 3 and 5, as shown at the bottom. However, the same set of three numbers could also have been obtained in a different order, giving a different permutation such as (5, 3, 1), but giving the same spike sequence or combination. As each combination (e.g. {1, 3, 5}) implies a single spike sequence, the number of different spike sequences that contain n spikes is the same as the number of combinations that contain n digits. 3.2 The Binomial Coefficient We describe how the binomial coefficient is derived. equation (12) on trust can skip this section. Readers who are happy to take We begin by placing M digits 1... M in M bins, in ascending order, as shown in Figure (6). Each digit corresponds to a bin, so choosing a digit is synomymous with choosing the position of a single spike. If we choose n M digits, how many different ordered sets or permutations could we obtain? This is the number P M,n of different spike sequences that would exist if each spike were labelled by a unique digit; so it does not really adddress our central problem. However, we can use P M,n to answer a more interesting question: how many different unordered sets (e.g. with the same digits) of size n are there, given that each set or combination is chosen from M digits? This is the number of sequences of length M that contain exactly n spikes. Basically, if each spike has a distinct identity given by a unique digit between 1 and M then we can find out how many of these labelled spike sequences (permutations) there are. However, this would be an odd sort of sequence, because each spike would be represented by a unique labelled digit. If we treat all spikes as if they are equivalent or anonymous then we can find out how many spike sequences there are that contain n anonymous spikes. Now, let s see how that works. As an example, we will use M = 5 bins (1, 2, 3, 4, 5), and we will choose n = 3 digits at random to obtain a permutation of numbers, such as (5, 3, 1). This permutation effectively places spikes in bins 1, 3 and 5 to yield the sequence (10101). Note the use of 7

8 Figure 6: From spike sequences to spike statistics. Each spike sequence corresponds to exactly one unordered set or combination, such as {1, 3, 5}. The number of spike sequences with M = 5 bins that contain exactly n = 3 spikes is given by the binomial coefficient C M,n = 10. If the probability that a spike occurs in each of M = 5 bins is b then the proportion of spike sequences that contain exactly n = 3 spikes is 10 b 3 (1 b) 2 (see equation (12)). parenthtical brackets ( is used to denote an ordered set or permutation, whereas curly brackets } are used to denote an unordered set or combination. Choosing labelled sequences: If we choose n digits (equivalently, bins or spike positions) from our ordered list of M digits then there are M ways to choose the first digit; that is, we have a choice of M digits from which to choose the first digit of our permutation. Having chosen the first digit, there are then (M 1) ways to choose the second digit. It follows that there are M(M 1) ways to choose the first pair of digits. Note that each pair of digits effectively defines the positions of two spikes in our sequence of M bins. Continuing this line of reasoning, there are M(M 1)... (M n + 1) ways to choose n digits from M digits. Moreover, each time we choose n digits we define a set of n positions for our spikes. Recall that each ordered set of digits is called a permutation. Of course, some permutations inevitably contain the same numbers as other permutations, albeit in a different order. For example, the permutation (1, 3, 5) contains the same digits as the permutation (5, 3, 1), but in a different order. But both permutations imply the same unordered set or combination {1, 3, 5}, and both permutations imply spikes are present in bin numbers 1, 3 and 5 in our sequence of bins; that is, both permutations imply the spike sequence (10101). Thus each combination implies a single spike sequence. So, if we can find the number of combinations then we would have the number of spike sequences. Choosing anonymous sequences: Now, for a given combination of n digits there are n! ways to order each combination to yield n! permutations. For example, the combination {1, 3, 5} yields 3! = = 6 permutations ((1, 3, 5), (3, 5, 1), (5, 1, 3) (5, 3, 1) (1, 5, 3) and (3, 1, 5)). By definition, each of these n! permutations contains the same n digits, and each implies the same set of bins with spikes. Therefore, for each combination there are n! permutations. This implies that if there are C M,n combinations then there are n!c M,n permutations. Now, we know that the number of permutation is P M,n, which we have just shown must be equal to n!c M,n : P M,n = n!c M,n. (13) 8

9 This implies that the number of combinations is C M,n = P M,n /n! (recall that this is also the number of anonymous spike sequences). Given that P M,n = M(M 1)... (M n+1), it follows that C M,n = [M(M 1)... (M n + 1)]/n!. (14) So there we have it. If we have n spikes in M bins, then the number of different sequences that contain exactly n spikes is C M,n. Finally, we can clean this up by re-wrting P M,n M(M 1)(M 2)... (M n + 1) = M! (M n)!. (15) For example, if M = 5 and n = 3 then we have M(M 1)(M 2)... (M n + 1) = (16) = 60. (17) Putting all this together yields C M,n = M! n!(m n)!, (18) which is the well-known equation for the number of combinations, also known as the binomial coefficient 4 The Poisson Distribution The binomial distribution defined in equation (12) is quite a cumbersome expression. Fortunately, under fairly mild conditions, the binomial distribution is well approximated by two more tractable distributions, the Poisson and the Gaussian. We consider the Poisson distribution here, and the Gaussian distribution in the next section. If M is large and b is small, but Mb is finite, then the binomial distribution is approximated by the Poisson distribution p(n) = λn n! e λ, (19) where λ = Mb. As might be expected given that the mean of the binomial is Mb = λ, so the mean of the Poisson is also λ, and is the mean number of spikes in a period T. The Poisson is a one parameter distribution, with both its mean and variance given by λ. If the Poisson is a good model of spike statistics then this implies that as the firing rate increases, so does the variance in the firing rate (which it does, although not exactly in proportion to λ). If we substitute n = rt and λ = f(s)t = rt in equation (19) (repeated below) then we obtain an expression for the likelihood p(r s) in terms of the stimulus value s and the 9

10 observed firing rate r p(n) = λn n! e λ, (20) = λrt (rt )! e λ (21) = (f(s)t )rt e f(s)t (rt )! (22) = (rt )rt (rt )! e rt (23) = p(r s). (24) This is a discrete distribution, meaning that it can only adopt non-zero values of p(n) at integer values of n. However, for large values of λ, it can be treated as a continuous distribution (see Cowan p29, 145), meaning that it can adopt non-zero values for real values of n. This has been done implicitly above in the transition from p(n) to p(r s). Additionally, this permits us to approximate this discrete distribution using the Gaussian distribution, which is continuous. 5 The Gaussian Distribution If the expected number of of heads (spikes) λ is large then the Poisson distribution given in equation (23) is approximated by the Gaussian distribution p(r s) = ke (r r)2 2σ 2, (25) where r = f(s) is the mean, σ is the standard deviation of the Gaussian distribution, and the constant k = 1/(σ (2π)). Figure 7: The Gaussian distribution. If bm for the binomial distribution is large (e.g. if M is large and b is small) then λ = bm for the Poisson distribution is large, which implies that the Poisson is approximated by the Gaussian distribution. The Poisson distribution for λ = 10 is shown as a histogram (red), and the corresponding Guassian distribution with mean r = 10 and σ = 10 is overlaid as a solid curve. 10

11 6 Summary We began by asking why we need to know about the statistics of spikes. The answer is that we need to know in order to infer what a specific firing rate tells us (and the brain) what stimulus could have given rise to the observed firing rate. The starting point for analysing spike statistics was to slice a time period T into a large number of very narrow bins, so that the probability of observing a spike in each bin is r t, where r is the firing rate, and t is the width of each bin. Assuming that spikes are mutually independent, we can model the state of each bin as the flip of a coin. Thus begins the process of counting heads (spikes), which results in the binomial distribution. We then showed that, under fairly mild conditions, the binomial distribution is approximated by the Poisson distribution, and that this is approximated by the Gaussian distribution. From a mathematical perspective, the Poisson and Gaussian distributions are much easier to use than the binomial distribution. Reference Recommended Dayan and Abbott, Chapter 1, especially pp14-17, pp24-27, pp31-34 (beware their notation for firing rate, and mean firing rate, on pages 9-10). Background Rieke, Spikes, pp49-54, and if you want the details then see pp (appendix) Cowan, Statistical data analysis, p DeGroot, Probability and Statistics, p