on friday, sebastian seung stopped by ucsd and the salk. he gave two talks, one on machine learning and a second on connectomics (see this review). this post won’t be critical. these are notes from his second talk. facts that should be noted before starting are that 1) the experimental work was done by winfried denk and kevin briggman, a ucsd comp neuro grad, and 2) the theoretical work was done by machine learning theorists long ago. also, this is just in-seminar note-taking; there’s been no attempt to make it appreciable to people unfamiliar with the topic (i’ll give you a post soon). enjoy!
- two problems of connectomics (the complete connection matrix of all cells) are
- determining what constitute synapses
- determining to which cells each axon is synapsed.
- techniques: use diamond knife, image slice with TEM, or (denk) image block with SEM (serial block face-sem… “the denktome”) (denk and horstmann, PLoS Biology, 2004; denk and briggman, Current Opinion in Neurobiology, 2006)
- limitations: staining a block is hard, electrons going too deep in the block (if too high energy) which will blur the z-resolution. so the x-y resolution is less, ~25-30 nm) (consider)
- nematode is example of successful reconstruction, but done by hand.
- near future: teravoxel datasets. 1 cubic mm, entire brains of small animals, small brain areas of large animals (the lore: c. elegans shows no individual diversity.)
- accuracy- (following neurites: image segmentation (contour completion is easier for humans than for algorithms. which is a bad omen for 3d images, since even humans stink at 3d contour completion); recognizing synapses (object recognition).
- note that medical imagers have somewhat solved this. but the problems they solve are easier. note: 6 sigma imaging (reference to engineering: wikipedia article)
- speed: teravoxel datasets
- NICE: briggman came up with a stain that only stains the membrane = contours (HRP), making comp sci problem easier. machine learning approach: train neural network on sample data (contour completion done by hand by undergrads) and generalize to whole set. in computer vision, david marr thought everything could be bottom-up. that failed. high-level knowledge is required. same for speech processing. harvey brings up the importance of light microscopy.
- the neural network details: convolutional network (related to a late 80s concept, the neocognitron, designed by kunihiko fukushima, a bell labs guy).
- network of convolution operations in 3D. progressively different convolutions (filters in feature maps). linear filter, folllowed by sigmoidal “squashing function”. to arrive at the filters, all parameters are adjustable by training (30,000 parameters, using backprop “gradient learning”, as from rumelhart and hinton. (n.b. from dm: see also werbos).
- each layer (from fukushima): reduced the resolution in each layer (like areas of cortex). this yields spatially invariant representations. here, their algorithm has a biological analog in multiple time steps in V1. “unfolding in time”.
- there are unsupervised approaches that one could do. they did this because it was guaranteed to work.
- shows a fly-through of reconstructed neurons to the bladerunner soundtrack. “though we can’t do science yet, we can do movies”. only 20 um
- structure-function relationships in neural networks
- neural development - how the brain wires and rewires itself
- “connectopathies” - subtle neuropathologies underlying psychiatric disorders.
- example. avian brain area HVC (test synaptic chain theory of motor sequences)
- as compared with lichtman’s projectome.
- 30 um slices of raw images. 50,000 petavoxels (1mil for 1 petabytes); connectome is 5 petabytes, which would fit in google’s storage (around 15 petabytes); 10^11 neurons, 37 bit address (5 bytes), 10^4 connetions/ neuron. in a few decades of moore’s law this’ll work. but this will require nanelectronics.
- terry’s less optimistic that even in decades they’ll be able to do this. only harvey karten can do 3d intuition… the rest of us are not evolved for it.
References.
- K. Fukushima. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4): 93-202, 1980.
- Denk W, Horstmann H. Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biol. 2004 Nov; 2(11): e329.
- Briggman KL, Denk W. Towards neural circuit reconstruction with volume electron microscopy techniques. 2006 Oct; 16(5): 562-70.