Day 4: Self-organization and computation: Alex Mordvintsev, Stan Kerstjens, Vijay Balasubramanian

Yesterday afternoon and evening there were several interesting tutorials and discussions. One in particular that I caught from Nuno de Costa and Ben Pedigo showed people the incredible Allen Brain Atlas capabilities. You can dynamically and visually explore individual neurons and their labeled synaptic connections and plot connection matrices.

And then late last night an intrepid 4 of us (which started out as 5 but one chickened out after a few moments in the water) went night snorkeling with headlamps. Fish all seemed to be sleeping and no sightings of octopus or cuttlefish (very unfortunately).

Morning brought calm sunny skies. After the usual excellent breakfast buffet, we started off discussing algorithms.

Florian Engert MC'ed this session and introduced the topic: Getting functionality from brain structure. He likened the difference between self-organized brain architecture and learned behavior to the difference between art and porno. You know it when you see it.

Alex Mordvintsev (a creator of Inception and Deep Dream at Google) kicked things off with simplified examples of machines that follow simple rules based on instructions that are defined by bytes operating on BF tapes.  Here is what he drew:

(Alex's sketch of BF and BFF)

BFF (arxiv paper) is a version of the BF language enhanced to consider 2 heads: data and code pointers. (From the BFF paper: "Brainfuck (BF) is an esoteric programming language widely known for its obscure minimalism".)

Now there is instruction pointer and data pointer. Adds {} and , operators.  Tapes have 32 or 64 cells. Each tape of a big bunch is randomly initialized and executed as BFF program. The digital soup of tapes are concatenated, run for some time, and then split to original lengths.  This process is repeated many times.

All kinds of interesting things happen: Emergence of compressibility measured by gzip, emergence of replicators and reverse replicators, sudden jumps in repeated sequences, and emergence of self-replicating programs. Sometimes special 0 (zeros) emerge and prevent copying. Then nested loops appear to deal with this. It seems like this system seems to nearly always move in the direction of decreased entropy driven by the "energy" of the free operations. Experiments to see the effect of limited energy (like ATP) have not been done yet.

By odd coincidence, I have just been finishing Dan Simmons's awesome Hyperion cantos. In the last book, the messianic heroine of story (Anaea) relates how Ed O Wilson's self-taught student Tom Ray studied these types of automata during the early 1990s and observed in his Tierra developments incredible behaviors of self-mutating and shorter and shorter and more efficient tape machines that would win out. From a handcrafted 80 byte machine there emerged a 45 byte then a 51 byte that was a parasite on the 45-byte. Then 22-byte hyperlife machines (like viruses) emerged. What to make of these modern developments in light of the very long history of the studies of artificial life in the form of core war and GOL over many decades is unclear to me, but interest keeps coming back over and over.

Stan Kerstjens, who partnered with Rodney Douglas (unfortunately not with us this year) on models of cortical development, continued after this interesting game-of-life start. He drew an illustration of a cell, and how a so-called "H-tree" of branches of cells (the lineage) can describe their development.  (The H-tree is a tree specialized for dividing cells on 2d plane.)

A class of cells, e.g. inhibitory, can appear on one main branch. You can think of the entire tree as the mitotic cell lineage. The genetic program must have certain basic capabilities: replication, mobility, lineage.  And it must start from a single cell (with all its machinery) that can build entire animals that replicate themselves.

Now discussing a specific case: Development of the projection from retina to tectum in reptiles.  One tectal axis has gradient of expression of some protein, the other axis has some other protein. How do these gradients emerge? This was not discussed. How do cells follow the extremely weak gradients? The answer is they probably do not, rather they may follow the scent like insects find food by searching odor plumes as shown in tiny vanilla RNNs studied by Bing Bruntun. 

What would be really nice is if this gradient could encode the location throughout the brain. The trick is that a gene expression pattern has x,y,z AND time, so 4d coordinates. So, like a neuron population can encode multiple features, so what if a gene expression population can encode multiple aspects analogously. 

The ABI has a gene expression atlas of about 2k genes selected to be potentially relevant during development of a mouse.  Stan found out that using the principal component projections of this 2k vector can compute an "eigengene" along the dorsal ventral axes of a developing mouse train gene expression vector voxel image. It clearly shows that these eigengenes are encoded along the dorsal-ventral principal physical developing brain axis.

One question is how growth cones navigate on short time scales. The other is how the gradients are established in the first place. Regarding the 2nd question, there will be some gradient of some eigengene dorsal-ventral. If you look at an embryonic mouse, the eigengene projection is very stable over development, in particular the variance is small. It continues to encode the same axis over complete development. The hypothesis is that the growth cones can detect this eigengene projection somehow, by as-yet unknown mechanisms. This type of eigengene dorsal ventral gradient has been observed over 3 scales, mouse embryo, late mouse brain, and larval zebrafish.

A common development question is how time is encoded: The eigengene also encodes development time very clearly, which could allow the development to occur in correct order. There are 8 stages of mouse development classified by the community that are clearly correlated by at least 0.7 covariance with these 8 development points.

After the coffee break Vijay Balasubramanian went on with considering self-organization of brain development. He invited Matthew Cook to provide initial comments but I did not catch the point. Vijay talked about self-organization. Brains need to solve the problem of structural layout using approx 5k cell types. If you consider motor cortex, a large chunk is devoted to face control in social animals. The layout is determined during development, and since so much is devoted to face control, strokes often cause face sagging. 

The retina is a brain area with ~75 cell types (photoreceptor, bipolar cells, an inner plexiform layer with the vastly complex 35 classes of amacrine cells, then 20 classes of ganglion cells (GCs)). The GCs are feature detectors, e.g. ON and OFF center cells, color-opponent and motion selective cells. All come from stem cells that move up and down during development during it's self-organization. 

If you look at worms and flies, each single neuron is in the same place and has essentially the same connections. 

So what needs to self-organize? He drew this list

(list of things that need to self-organize)

He then went on to show how natural vision statistics such as spatial correlation can result in ratios of ON and OFF GCs of particular values based on information theory where you test information content via I=Ion+Ioff-Ijoint. In particular the ratio you see is about 1.7 for OFF/ON ratio, which can be traced back to natural statistics that small dark features are more common that small bright features.

How can you consider self-organization by adding more cell types?  In transcriptomic space, you have local minima of some potential  The 'potential' in this space is evolved. There is some hypothesized dynamics. It is a gedanken experiment cartoon to allow speculating how cells that might move around this space. I didn't really understand the point here unfortunately.

Now the ON and OFF signals need to get from retina to thalamus to cortex. And the cells need to line up in cortex to make simple-type edge detectors. This self-organization is driven before the eyes open by waves of activity in the developing retina, even before there are photoreceptors. 

This is a baby version, next he considered this idea in spades, in particular related to language development in Broca's area. The idea in this model  of speech is that Broca's area is related to articulation, understanding, and concepts

There are 3 theories, but unfortunately I forgot to take a picture.

(List of theories to be added.)

In speech production, there are nerve tracts. Then if you do semantic processing there is another tract. Then there is auditory compression, then working memory. This "integrated model" of speech understanding and production is very flexible because there is a fixed hierarchy, not a fixed architecture. The integrated model is modular with different modules that have interfaces. 

Evolution has produced a set of procedures (modules) that needs to be wired and dynamically reconfigurable. Multiple FMRI studies show these brain networks, but how they are controlled is currently unknown. 

Vijay concluded the morning session (using up all of the time promised to Alex) with an interesting description of a specific example related to development of grid cells in the tectum of the entorhinal cortex of a rat brain. He sketched this picture:

The tectum is organized in 3 layers and the grid cells differ in scale by about a factor of 1.5. A particular layer has a triangular lattice of grid cells that appear as soon as the animal is put in a new environment, e.g. a new room. As long as the room has features that allow the animal to orient itself the grid cells appear within a few minutes. How does this happen? Vijay showed that very simple rules for neutral inhibition driven by tonic excitation with some small variability or noise cause this lattice of cells to quickly appear. The grid cells are presumably activated by some set of visual or other cues. When the animal moves, there are inputs to these grid cells that cause the bump of activity to move along the surface of the lattice.

But how do the different scales of the grid cells appear? Vijay showed that results from physics condensed matter research can explain this emergence via a push-pull mutual excitation between the layers with short range excitatory and longer range inhibitory scales. 

After this very interesting morning session we broke for snorkeling, lunch, and afternoon activities.

After lunch, Alex continued with a very popular tutorial in the disco about other artificial life studies. Here is a short video snippet of the tutorial:

That concludes today's blog. See you tomorrow! :)