Bayesian Inference, Dynamical Systems and the Brain


FIL Seminar Room, Thursdays 11am-1pm, 2011.


Useful Resources:


Probabilistic and Unsupervised Learning (Gatsby)


Theoretical Neuroscience (Gatsby)


Matrix Reference Manual


The PDF notes for each lecture contain a reference list.


1. Bayesian Inference, 24 Feb


      Bayes rule for Gaussians. Directed Acyclic Graphs

Joint Densities and Marginalisation. Medical Decision Making.

      Perception as statistical inference. Decision Making Dynamics.

      Bayesian Sensory Integration. Explaining Away.




2.  Empirical Bayes, 3 Mar


      Linear Models.  Maximum Likelihood. fMRI analysis.

Delta rule learning.  Newton method.

      Weighted Least Squares. Marginal Likelihood.

MEG source reconstruction. Empirical Bayes.

      Restricted Maximum Likelihood (ReML)




3. Sparse and Hierchical models, 24 Mar


      Relevance Vector Regression. Automatic Relevance Determination.

      Recurrent Lateral Inhibition. Linear predictive coding. Hebbian Learning. 

Sparse coding of natural visual images. MAP learning. Cauchy priors.



            Linear Hierarchical Models. Superficial versus Deep Cortical Laminae.

            Hierarchical predictive coding. Increasing receptive field size. Simple cells.

            End Stopping. Nonlinear hierarchical models.


hier.pdf  (this file is too big for most browsers to embed – instead, right click and ‘save as’)


4. Approximate Inference, 31 Mar


      Information, Entropy,Source Coding Theorem, Prefix Coding,

Kullback-Liebler Divergence, Asymmetry,

      Multimodality,Variational Free Energy, Factorised Approximations,

      Mean Field Approach, Free-Form versus Fixed Form,

      Nonlinear Regression, Adaptive Step Size, Approach to Limit.




5. The Microscopic Brain, 7 Apr


      Linear differential equations. Matrix Exponential. Eigendecompostion.

Nodes, Saddles, Spirals, Centres. Feeback Inhibition. Stability.

Local linear stability analysis. Isoclines. Nonlinear Oscillations.

Spiking Neurons. Fitzhugh-Nagumo. Hodgkin-Huxley. Rose Hindmarsh.

Hopf and Saddle-Node Bifurcations. Type 1/2 Cells.



6. The Mesoscopic Brain, 14 Apr


      Integrate and Fire Neurons.Phase Reduction.

      Phase Response Curves. Weakly Coupled Oscillators.

Synchronization via excitation or inhibition.

Spike Frequency Adaptation (SFA).

Spike Time Dependent Plasticity (STDP).

Dynamic pattern recognition via transient synchronisation.

Spike-to-Spike versus LFP synchronization.



7. The Macroscopic Brain, 21 Apr


Neural Mass Model.  Alpha activity. Spike cycles. Bifurcations.

Hierarchical cortical connectivity. Delay differential equations.

Dynamic Causal Models (DCM)  for Event Related Potentials.

Auditory oddball. Linear Systems Analysis.

Network kernel function. Frequency response. Cross Spectral Density.

DCM for steady state responses. Inferring synaptic physiology.



8. Stochastic Processes, 19 May


      Wiener, Ornstein-Uhlenbeck (OU) and Gaussian processes.

      Stochastic Differential Equations. Stochastic chain rule.

      Mean and variance functions.

Rodriguez-Tuckwell method for multivariate Gaussian

approximation of population density evolution.

Population of Fitzhugh-Nagumo neurons.

Fokker Planck Equation. Population of stochastic integrate

and fire cells. 2AFC tasks. Drift Diffusion Models.

Reaction Times and Error Rates.

      Continuum limit of discrete time Bayesian model.




9. Hierarchical Dynamic Models, 26 May


      OU(p) processes. Covariance functions. MEG alpha rhythms.

Embedding  and Generalised Coordinates.

Continuous time latent variable models.

      Filtering. Variational Energies and Actions.

Dynamic Expectation Maximisation. Linear Convolution Model.

Hierarchical Dynamic Models.




10. Bayesian Model Comparison, 2 June


      Posterior model probability, Bayes factors, Odds Ratios

      Model evidence. Free Energy. Approximation.

Accuracy and Complexity Decompositions.

AIC and BIC. Linear Models. fMRI example simulations.

DCM for fMRI example. Priors. Simulations.

      Inference for groups of subjects. Fixed Effects.

      Random effects generative model. Gibbs sampling.

      Exceedance probabilities.