Creation for weight matrices for initialising the first layer of a neural network

Implemented methods

• PCA: Principal Component Analysis
• ICA: Independent Component Analysis
• SC: Sparse Coding (Dictionary Learning with positive code)
• RG: Random Gabor filters (Gabor filters with randomly drawn parameters)
• RP: Random Projections drawn from a Gaussian distribution

All weight matrices are normalised such that the input vector (receptive field) of each unit in the hidden layer is normalised to L2-norm = 1.

Dependencies & Environment

• Conda needs to be installed
• Setup the conda environment weightmatrices by running:
  
  ` bash setup_dependencies.sh `

Usage

• To run the main script(s) that produce the weight matrices run:
  
  ` bash run.sh `
  
  If you want to directly run main.py to use command line args (e.g. python main.py --methods 'pca' --nhidden 10 25), do not forget to activate the environment by running:
  
  ` conda activate weightmatrices `

Remarks

By default, Datasets are downloaded and put into a /datasets directory. To avoid this you can:

1. Change the root variable in the torchvision data loader function.
2. Create the /datasets folder and corresponding subfolder and put the datasets (or a link) in there.