Annotating regions of the genome for occurences of DNA motifs (e.g transcription factor DNA binding sites)

Input:

• Regions of the genome to annotate, typically promoter regions (.bed)
  • Columns: chrom, window_start, window_end, identifier where the identifier is typically the ENSEMBL gene ID
  • Make sure that window_start and window_end are never out of the 0:chrom_size interval, otherwise twoBitToFa will raise an error. Note that UCSC tools all use 0-based indexing, with the end coordinate *not* included in the sequence.
• Reference genome
• MEME-formatted list of position-weight matrices (.txt)
• Number of cores for multiprocessing
• p-value threshold for FIMO

Output:

• Concatenated .txt file of FIMO output results, for further processing in pandas or R.

Key steps:

• Split the .bed file into multiple .bed files according to the number of cores used for multiprocessing
  • bedtools2/tools/split, equal records matters.

• Extract the fasta sequences from the reference genome, using the bedfile as an input:
  • twoBitToFa, with the .bed regions and the .2bit genome as inputs.

• Compute the background frequencies for the regions of interest and store the results in a 0-order Markov background model as specified by the MEME suite

• Launch parallel processes of FIMO on the newly created .fasta files.
  • FIMO is case insensitive -> no risks with masked regions

• Concatenate FIMO outputs into a single .txt file
• Sort FIMO outputs by gene, motif, chr and start.
• Save results on /home

Practical notes:

• Use environment variables: $HOME, $USER for compatibility with other usernames
• Work in /scratch/$USER/, save results in $HOME
• Running the annotation on 57700 human genes with 5500+ motifs ran in a bit more than 70 minutes

Reproduce the work on SCITAS: 0) check the SCITAS modules and python 3 requirements needed.

• Load modules and activate source environment

SCRIPTS HAVE TO BE RAN FROM WITHIN THE /scripts/ DIR !!!

1. execute 'download_ref_genome.py'
2. execute 'download_ucsc_binaries.py'
3. execute 'compute_background_model.py'
4. execute 'split_bed_files.py'
5. execute 'extract_fasta.py'
6. modify and execute 'run_fimo.sh'
   • launches run_fimo.py