This method will estimate the site frequency spectrum, the method is described in Nielsen2012. The theory behind the model is briefly described here

This is a 2 step procedure first generate a ".saf" file (site allele frequency likelihood), followed by an optimization of the .saf file which will estimate the Site frequency spectrum (SFS).

For the optimization we have implemented 2 different approaches both found in the misc folder. The diagram below shows the how the method goes from raw bam files to the SFS.

NB the ancestral state needs to be supplied for the full SFS, but you can use the -fold 1 to estimate the folded SFS and then use the reference as ancestral.

The information on this page relates to versions 0.570 or higher.

In earlier version -doSaf was called -realSFS

[sequence data{bg:orange}]->GL[genotype likelihoods|SAMtools;GATK;SOAPsnp;Kim et.al]

[genotype likelihoods|SAMtools;GATK;SOAPsnp;Kim et.al]->doSaf[.saf file{bg:blue}] [.saf file{bg:blue}]->optimize('emOptim2')[.saf.ml file{bg:red}]

</classdiagram>

Brief Overview

./angsd -doSaf 
	-> angsd version: 0.569	 build(Dec 17 2013 12:40:00)
	-> Analysis helpbox/synopsis information:
--------------
angsd_realSFS.cpp:
	-doSaf		0
	1: perform multisample GL estimation
	2: use an inbreeding version
	3: calculate genotype probabilities (beta)
	4: Assume genotype posteriors as input (still beta) 
	-doThetas		0 (calculate thetas)
	-underFlowProtect	0
	-fold			0 (deprecated)
	-anc			(null) (ancestral fasta)
	-noTrans		0 (remove transitions)
	-pest			(null) (prior SFS)

misc/emOptim2
./emOptim2 afile.sfs nChr [-start FNAME -P nThreads -tole tole -maxIter  -nSites ]
nChr is the number of chromosomes. (twice the number of diploid invididuals)

Options

-doSaf 1: Calculate the Site allele frequency likelihood based on individual genotype likelihoods assuming HWE

-doSaf 2: (version above 0.503) Calculate the Site allele frequency likelihood based on individual genotype likelihoods while taking into account individual inbreeding coefficients. This is implemented by Filipe G. Vieira. You need to supply a file containing all the inbreeding coefficients. -indF

-doSaf 3: Calculate the posterior probabilities of the sample allele frequency distribution for each site. This needs a prior distribution of the SFS (which can be obtained from -doSaf 1/emOptim2)

-doSaf 4: Calculate the posterior probabilities of the sample allele frequency distribution for each site based on genotype probabilities. The genotype probabilities should be provided by the using using the -beagle options. Often the genotype probabilities will be obtained by haplotype imputation.

-underFlowProtect [INT]

0: (default) no underflow protection. 1: use underflow protection. For large data sets (large number of individuals) underflow projection is needed.

Example

A full example is shown below, here we use GATK genotype likelihoods and hg19.fa as the ancestral. The emOptim2 can be found in the misc subfolder.

first generate .saf file with 4 threads

./angsd -bam bam.filelist -doSaf 1 -out small -anc  hg19.fa -GL 2 -P 4

Obtain a maximum likelihood estimate of the SFS using EM algorithm

misc/emOptim2 small.saf 20 -maxIter 100 -P 4 >small.sfs

We always recommend that you filter out the bad qscore bases and meaningless mapQ reads. eg -minMapQ 1 -minQ 20.

What does the 20 mean? This is the number of chromosomes

If you have say 10 diploid samples, then you should choose 20
if you have say 12 diploid samples, then you should choose 24.
If you have say 10 diploid samples, and you fold the spectrum then you should choose 10

Interpretation of the emOptim2 output file

The output from emOptim2 (small.sfs) is the SFS in logscale. This is to be interpreted as:

column1 := probabilty of sampling zero derived alleles (on logscale)

column2 := probabilty of sampling one derived allele (on logscale)

column3 := probabilty of sampling two derived allele (on logscale)

column4 := probabilty of sampling three derived allele (on logscale)

etc

each row is a region of the genome (see below).

NB

The generation of the .saf file contains a saf for each site, whereas the optimization requires information for a region of the genome. The optimization will therefore use large amounts of memory. The program defaults to 50megabase regions, and will loop over the genome using 50 megebase chunks. You can increase this adding -nSites 500000000. Which will then use 500megabase regions.

Folded spectra

If you don't have the ancestral state, you can instead estimate the folded SFS. This is done by supplying the -anc with the reference genome and also supply -fold 1. Then you should remember to change the second parameter to the emOptim2 function as the number of individuals instead of the number of chromosomes.

The above example would then be

#first generate .saf file
./angsd -bam bam.filelist -doSaf 1 -out small -anc  hg19.fa -GL 2 -fold 1
#now try the EM optimization with 4 threads
misc/emOptim2 small.saf 10 -maxIter 100 -P 4 >small.sfs

Posterior of the per-site distributions of the sample allele frequency

If you supply a prior for the SFS (which can be obtained from the -doSaf/emOptim2 analysis), the output of the .saf file will no longer be site allele frequency likelihoods but instead will be the log posterior probability of the sample allele frequency for each site.

Format specification of binary .saf file

The information in this section is only usefull for people who wants to work with the "multisample GL"/"sample allele frequencies" for the individual sites.

Assuming 'n' individuals we have (2n+1) categories for each site, each value is encoded as ctype double which on 'all known' platforms occupies 8 bytes. The (2n+1) values are log scaled like ratios, which means that the most likely category will have a value of 0.

The information for the first site is the first (2n+1)sizeof(double) bytes. The information for the next site is the next (2n+1) bytes etc. The positions are given by the ascii file called '.saf.pos'

#sample code to read .saf in c/c++ assuming 10 individuals
FILE *fp = fopen("mySFS.saf","rb");
int nInd = 10;
double persite[2*nInd+1];
fread(persite,sizeof(double)*(2*nInd+1),1,fp);
for(int i=0;i<2*nind+1;i++)
   fprintf(stderr,"cat: %d = %f\n",i,persite[i]);

If the -fold 1 has been set, then the dimension is no longer 2*nInd+1 but nInd+1
If the -pest parameter has been supplied the output is no longer likelihoods but log posterior site allele frequencies

How to plot

Assuming the we have obtained a single global sfs(only one line in the output) from emOptim2 program, and this is located in file.saf.sfs, then we can plot the results simply like:

sfs<-exp(scan("small.sfs")) #read in the log sfs
barplot(sfs[-c(1,length(sfs))]) #plot variable sites

We can make it more fancy like below:

#function to normalize
norm <- function(x) x/sum(x)
#read data
sfs <- exp(scan("small.sfs"))
#the variability as percentile
pvar<- (1-sfs[1]-sfs[length(sfs)])*100
#the variable categories of the sfs
sfs<-norm(sfs[-c(1,length(sfs))]) 
barplot(sfs,legend=paste("Variability:= ",round(pvar,3),"%"),xlab="Chromosomes",
names=1:length(sfs),ylab="Proportions",main="mySFS plot",col='blue')

If your output from emOptim2 contains more than one line, it is because you have estimated multiple local SFS's. Then you can't use the above commands directly but should first pick a specific row.

sfs<-exp(as.numeric(read.table("multiple.saf.sfs")[1,])) #first region.
#do the above
sfs<-exp(as.numeric(read.table("multiple.saf.sfs")[2,])) #second region.

SFS Estimation

Contents