SFS Estimation

This method will estimate the site frequency spectrum, the method is described in Nielsen2012.

This is a 2 step procedure first generate a ".saf" file, followed by an optimization of the .saf file which will estimate the Site frequency spectrum.

For the optimization we have implemented 2 different approaches both found in the misc subdir of the root subdir.This is shown in the diagram below.

NB the ancestral state needs to be supplied for the full SFS, but you can use the -fold 1 to estimate the folded SFS and then use the reference as ancestral.

The information on this page relates to versions 0.551 or higher.

<classdiagram type="dir:LR">

[sequence data{bg:orange}]->GL[genotype likelihoods|SAMtools;GATK;SOAPsnp;Kim et.al]

[genotype likelihoods|SAMtools;GATK;SOAPsnp;Kim et.al]->doSaf[.saf file{bg:blue}] [.saf file{bg:blue}]->optimize('emOptim2')[.saf.ml file{bg:red}]

</classdiagram>

Brief Overview

./angsd -doSaf 
	-> angsd version: 0.569	 build(Dec 17 2013 12:40:00)
	-> Analysis helpbox/synopsis information:
--------------
angsd_realSFS.cpp:
	-doSaf		0
	1: perform multisample GL estimation
	2: use an inbreeding version
	3: calculate genotype probabilities
	4: Assume genotype posteriors as input (still beta) 
	-doThetas		0 (calculate thetas)
	-underFlowProtect	0
	-fold			0 (deprecated)
	-anc			(null) (ancestral fasta)
	-noTrans		0 (remove transitions)
	-pest			(null) (prior SFS)

Options

-realSFS 1

an saf file will be generated.

-realSFS 2

(version above 0.503) Taking into account perIndividual inbreeding coefficients. This is the work of Filipe G. Vieira

-realSFS 3

Calculate genotype probabilities

-realSFS 4

Assume genotype probs as intput (beta)

For the inbreeding version you need to supply a file containing all the inbreeding coefficients. -indF

-underFlowProtect [INT]

a very basic underflowprotection

Example

A full example is shown below, here we use GATK genotype likelihoods and hg19.fa as the ancestral. The emOptim2 can be found in the misc subfolder.

#first generate .sfs file
./angsd -bam smallBam.filelist -realSFS 1 -out small -anc  hg19.fa -GL 2 [options]
#now try the EM optimization with 4 threads
./emOptim2 small.sfs 20 -maxIter 100 -P 4 >small.sfs.em.ml

We always recommend that you filter out the bad qscore bases and meaningless mapQ reads. eg -minMapQ 1 -minQ 20.

If you have say 10 diploid samples, then you should do -nChr 20
if you have say 12 diploid samples, then you should do -nChr 24.

Interpretation of output file

The outpiles are then called small.em.ml. This will be the SFS in logscale. This is to be interpreted as:

column1 := probabilty of sampling zero derived alleles

column2 := probabilty of sampling one derived allele

column3 := probabilty of sampling two derived allele

column4 := probabilty of sampling three derived allele

etc

NB

The generation of the .sfs file is done on a persite basis, whereas the optimization requires information for a region of the genome. The optimization will therefore use large amounts of memory. The program defaults to 50megabase regions, and will loop over the genome using 50 megebase chunks. You can increase this adding -nSites 500000000. Which will then use 500megabase regions.

Folded spectra

Below is for version 0.556 and above

If you don't have the ancestral state, you can instead estimate the folded SFS. This is done by supplying the -anc with the reference and also supply -fold 1. Then you should remember to change the second parameter to the emOptim2 function as the number of individuals instead of the number of chromosomes.

The above example would then be

#first generate .sfs file
./angsd -bam smallBam.filelist -realSFS 1 -out small -anc  hg19.fa -GL 2 -fold 1 [options]
#now try the EM optimization with 4 threads
./emOptim2 small.sfs 10 -maxIter 100 -P 4 >small.sfs.em.ml

Posterior per-site distributions of the sample allele frequency

This is assuming version 0.556 or higher. If you supply a prior for the -realSFS analysis, the output of the .sfs file will no longer be loglikeratios of the different categories but log posteriors of the different categories.

Format specification of binary .sfs file

The information in this section is only usefull for people who wants to work with the "multisample GL"/"sample allele frequencies" for the individual sites.

Assuming 'n' individuals we have (2n+1) categories for each site, each value is encoded as ctype double which on 'all known' platforms occupies 8 bytes. The (2n+1) values are log scaled like ratios, which means that the most likely category will have a value of 0.

The information for the first site is the first (2n+1)sizeof(double) bytes. The information for the next site is the next (2n+1) bytes etc. The positions are given by the ascii file called '.sfs.pos'

#sample code to read .sfs in c/c++ assuming 10 individuals
FILE *fp = fopen("mySFS.sfs","rb");
int nInd = 10;
double persite[2*nInd+1];
fread(persite,sizeof(double)*(2*nInd+1),1,fp);
for(int i=0;i<2*nind+1;i++)
   fprintf(stderr,"cat: %d = %f\n",i,persite[i]);

• If the -fold 1 has been set, then the dimension is no longer 2*nInd+1 but nInd+1
• If the -pest parameter has been supplied the output is no longer like ratios to the most likely but log posts of the different categories.

Theory

We will try to elaborate on the theory behind the methods. Below is only a preliminary version of the theory.

This method is described in Nielsen2012.

SFS definition

For 'n' diploid samples, the site frequency spectrum (SFS) is the (2n+1) vector containing the proportion of site carrying 'k'-mutations. This means that the first element in the SFS is the proportion of sites where we don't observe any mutations, The second value is the proportion of sites where we observe 1 mutations. The last value is the proportion of sites we only observe mutations. It follows that the first and last column are the invariable categories and assuming that the SFS contains relative frequencies the variability in the sample can be estimated by:

$pvar=1-sfs_{0}-sfs_{2n}=\sum _{i=1}^{2n-1}sfs_{i}$

Sample allele frequency/Multisample GL

By supplying the -realSFS 1, flag to angsd. Angsd will calculate the likelihood of the sample allele frequency for each site and dump these into the .sfs file. The likelihood of the sample allele frequency are in this context the likelihood of sampling k-derived alleles. This is estimated on the basis of the 10 possible genotype likelihoods for all individuals by summing over all combinations. This is done using the recursive algorithm described in Nielsen2012. This we write as $p(X^{s}\mid j)$ meaning the likelihood of sampling j derived alleles for site s. And we calculate the folded as

$p_{fold}(x^{s}\mid j)=p(x^{s}\mid j)+p(x^{s}\mid 2n-j),\qquad j\in \{0,1,3,\ldots ,n-1\},$

$p_{fold}(x^{s}\mid j)=2p(x^{s}\mid j),\qquad j=n$

Likelihood of the SFS

The likelihood of the sfs is then given as:

$p(X|\theta )=\prod _{s=0}^{S}\sum _{i=0}^{2n}p(X^{s}\mid i)\theta _{i}$

Here $\theta$ is our sfs. In the case of the folded sfs, we use n instead of 2n in the summation. We can find the MLE of the SFS by using either an BFGS approach that uses derivatives or by using en EM algorithm. Both is implemented in the emOptim2 program.