ANGSD: Analysis of next generation Sequencing Data

Latest tar.gz version is (0.938/0.939 on github), see Change_log for changes, and download it here.

Sites: Difference between revisions

From angsd
Jump to navigation Jump to search
 
(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page describes the '''-sites''' filtering that angsd allows. This functionality allows the user to supply a list of sites for which the analysis will be limited to. If you are interested in regions you should consider to use the '''-r/-rf''' options, as described in [[Filters]]. The '''-sites''' will loop over all input data, where as the '''-r/-rf''', will use the indexing of BAM files. The '''-sites''' and '''-r/-rf''' can be used in combination.
This page describes the '''-sites''' filtering that angsd allows. This functionality allows the user to supply a list of sites for which the analysis will be limited to. If you are interested in regions you could consider to use the '''-r/-rf''' options, as described in [[Filters]]. The '''-sites''' will read all input data (but limit the analyses to those defined by -sites), where as the '''-r/-rf''', will use the indexing of BAM files. The '''-sites''' and '''-r/-rf''' can be used in combination.
 
You should therefore limit your analyses to the chromosomes/scaffolds for which you have sites you want to analyse (beeing the chromsomes in your -sites file).
This can be done with (assuming sites.txt is your -sites sites.txt file).
 
;NB the positions are one indexed. Which means there should never be a position of zero.
<pre>
cut -f1 sites.txt |sort|uniq >chrs.txt
</pre>
And append '''-rf chrs.txt''' to your argument list.


=Brief overview=
=Brief overview=
<pre>
<pre>
/angsd -sites
-> angsd version: 0.569 build(Dec 11 2013 13:38:47)
-> Analysis helpbox/synopsis information:
--------------
--------------
analysisKeepList.cpp:
abcFilter.cpp:
-sites (null) (File containing sites to keep (chr tab pos))
-sites (null) (File containing sites to keep (chr pos))
-sites (null) (File containing sites to keep (chr regStart regStop))
-sites (null) (File containing sites to keep (chr pos major minor))
-minInd 0 Only use site if atleast minInd of samples has data
-minInd 0 Only use site if atleast minInd of samples has data
You can force major/minor by -doMajorMinor 3
1) You can force major/minor by -doMajorMinor 3
And make sure file contains 4 columns (chr tab pos tab major tab minor)
And make sure file contains 4 columns (chr tab pos tab major tab minor)
</pre>
The -sites file is a nice ascii text file. ANGSD requires that a binary index version is generated. This is done by
<pre>
angsd sites index your.file
</pre>
</pre>


=Details=
=Details=
We support 2 different kinds of inputfiles for filtering.  
;-sites filename
File containing the sites to include in analysis. If the site does not exist in the sequencing data, then the sites will of course not be included in the output.
;-minInd [int]
Only keep those sites where we have data for at least this number of individuals.
 
 
We support 3 different kinds of inputfiles for filtering.  


# Either the user supply a file containing chromsome and positions
# Either the user supply a file containing chromsome and positions
# Or the user supply a file containing chromosome,position, major and minor
# Or supply regions like chr tab regStart tab regSTop
# Or supply a file containing chromosome,position, major and minor


Only sites contained in the filter file will be outputted. If you supply an augmented filter for the purpose of forcing a major and minor state then remember to supply '-doMajorMinor 3'
Only sites contained in the filter file will be outputted. If you supply an augmented filter for the purpose of forcing a major and minor state then remember to supply '-doMajorMinor 3'
Line 24: Line 43:
A filter file is supplied to ANGSD with the command
A filter file is supplied to ANGSD with the command
<pre>
<pre>
-filter filename
-sites filename
</pre>
</pre>
And this file should be indexed before hand


 
<pre>
Example of a filter file. File must be tab seperated.
angsd sites index filename
</pre>
==Example of filter file for single sites==
Example of a filter file.  
<pre>
<pre>
chr1  100001
chr1  100001
chr1  2500000
chr1  2500000
chr1  347348
chr1  347348
</pre>
==Example of filter file for regions==
Example of a filter file.
<pre>
chr1  100 220
chr1  1234 2234
chr1  1000000 2000000
</pre>
</pre>


 
==Example of augmented filterfile==
Example of a file containing information of major and minor. File must be tab seperated.
Example of a file containing information of major and minor. File must be tab seperated.
<pre>
<pre>
Line 52: Line 82:


==Internal representation==
==Internal representation==
if a filter file has been supplied as '-filter filter.txt', then ANGSD will parse the entire filter.txt file and generate binary representations and dump these in the outputfiles called
If a filter file has been supplied as '-sites filter.txt', then ANGSD will parse the entire filter.txt file and generate binary representations and dump these in the outputfiles called
# filter.txt.bin
# filter.txt.bin
# filter.txt.idx
# filter.txt.idx


Therefore remember to purge old versions of these files, if you have updated the filter.txt file.
These can be printed using the command
 
=Allele frequencies=
; -minMaf [float]: only work with sites with a maf above 'float'
 
=polymorphic sites=
 
; -minLRT [float]: only work with sits with an LRT>float
 
=Number of non missing individuals=
 
 
; -minInd [int]: only work with sites with information from atleast int individiduals, requires -doCounts 1
 
 
 
 
 
First we do a run with no filters
<pre>
<pre>
./angsd -doMaf 2 -doMajorMinor 1 -out TSK -bam bam.filelist -GL 1 -r 1:
angsd sites print filter.txt
...
head TSK.mafs
chromo position major minor knownEM nInd
1 13999919 A C 0.000008 1
1 13999920 G A 0.000008 1
1 13999921 G A 0.000008 1
1 13999922 C A 0.000008 1
1 13999923 A C 0.000008 1
1 13999924 G A 0.000008 1
1 13999925 G A 0.000008 1
1 13999926 A C 0.000008 1
1 13999927 G A 0.000008 1
</pre>
</pre>


Now we do a filter with MAF cutoff of 1\%
=Requirements=
 
The file should be sorted according to column1.
This can be achieved by:
<pre>
<pre>
../angsd0.3/angsd -doMaf 2 -doMajorMinor 1 -out TSK -bam bam.filelist -GL 1 -r 1: -minMaf 0.01
sort -k1 unsorted.txt >sorted.txt
head TSK.mafs
chromo position major minor knownEM nInd
1 13999950 T G 0.495291 2
1 14000019 G T 0.047247 9
1 14000056 C T 0.055851 10
1 14000127 G T 0.060760 10
1 14000170 C T 0.052388 9
1 14000176 G A 0.047928 10
1 14000202 G A 0.279722 9
1 14000262 C T 0.058555 9
1 14000322 A G 0.040471 8
</pre>
</pre>
=Bedfiles?=
If you want to use bed files, you need to convert to the native angsd format.
Bedfiles are chr pos1 pos2 value, and are zero indexed. Furthermore pos1 is included, but pos2 is not included.
You can convert to angsd format with


Similar if we only want sites with information for atleast 5 samples
<pre>
../angsd0.3/angsd -doMaf 2 -doMajorMinor 1 -out TSK -bam bam.filelist -GL 1 -r 1: -minKeepInd 5
head TSK.mafs
chromo position major minor knownEM nInd
1 13999971 T A 0.000007 6
1 13999972 G A 0.000007 6
1 13999973 C A 0.000005 5
1 13999974 G A 0.000006 6
1 13999975 C A 0.000002 5
1 13999976 C A 0.000004 7
1 13999977 A C 0.000005 8
1 13999978 C A 0.000005 8
1 13999979 T A 0.000005 8
</pre>
If we are interested in all sites with a p-value of 10^(-6) of being variable
<pre>
<pre>
../angsd0.3/angsd -doMaf 2 -doMajorMinor 1 -out TSK -bam bam.filelist -GL 1 -r 1: -minLRT 24 -doSNP 1
awk '{print $1"\t"$2+1"\t"$3}' input.bed >angsd.file
head TSK.mafs
chromo position major minor knownEM pK-EM nInd
1 14000202 G A 0.279722 42.623150 9
1 14000873 G A 0.212120 79.118476 10
1 14001018 T C 0.333736 89.040311 8
1 14001867 A G 0.200232 47.195423 10
1 14002422 A T 0.167692 43.196259 9
1 14003581 C T 0.207404 58.593208 9
1 14004623 T C 0.219838 102.856433 10
1 14007493 A G 0.453217 28.398647 9
1 14007558 C T 0.395670 80.236777 7
 
</pre>
</pre>
==Deprecated options==
These options should either be included (as is) or be discarded
;-minDepth:
;-maxDepth:

Latest revision as of 15:00, 16 November 2015

This page describes the -sites filtering that angsd allows. This functionality allows the user to supply a list of sites for which the analysis will be limited to. If you are interested in regions you could consider to use the -r/-rf options, as described in Filters. The -sites will read all input data (but limit the analyses to those defined by -sites), where as the -r/-rf, will use the indexing of BAM files. The -sites and -r/-rf can be used in combination.

You should therefore limit your analyses to the chromosomes/scaffolds for which you have sites you want to analyse (beeing the chromsomes in your -sites file). This can be done with (assuming sites.txt is your -sites sites.txt file).

NB the positions are one indexed. Which means there should never be a position of zero.
cut -f1 sites.txt |sort|uniq >chrs.txt

And append -rf chrs.txt to your argument list.

Brief overview

--------------
abcFilter.cpp:
	-sites		(null)	(File containing sites to keep (chr pos))
	-sites		(null)	(File containing sites to keep (chr regStart regStop))
	-sites		(null)	(File containing sites to keep (chr pos major minor))
	-minInd		0	Only use site if atleast minInd of samples has data
	1) You can force major/minor by -doMajorMinor 3
	And make sure file contains 4 columns (chr tab pos tab major tab minor)

The -sites file is a nice ascii text file. ANGSD requires that a binary index version is generated. This is done by

angsd sites index your.file

Details

-sites filename

File containing the sites to include in analysis. If the site does not exist in the sequencing data, then the sites will of course not be included in the output.

-minInd [int]

Only keep those sites where we have data for at least this number of individuals.


We support 3 different kinds of inputfiles for filtering.

  1. Either the user supply a file containing chromsome and positions
  2. Or supply regions like chr tab regStart tab regSTop
  3. Or supply a file containing chromosome,position, major and minor

Only sites contained in the filter file will be outputted. If you supply an augmented filter for the purpose of forcing a major and minor state then remember to supply '-doMajorMinor 3'

A filter file is supplied to ANGSD with the command

-sites filename

And this file should be indexed before hand

angsd sites index filename

Example of filter file for single sites

Example of a filter file.

chr1  100001
chr1  2500000
chr1  347348

Example of filter file for regions

Example of a filter file.

chr1  100 220
chr1  1234 2234
chr1  1000000 2000000

Example of augmented filterfile

Example of a file containing information of major and minor. File must be tab seperated.

1	728951	T	C
1	752721	A	G
1	754182	A	G
1	754334	T	C
1	760912	C	T
1	776546	G	A
1	779322	G	A
1	838555	A	C

The major and minor state can also be encoded as 0,1,2,3,4. With 0=A,1=C,2=G,3=T,4=N

We do not require the positions to be sorted, but we require that the file is grouped by chromosome name.

Internal representation

If a filter file has been supplied as '-sites filter.txt', then ANGSD will parse the entire filter.txt file and generate binary representations and dump these in the outputfiles called

  1. filter.txt.bin
  2. filter.txt.idx

These can be printed using the command

angsd sites print filter.txt

Requirements

The file should be sorted according to column1. This can be achieved by:

sort -k1 unsorted.txt >sorted.txt

Bedfiles?

If you want to use bed files, you need to convert to the native angsd format.

Bedfiles are chr pos1 pos2 value, and are zero indexed. Furthermore pos1 is included, but pos2 is not included. You can convert to angsd format with

awk '{print $1"\t"$2+1"\t"$3}' input.bed >angsd.file