Merge pull request #51 from drneavin/v3.0.0

V3.0.0

Author: drneavin

Singularity.Demuxafy

@@ -4,10 +4,10 @@
 Contact
 =======
 
-.. _preprint: https://www.biorxiv.org/content/10.1101/2022.03.07.483367v1
+.. _publication: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-024-03224-8
 
 Demuxafy has been developed by Drew Neavin in Joseph Powell's Lab at the Garvan Institute of Medical Research.
 
 You can contact us with questions, issues or recommendations with a `Github issue <https://github.com/drneavin/Demultiplexing_Doublet_Detecting_Docs/issues>`__.
 
-If you use this resource, please cite our preprint_.
+If you use this resource, please cite our publication_.

Singularity.Demuxafy.3.0.0_ubuntu2004.def

@@ -77,7 +77,10 @@ For 1000G, use the instructions at the above link to access the data per your pr
 Preparing your own SNP Genotype Data
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-It is best to filter the SNP genotypes for common SNPs (generally > 1% or > 5% minor allele frequency) that overlap exons.
+It is best to filter the SNP genotypes for common SNPs (generally > 1% or > 5% minor allele frequency) that overlap either exons or genes.
+We typically suggest filtering for exons since it typically resultsin in ~250k SNPs to remain following filtering which is sufficient for demultiplexing without using too many SNPs which can slow down the demultiplexing softwares.
+However, some capture types might be better suited to look for SNPs overlapping genes such as single nuclei RNA-seq.
+For relative numbers of SNPs in the exons and introns, see the `issue raised by @jamesnemesh <https://github.com/drneavin/Demultiplexing_Doublet_Detecting_Docs/issues/49#issue-2182195018>`__.
 Here we provide an example of how to do this filtering. 
 We built the required softwares into the singularity image so you can run these filtering steps with the image.
 
@@ -237,7 +240,7 @@ You can download the dataset with one of the following commands:
 
     .. code-block:: bash
 
-      tar -xvf TestData4PipelineFull.tar.gz
+      tar -xvf TestData4PipelineSmall.tar.gz
 
     This should unzip the ``TestData4PipelineSmall`` directory where you will have the following file structure:
 

docs/source/Contact.rst

@@ -4,7 +4,7 @@ Overview of Demultiplexing Softwares
 
 Demultiplexing softwares use the inherent genetic differences between donors multiplexed in a single pool to assign droplets to each donor and to identify doublets.
 There are five demultiplexing softwares that have different capabilities and advantages depending on your dataset.
-As you can see from this table, only :ref:`Demuxlet <Demuxlet-docs>` absolutely requires reference SNP genotypes for the donors multiplexed in your pool.
+As you can see from this table, only :ref:`Demuxlet <Demuxlet-docs>`, :ref:`Demuxalot <Demuxalot-docs>` and :ref:`Dropulation <Dropulation-docs>` absolutely requires reference SNP genotypes for the donors multiplexed in your pool.
 However, :ref:`Souporcell <Souporcell-docs>` and :ref:`Vireo <Vireo-docs>` are also capable of accomodating reference SNP genotypes as well.
 
 +--------------------------------------+------------------------------------------+------------------------------------------+------------------------------------------+
@@ -26,8 +26,9 @@ However, :ref:`Souporcell <Souporcell-docs>` and :ref:`Vireo <Vireo-docs>` are a
 |:ref:`Vireo <Vireo-docs>`             | .. centered:: |:heavy_multiplication_x:| | .. centered:: |:heavy_check_mark:|       | .. centered:: |:heavy_check_mark:|       |
 +--------------------------------------+------------------------------------------+------------------------------------------+------------------------------------------+
 
-We highly recommend using :ref:`Souporcell <Souporcell-docs>` if only to estimate the percentage of ambient RNA in your pool.
+We highly recommend using :ref:`Souporcell <Souporcell-docs>` or :ref:`Vireo <Vireo-docs>` if only to estimate the percentage of ambient RNA in your pool.
+:ref:`Souporcell <Souporcell-docs>` will estimate ambient RNA for the pool as a whole while :ref:`Vireo <Vireo-docs>` will estimate ambient RNA for each cell.
 As far as we are aware, this is the only software that leverages SNP genotype data to estimate ambient RNA in multiplexed pools and it is helpful to identify high ambient RNA which is sometimes undetectable with basic QC metrics.
 We view this as supplementary to other ambient RNA methods that use the transcriptional profile to estimate and remove ambient RNA per droplet.
 
-If you don't know which demultiplexing software(s) to run, take a look at our :ref:`Software Selection Recommendations <SoftwareSelection-docs>` based on your dataset or use our **add widget link here**
+If you don't know which demultiplexing software(s) to run, take a look at our :ref:`Software Selection Recommendations <SoftwareSelection-docs>` based on your dataset or use our `Software Selector and Doublet Estimator Tool <https://demultiplexing-doublet-detecting-docs.readthedocs.io/en/latest/Calculator_final_version.html>`__`

docs/source/DataPrep.rst

@@ -5,7 +5,7 @@ Demuxalot
 ===========================
 
 .. _Demuxalot: https://pypi.org/project/demuxalot/
-.. _preprint: https://www.biorxiv.org/content/10.1101/2022.03.07.483367v1
+.. _publication: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-024-03224-8
 
 
 
@@ -40,6 +40,15 @@ This is the data that you will need to have prepare to run Demuxalot_:
 
       - For example, this is the :download:`individual file <_download_files/Individuals.txt>` for our example dataset
 
+.. admonition:: Optional
+
+    - The SAM tag used in the Bam file to annotate the aligned single cell reads with their corresponding cell barcode (``$CELL_TAG``)
+
+      - If not specified, _Demuxalot defaults to using ``CB`` as that flag is used by Cell Ranger.
+
+    - The SAM tag used in the Bam file to annotate the aligned single cell reads with their corresponding unique molecular identifier (UMI) (``$UMI_TAG``)
+
+      - If not specified, _Demuxalot defaults to using ``UB`` as that flag is used by Cell Ranger.
 
 
 Run Demuxalot
@@ -75,7 +84,7 @@ Demultiplex with Demuxalot
 
   .. tab:: With Refinement
 
-    This will run the first phase of Demuxalot_ as well as the subsequent refinement:
+    This will run the first phase of Demuxalot_ as well as the subsequent refinement, provided an appropriate thread number (``$THREADS``) for your system:
 
     .. code-block:: bash
 
@@ -85,6 +94,9 @@ Demultiplex with Demuxalot
               -n $INDS \
               -v $VCF \
               -o $DEMUXALOT_OUTDIR \
+              -p $THREADS \
+              ${CELL_TAG:+-c $CELL_TAG} \
+              ${UMI_TAG:+-u $UMI_TAG} \
               -r True
 
     .. admonition:: HELP! It says my file/directory doesn't exist!
@@ -94,6 +106,8 @@ Demultiplex with Demuxalot
       This is easy to fix.
       The issue and solution are explained in detail in the :ref:`Notes About Singularity Images <Singularity-docs>`
 
+    Setting ``$THREADS`` to ``-1`` results in Demuxalot_ using all available CPUs/threads.
+
     If Demuxalot_ is successful, you will have these new files in your ``$DEMUXALOT_OUTDIR``:
 
     .. code-block:: bash
@@ -109,7 +123,7 @@ Demultiplex with Demuxalot
 
   .. tab:: Without Refinement
 
-    This will run the first phase of Demuxalot_ only without any refinement:
+    This will run the first phase of Demuxalot_ only without any refinement, provided an appropriate thread number (``$THREADS``) for your system:
 
     .. code-block:: bash
 
@@ -119,6 +133,7 @@ Demultiplex with Demuxalot
               -n $INDS \
               -v $VCF \
               -o $DEMUXALOT_OUTDIR \
+              -p $THREADS \
               -r False
 
     .. admonition:: HELP! It says my file/directory doesn't exist!
@@ -128,6 +143,8 @@ Demultiplex with Demuxalot
       This is easy to fix.
       The issue and solution are explained in detail in the :ref:`Notes About Singularity Images <Singularity-docs>`
 
+    Setting ``$THREADS`` to ``-1`` results in Demuxalot_ using all available CPUs/threads.
+
     If Demuxalot_ is successful, you will have these new files in your ``$DEMUXALOT_OUTDIR``:
 
     .. code-block:: bash
@@ -303,6 +320,6 @@ See :ref:`Combine Results <Combine-docs>`.
 
 Citation
 --------
-If you used the Demuxafy platform for analysis, please reference our preprint_ as well as Demuxalot_.
+If you used the Demuxafy platform for analysis, please reference our publication_ as well as Demuxalot_.
 
 

docs/source/DemultiplexingSoftwares.rst

@@ -5,7 +5,7 @@ Demuxlet
 ===========================
 
 .. _Demuxlet: https://github.com/statgen/popscle
-.. _preprint: https://www.biorxiv.org/content/10.1101/2022.03.07.483367v1
+.. _publication: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-024-03224-8
 
 Demuxlet_ is a genotype demultiplexing software that requires reference genotypes to be available for each individual in the pool. 
 Therefore, if you don't have reference genotypes, you may want to demultiplex with one of the softwares that do not require reference genotype data
@@ -46,6 +46,14 @@ This is the data that you will need to have prepare to run Demuxlet_:
 
       - For example, this is the :download:`individual file <_download_files/Individuals.txt>` for our example dataset
 
+    - The SAM tag used in the Bam file to annotate the aligned single cell reads with their corresponding cell barcode (``$CELL_TAG``)
+
+      - If not specified, _Demuxlet defaults to using ``CB``.
+
+    - The SAM tag used in the Bam file to annotate the aligned single cell reads with their corresponding unique molecular identifier (UMI) (``$UMI_TAG``)
+
+      - If not specified, _Demuxlet defaults to using ``UB``.
+
 
 Run Demuxlet
 ------------
@@ -77,6 +85,8 @@ Popscle Pileup
 
 First we will need to identify the number of reads from each allele at each SNP location.
 
+Please note that the ``\`` at the end of each line is purely for readability to put a separate parameter argument on each line.
+
 .. tabs::
 
   .. tab:: With ``$INDS`` file
@@ -85,7 +95,14 @@ First we will need to identify the number of reads from each allele at each SNP
 
     .. code-block:: bash
 
-      singularity exec Demuxafy.sif popscle dsc-pileup --sam $BAM --vcf $VCF --group-list $BARCODES --out $DEMUXLET_OUTDIR/pileup --sm-list $INDS
+      singularity exec Demuxafy.sif popscle_pileup.py \
+              --sam $BAM \
+              --vcf $VCF \
+              --group-list $BARCODES \
+              --tag-group $CELL_TAG \
+              --tag-UMI $UMI_TAG \
+              --out $DEMUXLET_OUTDIR/pileup \
+              --sm-list $INDS
 
     .. admonition:: HELP! It says my file/directory doesn't exist!
       :class: dropdown
@@ -103,7 +120,14 @@ First we will need to identify the number of reads from each allele at each SNP
 
     .. code-block:: bash
 
-      singularity exec Demuxafy.sif popscle dsc-pileup --sam $BAM --vcf $VCF --group-list $BARCODES --out $DEMUXLET_OUTDIR/pileup
+      singularity exec Demuxafy.sif popscle dsc-pileup \
+              --sam $BAM \
+              --vcf $VCF \
+              --group-list $BARCODES \
+              --tag-UMI $UMI_TAG \
+              --tag-group $CELL_TAG \
+              --out $DEMUXLET_OUTDIR/pileup
+
 
     .. admonition:: HELP! It says my file/directory doesn't exist!
       :class: dropdown
@@ -135,6 +159,7 @@ Popscle Demuxlet
 
 Once you have run ``popscle pileup``, you can demultiplex your samples:
 
+    Please note that the ``\`` at the end of each line is purely for readability to put a separate parameter argument on each line.
 
 .. tabs::
 
@@ -144,7 +169,18 @@ Once you have run ``popscle pileup``, you can demultiplex your samples:
 
     .. code-block:: bash
 
-      singularity exec Demuxafy.sif popscle demuxlet --plp $DEMUXLET_OUTDIR/pileup --vcf $VCF --field $FIELD --group-list $BARCODES --geno-error-coeff 1.0 --geno-error-offset 0.05 --out $DEMUXLET_OUTDIR/demuxlet --sm-list $INDS
+      singularity exec Demuxafy.sif popscle demuxlet \
+              --plp $DEMUXLET_OUTDIR/pileup \
+              --vcf $VCF \
+              --field $FIELD \
+              --group-list $BARCODES \
+              --tag-group $CELL_TAG \
+              --tag-UMI $UMI_TAG \
+              --geno-error-coeff 1.0 \
+              --geno-error-offset 0.05 \
+              --out $DEMUXLET_OUTDIR/demuxlet \
+              --sm-list $INDS
+
 
     .. admonition:: HELP! It says my file/directory doesn't exist!
       :class: dropdown
@@ -161,7 +197,17 @@ Once you have run ``popscle pileup``, you can demultiplex your samples:
 
     .. code-block:: bash
 
-      singularity exec Demuxafy.sif popscle demuxlet --plp $DEMUXLET_OUTDIR/pileup --vcf $VCF --field $FIELD --group-list $BARCODES --geno-error-coeff 1.0 --geno-error-offset 0.05 --out $DEMUXLET_OUTDIR/demuxlet
+      singularity exec Demuxafy.sif popscle demuxlet \
+              --plp $DEMUXLET_OUTDIR/pileup \
+              --vcf $VCF \
+              --field $FIELD \
+              --group-list $BARCODES \
+              --tag-group $CELL_TAG \
+              --tag-UMI $UMI_TAG \
+              --geno-error-coeff 1.0 \
+              --geno-error-offset 0.05 \
+              --out $DEMUXLET_OUTDIR/demuxlet
+              
 
     .. admonition:: HELP! It says my file/directory doesn't exist!
       :class: dropdown
@@ -285,7 +331,7 @@ See :ref:`Combine Results <Combine-docs>`.
 
 Citation
 --------
-If you used the Demuxafy platform for analysis, please reference our preprint_ as well as `Demuxlet <https://www.nature.com/articles/nbt.4042>`__.
+If you used the Demuxafy platform for analysis, please reference our publication_ as well as `Demuxlet <https://www.nature.com/articles/nbt.4042>`__.
 
 
 

docs/source/Demuxalot.rst

@@ -4,7 +4,7 @@ DoubletDecon
 ===========================
 
 .. _DoubletDecon: https://github.com/EDePasquale/DoubletDecon
-.. _preprint: https://www.biorxiv.org/content/10.1101/2022.03.07.483367v1
+.. _publication: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-024-03224-8
 
 DoubletDecon_ is a transcription-based doublet detection software that uses deconvolution to identify doublets using the `R` statistical software.
 We have provided a wrapper script that takes common arguments for DoubletDecon_ and also provide example code for you to run manually if you prefer.
@@ -271,4 +271,4 @@ See :ref:`Combine Results <Combine-docs>`.
 
 Citation
 --------
-If you used the Demuxafy platform for analysis, please reference our preprint_ as well as `DoubletDecon <https://www.sciencedirect.com/science/article/pii/S2211124719312860>`__.
+If you used the Demuxafy platform for analysis, please reference our publication_ as well as `DoubletDecon <https://www.sciencedirect.com/science/article/pii/S2211124719312860>`__.

docs/source/Demuxlet.rst

@@ -21,7 +21,7 @@ The table bellow provides a comparison of the different methods.
 +--------------------------------------------------+------------------------------------------+------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
 | :ref:`scDblFinder <scDblFinder-docs>`            | .. centered:: |:heavy_multiplication_x:| | .. centered:: |:heavy_multiplication_x:| | Gradient boosted trees trained with number neighboring doublets and QC metrics to classify doublets                                                       |
 +--------------------------------------------------+------------------------------------------+------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
-| :ref:`Scds <Scds-docs>`                          | .. centered:: |:heavy_multiplication_x:| | .. centered:: |:heavy_multiplication_x:| | **cxds**: Uses genes pairs that are typically not expressed in the same droplet to rank droplets based on co-expression of all pairs. |br|                 |
+| :ref:`Scds <Scds-docs>`                          | .. centered:: |:heavy_multiplication_x:| | .. centered:: |:heavy_multiplication_x:| | **cxds**: Uses genes pairs that are typically not expressed in the same droplet to rank droplets based on co-expression of all pairs. |br|                |
 |                                                  |                                          |                                          | **bcds**: Uses highly variable genes and simulated doublets to train a binary classification algorithm and return probability of droplet being a doublet. |
 +--------------------------------------------------+------------------------------------------+------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
 | :ref:`Scrublet <Scrublet-docs>`                  | .. centered:: |:heavy_multiplication_x:| | .. centered:: |:heavy_multiplication_x:| | Identifies the number of neighboring simulated doublets for each droplet and uses bimodal distribution of scores to classify singlets and doublets.       |

docs/source/DoubletDecon.rst

@@ -4,7 +4,7 @@ DoubletDetection
 ===========================
 
 .. _DoubletDetection: https://github.com/JonathanShor/DoubletDetection
-.. _preprint: https://www.biorxiv.org/content/10.1101/2022.03.07.483367v1
+.. _publication: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-024-03224-8
 
 
 DoubletDetection_ is a transcription-based doublet detection software.
@@ -405,4 +405,4 @@ See :ref:`Combine Results <Combine-docs>`.
 
 Citation
 --------
-If you used the Demuxafy platform for analysis, please reference our preprint_ as well as `DoubletDetection <https://zenodo.org/record/4359992>`__.
+If you used the Demuxafy platform for analysis, please reference our publication_ as well as `DoubletDetection <https://zenodo.org/record/4359992>`__.