mirror of
https://github.com/chanzuckerberg/cellxgene.git
synced 2026-09-17 05:47:58 +08:00
[WIP] Overhaul the documentation site (#1030)
Overhaul the whole documentation site
This commit is contained in:
Sidney Bell
@@ -88,3 +88,7 @@ We are eager to explore integrations with other computational backends such as [
|
||||
## reuse
|
||||
|
||||
This project was started with the sole goal of empowering the scientific community to explore and understand their data. As such, we encourage other scientific tool builders in academia or industry to adopt the patterns, tools, and code from this project, and reach out to us with ideas or questions. All code is freely available for reuse under the [MIT license](https://opensource.org/licenses/MIT).
|
||||
|
||||
## security
|
||||
|
||||
If you believe you have found a security issue, we would appreciate notification. Please send email to <security@chanzuckerberg.com>.
|
||||
|
||||
@@ -1,11 +1,31 @@
|
||||
theme: jekyll-theme-cayman
|
||||
theme: jekyll-theme-minimal
|
||||
show_downloads: false
|
||||
baseurl: /cellxgene
|
||||
baseurl: ""
|
||||
|
||||
logo: cellxgene-logo.png
|
||||
|
||||
nav:
|
||||
- title: Getting Started
|
||||
url: getting-started.html
|
||||
- title: Data
|
||||
url: data.html
|
||||
- title: FAQ
|
||||
url: faq.html
|
||||
- title: Installation
|
||||
url: posts/install
|
||||
- title: Gallery
|
||||
url: posts/gallery
|
||||
- title: Demo datasets
|
||||
url: posts/demo-data
|
||||
- title: Preparing your data
|
||||
url: posts/prepare
|
||||
- title: Launching cellxgene
|
||||
url: posts/launch
|
||||
- title: Hosting cellxgene
|
||||
url: posts/hosted
|
||||
- title: Annotating data
|
||||
url: posts/annotations
|
||||
- title: Methods
|
||||
url: posts/methods
|
||||
- title: Troubleshooting
|
||||
url: posts/troubleshooting
|
||||
- title: Roadmap
|
||||
url: posts/roadmap
|
||||
- title: Contributing (ideas or code)
|
||||
url: posts/contribute
|
||||
- title: Contact & finding help
|
||||
url: posts/contact
|
||||
|
||||
@@ -1,50 +1,61 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="{{ site.lang | default: "en-US" }}">
|
||||
<head>
|
||||
|
||||
{% if site.google_analytics %}
|
||||
<script async src="https://www.googletagmanager.com/gtag/js?id={{ site.google_analytics }}"></script>
|
||||
<script>
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', '{{ site.google_analytics }}');
|
||||
</script>
|
||||
{% endif %}
|
||||
<meta charset="UTF-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
|
||||
{% seo %}
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="theme-color" content="#157878">
|
||||
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
|
||||
<link rel="shortcut icon" type="image/x-icon" href="cellxgene-favicon.png">
|
||||
<link rel="stylesheet" href="{{ '/assets/css/style.css?v=' | append: site.github.build_revision | relative_url }}">
|
||||
<link rel="stylesheet" href="{{ "/assets/css/style.css?v=" | append: site.github.build_revision | relative_url }}">
|
||||
<!--[if lt IE 9]>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
|
||||
<![endif]-->
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrapper">
|
||||
<header>
|
||||
<img src="{{site.logo | relative_url}}" alt="cellxgene" />
|
||||
|
||||
<p>{{ site.description | default: site.github.project_tagline }}</p>
|
||||
<p>
|
||||
|
||||
{% if page.url == "/" %}
|
||||
<a href="{{ site.baseurl }}/" class="btn"><b>Quick start</b></a><br>
|
||||
{% else %}
|
||||
<a href="{{ site.baseurl }}/" class="btn">Quick start</a><br>
|
||||
{% endif %}
|
||||
|
||||
<header class="page-header" role="banner">
|
||||
<h1 class="project-name">{{ site.title | default: site.github.repository_name }}</h1>
|
||||
<h2 class="project-tagline">{{ site.description | default: site.github.project_tagline }}</h2>
|
||||
{% if site.nav %}
|
||||
<a href="{{ site.baseurl }}/" class="btn">Home</a>
|
||||
{% for item in site.nav %}
|
||||
<a href="{{ item.url }}" class="btn">{{ item.title }}</a>
|
||||
{% if page.url contains item.url %}
|
||||
<a href="{{ item.url | relative_url}}" class="btn"><b>{{ item.title }}</b></a><br>
|
||||
{% else %}
|
||||
<a href="{{ item.url | relative_url}}" class="btn">{{ item.title }}</a><br>
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if site.github.is_project_page %}
|
||||
|
||||
<a href="{{ site.github.repository_url }}" class="btn" target="_blank">Code</a>
|
||||
{% endif %}
|
||||
</p>
|
||||
</header>
|
||||
|
||||
<main id="content" class="main-content" role="main">
|
||||
<section>
|
||||
{{ content }}
|
||||
|
||||
<footer class="site-footer">
|
||||
</section>
|
||||
<footer>
|
||||
{% if site.github.is_project_page %}
|
||||
<span class="site-footer-owner"><a href="{{ site.github.repository_url }}">{{ site.github.repository_name }}</a> is maintained by <a href="{{ site.github.owner_url }}">{{ site.github.owner_name }}</a>.</span>
|
||||
<p>This project is maintained by <a href="{{ site.github.owner_url }}">{{ site.github.owner_name }}</a></p>
|
||||
{% endif %}
|
||||
<span class="site-footer-credits">This page was generated by <a href="https://pages.github.com">GitHub Pages</a>.</span>
|
||||
</footer>
|
||||
</main>
|
||||
</div>
|
||||
<script src="{{ "/assets/js/scale.fix.js" | relative_url }}"></script>
|
||||
{% if site.google_analytics %}
|
||||
<script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
||||
ga('create', '{{ site.google_analytics }}', 'auto');
|
||||
ga('send', 'pageview');
|
||||
</script>
|
||||
{% endif %}
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -2,3 +2,9 @@
|
||||
---
|
||||
|
||||
@import "{{ site.theme }}";
|
||||
body { color: #3b3a3a;}
|
||||
section { width: 500px; float: right; padding-bottom: 50px; padding-top:15px}
|
||||
a { color: #9E2364; text-decoration: none; }
|
||||
a:hover, a:focus { color: #9E2364; font-weight: bold; }
|
||||
a small { font-size: 11px; color: #9E2364; margin-top: -0.3em; display: block; }
|
||||
a:hover small { color: #9E2364; }
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 312 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 285 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 6.0 MiB |
BIN
docs/cellxgene-logo.png
Normal file
BIN
docs/cellxgene-logo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 4.3 KiB |
@@ -1,52 +0,0 @@
|
||||
<svg width="315" height="106" viewBox="0 0 315 106" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<rect width="315" height="106" fill="white"/>
|
||||
<path d="M93.9375 63.375C95.0469 63.375 95.9062 63.0781 96.5156 62.4844C97.125 61.875 97.4219 61.0312 97.4062 59.9531H104.812C104.812 62.7344 103.812 65.0156 101.812 66.7969C99.8281 68.5781 97.2656 69.4688 94.125 69.4688C90.4375 69.4688 87.5312 68.3125 85.4062 66C83.2812 63.6875 82.2188 60.4844 82.2188 56.3906V56.0625C82.2188 53.5 82.6875 51.2422 83.625 49.2891C84.5781 47.3203 85.9453 45.8125 87.7266 44.7656C89.5078 43.7031 91.6172 43.1719 94.0547 43.1719C97.3516 43.1719 99.9688 44.0859 101.906 45.9141C103.844 47.7422 104.812 50.2188 104.812 53.3438H97.4062C97.4062 52.0312 97.0859 51.0234 96.4453 50.3203C95.8047 49.6172 94.9453 49.2656 93.8672 49.2656C91.8203 49.2656 90.625 50.5703 90.2812 53.1797C90.1719 54.0078 90.1172 55.1484 90.1172 56.6016C90.1172 59.1484 90.4219 60.9141 91.0312 61.8984C91.6406 62.8828 92.6094 63.375 93.9375 63.375ZM120.422 69.4688C116.531 69.4688 113.383 68.3125 110.977 66C108.57 63.6719 107.367 60.6484 107.367 56.9297V56.2734C107.367 53.6797 107.844 51.3906 108.797 49.4062C109.766 47.4219 111.172 45.8906 113.016 44.8125C114.859 43.7188 117.047 43.1719 119.578 43.1719C123.141 43.1719 125.953 44.2812 128.016 46.5C130.078 48.7031 131.109 51.7812 131.109 55.7344V58.8047H115.406C115.688 60.2266 116.305 61.3438 117.258 62.1562C118.211 62.9688 119.445 63.375 120.961 63.375C123.461 63.375 125.414 62.5 126.82 60.75L130.43 65.0156C129.445 66.375 128.047 67.4609 126.234 68.2734C124.438 69.0703 122.5 69.4688 120.422 69.4688ZM119.531 49.2656C117.219 49.2656 115.844 50.7969 115.406 53.8594H123.375V53.25C123.406 51.9844 123.086 51.0078 122.414 50.3203C121.742 49.6172 120.781 49.2656 119.531 49.2656ZM142.641 69H134.719V33H142.641V69ZM155.859 69H147.938V33H155.859V69ZM185.414 56.1328C185.414 53.5703 185.836 51.3047 186.68 49.3359C187.523 47.3672 188.734 45.8516 190.312 44.7891C191.891 43.7109 193.727 43.1719 195.82 43.1719C198.461 43.1719 200.523 44.0625 202.008 45.8438L202.289 43.6406H209.461V68.0391C209.461 70.2734 208.938 72.2109 207.891 73.8516C206.859 75.5078 205.359 76.7734 203.391 77.6484C201.422 78.5391 199.141 78.9844 196.547 78.9844C194.688 78.9844 192.883 78.6328 191.133 77.9297C189.398 77.2266 188.07 76.3125 187.148 75.1875L190.453 70.5469C191.922 72.2812 193.859 73.1484 196.266 73.1484C199.766 73.1484 201.516 71.3516 201.516 67.7578V66.9609C200 68.6328 198.086 69.4688 195.773 69.4688C192.68 69.4688 190.18 68.2891 188.273 65.9297C186.367 63.5547 185.414 60.3828 185.414 56.4141V56.1328ZM193.336 56.625C193.336 58.7031 193.727 60.3516 194.508 61.5703C195.289 62.7734 196.383 63.375 197.789 63.375C199.492 63.375 200.734 62.8047 201.516 61.6641V51C200.75 49.8438 199.523 49.2656 197.836 49.2656C196.43 49.2656 195.328 49.8984 194.531 51.1641C193.734 52.4141 193.336 54.2344 193.336 56.625ZM226.219 69.4688C222.328 69.4688 219.18 68.3125 216.773 66C214.367 63.6719 213.164 60.6484 213.164 56.9297V56.2734C213.164 53.6797 213.641 51.3906 214.594 49.4062C215.562 47.4219 216.969 45.8906 218.812 44.8125C220.656 43.7188 222.844 43.1719 225.375 43.1719C228.938 43.1719 231.75 44.2812 233.812 46.5C235.875 48.7031 236.906 51.7812 236.906 55.7344V58.8047H221.203C221.484 60.2266 222.102 61.3438 223.055 62.1562C224.008 62.9688 225.242 63.375 226.758 63.375C229.258 63.375 231.211 62.5 232.617 60.75L236.227 65.0156C235.242 66.375 233.844 67.4609 232.031 68.2734C230.234 69.0703 228.297 69.4688 226.219 69.4688ZM225.328 49.2656C223.016 49.2656 221.641 50.7969 221.203 53.8594H229.172V53.25C229.203 51.9844 228.883 51.0078 228.211 50.3203C227.539 49.6172 226.578 49.2656 225.328 49.2656ZM247.359 43.6406L247.617 46.6172C249.367 44.3203 251.781 43.1719 254.859 43.1719C257.5 43.1719 259.469 43.9609 260.766 45.5391C262.078 47.1172 262.758 49.4922 262.805 52.6641V69H254.883V52.9922C254.883 51.7109 254.625 50.7734 254.109 50.1797C253.594 49.5703 252.656 49.2656 251.297 49.2656C249.75 49.2656 248.602 49.875 247.852 51.0938V69H239.953V43.6406H247.359ZM279.469 69.4688C275.578 69.4688 272.43 68.3125 270.023 66C267.617 63.6719 266.414 60.6484 266.414 56.9297V56.2734C266.414 53.6797 266.891 51.3906 267.844 49.4062C268.812 47.4219 270.219 45.8906 272.062 44.8125C273.906 43.7188 276.094 43.1719 278.625 43.1719C282.188 43.1719 285 44.2812 287.062 46.5C289.125 48.7031 290.156 51.7812 290.156 55.7344V58.8047H274.453C274.734 60.2266 275.352 61.3438 276.305 62.1562C277.258 62.9688 278.492 63.375 280.008 63.375C282.508 63.375 284.461 62.5 285.867 60.75L289.477 65.0156C288.492 66.375 287.094 67.4609 285.281 68.2734C283.484 69.0703 281.547 69.4688 279.469 69.4688ZM278.578 49.2656C276.266 49.2656 274.891 50.7969 274.453 53.8594H282.422V53.25C282.453 51.9844 282.133 51.0078 281.461 50.3203C280.789 49.6172 279.828 49.2656 278.578 49.2656Z" fill="#E9429A" fill-opacity="0.5"/>
|
||||
<path d="M160.617 61.2891L168.352 53.4141L160.664 45.5625L163.453 42.6797L171.164 50.5312L178.875 42.6797L181.664 45.5625L173.977 53.4141L181.711 61.2891L178.922 64.1719L171.164 56.2734L163.406 64.1719L160.617 61.2891Z" fill="white" fill-opacity="0.5"/>
|
||||
<g filter="url(#filter0_d)">
|
||||
<rect x="22" y="27" width="48" height="48" rx="3" fill="#E9429A"/>
|
||||
<rect x="22.5" y="27.5" width="47" height="47" rx="2.5" stroke="white"/>
|
||||
</g>
|
||||
<mask id="mask0" mask-type="alpha" maskUnits="userSpaceOnUse" x="22" y="27" width="48" height="48">
|
||||
<rect x="22" y="27" width="48" height="48" rx="3" fill="white"/>
|
||||
</mask>
|
||||
<g mask="url(#mask0)">
|
||||
<rect x="37" y="42" width="28" height="28" fill="white"/>
|
||||
<rect x="42" y="47" width="18" height="18" fill="#E9429A"/>
|
||||
</g>
|
||||
<rect x="27" y="42" width="6" height="28" fill="white"/>
|
||||
<rect x="37" y="32" width="28" height="6" fill="white"/>
|
||||
<g filter="url(#filter1_d)">
|
||||
<rect x="22" y="27" width="48" height="48" rx="3" fill="#E9429A"/>
|
||||
<rect x="23" y="28" width="46" height="46" rx="2" stroke="white" stroke-width="2"/>
|
||||
</g>
|
||||
<mask id="mask1" mask-type="alpha" maskUnits="userSpaceOnUse" x="22" y="27" width="48" height="48">
|
||||
<rect x="22" y="27" width="48" height="48" rx="3" fill="white"/>
|
||||
</mask>
|
||||
<g mask="url(#mask1)">
|
||||
<rect x="41" y="46" width="22" height="22" fill="white"/>
|
||||
<rect x="46" y="51" width="12" height="12" fill="#E9429A"/>
|
||||
</g>
|
||||
<rect x="29" y="46" width="7" height="22" fill="white"/>
|
||||
<rect x="41" y="34" width="22" height="7" fill="white"/>
|
||||
<path d="M160.586 64.2891L168.32 56.4141L160.633 48.5625L163.422 45.6797L171.133 53.5312L178.844 45.6797L181.633 48.5625L173.945 56.4141L181.68 64.2891L178.891 67.1719L171.133 59.2734L163.375 67.1719L160.586 64.2891Z" fill="#E9429A" fill-opacity="0.5"/>
|
||||
<defs>
|
||||
<filter id="filter0_d" x="18" y="23" width="60" height="60" filterUnits="userSpaceOnUse" color-interpolation-filters="sRGB">
|
||||
<feFlood flood-opacity="0" result="BackgroundImageFix"/>
|
||||
<feColorMatrix in="SourceAlpha" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0"/>
|
||||
<feOffset dx="2" dy="2"/>
|
||||
<feGaussianBlur stdDeviation="3"/>
|
||||
<feColorMatrix type="matrix" values="0 0 0 0 0.572052 0 0 0 0 0.572052 0 0 0 0 0.572052 0 0 0 0.25 0"/>
|
||||
<feBlend mode="normal" in2="BackgroundImageFix" result="effect1_dropShadow"/>
|
||||
<feBlend mode="normal" in="SourceGraphic" in2="effect1_dropShadow" result="shape"/>
|
||||
</filter>
|
||||
<filter id="filter1_d" x="18" y="23" width="60" height="60" filterUnits="userSpaceOnUse" color-interpolation-filters="sRGB">
|
||||
<feFlood flood-opacity="0" result="BackgroundImageFix"/>
|
||||
<feColorMatrix in="SourceAlpha" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0"/>
|
||||
<feOffset dx="2" dy="2"/>
|
||||
<feGaussianBlur stdDeviation="3"/>
|
||||
<feColorMatrix type="matrix" values="0 0 0 0 0.572052 0 0 0 0 0.572052 0 0 0 0 0.572052 0 0 0 0.25 0"/>
|
||||
<feBlend mode="normal" in2="BackgroundImageFix" result="effect1_dropShadow"/>
|
||||
<feBlend mode="normal" in="SourceGraphic" in2="effect1_dropShadow" result="shape"/>
|
||||
</filter>
|
||||
</defs>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 7.7 KiB |
108
docs/data.md
108
docs/data.md
@@ -1,108 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: data
|
||||
description: Data
|
||||
---
|
||||
|
||||
# Using `cellxgene prepare`
|
||||
|
||||
#### What is `cellxgene prepare`?
|
||||
|
||||
`prepare` offers an easy command line interface (CLI) to preliminarily wrangle your data into the required format for previewing it with `cellxgene`.
|
||||
|
||||
#### What is `cellxgene prepare` _not_?
|
||||
|
||||
`cellxgene prepare` is not meant as a way to formally process or analyze your data. It's simply a utility for quickly wrangling your data into cellxgene-compatible format and computing a "vanilla" embedding so you can try out `cellxgene` and get a general sense of a dataset.
|
||||
|
||||
#### How do I install `cellxgene prepare`?
|
||||
|
||||
The `cellxgene prepare` command is an optional install that you can install alongside `cellxgene launch` by running
|
||||
|
||||
```
|
||||
pip install cellxgene[prepare]
|
||||
```
|
||||
|
||||
#### What input formats does it accept?
|
||||
|
||||
Currently, we accept `h5ad` and `loom` files, as well as `10x` directories, and are hoping to accept more formats in the future.
|
||||
|
||||
While we'd like to support quick conversion from seurat and bioconductor, these packages don't currently output a python-parseable intermediate file type. In the meantime, you might check out the [converters](https://satijalab.org/seurat/v3.0/conversion_vignette.html) that are under early development.
|
||||
|
||||
#### What can `cellxgene prepare` do?
|
||||
|
||||
`prepare` uses scanpy to:
|
||||
|
||||
- Handle simple data normalization (from a [recipe](https://www.pydoc.io/pypi/scanpy-0.2.3/autoapi/preprocessing/recipes/index.html))
|
||||
- Do basic preprocessing to run PCA and compute the neighbor graph
|
||||
- Infer clusters
|
||||
- Reduce dimensionality to generate embeddings.
|
||||
You can control which steps to run and their methods (when applicable), via the CLI. The CLI also includes options for computing QC metrics, enforcing matrix sparcity, specifying index names, and plotting output.
|
||||
|
||||
**To see a full list of available arguments and options, run `cellxgene prepare --help`.**
|
||||
|
||||
#### How do I use `cellxgene prepare`?
|
||||
|
||||
As a quick example, let's construct a command to use `prepare` to take a raw expression matrix and generate a processed `h5ad` ready to visualize with cellxgene.
|
||||
|
||||
We'll start off using the raw data from the pbmc3k dataset. This dataset is described [here](https://icb-scanpy.readthedocs-hosted.com/en/stable/api/scanpy.datasets.pbmc3k.html), and is available as part of the scanpy API. For this example, we'll assume this raw data is stored in a file called `pbmc3k-raw.h5ad`.
|
||||
|
||||
Our `prepare` compose our command looks like this:
|
||||
|
||||
```
|
||||
cellxgene prepare pbmc3k-raw.h5ad \
|
||||
--run-qc \ # (A)
|
||||
--recipe seurat \ # (B)
|
||||
--layout tsne --layout umap \ # (C)
|
||||
--output pbmc3k-prepared.h5ad # (D)
|
||||
```
|
||||
|
||||
Let's look at what `prepare` is doing to our data, and how each step relates to the command above. You can see a walkthrough of what's going on under the hood for this example in [this notebook](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/pbmc3k-prepare-example.ipynb).
|
||||
|
||||
**1 - Compute quality control metrics and store this in our `AnnData` object for later inspection (A)**
|
||||
**2 - Normalize the expression matrix using a basic preprocessing recipe (B)**
|
||||
**3 - Do some preprocessing to run PCA and compute the neighbor graph (auto)**
|
||||
**4 - Infer clusters with the Louvain algorithm and store these labels to visualize later (auto)**
|
||||
**5 - Compute and store umap and tsne embeddings (C)**
|
||||
**6 - Write results to file (D)**
|
||||
|
||||
# Example datasets to use with cellxgene
|
||||
|
||||
**To download and use these datasets, run:**
|
||||
`curl -O [URL]`
|
||||
`unzip [filename.zip]`
|
||||
`cellxgene launch [filename.h5ad] --open`
|
||||
|
||||
### Peripheral blood mononuclear cells
|
||||
|
||||
Healthy human PBMCs (10X).
|
||||
|
||||
- Source: [10X genomics](https://support.10xgenomics.com/single-cell-gene-expression/datasets/1.1.0/pbmc3k)
|
||||
- Cells: 2,638
|
||||
- File size: 19MB
|
||||
- [Raw data](http://cf.10xgenomics.com/samples/cell-exp/1.1.0/pbmc3k/pbmc3k_filtered_gene_bc_matrices.tar.gz)
|
||||
- [Processing](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/pbmc3k-processing.ipynb)
|
||||
- Download: `curl -O https://cellxgene-example-data.czi.technology/pbmc3k.h5ad.zip`
|
||||
|
||||
### Tabula muris
|
||||
|
||||
20 organs and tissues from healthy mice (Smart-Seq2).
|
||||
Rich metadata and annotations.
|
||||
|
||||
- Source: [bioRxiv, CZBiohub](https://www.biorxiv.org/content/10.1101/237446v2)
|
||||
- Cells: 45,423
|
||||
- File size: 174MB
|
||||
- [Raw data](https://figshare.com/projects/Tabula_Muris_Transcriptomic_characterization_of_20_organs_and_tissues_from_Mus_musculus_at_single_cell_resolution/27733)
|
||||
- [Processing](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/tabula-muris-processing.ipynb)
|
||||
- Download: `curl -O https://cellxgene-example-data.czi.technology/tabula-muris.h5ad.zip`
|
||||
|
||||
### Tabula muris senis
|
||||
|
||||
22 organs and tissues from healthy mice at ages 3mo, 18mo, 21mo, and 24mo (Smart-Seq2).
|
||||
Rich metadata and annotations.
|
||||
|
||||
- Source: [bioRxiv, CZBiohub](https://www.biorxiv.org/content/10.1101/661728v1)
|
||||
- Cells: 81,478
|
||||
- File size: 3.9GB
|
||||
- Raw data [geo link coming soon!]
|
||||
- [Processing](https://www.biorxiv.org/content/10.1101/661728v1)
|
||||
- Download: `curl -O https://cellxgene-example-data.czi.technology/tabula-muris-senis.h5ad.zip`
|
||||
BIN
docs/diffexp.gif
BIN
docs/diffexp.gif
Binary file not shown.
|
Before Width: | Height: | Size: 22 MiB |
130
docs/faq.md
130
docs/faq.md
@@ -1,130 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: FAQ
|
||||
description: Frequently Asked Questions
|
||||
---
|
||||
|
||||
# Data formatting
|
||||
|
||||
#### What file formats can I use with _cellxgene_?
|
||||
|
||||
Currently, you can go straight into `cellxgene launch` with your own analyzed data in h5ad format, after you have performed dimenstionality reduction (tsne, umap) and clustering (louvain).
|
||||
|
||||
If your data is in a different format, and/or you still need to perform dimensionality reduction and clustering, `cellxgene` can do that for you with the `prepare` command. `cellxgene prepare` runs `scanpy` under the hood and can read in any format that is currently supported by `scanpy` (including mtx, loom, and more listed [here](https://scanpy.readthedocs.io/en/latest/api/index.html#reading)).
|
||||
|
||||
To add `cellxgene prepare` to your cellxgene installation run `pip install cellxgene[prepare]`.
|
||||
|
||||
The output of `cellxgene prepare` is a h5ad file with your computed clusters and tsne/umap projections that can be used in `cellxgene launch`.
|
||||
|
||||
#### I have a directory of 10X-Genomics data with _mtx_ files and I've never used _scanpy_, can I use _cellxgene_?
|
||||
|
||||
Yep! This should only take a couple steps. We'll assume your data is in a folder called `data/` and you've successfully installed `cellxgene` with the `prepare` packages as described above. Just run
|
||||
|
||||
```
|
||||
cellxgene prepare data/ --output=data-processed.h5ad --embedding=umap
|
||||
```
|
||||
|
||||
Depending on the size of the dataset, this may take some time. Once it's done, call
|
||||
|
||||
```
|
||||
cellxgene launch data-processed.h5ad --embedding=umap --open
|
||||
```
|
||||
|
||||
And your web browser should open with an interactive view of your data.
|
||||
|
||||
#### I have extra metadata that I want to add to my dataset
|
||||
|
||||
Currently this is not supported directly, but you should be able to do this yourself using `scanpy`. For example, this [notebook](https://github.com/falexwolf/fun-analyses/blob/master/tabula_muris/tabula_muris.ipynb) shows adding the contents of a `csv` file with metadata to an `anndata` object. For now, you could do this manually on your data in the same way and then save out the result before loading into `cellxgene`.
|
||||
|
||||
#### What part of the _anndata_ objects does cellxgene pull in for visualization?
|
||||
|
||||
- `.obs` and `.var` annotations are use to extract metadata for filtering
|
||||
- `.X` is used to display expression (histograms, scatterplot & colorscale) and to compute differential expression
|
||||
- `.obsm` is used for embedding(s). If an embedding has more than two components, the first two will be used for visualization.
|
||||
|
||||
#### I have a BIG dataset - how can I make cellxgene run as fast as possible?
|
||||
|
||||
If your dataset requires gigabytes of disk space, you may need to select an appropriate storage format in order to effectively utilize `cellxgene`. Tips and tricks:
|
||||
|
||||
- `cellxgene` is optimized for columnar data access. For large datasets, format the expression matrix (`.X`) as either a [SciPy CSC sparse matrix](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csc_matrix.html) or a dense Numpy array (whichever creates a smaller `h5ad` file). If you are using `cellxgene prepare`, include the `--sparse` flag to ensure `.X` is formatted as a CSC sparse matrix (by default, `.X` will be a dense matrix).
|
||||
- `cellxgene` start time is directly proportional to `h5ad` file size and the speed of your file system. Expect that large (eg, million cell) datasets will take minutes to load, even on relatively fast computers with a high performance local hard drive. Once loaded, exploring metadata should still be quick.
|
||||
- If your dataset size exceeds the size of memory (RAM) on the host computer, differential expression calculations will be extremely slow (or fail, if you run out of virtual memory).
|
||||
|
||||
# Algorithms
|
||||
|
||||
#### How are you computing and sorting differential expression results?
|
||||
|
||||
We use a [Welch's _t_-test](https://en.wikipedia.org/wiki/Welch%27s_t-test) implementation including the same variance overestimation correction as used in `scanpy`. We sort the `tscore` to identify the top N genes, and then filter to remove any that fall below a cutoff log fold change value, which can help remove spurious test results. The default threshold is `0.01` and can be changed using the option `--diffexp-lfc-cutoff`.
|
||||
|
||||
# Problems, errors, & bugs
|
||||
|
||||
#### How do I create a Python environment for _cellxgene_?
|
||||
|
||||
If you use conda and want to create a [conda environment](https://conda.io/docs/user-guide/tasks/manage-environments.html) for _cellxgene_ you can use the following commands
|
||||
|
||||
```
|
||||
conda create --yes -n cellxgene python=3.7
|
||||
conda activate cellxgene
|
||||
pip install cellxgene
|
||||
```
|
||||
|
||||
Or you can create a virtual environment by using
|
||||
|
||||
```
|
||||
ENV_NAME=cellxgene
|
||||
python3.7 -m venv ${ENV_NAME}
|
||||
source ${ENV_NAME}/bin/activate
|
||||
pip install cellxgene
|
||||
```
|
||||
|
||||
#### I ran _prepare_ and I'm getting results that look unexpected
|
||||
|
||||
You might want to try running one of the preprocessing recipes included with `scanpy` (read more about them [here](https://scanpy.readthedocs.io/en/latest/api/index.html#recipes)). You can specify this with the `--recipe` option, such as
|
||||
|
||||
```
|
||||
cellxgene prepare data/ --output=data-processed.h5ad --recipe=zheng17
|
||||
```
|
||||
|
||||
It should be easy to run `prepare` then call `cellxgene launch` a few times with different settings to explore different behaviors. We may explore adding other preprocessing options in the future.
|
||||
|
||||
#### I tried to `pip install cellxgene` and got a weird error I don't understand
|
||||
|
||||
This may happen, especially as we work out bugs in our installation process! Please create a new [Github issue](https://github.com/chanzuckerberg/cellxgene/issues), explain what you did, and include all the error messages you saw. It'd also be super helpful if you call `pip freeze` and include the full output alongside your issue.
|
||||
|
||||
#### I'm following the developer instructions and get an error about "missing files and directories” when trying to build the client
|
||||
|
||||
This is likely because you do not have node and npm installed, we recommend using [nvm](https://github.com/creationix/nvm) if you're new to using these tools.
|
||||
|
||||
# Data access
|
||||
|
||||
#### Can I use a _s3:_ or _gs:_ URL with `cellxgene launch`?
|
||||
|
||||
Yes. Support for S3 and GCS is not enabled by default. If you wish to directly access S3 or GFS, install one or both of the following packages using `pip`:
|
||||
|
||||
- [s3fs](https://s3fs.readthedocs.io/en/latest/) for S3 support
|
||||
- [gcsfs](https://gcsfs.readthedocs.io/en/latest/) for GCS support
|
||||
|
||||
For example:
|
||||
|
||||
```
|
||||
pip install s3fs
|
||||
cellxgene launch s3://mybucket.s3-us-west-2.amazonaws.com/mydata.h5ad
|
||||
```
|
||||
|
||||
#### What does the command line option `--backed` do?
|
||||
|
||||
The `--backed` option instructs `cellxgene launch` to read the H5AD file in "backed" mode (for more information, see the
|
||||
[anndata.read_h5ad() documentation](https://anndata.readthedocs.io/en/latest/anndata.read_h5ad.html#anndata.read_h5ad)).
|
||||
|
||||
By default, cellxgene will read the entire H5AD will be into memory at startup, improving application speed and performance.
|
||||
Very large datasets may not fit in memory. The "--backed" mode will read the file incrementally, reducing memory
|
||||
use, and for large files, improving startup speed. _However_, this option will also significantly slow
|
||||
down access to gene expression histograms, and may render differential expression calculations too slow
|
||||
to use (see `--disable-diffexp` for an option to disable this feature).
|
||||
|
||||
#### What does the command line option `--disable-diffexp` do?
|
||||
|
||||
The `--disable-diffexp` option will disable and hide the `Compute Differential Expression` feature.
|
||||
For large datasets, or datasets loaded with the `--backed` option, computing differential expression may
|
||||
be extremely slow or use excessive reources on the host computer (eg, memory thrasing).
|
||||
Disabling the feature will ensure that the end-user does not accidentally initiate this computation.
|
||||
@@ -1,122 +0,0 @@
|
||||
## getting started
|
||||
|
||||
You'll need **python 3.6+** and **Google Chrome**.
|
||||
The web UI is tested on OSX and Windows using Chrome, and the python CLI is tested on OSX and Ubuntu (via WSL/Windows). It should work on other platforms, but if you run into trouble let us know.
|
||||
|
||||
To install run
|
||||
|
||||
```
|
||||
pip install cellxgene
|
||||
```
|
||||
|
||||
To start exploring a dataset call
|
||||
|
||||
```
|
||||
cellxgene launch dataset.h5ad --open
|
||||
```
|
||||
|
||||
If you want an example dataset download [this file](https://github.com/chanzuckerberg/cellxgene/raw/master/example-dataset/pbmc3k.h5ad) and then call
|
||||
|
||||
```
|
||||
cellxgene launch pbmc3k.h5ad --open
|
||||
```
|
||||
|
||||
You can also directly specify URLs as a data source, and the data will be downloaded during launch
|
||||
|
||||
```
|
||||
cellxgene launch https://raw.githubusercontent.com/chanzuckerberg/cellxgene/master/example-dataset/pbmc3k.h5ad
|
||||
```
|
||||
|
||||
On Mac OS and Ubuntu, you should see your web browser open with the following
|
||||
|
||||
<img width="450" src="https://raw.githubusercontent.com/chanzuckerberg/cellxgene/master/docs/cellxgene-opening-screenshot.png" pad="50px">
|
||||
|
||||
**Note**: automatic opening of the browser with the `--open` flag only works on OS X, on other platforms you'll need to directly point to the provided link in your browser.
|
||||
|
||||
There are several options available, such as:
|
||||
|
||||
- `--embedding` to restrict available emdeddings in the UI, eg, `tsne`, `umap`, `diffmap`, `phate`, `draw_graph_fa`, or `draw_graph_fr`
|
||||
- `--title` to show a title on the explorer
|
||||
- `--open` to automatically open the web browser after launching (OS X only)
|
||||
|
||||
To see all options call
|
||||
|
||||
```
|
||||
cellxgene launch --help
|
||||
```
|
||||
|
||||
There is an additional subcommand called `cellxgene prepare` that takes an existing dataset in one of several formats and applies minimal preprocessing and reformatting so that `launch` can use it (see [the next section](#data-formatting) for more info on `prepare`).
|
||||
|
||||
## data formatting
|
||||
|
||||
### requirements
|
||||
|
||||
The `launch` command assumes that the data is stored in the `.h5ad` format from the [`anndata`](https://anndata.readthedocs.io/en/latest/index.html) library. It also assumes that certain computations have already been performed. Briefly, the `.h5ad` format wraps a two-dimensional `ndarray` and stores additional metadata as "annotations" for either observations (referred to as `obs` and `obsm`) or variables (`var` and `varm`). `cellxgene launch` makes the following assumptions about your data (we recommend loading and inspecting your data using `scanpy` to validate these assumptions)
|
||||
|
||||
- an `obs` field has a unique identifier for every cell (you can specify which field to use with the `--obs-names` option, by default it will use the value of `data.obs_names`)
|
||||
- a `var` field has a unique identifier for every gene (you can specify which field to use with the `--var-names` option, by default it will use the value of `data.var_names`)
|
||||
- an `obsm` field contains the two-dimensional coordinates for the embedding that you want to render (e.g. `X_umap` for the `umap` embedding)
|
||||
- any additional `obs` fields will be rendered as per-cell continuous or categorical metadata by the app (e.g. `louvain` cluster assignments)
|
||||
|
||||
### prepare
|
||||
|
||||
The `prepare` command is included to help you format your data. It uses `scanpy` under the hood. This is especially useful if you are starting with raw unanalyzed data and are unfamiliar with `scanpy`.
|
||||
|
||||
To install `cellxgene prepare` alongside `cellxgene launch`
|
||||
|
||||
```
|
||||
pip install cellxgene[prepare]
|
||||
```
|
||||
|
||||
If the aforementioned optional package installation fails, you can also install these packages directly:
|
||||
|
||||
```
|
||||
pip install scanpy>=1.3.7 python-igraph louvain>=0.6
|
||||
```
|
||||
|
||||
To prepare from an existing `.h5ad` file use
|
||||
|
||||
```
|
||||
cellxgene prepare dataset.h5ad --output=dataset-processed.h5ad
|
||||
```
|
||||
|
||||
This will load the input data, perform PCA and nearest neighbor calculations, compute `umap` and `tsne` embeddings and `louvain` cluster assignments, and save the results in a new file called `dataset-processed.h5ad` that can be loaded using `cellxgene launch`. Data can be loaded from several formats, including `.h5ad` `.loom` and a `10-Genomics-formatted` `mtx` directory. Several options are available, including running one of the preprocessing `recipes` included with `scanpy`, which include steps like cell filtering and gene selection. To learn more about the `recipes` please see the `scanpy` [documentation](https://scanpy.readthedocs.io/en/latest/api/index.html#recipes).
|
||||
|
||||
Depending on the options chosen, `prepare` can take a long time to run (a few minutes for datasets with 10-100k cells, up to an hour or more for datasets with >100k cells). If you want `prepare` to run faster we recommend using the `sparse` option and only computing the embedding for `umap`, using a call like this
|
||||
|
||||
```
|
||||
cellxgene prepare dataset.h5ad --output=dataset-processed.h5ad --embedding=umap --sparse
|
||||
```
|
||||
|
||||
To see all options call
|
||||
|
||||
```
|
||||
cellxgene prepare --help
|
||||
```
|
||||
|
||||
## conda and virtual environments
|
||||
|
||||
If you use conda and want to create a conda environment for `cellxgene` you can use the following commands
|
||||
|
||||
```
|
||||
conda create --yes -n cellxgene python=3.7
|
||||
conda activate cellxgene
|
||||
pip install cellxgene
|
||||
```
|
||||
|
||||
Or you can create a virtual environment by using
|
||||
|
||||
```
|
||||
ENV_NAME=cellxgene
|
||||
python3.7 -m venv ${ENV_NAME}
|
||||
source ${ENV_NAME}/bin/activate
|
||||
pip install cellxgene
|
||||
```
|
||||
|
||||
## docker
|
||||
|
||||
We have included a dockerfile to conveniently run cellxgene from docker.
|
||||
|
||||
1. Build the image `docker build . -t cellxgene`
|
||||
2. Run the container and mount data `docker run -v "$PWD/example-dataset/:/data/" -p 5005:5005 cellxgene launch --host 0.0.0.0 data/pbmc3k.h5ad`
|
||||
- You will need to use --host 0.0.0.0 to have the container listen to incoming requests from the browser
|
||||
BIN
docs/images/category-breakdown.gif
Normal file
BIN
docs/images/category-breakdown.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 6.6 MiB |
|
Before Width: | Height: | Size: 644 KiB After Width: | Height: | Size: 644 KiB |
BIN
docs/images/compare-genes.gif
Normal file
BIN
docs/images/compare-genes.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 3.7 MiB |
BIN
docs/images/crossfilter.gif
Normal file
BIN
docs/images/crossfilter.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 6.6 MiB |
BIN
docs/images/diffexp.gif
Normal file
BIN
docs/images/diffexp.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 5.8 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 18 MiB After Width: | Height: | Size: 12 MiB |
BIN
docs/images/switch-embeddings.gif
Normal file
BIN
docs/images/switch-embeddings.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 3.5 MiB |
@@ -1,17 +1,11 @@
|
||||
<img src="cellxgene-logo.svg" width="300">
|
||||
|
||||
_an interactive explorer for single-cell transcriptomics data_
|
||||
---
|
||||
title: Index
|
||||
subtitle: Index
|
||||
layout: default
|
||||
---
|
||||
# Quick start
|
||||
|
||||
Whether you need to visualize one thousand cells or one million, cellxgene helps you gain insight into your single-cell data.
|
||||
## features
|
||||
|
||||
#### flexible selections, coloring, and differential expression of your selected sets of cells
|
||||
<img src="diffexp.gif" width="600"/>
|
||||
|
||||
#### single-gene analyses (e.g. expression analysis)
|
||||
<img src="customGene.gif" width="600" />
|
||||
|
||||
## quick start
|
||||
|
||||
To install cellxgene you need Python 3.6+. We recommend [installing cellxgene into a conda or virtual environment.](/faq.html#how-do-i-create-a-python-environment-for-cellxgene)
|
||||
|
||||
@@ -23,19 +17,20 @@ pip install cellxgene
|
||||
Download an example [anndata](https://anndata.readthedocs.io/en/latest/) file
|
||||
|
||||
``` bash
|
||||
curl -o pbmc3k.h5ad https://raw.githubusercontent.com/chanzuckerberg/cellxgene/master/example-dataset/pbmc3k.h5ad
|
||||
curl -o tabula-muris.h5ad https://cellxgene-example-data.czi.technology/tabula-muris.h5ad.zip
|
||||
unzip tabula-muris.h5ad.zip
|
||||
```
|
||||
|
||||
Launch cellxgene
|
||||
``` bash
|
||||
cellxgene launch pbmc3k.h5ad --open
|
||||
cellxgene launch tabula-muris.h5ad --open
|
||||
```
|
||||
|
||||
To explore more datasets already formatted for cellxgene, see [Data](data) or
|
||||
visit [Getting Started](getting-started) to learn more about formatting your own
|
||||
To explore more datasets already formatted for cellxgene, check out the [Demo data](demo-data) or
|
||||
see [Preparing your data](prepare) to learn more about formatting your own
|
||||
data for cellxgene.
|
||||
|
||||
## getting help
|
||||
# Getting help
|
||||
|
||||
We'd love to hear from you!
|
||||
|
||||
|
||||
91
docs/posts/annotations.md
Normal file
91
docs/posts/annotations.md
Normal file
@@ -0,0 +1,91 @@
|
||||
---
|
||||
layout: default
|
||||
title: annotations
|
||||
description: Creating annotations
|
||||
---
|
||||
|
||||
# Creating annotations in cellxgene
|
||||
|
||||
We are _piloting_ a new feature in cellxgene that enables users to create and edit categorical annotations within the app. We'd love for you to try it out and [give us feedback](contact)!
|
||||
|
||||
## Quick start for annotations (RECOMMENDED for most users)
|
||||
|
||||
You can enable this experimental feature like so:
|
||||
|
||||
`cellxgene launch mydata.h5ad --experimental-annotations`
|
||||
|
||||
To preserve data provenance, **`cellxgene` does not alter the input h5ad file**. Rather, newly-created annotations are saved in a specified CSV file:
|
||||
- You will be prompted to enter a name for your annotations the first time you create a new category.
|
||||
- We also assign a unique identifier in the form of an 8-character suffix, `########`; this helps cellxgene identify your file to avoid overwriting your work.
|
||||
- Any annotations you create in the application will be autosaved in `cwd/name-########.csv`, where `cwd` is your current working directory (i.e., the directory you were in when you started cellxgene).
|
||||
|
||||
If you quit cellxgene and relaunch it with the same h5ad, we will check for this annotations csv and load it in editable mode alongside.
|
||||
|
||||
## Data management
|
||||
|
||||
### Loading, editing and updating existing draft annotations
|
||||
|
||||
If you'd like to specify the complete file path for your annotations, you can do so by running:
|
||||
```
|
||||
cellxgene launch mydata.h5ad --experimental-annotations-file path/to/myfile.csv
|
||||
```
|
||||
|
||||
If this file already exists and contains compatible annotations, these annotations will be loaded as editable categories that you can update directly. Compatible annotations are tabular, with category names as column headers; `anndata.obs.index` as the index; and categorical values (i.e., fewer unique values per column than specified in `--max-category-items`, default 1000).
|
||||
|
||||
Any changes you make will be reflected in the original CSV (which will be overwritten). This is helpful if you wish to annotate over multiple sessions.
|
||||
|
||||
If the file does not exist, it will be created.
|
||||
|
||||
### Annotations by multiple users
|
||||
|
||||
An alternative to specifying the file path is to specify the output directory, and allow cellxgene to assign filenames. This is most useful for situations where the same cellxgene instance is being used by multiple users to create annotations.
|
||||
|
||||
As described in the [hosted](hosted) section, we do not officially support hosted or multi-user use of cellxgene. However, we recognize that the app is often adapted for this purpose, and have tried to provide a "safe path" for multi-user setups that avoids overwriting data.
|
||||
|
||||
To specify an output directory, run:
|
||||
```
|
||||
cellxgene launch mydata.h5ad --experimental-annotations-output-dir path/to/annotations-directory/
|
||||
```
|
||||
|
||||
For each user, annotations will be saved as follows:
|
||||
- Each user will be prompted to enter a name for their annotations the first time they create a new category.
|
||||
- We also assign a unique identifier in the form of an 8-character suffix, `########`; this helps cellxgene identify their specific file to avoid overwriting others' work.
|
||||
- Any annotations created in the application will be autosaved in `annotations-directory/name-########.csv`
|
||||
|
||||
|
||||
### Merging draft annotations with the main h5ad file
|
||||
|
||||
Once you're finished with your annotations, you should finalize and preserve your work by merging your `csv` into your main `h5ad` file.
|
||||
|
||||
You can do so like this:
|
||||
```
|
||||
import pandas as pd
|
||||
import scanpy as sc
|
||||
|
||||
new_annotations = pd.read_csv('myannotations.csv',
|
||||
comment='#',
|
||||
dtype='category',
|
||||
index_col=0)
|
||||
anndata = sc.read('mydata.h5ad')
|
||||
anndata.obs = anndata.obs.join(new_annotations)
|
||||
```
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I know my annotations are saved?
|
||||
`cellxgene` autosaves any changes made to your annotations every 3 seconds.
|
||||
|
||||
### I think I deleted my annotations! Oh noes!
|
||||
Not to worry! We save the last 10 versions of your annotations in `annotations-directory/NAME-backups/`
|
||||
|
||||
### What about creating continuous annotations?
|
||||
Continuous metadata is important! However, these values (e.g., pseudotime) are the result of statistical analyses that are beyond cellxgene's visualization- and exploration-focused scope. We do, of course, provide visualization of continuous metadata values computed elsewhere and stored in `anndata.obs`.
|
||||
|
||||
### I keep getting weird index errors when trying to join my annotations to my anndata??
|
||||
This is most likely because the h5ad file you are working with is not the original file used to generate the annotations! We recommend merging new annotations in on a regular basis for this reason.
|
||||
|
||||
### How do you remember my unique ID to match my cellxgene session with my annotations file?
|
||||
We place a small cookie (file) in your browser that identifies where your draft annotations are saved. This file never leaves your machine, and is never sent to the cellxgene team or anyone else.
|
||||
|
||||
### I have feedback and ideas for you!
|
||||
Wonderful! This is a very new and complex feature; we would _love_ to [hear your feedback](contact) :)
|
||||
26
docs/posts/contact.md
Normal file
26
docs/posts/contact.md
Normal file
@@ -0,0 +1,26 @@
|
||||
---
|
||||
layout: default
|
||||
title: Contact
|
||||
description: Contact
|
||||
---
|
||||
# Get in touch
|
||||
|
||||
We'd love to hear from you!
|
||||
|
||||
For questions, suggestions, or accolades, [join the `#cellxgene-users` channel on the CZI Science Slack](https://join-cellxgene-users.herokuapp.com/) and say "hi!".
|
||||
|
||||
For any errors, [report bugs on Github](https://github.com/chanzuckerberg/cellxgene/issues).
|
||||
|
||||
# Team
|
||||
|
||||
The current core team:
|
||||
|
||||
- Colin Megill, frontend & product design
|
||||
- Bruce Martin, software engineer
|
||||
- Sidney Bell, computational biologist
|
||||
- Lia Prins, designer
|
||||
- Severiano Badajoz, software engineer
|
||||
|
||||
We would also like to gratefully acknowledge contributions from past core team members:
|
||||
|
||||
- Charlotte Weaver, software engineer
|
||||
70
docs/posts/contribute.md
Normal file
70
docs/posts/contribute.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# Code of conduct
|
||||
We warmly welcome contributions from the community!
|
||||
|
||||
To ensure a welcoming experience for our entire community, this project adheres to the Contributor Covenant
|
||||
[code of conduct](https://github.com/chanzuckerberg/.github/tree/master/CODE_OF_CONDUCT.md).
|
||||
By participating, you are expected to uphold this code. Please report unacceptable behavior
|
||||
to opensource@chanzuckerberg.com.
|
||||
|
||||
If you have any questions about any of this stuff, just ask! :)
|
||||
|
||||
# Contributing ideas and issues
|
||||
|
||||
We'd love to hear from you! Please submit any bug reports and feature requests through [Github issues](https://github.com/chanzuckerberg/cellxgene/issues).
|
||||
|
||||
# Direct contributions
|
||||
## Getting started
|
||||
|
||||
If you are interested in working on `cellxgene` development, you'll need to use git to make a copy of the [project repository](https://www.youtube.com/watch?v=A-4WltCTVms&list=PLe6EXFvnTV7-_41SpakZoTIYCgX4aMTdU&index=2&t=0s) and share your changes.
|
||||
If you're new to git, we recommend [GitKraken](https://www.gitkraken.com/) for an intuitive interface.
|
||||
|
||||
Please submit any direct contributions by [forking the repository](https://www.youtube.com/watch?v=Lb4yvfrX_7I&list=PLe6EXFvnTV7-_41SpakZoTIYCgX4aMTdU&index=3&t=9s), creating a branch, and [submitting a Pull Request](https://www.youtube.com/watch?v=2VX1ISk9XH8&list=PLe6EXFvnTV7-_41SpakZoTIYCgX4aMTdU&index=9&t=0s).
|
||||
|
||||
First, you'll need the following installed on your machine
|
||||
|
||||
- python 3.6+
|
||||
- node and npm (we recommend using [nvm](https://github.com/creationix/nvm) if this is your first time with node)
|
||||
|
||||
Then clone the project
|
||||
|
||||
```
|
||||
git clone https://github.com/chanzuckerberg/cellxgene.git
|
||||
```
|
||||
|
||||
This is enough to get you started with editing documentation. If you'd like to contribute code:
|
||||
|
||||
Build the client web assets by calling `make` from inside the `cellxgene` folder
|
||||
|
||||
```
|
||||
make
|
||||
```
|
||||
|
||||
Install all requirements (we recommend doing this inside a [virtual environment](install))
|
||||
|
||||
```
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
You can start the app while developing either by calling `cellxgene` or by calling `python -m server`. We recommend using the `--debug` flag to see more output, which you can include when reporting bugs.
|
||||
|
||||
If you have any questions about developing or contributing, come hang out with us by joining the [CZI Science Slack](https://join-cellxgene-users.herokuapp.com/) and posting in the `#cellxgene-dev` channel.
|
||||
|
||||
## Contributing code
|
||||
|
||||
This project has made a few key design choices:
|
||||
|
||||
- The front-end is built with [`regl`](https://github.com/regl-project/regl) (a webgl library), [`react`](https://reactjs.org/), [`redux`](https://redux.js.org/), [`d3`](https://github.com/d3/d3), and [`blueprint`](https://blueprintjs.com/docs/#core) to handle rendering large numbers of cells with lots of complex interactivity
|
||||
- The app is designed with a client-server model that can support a range of existing analysis packages for Python-based backend computational tasks (currently built for [scanpy](https://github.com/theislab/scanpy))
|
||||
- The client uses fast cross-filtering to handle selections and comparisons across subsets of data
|
||||
|
||||
Depending on your background and interests, you might want to contribute to the frontend, or backend, or both!
|
||||
|
||||
Please submit any direct contributions via a Pull Request. It'd be great for PRs to include test cases and documentation updates where relevant, though we know the core test suite is itself still a work in progress.
|
||||
|
||||
## Contributing documentation
|
||||
|
||||
The documentation is written in [markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet), and lives in the directory `cellxgene/docs/posts`. You can directly edit or add to these files and submit a Pull Request as described above.
|
||||
|
||||
To preview your changes on your local machine, you'll need to install Jekyll and Ruby using [these instructions](https://jekyllrb.com/docs/installation/) (you don't have to know how to program in Ruby, just install it).
|
||||
|
||||
You can then preview your changes by running `cellxgene/docs$ bundle exec jekyll serve` and navigating to the url indicated in the terminal.
|
||||
50
docs/posts/demo-data.md
Normal file
50
docs/posts/demo-data.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
layout: default
|
||||
title: demo-data
|
||||
description: Demo datasets
|
||||
---
|
||||
|
||||
# Example datasets
|
||||
**To use these datasets, run:**
|
||||
`cellxgene launch [filename.h5ad] --open`
|
||||
|
||||
### Peripheral blood mononuclear cells
|
||||
Healthy human PBMCs (10X).
|
||||
|
||||
- Source: [10X genomics](https://support.10xgenomics.com/single-cell-gene-expression/datasets/1.1.0/pbmc3k)
|
||||
- Cells: 2,638
|
||||
- File size: 19MB
|
||||
- [Raw data](http://cf.10xgenomics.com/samples/cell-exp/1.1.0/pbmc3k/pbmc3k_filtered_gene_bc_matrices.tar.gz)
|
||||
- [Processing](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/pbmc3k-processing.ipynb)
|
||||
- Launch:
|
||||
```
|
||||
cellxgene launch https://cellxgene-example-data.czi.technology/pbmc3k.h5ad
|
||||
```
|
||||
|
||||
### Tabula muris
|
||||
20 organs and tissues from healthy mice (Smart-Seq2).
|
||||
Rich metadata and annotations.
|
||||
|
||||
- Source: [bioRxiv, CZBiohub](https://www.biorxiv.org/content/10.1101/237446v2)
|
||||
- Cells: 45,423
|
||||
- File size: 174MB
|
||||
- [Raw data](https://figshare.com/projects/Tabula_Muris_Transcriptomic_characterization_of_20_organs_and_tissues_from_Mus_musculus_at_single_cell_resolution/27733)
|
||||
- [Processing](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/tabula-muris-processing.ipynb)
|
||||
- Launch:
|
||||
```
|
||||
cellxgene launch https://cellxgene-example-data.czi.technology/tabula-muris.h5ad
|
||||
```
|
||||
|
||||
### Tabula muris senis
|
||||
22 organs and tissues from healthy mice at ages 3mo, 18mo, 21mo, and 24mo (Smart-Seq2).
|
||||
Rich metadata and annotations.
|
||||
|
||||
- Source: [bioRxiv, CZBiohub](https://www.biorxiv.org/content/10.1101/661728v1)
|
||||
- Cells: 81,478
|
||||
- File size: 3.9GB
|
||||
- Raw data [geo link coming soon!]
|
||||
- [Processing](https://www.biorxiv.org/content/10.1101/661728v1)
|
||||
- Launch:
|
||||
```
|
||||
cellxgene launch https://cellxgene-example-data.czi.technology/tabula-muris-senis.h5ad
|
||||
```
|
||||
34
docs/posts/gallery.md
Normal file
34
docs/posts/gallery.md
Normal file
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Gallery
|
||||
subtitle: Features & examples
|
||||
layout: default
|
||||
---
|
||||
|
||||
# Features
|
||||
|
||||
### Cross-filter cells and color by metadata
|
||||
<img src="../images/crossfilter.gif" width="600"/>
|
||||
|
||||
### Compare groups of cells with differential expression
|
||||
<img src="../images/diffexp.gif" width="600"/>
|
||||
|
||||
### See how metadata and gene expression break down across categories
|
||||
<img src="../images/category-breakdown.gif" width="600"/>
|
||||
|
||||
### Find cells where a gene is expressed
|
||||
<img src="../images/gene-expression.gif" width="600"/>
|
||||
|
||||
### Compare expression of multiple genes
|
||||
<img src="../images/compare-genes.gif" width="600"/>
|
||||
|
||||
# Interactive demos
|
||||
|
||||
Several groups have independently [deployed](hosted) various versions of cellxgene to the web.
|
||||
Check out the cool data that our users are using cellxgene to explore!
|
||||
|
||||
### [Kidney cell atlas](https://www.kidneycellatlas.org/)
|
||||
### [Tabula muris senis](https://tabula-muris-senis.ds.czbiohub.org/)
|
||||
### [Hemocytes](https://hemocytes.cellgeni.sanger.ac.uk/)
|
||||
### [Melanoma](https://melanoma.cellgeni.sanger.ac.uk/)
|
||||
|
||||
_Want us to link to your dataset here? [Just send us a note!](contact)_
|
||||
96
docs/posts/hosted.md
Normal file
96
docs/posts/hosted.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Hosting cellxgene on the web
|
||||
|
||||
Cellxgene is intended to be used by researchers on their local machines. However, we recognize that sharing and exploring data on the web is important. We're exploring how we could better support this in the future, and [would welcome your input](https://github.com/chanzuckerberg/cellxgene/issues/875)!
|
||||
|
||||
In the meantime, you can see examples of how other groups have approached this in our [gallery](gallery). While we don't officially support web deployment, we've offered some guidance below on one way to deploy cellxgene to the web.
|
||||
|
||||
## General notes and cautions
|
||||
|
||||
Please consider the following when deploying cellxgene in any "hosted" environment, especially where access from the broader Internet is possible:
|
||||
|
||||
- Information security requires careful configuration of the host environment, including firewall, logging, etc. Please follow best practices.
|
||||
- cellxgene includes features which may be inappropriate for a hosted deployment. You may wish to use the following command line option: `--disable-diffexp`.
|
||||
- `cellxgene launch` currently uses Flask's development server, which is not recommended for hosted deployment (see the [Flask documentation](https://flask.palletsprojects.com/en/1.1.x/tutorial/deploy/#run-with-a-production-server))
|
||||
- We have no testing or official support for deployments where multiple users are accessing the same cellxgene instance.
|
||||
- Your cellxgene instance is likely to hang or crash if too many people access it at the same time, especially if they using functions that call the Python backend (such as differential expression, noted above).
|
||||
- cellxgene only supports one instance per dataset
|
||||
|
||||
If you believe you have found a security-related issue with cellxgene, please report the issue immediately to <security@chanzuckerberg.com>.
|
||||
|
||||
## Configuration options
|
||||
|
||||
The following configuration options require special consideration in any multi-user or hosted environment:
|
||||
|
||||
`--disable-diffexp`: the differential expression computation can be resource intensive, in particular for large datasets. If many differential expression calculation requests are made in rapid sequence, it may cause the server CPU or memory resources to be exhausted, and impact the ability of other users to access data. This command line option will disable the differential expression feature, including the removal of the `Differential expression` button.
|
||||
|
||||
`--experimental-annotations`: this feature, which is disabled by default, may not be appropriate for hosted environments. It will write to the local file system, and in extreme cases could be used to abuse (or exceed) file system capacity on the hosting server.
|
||||
|
||||
`--experimental-annotations-file`: this specifies a single file for all end-user annotations, and is incompatible with hosted or multi-user use of cellxgene. Using it will cause loss of user annotation data (ie, the CSV file will be overwritten). If you wish to explore using the _experimental_ annotations feature in a multi-user environment, please refer to the [annotations documentation](annotations).
|
||||
|
||||
## Community software projects
|
||||
|
||||
There are a number of teams building tools or infrastructure to better utilize cellxgene in a multiple user environment. While we do not endorse any particular solution, you may find the following helpful.
|
||||
|
||||
- [Novartis Cellxgene Gateway](https://github.com/Novartis/cellxgene-gateway) - a multiple-user and multiple-dataset gateway for cellxgene.
|
||||
- Interactive Enviroment in the [Galaxy Project](https://galaxyproject.org/) ([patch notes](https://docs.galaxyproject.org/en/release_19.05/releases/19.05_announce.html))
|
||||
|
||||
If you know of other solutions, drop us a note and we'll add to this list.
|
||||
|
||||
# Deploying cellxgene with Heroku
|
||||
|
||||
## Quickstart
|
||||
|
||||
Clicking on the following button will forward you to Heroku to begin the deployment process:
|
||||
|
||||
<a href="https://heroku.com/deploy?template=https://github.com/chanzuckerberg/cellxgene/tree/heroku">
|
||||
<img src="https://www.herokucdn.com/deploy/button.svg" alt="Deploy">
|
||||
</a>
|
||||
|
||||
If not already logged in to Heroku, there you will be prompted to log in or sign up for an account.
|
||||
|
||||
Once logged in you will be sent to the setup page. Here you can set some of the basic settings for the app:
|
||||
|
||||
### Default settings
|
||||
|
||||
- `App name`: the unique name for your deployment
|
||||
- This will also serve as the default URL (e.g. https://cellxgene.herokapp.com/)
|
||||
- `App owner`: Who will own this app. Either you personally or an organization/team
|
||||
- `Region`: Location of the server where the app will be deployed (EU or US)
|
||||
|
||||
### Configuration
|
||||
|
||||
- `DATASET`: A _publicly_ accessible URL pointing to a .h5ad file to view
|
||||
- This defaults to pbm3k.h5ad
|
||||
|
||||
After filling out the settings and pressing the `Deploy app` button Heroku will begin building your deployment. This process will take a few minutes, but once completed you will have a personal free hosted version of cellxgene!
|
||||
|
||||
## What is Heroku?
|
||||
|
||||
Heroku is a quick and easy way to host applications on the cloud.
|
||||
|
||||
A Heroku deployment of cellxgene means that the app is not running on your local machine. Instead, the app is installed, configured, and run on the Heroku servers (read: cloud).
|
||||
|
||||
On Heroku's servers, applications run on a [dyno](https://www.heroku.com/dynos) which are Heroku's implementation and abstraction of containers.
|
||||
|
||||
Heroku is one of many options available for hosting instances of cellxgene on the web.
|
||||
Some other options include: Amazon Web Services, Google Cloud Platform, Digital Ocean, and Microsoft Azure.
|
||||
|
||||
## Why use Heroku to deploy cellxgene?
|
||||
|
||||
What Heroku enables is a quick, non-technical method of setting up a cellxgene instance. No command line knowledge needed. This also allows machines to access the instance via the internet, so sharing a visualized dataset is as simple as sharing a link.
|
||||
|
||||
Because cellxgene currently heavily relies on its Python backend for providing the viewer with the necessary data and tooling, it is currently not possible to host cellxgene as a static webpage.
|
||||
|
||||
This is a good option if you want to quickly deploy an instance of cellxgene to the web. Heroku deployments are free for small datasets up to around 250MBs in size. See below regarding larger datasets.
|
||||
|
||||
## When should I not deploy with Heroku?
|
||||
|
||||
- The default free dyno offered by Heroku is limited in memory to 512 MBs
|
||||
- The amount of memory needed for the dyno is roughly the same size as the h5ad file
|
||||
- Heroku offers tiered paid dynos. More can be found on the [Heroku pricing page](https://www.heroku.com/pricing)
|
||||
- Note that this can get _very_ expensive for larger datasets (\$25+ a month)
|
||||
- On the free dyno, after 30 minutes of inactivity, Heroku will put your app into a hibernation mode. On the next access, Heroku will need time to boot the dyno back online.
|
||||
- Having multiple simultaneous users requires more memory. This means that the free container size is easily overwhelmed by multiple users, even with small datasets; this can be addressed by purchasing a larger container size
|
||||
- For this facilitated Heroku deployment to work, your dataset must be hosted on a publicly accessible URL
|
||||
- By default, Heroku publically shares your instance to anyone with the URL.
|
||||
- There are many ways of securing your instance. One quick and simple way is by installing [wwwhisper](https://elements.heroku.com/addons/wwwhisper), a Heroku addon
|
||||
109
docs/posts/index.html
Normal file
109
docs/posts/index.html
Normal file
@@ -0,0 +1,109 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en-US">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
|
||||
<!-- Begin Jekyll SEO tag v2.5.0 -->
|
||||
<title>Index | cellxgene</title>
|
||||
<meta name="generator" content="Jekyll v3.8.5" />
|
||||
<meta property="og:title" content="Index" />
|
||||
<meta property="og:locale" content="en_US" />
|
||||
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
|
||||
<meta property="og:description" content="An interactive explorer for single-cell transcriptomics data" />
|
||||
<link rel="canonical" href="http://localhost:4000/" />
|
||||
<meta property="og:url" content="http://localhost:4000/" />
|
||||
<meta property="og:site_name" content="cellxgene" />
|
||||
<script type="application/ld+json">
|
||||
{"description":"An interactive explorer for single-cell transcriptomics data","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"http://localhost:4000/cellxgene-logo.svg"}},"@type":"WebSite","url":"http://localhost:4000/","name":"cellxgene","headline":"Index","@context":"http://schema.org"}</script>
|
||||
<!-- End Jekyll SEO tag -->
|
||||
|
||||
<link rel="stylesheet" href="/assets/css/style.css?v=d12c1b6de7e247cdbabc4ce83364f9ea50978892">
|
||||
<!--[if lt IE 9]>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
|
||||
<![endif]-->
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrapper">
|
||||
<header>
|
||||
<img src="/cellxgene-logo.svg" alt="cellxgene" />
|
||||
|
||||
<p>An interactive explorer for single-cell transcriptomics data</p>
|
||||
<p>
|
||||
|
||||
<a href="/" class="btn">Quick start</a><br>
|
||||
|
||||
<a href="/posts/gallery" class="btn">Gallery</a><br>
|
||||
|
||||
<a href="/posts/install" class="btn">Installation</a><br>
|
||||
|
||||
<a href="/posts/demo-data" class="btn">Demo datasets</a><br>
|
||||
|
||||
<a href="/posts/prepare" class="btn">Preparing your data</a><br>
|
||||
|
||||
<a href="/posts/launch" class="btn">Launching cellxgene</a><br>
|
||||
|
||||
<a href="/posts/hosted" class="btn">Hosting cellxgene</a><br>
|
||||
|
||||
<a href="/posts/contribute" class="btn">Contributing (ideas or code)</a><br>
|
||||
|
||||
<a href="/posts/methods" class="btn">Methods</a><br>
|
||||
|
||||
<a href="/posts/faq" class="btn">FAQ</a><br>
|
||||
|
||||
<a href="/posts/roadmap" class="btn">Roadmap</a><br>
|
||||
|
||||
<a href="/posts/contact" class="btn">Contact & finding help</a><br>
|
||||
|
||||
|
||||
|
||||
<a href="https://github.com/chanzuckerberg/cellxgene" class="btn" target="_blank">Code</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
|
||||
<section>
|
||||
<h1 id="quick-start">Quick start</h1>
|
||||
|
||||
<p>Whether you need to visualize one thousand cells or one million, cellxgene helps you gain insight into your single-cell data.</p>
|
||||
|
||||
<p>To install cellxgene you need Python 3.6+. We recommend <a href="/faq.html#how-do-i-create-a-python-environment-for-cellxgene">installing cellxgene into a conda or virtual environment.</a></p>
|
||||
|
||||
<p>Install the package.</p>
|
||||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install cellxgene
|
||||
</code></pre></div></div>
|
||||
|
||||
<p>Download an example <a href="https://anndata.readthedocs.io/en/latest/">anndata</a> file</p>
|
||||
|
||||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-o</span> tabula-muris.h5ad https://cellxgene-example-data.czi.technology/tabula-muris.h5ad.zip
|
||||
unzip tabula-muris.h5ad.zip
|
||||
</code></pre></div></div>
|
||||
|
||||
<p>Launch cellxgene</p>
|
||||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cellxgene launch tabula-muris.h5ad <span class="nt">--open</span>
|
||||
</code></pre></div></div>
|
||||
|
||||
<p>To explore more datasets already formatted for cellxgene, check out the <a href="demo-data">Demo data</a> or
|
||||
see <a href="prepare">Preparing your data</a> to learn more about formatting your own
|
||||
data for cellxgene.</p>
|
||||
|
||||
<h1 id="getting-help">Getting help</h1>
|
||||
|
||||
<p>We’d love to hear from you!</p>
|
||||
|
||||
<p>For questions, suggestions, or accolades, <a href="https://join-cellxgene-users.herokuapp.com/">join the <code class="highlighter-rouge">#cellxgene-users</code> channel on the CZI Science Slack</a> and say “hi!”.</p>
|
||||
|
||||
<p>For any errors, <a href="https://github.com/chanzuckerberg/cellxgene/issues">report bugs on Github</a>.</p>
|
||||
|
||||
</section>
|
||||
<footer>
|
||||
|
||||
<p>This project is maintained by <a href="https://github.com/chanzuckerberg">chanzuckerberg</a></p>
|
||||
|
||||
</footer>
|
||||
</div>
|
||||
<script src="/assets/js/scale.fix.js"></script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
109
docs/posts/install.md
Normal file
109
docs/posts/install.md
Normal file
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: Install
|
||||
subtitle: Install
|
||||
layout: default
|
||||
---
|
||||
|
||||
# Installing cellxgene
|
||||
|
||||
Cellxgene has two parts:
|
||||
|
||||
- [`cellxgene`](launch) is the main explorer application, which takes an already-processed `h5ad` file as input. This is installed by default.
|
||||
- [`cellxgene prepare`](prepare) provides auxiliary functionality for preparing your dataset. This is _not_ installed by default.
|
||||
|
||||
## Requirements
|
||||
|
||||
You'll need **python 3.6+** and an up-to-date version of **Google Chrome**.
|
||||
The web UI is tested on OSX and Windows using Chrome, and the python CLI is tested on OSX and Ubuntu (via WSL/Windows).
|
||||
It should work on other platforms, but if you run into trouble let us know.
|
||||
|
||||
[Python.org](https://www.python.org/downloads/) has help on installing a recent
|
||||
version of Python, including the pip package manager. Chrome is available at
|
||||
[Google.com/chrome](https://google.com/chrome).
|
||||
|
||||
## Basic install using pip
|
||||
|
||||
To install the `cellxgene` explorer alone, run:
|
||||
|
||||
```
|
||||
pip install cellxgene
|
||||
```
|
||||
|
||||
To install `cellxgene` and the optional `cellxgene prepare`, run:
|
||||
|
||||
```
|
||||
pip install cellxgene[prepare]
|
||||
```
|
||||
|
||||
_Note: if the aforementioned optional `prepare` package installation fails, you can also install these packages directly:_
|
||||
|
||||
```
|
||||
pip install scanpy>=1.3.7 python-igraph louvain>=0.6
|
||||
```
|
||||
|
||||
_On various Linux platforms, you may also need to install build dependencies first:_
|
||||
|
||||
```
|
||||
sudo apt-get install build-essential python-dev
|
||||
pip install scanpy>=1.3.7 python-igraph louvain>=0.6
|
||||
```
|
||||
|
||||
If you already have `cellxgene` installed, you can update to the most recent version by running:
|
||||
|
||||
```
|
||||
pip install cellxgene --upgrade
|
||||
```
|
||||
|
||||
## Using a conda environment
|
||||
|
||||
To install `cellxgene` alone, run:
|
||||
|
||||
```
|
||||
conda create --yes -n cellxgene python=3.7
|
||||
conda activate cellxgene
|
||||
pip install cellxgene
|
||||
```
|
||||
|
||||
To install `cellxgene` and the optional `cellxgene prepare`, run:
|
||||
|
||||
```
|
||||
conda create --yes -n cellxgene python=3.7
|
||||
conda activate cellxgene
|
||||
pip install cellxgene[prepare]
|
||||
```
|
||||
|
||||
## Using a virtual environment
|
||||
|
||||
To install `cellxgene` alone, run:
|
||||
|
||||
```
|
||||
ENV_NAME=cellxgene
|
||||
python3.7 -m venv ${ENV_NAME}
|
||||
source ${ENV_NAME}/bin/activate
|
||||
pip install cellxgene
|
||||
```
|
||||
|
||||
To install `cellxgene` and `cellxgene prepare`, run:
|
||||
|
||||
```
|
||||
ENV_NAME=cellxgene
|
||||
python3.7 -m venv ${ENV_NAME}
|
||||
source ${ENV_NAME}/bin/activate
|
||||
pip install cellxgene[prepare]
|
||||
```
|
||||
|
||||
## Using docker
|
||||
|
||||
Build the image
|
||||
|
||||
```
|
||||
docker build . -t cellxgene
|
||||
```
|
||||
|
||||
Run the container and mount data (change data location, `--port` and `--host` parameters as needed)
|
||||
|
||||
```
|
||||
docker run -v "$PWD/example-dataset/:/data/" -p 5005:5005 cellxgene launch --host 0.0.0.0 data/pbmc3k.h5ad
|
||||
```
|
||||
|
||||
You will need to use `--host 0.0.0.0` to have the container listen to incoming requests from the browser
|
||||
77
docs/posts/launch.md
Normal file
77
docs/posts/launch.md
Normal file
@@ -0,0 +1,77 @@
|
||||
---
|
||||
layout: default
|
||||
title: demo-data
|
||||
description: Demo datasets
|
||||
---
|
||||
|
||||
# Launching cellxgene with your dataset
|
||||
|
||||
## Quickstart
|
||||
|
||||
Once you've [prepared your data](prepare) for cellxgene, you can launch the app using
|
||||
|
||||
```
|
||||
cellgene launch mydataset.h5ad --open
|
||||
```
|
||||
|
||||
You should see your web browser open with the following
|
||||
|
||||
<img width="450" src="https://raw.githubusercontent.com/chanzuckerberg/cellxgene/master/docs/cellxgene-opening-screenshot.png" pad="50px">
|
||||
|
||||
**Note**: automatic opening of the browser with the `--open` flag only works on some platforms (eg, OSX). On other platforms you'll need to directly point to the provided link in your browser.
|
||||
|
||||
## Launching from a URL
|
||||
|
||||
You can also launch from a URL directly like this:
|
||||
|
||||
```
|
||||
cellxgene launch https://github.com/chanzuckerberg/cellxgene/blob/master/example-dataset/pbmc3k.h5ad
|
||||
```
|
||||
|
||||
Support for S3 and GCS is not enabled by default. If you wish to directly access S3 or GFS, install one or both of the following packages:
|
||||
|
||||
- [s3fs](https://s3fs.readthedocs.io/en/latest/) for S3 support
|
||||
- [gcsfs](https://gcsfs.readthedocs.io/en/latest/) for GCS support
|
||||
|
||||
For example:
|
||||
|
||||
```
|
||||
pip install s3fs
|
||||
cellxgene launch s3://mybucket.s3-us-west-2.amazonaws.com/mydata.h5ad
|
||||
```
|
||||
|
||||
## Options for cellxgene `launch`
|
||||
|
||||
**For the most up-to-date and comprehensive list of options, run `cellxgene launch --help`**
|
||||
|
||||
`--open` automatically opens the web browser after launching (caveat: only works on some operating systems).
|
||||
|
||||
`--experimental-annotations`, `--experimental-annotations-file` & `--experimental-annotations-output-dir` all have to do with an experimental feature to allow users to create new categorical annotations in the application. We have a [whole separate page](annotations) about their usage! :)
|
||||
|
||||
`--diffexp-lfc-cutoff` as explained [in the methods](methods), genes are only returned in differential expression if the effect size is above the specified threshold for log fold change. Defaults to 0.01.
|
||||
|
||||
`--disable-diffexp` will disable and hide the `Compute Differential Expression` feature.
|
||||
For large datasets, or datasets loaded with the `--backed` option, computing differential expression may
|
||||
be extremely slow or use excessive resources on the host computer (e.g., memory thrashing).
|
||||
Disabling the feature will ensure that this computation is not initiated accidentally.
|
||||
|
||||
`--backed` option instructs `cellxgene launch` to read the H5AD file in "backed" mode (for more information, see the
|
||||
[anndata.read_h5ad() documentation](https://anndata.readthedocs.io/en/latest/anndata.read_h5ad.html#anndata.read_h5ad)).
|
||||
|
||||
By default, cellxgene will read the entire H5AD will be into memory at startup, improving application speed and performance.
|
||||
Very large datasets may not fit in memory. The "--backed" mode will read the file incrementally, reducing memory
|
||||
use, and for large files, improving startup speed. _However_, this option will also significantly slow
|
||||
down access to gene expression histograms, and may render differential expression calculations too slow
|
||||
to use (see `--disable-diffexp` for an option to disable this feature).
|
||||
|
||||
`--embedding` restricts which embeddings will be available in the viewer. By default, all embeddings specified in `anndata.obsm['X_name']` will be loaded; if you have many embeddings, you may wish to restrict this list for a speedier launch.
|
||||
|
||||
`--title` adds a title to the viewer. Defaults to file name.
|
||||
|
||||
`--about` adds a link where users can go to find more infomation about the dataset. Requires `https`.
|
||||
|
||||
`--obs-names` allows you to specify which column in `anndata.obs` to use as `anndata.obs.index`.
|
||||
|
||||
`--var-names` allows you to specify which column in `anndata.var` to use as `anndata.var.index`.
|
||||
|
||||
`--max-category-items` omits categorical metadata fields that contain more than N _distinct_ values. Defaults to 1000.
|
||||
21
docs/posts/methods.md
Normal file
21
docs/posts/methods.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# Methods
|
||||
|
||||
## Data structure: _anndata_ fields used for visualization
|
||||
|
||||
### Expression data
|
||||
|
||||
Gene expression values are pulled from `anndata.X`. These feed into the histograms, scatterplot, colorscale, and differential expression calculations. We're [working on ways](https://github.com/chanzuckerberg/cellxgene/issues/689) to incorporate `anndata.raw` and other `anndata.layers`!
|
||||
|
||||
### Metadata
|
||||
|
||||
Categorical (e.g., cluster labels) and continuous (e.g., pseudotime) metadata are pulled from `anndata.obs`. Any column added here will be available for visualization in cellxgene. You can also [create new categorical annotations](annotations) within the application.
|
||||
|
||||
### Embeddings
|
||||
|
||||
cellxgene looks for embeddings (e.g., tSNE, UMAP, PCA, spatial coordinates) in `anndata.obsm`. These fields must follow the scanpy convention of starting with `X_`, e.g., `anndata.obsm['X_umap']`. If an embedding has more than two components, the first two will be used for visualization.
|
||||
|
||||
## Differential expression
|
||||
|
||||
We're actively working on how to improve differential expression within the app.
|
||||
|
||||
Currently, we use a [Welch's _t_-test](https://en.wikipedia.org/wiki/Welch%27s_t-test) implementation, including the same variance overestimation correction as used in `scanpy`. We sort the `tscore` to identify the top 15 genes, and then filter to remove any that fall below a cutoff log fold change value, which can help remove spurious test results. The default threshold is `0.01` and can be changed using the option `--diffexp-lfc-cutoff`.
|
||||
103
docs/posts/prepare.md
Normal file
103
docs/posts/prepare.md
Normal file
@@ -0,0 +1,103 @@
|
||||
---
|
||||
layout: default
|
||||
title: prepare
|
||||
description: Preparing your data
|
||||
---
|
||||
# Data format requirements
|
||||
|
||||
If your data is in `h5ad` file (from the [`anndata`](https://anndata.readthedocs.io/en/latest/index.html) library) and meets the following requirements, you can go straight to `cellxgene launch`:
|
||||
|
||||
- Expression values (raw or normalized) in `anndata.X`
|
||||
- At least one embedding (e.g., tSNE, UMAP) in `anndata.obsm`, specified with the prefix `X_` (e.g., by default scanpy stores UMAP coordinates in `anndata.obsm['X_umap']`)
|
||||
- A unique identifier for every cell is available in an `anndata.obs` field (you can specify this with the `--obs-names` option)
|
||||
- A unique identifier for every gene is available in an `anndata.var` field (you can specify which field to use with the `--var-names` option)
|
||||
|
||||
#### What about R objects from seurat / bioconductor!?
|
||||
We hear you! We'd also love to be able to ingest these files directly. This isn't currently possible, but in the meantime, you can use one of [these handy adapters](https://satijalab.org/seurat/v3.0/conversion_vignette.html) to convert to `h5ad`.
|
||||
|
||||
#### Can I use data hosted on the web somewhere?
|
||||
Yes! You can launch from a URL instead of a filepath. The same data format requirements apply. Please see [here](launch) for more details.
|
||||
|
||||
# Using `cellxgene prepare`
|
||||
|
||||
If your data is in a different format, and/or you still need to perform dimensionality reduction and/or clustering, `cellxgene` can do that for you with the `prepare` command.
|
||||
|
||||
## What is `cellxgene prepare`?
|
||||
|
||||
`cellxgene prepare` offers an easy command line interface (CLI) to preliminarily wrangle your data into the required format for previewing it with cellxgene. It runs `scanpy` under the hood and can read in any format that is currently supported by `scanpy` (including mtx, loom, and more listed [in the scanpy documentation](https://scanpy.readthedocs.io/en/latest/api/index.html#reading)).
|
||||
|
||||
`prepare` uses scanpy to:
|
||||
|
||||
- Handle simple data normalization (from a [recipe](https://www.pydoc.io/pypi/scanpy-0.2.3/autoapi/preprocessing/recipes/index.html))
|
||||
- Do basic preprocessing to run PCA and compute the neighbor graph
|
||||
- Reduce dimensionality to generate embeddings
|
||||
- Infer clusters
|
||||
|
||||
You can control which steps to run and their methods (when applicable), via the CLI. The CLI also includes options for computing QC metrics, enforcing matrix sparcity, specifying index names, and plotting output.
|
||||
|
||||
## What is cellxgene `prepare` _not_?
|
||||
|
||||
`cellxgene prepare` is not meant as a way to formally process or analyze your data. It's simply a utility for quickly wrangling your data into cellxgene-compatible format and computing a "vanilla" embedding so you can try out `cellxgene` and get a general sense of a dataset.
|
||||
|
||||
## Quickstart for `cellxgene prepare`
|
||||
To add `cellxgene prepare` to your [cellxgene installation](install), run
|
||||
`pip install cellxgene[prepare]`
|
||||
|
||||
Then run `prepare` on your data with:
|
||||
```
|
||||
cellxgene prepare dataset.h5ad --output=dataset-processed.h5ad
|
||||
```
|
||||
|
||||
This will load the input data, perform PCA and nearest neighbor calculations, compute `UMAP` and `tSNE` embeddings and `louvain` cluster assignments, and save the results in a new file called `dataset-processed.h5ad` that can be loaded using `cellxgene launch`.
|
||||
|
||||
## Example usage
|
||||
|
||||
As a quick example, let's construct a command to use `prepare` to take a raw expression matrix and generate a processed `h5ad` ready to visualize with cellxgene.
|
||||
|
||||
We'll start off using the raw data from the pbmc3k dataset. This dataset is described [here](https://icb-scanpy.readthedocs-hosted.com/en/stable/api/scanpy.datasets.pbmc3k.html), and is available as part of the scanpy package. For this example, we'll assume this raw data is stored in a file called `pbmc3k-raw.h5ad`.
|
||||
|
||||
Our `prepare` command looks like this:
|
||||
|
||||
```
|
||||
cellxgene prepare pbmc3k-raw.h5ad \
|
||||
--run-qc \ # (A)
|
||||
--recipe seurat \ # (B)
|
||||
--layout tsne --layout umap \ # (C)
|
||||
--output pbmc3k-prepared.h5ad # (D)
|
||||
```
|
||||
|
||||
Let's look at what `prepare` is doing to our data, and how each step relates to the command above. You can see a walkthrough of what's going on under the hood for this example in [this notebook](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/pbmc3k-prepare-example.ipynb).
|
||||
|
||||
**(A) - Compute quality control metrics and store this in our `AnnData` object for later inspection**
|
||||
**(B) - Normalize the expression matrix using a basic preprocessing recipe**
|
||||
**(auto) - Do some preprocessing to run PCA and compute the neighbor graph**
|
||||
**(auto) - Infer clusters with the Louvain algorithm and store these labels to visualize later**
|
||||
**(C) - Compute and store UMAP and tSNE embeddings**
|
||||
**(D) - Write results to file**
|
||||
|
||||
## Options for cellxgene `prepare`
|
||||
|
||||
**For the most up-to-date and comprehensive list of options, run `cellxgene prepare --help`**
|
||||
|
||||
`--embedding` controls which dimensionality reduction algorithm is applies to your data.
|
||||
Options are `umap` and/or `tsne`. Defaults to both.
|
||||
|
||||
`--recipe` controls which normalization steps to apply to your data, based on one of the preprocessing `recipes` included with `scanpy`.
|
||||
These recipes include steps like cell filtering and gene selection; see the `scanpy` [documentation](https://scanpy.readthedocs.io/en/latest/api/index.html#recipes) for more details.
|
||||
Options are `none`, `seurat`, or `zheng17`. Defaults to `none`.
|
||||
|
||||
`--sparse` is a flag determines whether to enforce a sparse matrix. For large datasets, `prepare` can take a long time to run (a few minutes for datasets with 10-100k cells, up to an hour or more for datasets with >100k cells). If you want `prepare` to run faster we recommend using the `sparse` option.
|
||||
If this flag is not included, default is `False`
|
||||
|
||||
`--skip-qc` by default, `cellxgene prepare` will compute quality control metrics (saved to `anndata.obs` and `anndata.var`) as described in the `scanpy` [documentation](https://scanpy.readthedocs.io/en/stable/api/scanpy.pp.calculate_qc_metrics.html). Pass this flag if you would like to skip this step.
|
||||
|
||||
`--make-obs-names-unique` / `--make-var-names-unique` determine whether to rename `obs` (cell) / `var` (gene) names, respectively, to be unique.
|
||||
Default is `True`.
|
||||
|
||||
`--set-obs-names` controls which field in `anndata.obs` (cell metadata) is used as the _index_ for cells (e.g., a cell ID column).
|
||||
Default is `anndata.obs.names`
|
||||
|
||||
`--set-var-names` controls which field in `anndata.var` (gene metadata) is used as the _index_ for genes.
|
||||
Default is `anndata.var.names`
|
||||
|
||||
`--output` and `--overwrite` control where the processed data is saved.
|
||||
52
docs/posts/roadmap.md
Normal file
52
docs/posts/roadmap.md
Normal file
@@ -0,0 +1,52 @@
|
||||
---
|
||||
layout: default
|
||||
title: roadmap
|
||||
description: Roadmap
|
||||
---
|
||||
|
||||
# Roadmap
|
||||
|
||||
cellxgene makes it easier for biologists to collaboratively explore and understand their single-cell RNA-seq data.
|
||||
In the near term, we are focused on continuing to enable fast, interactive exploration of single-cell data, supporting collaborative workflows in single-cell analysis, and improving user support.
|
||||
If you have questions or feedback about this roadmap, please submit an issue on GitHub.
|
||||
Please note: this roadmap is subject to change.
|
||||
|
||||
Last updated: June 25, 2019
|
||||
|
||||
## Fast, interactive exploration of single-cell data
|
||||
|
||||
### Exposing Relationships Between Metadata and Data
|
||||
Biologists need to understand how variables (stored in metadata) are associated with one another and how they relate to changes in gene expression.
|
||||
Building upon visualization features that reveal categorical metadata relationships (cluster occupancy) and gene expression relationships (scatterplot), we plan to add exploratory visualization components that enable investigation of relationships between metadata and gene expression.
|
||||
See [issue #616](https://github.com/chanzuckerberg/cellxgene/issues/616) for more details.
|
||||
|
||||
### Contextualizing Genes
|
||||
While exploring a transcriptomics dataset, scientists need to understand the biological context of genes.
|
||||
This context may be provided by user-defined gene metadata or publicly available gene databases.
|
||||
We plan to support augmenting gene names with additional information that is useful to biologists.
|
||||
See [issue #96](https://github.com/chanzuckerberg/cellxgene/issues/96) for more detail.
|
||||
|
||||
## Support collaborative workflows in single-cell analysis
|
||||
|
||||
### Manual Annotations
|
||||
cellxgene offers exploratory visualizations that are critical for manual annotation workflows, especially in collaborative environments.
|
||||
We plan to support manually annotating cells with labels (i.e., cell type or QC flags), and their easy export for downstream analysis.
|
||||
See [issue #524](https://github.com/chanzuckerberg/cellxgene/issues/524) for more details.
|
||||
|
||||
### Simple Click to Launch [Paused]
|
||||
|
||||
Many biologists prefer not to interact with the command line and need an OS-native experience when using cellxgene.
|
||||
We plan to implement a point-and-click installation and launch experience so that users can easily load data into cellxgene.
|
||||
See [issue #687](https://github.com/chanzuckerberg/cellxgene/issues/687) for details.
|
||||
|
||||
### Python API [Paused]
|
||||
For computational biologists, saving h5ad files then loading them into cellxgene is a point of friction.
|
||||
We plan to support importing cellxgene as a Python package so that users can launch cellxgene directly from an interactive environment (such as Jupyter, IPython, or Spyder), and pass data to and from the cellxgene UI.
|
||||
|
||||
## Improving user support
|
||||
|
||||
### Improved documentation
|
||||
cellxgene has some specific expectations about how data is stored.
|
||||
We want to ensure that new users can get started easily and learn how to use cellxgene with their own data.
|
||||
We plan to improve documentation on getting started, installation, data, and contributing.
|
||||
See [issue #533](https://github.com/chanzuckerberg/cellxgene/issues/533) for more details.
|
||||
23
docs/posts/troubleshooting.md
Normal file
23
docs/posts/troubleshooting.md
Normal file
@@ -0,0 +1,23 @@
|
||||
---
|
||||
layout: default
|
||||
title: Troubleshooting
|
||||
description: Troubleshooting
|
||||
---
|
||||
|
||||
# Troubleshooting tips & tricks
|
||||
|
||||
#### I tried to `pip install cellxgene` and got a weird error I don't understand
|
||||
|
||||
This may happen, especially as we work out bugs in our installation process! Please create a new [Github issue](https://github.com/chanzuckerberg/cellxgene/issues), explain what you did, and include all the error messages you saw. It'd also be super helpful if you call `pip freeze` and include the full output alongside your issue.
|
||||
|
||||
#### I have a BIG dataset, how can I make cellxgene run as fast as possible?
|
||||
|
||||
If your dataset requires gigabytes of disk space, you may need to select an appropriate storage format in order to effectively utilize `cellxgene`. Tips and tricks:
|
||||
|
||||
- `cellxgene` is optimized for columnar data access. For large datasets, format the expression matrix (`.X`) as either a [SciPy CSC sparse matrix](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csc_matrix.html) or a dense Numpy array (whichever creates a smaller `h5ad` file). If you are using `cellxgene prepare`, include the `--sparse` flag to ensure `.X` is formatted as a CSC sparse matrix (by default, `.X` will be a dense matrix).
|
||||
- By default, `cellxgene` loads the dataset into memory, and start time is directly proportional to `h5ad` file size and the speed of your file system. Expect that large (e.g., million cell) datasets will take minutes to load, even on relatively fast computers with a high performance local hard drive. Once loaded, exploring metadata should still be quick. If this start time is a problem, try the `--backed` flag, which will attempt to lazily load data as needed (caveat: subsequent data access may be slower).
|
||||
- If your dataset size exceeds the size of memory (RAM) on the host computer, differential expression calculations will be extremely slow (or fail, if you run out of virtual memory). In this case, we recommend running with the `--disable-diffexp` flag. For datasets that are extremely large, you may also find the `--backed` flag improves your ability to explore them.
|
||||
|
||||
#### I'm following the developer instructions and get an error about "missing files and directories” when trying to build the client
|
||||
|
||||
This is likely because you do not have node and npm installed, we recommend using [nvm](https://github.com/creationix/nvm) if you're new to using these tools.
|
||||
Reference in New Issue
Block a user