fork/cellxgene
1
0
Fork 0
mirror of https://github.com/chanzuckerberg/cellxgene.git synced 2026-09-22 19:28:12 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into colinmegill/geneset-prototype

Browse Source
This commit is contained in:
Colin Megill
2020-08-17 10:49:57 -04:00
parent adb0b65058 508889f74b
commit bef631396c
271 changed files with 4546 additions and 1766 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.bumpversion.cfg
+1 -2
View File
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.15.0
current_version = 0.16.0
[bumpversion:file:setup.py]
search = version="{current_version}"
@@ -12,4 +12,3 @@ replace = "version": "{new_version}"
[bumpversion:file:server/__init__.py]
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"
.github/ISSUE_TEMPLATE/---bug-report.md
+32
View File
@@ -0,0 +1,32 @@
---
name: "\U0001F41E Bug report"
about: Create a report to identify a bug in cellxgene
title: "[BUG]"
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Version (please complete the following information):**
- Desktop or hosted?:
- Browser (if hosted) [e.g. chrome, safari]:
- Version [e.g. 0.13.0]:
**Additional context**
Add any other context about the problem here.
.github/ISSUE_TEMPLATE/---feature-request.md
+20
View File
@@ -0,0 +1,20 @@
---
name: "\U0001F4A1 Feature request"
about: Suggest an idea for this project
title: "[FEATURE REQUEST]"
labels: user request
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
.github/ISSUE_TEMPLATE/---question-clarification.md
+10
View File
@@ -0,0 +1,10 @@
---
name: "\U0001F9D0 Question/Clarification"
about: Ask a question or for a clarification
title: "[QUESTION]"
labels: question
assignees: ''
---
.github/workflows/deploy.yml
+1 -1
View File
@@ -10,4 +10,4 @@ jobs:
steps:
- name: repository dispatch
run: |
curl -XPOST -u mdunitz:${{secrets.SCINFRA_TOKEN}} -H "Accept: application/vnd.github.everest-preview+json" -H "Content-Type: application/json" https://api.github.com/repos/chanzuckerberg/single-cell-infra/dispatches --data '{"event_type": "cellxgene-hook"}'
curl -XPOST -u czi-sci-single-cell-eng:${{secrets.SCI_GITHUB_TOKEN}} -H "Accept: application/vnd.github.everest-preview+json" -H "Content-Type: application/json" https://api.github.com/repos/chanzuckerberg/single-cell-infra/dispatches --data '{"event_type": "cellxgene-hook"}'
.github/workflows/push_tests.yml
+2
View File
@@ -73,6 +73,7 @@ jobs:
smoke-tests:
runs-on: macos-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.7
@@ -102,6 +103,7 @@ jobs:
smoke-tests-annotations:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.7
.gitignore
+1
View File
@@ -44,6 +44,7 @@ __pycache__
*.DS_Store*
data
tags
myconfig.yaml
# Jekyll
docs/_site/
Dockerfile
+1 -1
View File
@@ -4,7 +4,7 @@ ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
RUN apt-get update && \
apt-get install -y build-essential libxml2-dev python3-dev python3-pip zlib1g-dev python3-requests && \
apt-get install -y build-essential libxml2-dev python3-dev python3-pip zlib1g-dev python3-requests python3-aiohttp && \
pip3 install cellxgene
ENTRYPOINT ["cellxgene"]
Makefile
+5
View File
@@ -60,6 +60,11 @@ smoke-test:
smoke-test-annotations:
cd client && $(MAKE) smoke-test-annotations
.PHONY: test-db
test-db:
cd server && $(MAKE) test-db
# FORMATTING CODE
.PHOHY: fmt
Procfile
-1
View File
@@ -1 +0,0 @@
web: gunicorn --chdir server/eb app:application --log-file -
README.md
+24 -7
View File
@@ -14,26 +14,30 @@ Whether you need to visualize one thousand cells or one million, cellxgene helps
<img src="https://github.com/chanzuckerberg/cellxgene/raw/main/docs/images/crossfilter.gif" width="350" height="200" hspace="30"><img src="https://github.com/chanzuckerberg/cellxgene/raw/main/docs/images/category-breakdown.gif" width="350" height="200" hspace="30">
# Getting started
### The comprehensive guide to cellxgene
[The cellxgene documentation is your one-stop-shop for information about cellxgene](https://chanzuckerberg.github.io/cellxgene/)! You may be particularly interested in:
* Seeing [what cellxgene can do](https://chanzuckerberg.github.io/cellxgene/posts/gallery)
* Learning more about cellxgene [installation](https://chanzuckerberg.github.io/cellxgene/posts/install) and [usage](https://chanzuckerberg.github.io/cellxgene/posts/launch)
* [Preparing your own data](https://chanzuckerberg.github.io/cellxgene/posts/prepare) for use in cellxgene
* Checking out [our roadmap](https://chanzuckerberg.github.io/cellxgene/posts/roadmap) for future development
* [Contributing](https://chanzuckerberg.github.io/cellxgene/posts/contribute) to cellxgene
- Seeing [what cellxgene can do](https://chanzuckerberg.github.io/cellxgene/posts/gallery)
- Learning more about cellxgene [installation](https://chanzuckerberg.github.io/cellxgene/posts/install) and [usage](https://chanzuckerberg.github.io/cellxgene/posts/launch)
- [Preparing your own data](https://chanzuckerberg.github.io/cellxgene/posts/prepare) for use in cellxgene
- Checking out [our roadmap](https://chanzuckerberg.github.io/cellxgene/posts/roadmap) for future development
- [Contributing](https://chanzuckerberg.github.io/cellxgene/posts/contribute) to cellxgene
### Quick start
To install cellxgene you need Python 3.6+. We recommend [installing cellxgene into a conda or virtual environment.](https://chanzuckerberg.github.io/cellxgene/posts/install)
Install the package.
``` bash
```bash
pip install cellxgene
```
Launch cellxgene with an example [anndata](https://anndata.readthedocs.io/en/latest/) file
``` bash
```bash
cellxgene launch https://cellxgene-example-data.czi.technology/pbmc3k.h5ad
```
@@ -41,6 +45,17 @@ To explore more datasets already formatted for cellxgene, check out the [Demo da
see [Preparing your data](https://chanzuckerberg.github.io/cellxgene/posts/prepare) to learn more about formatting your own
data for cellxgene.
### Supported browsers
cellxgene currently supports the following browsers:
- Google Chrome 61+
- Edge 15+
- Firefox 60+
- Safari 10.1+
Please [file an issue](https://github.com/chanzuckerberg/cellxgene/issues/new/choose) if you would like us to add support for an unsupported browser.
### Finding help
We'd love to hear from you!
@@ -51,6 +66,7 @@ For any errors, [report bugs on Github](https://github.com/chanzuckerberg/cellxg
# Developing with cellxgene
### Contributing
We warmly welcome contributions from the community! Please see our [contributing guide](https://chanzuckerberg.github.io/cellxgene/posts/contribute) and don't hesitate to open an issue or send a pull request to improve cellxgene.
This project adheres to the Contributor Covenant [code of conduct](https://github.com/chanzuckerberg/.github/blob/master/CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to opensource@chanzuckerberg.com.
@@ -64,6 +80,7 @@ This project was started with the sole goal of empowering the scientific communi
If you believe you have found a security issue, we would appreciate notification. Please send email to <security@chanzuckerberg.com>.
# About
### Core team
The current core team:
app.json
-29
View File
@@ -1,29 +0,0 @@
{
"name": "cellxgene",
"description": "An interactive explorer for single-cell transcriptomics data",
"repository": "https://github.com/chanzuckerberg/cellxgene",
"logo": "https://cellxgene-example-data.czi.technology/favicon.png",
"keywords": [
"scientific",
"visualization",
"scrna-seq",
"transcriptomics",
"dataviz"
],
"buildpacks": [
{
"url": "heroku/nodejs"
},
{
"url": "heroku/python"
}
],
"stack": "heroku-18",
"env": {
"DATASET": {
"description": "Link to dataset",
"value": "https://cellxgene-example-data.czi.technology/pbmc3k.h5ad",
"required": "true"
}
}
}
client/Makefile
+1 -1
View File
@@ -1,6 +1,6 @@
include ../common.mk
ANNOTATIONS := $(if $(ANNOTATIONS),$(ANNOTATIONS),../server/test/test_datasets/pbmc3k-annotations.csv)
ANNOTATIONS := $(if $(ANNOTATIONS),$(ANNOTATIONS),../server/test/fixtures/pbmc3k-annotations.csv)
ANNOTATIONS_FILENAME := $(shell basename $(ANNOTATIONS))
# Packaging
client/__tests__/e2e/__snapshots__/e2eAnnotations.test.js.snap
+9 -1
View File
@@ -3,7 +3,7 @@
exports[`annotations stacked bar graph renders 1`] = `
Array [
"<div class=\\"categorical__value___2RKaC\\" data-testclass=\\"categorical-row\\" style=\\"padding: 4px 0px 4px 7px; display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 2px; border-radius: 2px;\\"><div style=\\"margin: 0px; padding: 0px; user-select: none; width: 220px; display: flex; justify-content: space-between;\\"><div style=\\"display: flex; align-items: baseline;\\"><label for=\\"value-toggle-checkbox-TEST-CATEGORY-TEST-LABEL\\" class=\\"bp3-control bp3-checkbox\\" style=\\"margin: 0px;\\"><input id=\\"value-toggle-checkbox-TEST-CATEGORY-TEST-LABEL\\" data-testclass=\\"categorical-value-select\\" data-testid=\\"categorical-value-select-TEST-CATEGORY-TEST-LABEL\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"categorical-value-TEST-CATEGORY-TEST-LABEL\\" data-testclass=\\"categorical-value\\" aria-label=\\"TEST-LABEL\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 63px; color: black; font-style: normal; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 63px; color: black; font-style: normal; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">TEST-</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">LABEL</span><span style=\\"position: absolute; right: 0px; color: black;\\">LABEL</span></span></span></span></span></span></div><span style=\\"flex-shrink: 0;\\"></span></div><div><span><span data-testclass=\\"categorical-value-count\\" data-testid=\\"categorical-value-count-TEST-CATEGORY-TEST-LABEL\\" style=\\"color: black;\\">0</span><svg display=\\"none\\" style=\\"margin-left: 5px; width: 11px; height: 11px; background-color: inherit;\\"></svg><span><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><button type=\\"button\\" data-testclass=\\"seeActions\\" data-testid=\\"TEST-CATEGORY:TEST-LABEL:see-actions\\" class=\\"bp3-button bp3-minimal bp3-small\\" tabindex=\\"0\\" style=\\"margin-left: 2px; position: relative; top: -1px; min-height: 16px;\\"><span icon=\\"more\\" class=\\"bp3-icon bp3-icon-more\\"><svg data-icon=\\"more\\" width=\\"10\\" height=\\"10\\" viewBox=\\"0 0 16 16\\"><desc>more</desc><path d=\\"M2 6.03a2 2 0 100 4 2 2 0 100-4zM14 6.03a2 2 0 100 4 2 2 0 100-4zM8 6.03a2 2 0 100 4 2 2 0 100-4z\\" fill-rule=\\"evenodd\\"></path></svg></span></button></span></span></span></span></div></div>",
"<div class=\\"categorical__value___2RKaC\\" data-testclass=\\"categorical-row\\" style=\\"padding: 4px 0px 4px 7px; display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 2px; border-radius: 2px;\\"><div style=\\"margin: 0px; padding: 0px; user-select: none; width: 220px; display: flex; justify-content: space-between;\\"><div style=\\"display: flex; align-items: baseline;\\"><label for=\\"value-toggle-checkbox-TEST-CATEGORY-unassigned\\" class=\\"bp3-control bp3-checkbox\\" style=\\"margin: 0px;\\"><input id=\\"value-toggle-checkbox-TEST-CATEGORY-unassigned\\" data-testclass=\\"categorical-value-select\\" data-testid=\\"categorical-value-select-TEST-CATEGORY-unassigned\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"categorical-value-TEST-CATEGORY-unassigned\\" data-testclass=\\"categorical-value\\" aria-label=\\"unassigned\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">unass</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">igned</span><span style=\\"position: absolute; right: 0px; color: rgb(171, 171, 171);\\">igned</span></span></span></span></span></span></div><span style=\\"flex-shrink: 0;\\"><canvas class=\\"bp3-popover-targer\\" width=\\"100\\" height=\\"11\\" style=\\"margin-right: 5px; width: 100px; height: 11px;\\"></canvas></span></div><div><span><span data-testclass=\\"categorical-value-count\\" data-testid=\\"categorical-value-count-TEST-CATEGORY-unassigned\\" style=\\"color: rgb(171, 171, 171); font-style: italic;\\">2132</span><svg display=\\"none\\" style=\\"margin-left: 5px; width: 11px; height: 11px; background-color: inherit;\\"></svg><span><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><button type=\\"button\\" data-testclass=\\"seeActions\\" data-testid=\\"TEST-CATEGORY:unassigned:see-actions\\" class=\\"bp3-button bp3-minimal bp3-small\\" tabindex=\\"0\\" style=\\"margin-left: 2px; position: relative; top: -1px; min-height: 16px;\\"><span icon=\\"more\\" class=\\"bp3-icon bp3-icon-more\\"><svg data-icon=\\"more\\" width=\\"10\\" height=\\"10\\" viewBox=\\"0 0 16 16\\"><desc>more</desc><path d=\\"M2 6.03a2 2 0 100 4 2 2 0 100-4zM14 6.03a2 2 0 100 4 2 2 0 100-4zM8 6.03a2 2 0 100 4 2 2 0 100-4z\\" fill-rule=\\"evenodd\\"></path></svg></span></button></span></span></span></span></div></div>",
"<div class=\\"categorical__value___2RKaC\\" data-testclass=\\"categorical-row\\" style=\\"padding: 4px 0px 4px 7px; display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 2px; border-radius: 2px;\\"><div style=\\"margin: 0px; padding: 0px; user-select: none; width: 220px; display: flex; justify-content: space-between;\\"><div style=\\"display: flex; align-items: baseline;\\"><label for=\\"value-toggle-checkbox-TEST-CATEGORY-unassigned\\" class=\\"bp3-control bp3-checkbox\\" style=\\"margin: 0px;\\"><input id=\\"value-toggle-checkbox-TEST-CATEGORY-unassigned\\" data-testclass=\\"categorical-value-select\\" data-testid=\\"categorical-value-select-TEST-CATEGORY-unassigned\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"categorical-value-TEST-CATEGORY-unassigned\\" data-testclass=\\"categorical-value\\" aria-label=\\"unassigned\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">unass</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">igned</span><span style=\\"position: absolute; right: 0px; color: rgb(171, 171, 171);\\">igned</span></span></span></span></span></span></div><span style=\\"flex-shrink: 0;\\"><canvas class=\\"bp3-popover-targer\\" width=\\"100\\" height=\\"11\\" style=\\"margin-right: 5px; width: 100px; height: 11px;\\"></canvas></span></div><div><span><span data-testclass=\\"categorical-value-count\\" data-testid=\\"categorical-value-count-TEST-CATEGORY-unassigned\\" style=\\"color: rgb(171, 171, 171); font-style: italic;\\">2133</span><svg display=\\"none\\" style=\\"margin-left: 5px; width: 11px; height: 11px; background-color: inherit;\\"></svg><span><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><button type=\\"button\\" data-testclass=\\"seeActions\\" data-testid=\\"TEST-CATEGORY:unassigned:see-actions\\" class=\\"bp3-button bp3-minimal bp3-small\\" tabindex=\\"0\\" style=\\"margin-left: 2px; position: relative; top: -1px; min-height: 16px;\\"><span icon=\\"more\\" class=\\"bp3-icon bp3-icon-more\\"><svg data-icon=\\"more\\" width=\\"10\\" height=\\"10\\" viewBox=\\"0 0 16 16\\"><desc>more</desc><path d=\\"M2 6.03a2 2 0 100 4 2 2 0 100-4zM14 6.03a2 2 0 100 4 2 2 0 100-4zM8 6.03a2 2 0 100 4 2 2 0 100-4z\\" fill-rule=\\"evenodd\\"></path></svg></span></button></span></span></span></span></div></div>",
]
`;
@@ -13,3 +13,11 @@ Array [
"<div class=\\"categorical__value___2RKaC\\" data-testclass=\\"categorical-row\\" style=\\"padding: 4px 0px 4px 7px; display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 2px; border-radius: 2px;\\"><div style=\\"margin: 0px; padding: 0px; user-select: none; width: 220px; display: flex; justify-content: space-between;\\"><div style=\\"display: flex; align-items: baseline;\\"><label for=\\"value-toggle-checkbox-TEST-CATEGORY-unassigned\\" class=\\"bp3-control bp3-checkbox\\" style=\\"margin: 0px;\\"><input id=\\"value-toggle-checkbox-TEST-CATEGORY-unassigned\\" data-testclass=\\"categorical-value-select\\" data-testid=\\"categorical-value-select-TEST-CATEGORY-unassigned\\" type=\\"checkbox\\" checked=\\"\\"><span class=\\"bp3-control-indicator\\"></span></label><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><span data-testid=\\"categorical-value-TEST-CATEGORY-unassigned\\" data-testclass=\\"categorical-value\\" aria-label=\\"unassigned\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 63px; color: rgb(171, 171, 171); font-style: italic; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">unass</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">igned</span><span style=\\"position: absolute; right: 0px; color: rgb(171, 171, 171);\\">igned</span></span></span></span></span></span></div><span style=\\"flex-shrink: 0;\\"><canvas class=\\"bp3-popover-targer\\" width=\\"100\\" height=\\"11\\" style=\\"margin-right: 5px; width: 100px; height: 11px;\\"></canvas></span></div><div><span><span data-testclass=\\"categorical-value-count\\" data-testid=\\"categorical-value-count-TEST-CATEGORY-unassigned\\" style=\\"color: rgb(171, 171, 171); font-style: italic;\\">2638</span><svg display=\\"none\\" style=\\"margin-left: 5px; width: 11px; height: 11px; background-color: inherit;\\"></svg><span><span class=\\"bp3-popover-wrapper\\"><span class=\\"bp3-popover-target\\"><button type=\\"button\\" data-testclass=\\"seeActions\\" data-testid=\\"TEST-CATEGORY:unassigned:see-actions\\" class=\\"bp3-button bp3-minimal bp3-small\\" tabindex=\\"0\\" style=\\"margin-left: 2px; position: relative; top: -1px; min-height: 16px;\\"><span icon=\\"more\\" class=\\"bp3-icon bp3-icon-more\\"><svg data-icon=\\"more\\" width=\\"10\\" height=\\"10\\" viewBox=\\"0 0 16 16\\"><desc>more</desc><path d=\\"M2 6.03a2 2 0 100 4 2 2 0 100-4zM14 6.03a2 2 0 100 4 2 2 0 100-4zM8 6.03a2 2 0 100 4 2 2 0 100-4z\\" fill-rule=\\"evenodd\\"></path></svg></span></button></span></span></span></span></div></div>",
]
`;
exports[`annotations truncate midpoint whitespace 1`] = `"<span data-testid=\\"categorical-value-TEST-CATEGORY-123 456\\" data-testclass=\\"categorical-value\\" aria-label=\\"123 456\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 187px; color: black; font-style: normal; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 187px; color: black; font-style: normal; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">123</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">&nbsp;456</span><span style=\\"position: absolute; right: 0px; color: black;\\">&nbsp;456</span></span></span></span>"`;
exports[`annotations truncate midpoint whitespace 2`] = `"<span data-testid=\\"categorical-value-TEST-CATEGORY-123 456\\" data-testclass=\\"categorical-value\\" aria-label=\\"123 456\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 187px; color: black; font-style: normal; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 187px; color: black; font-style: normal; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">123</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\">&nbsp;456</span><span style=\\"position: absolute; right: 0px; color: black;\\">&nbsp;456</span></span></span></span>"`;
exports[`annotations truncate single character 1`] = `"<span data-testid=\\"categorical-value-TEST-CATEGORY-T\\" data-testclass=\\"categorical-value\\" aria-label=\\"T\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 187px; color: black; font-style: normal; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 187px; color: black; font-style: normal; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">T</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\"></span><span style=\\"position: absolute; right: 0px; color: black;\\"></span></span></span></span>"`;
exports[`annotations truncate single character 2`] = `"<span data-testid=\\"categorical-value-TEST-CATEGORY-T\\" data-testclass=\\"categorical-value\\" aria-label=\\"T\\" class=\\"\\" tabindex=\\"0\\" style=\\"width: 187px; color: black; font-style: normal; display: inline-block; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px;\\"><span style=\\"width: 187px; color: black; font-style: normal; display: flex; overflow: hidden; line-height: 1.1em; height: 1.1em; vertical-align: middle; margin-right: 16px; justify-content: flex-start;\\"><span style=\\"overflow: hidden; text-overflow: ellipsis; white-space: nowrap; flex-shrink: 1; min-width: 5px;\\">T</span><span style=\\"position: relative; overflow: hidden; white-space: nowrap;\\"><span style=\\"color: transparent;\\"></span><span style=\\"position: absolute; right: 0px; color: black;\\"></span></span></span></span>"`;
client/__tests__/e2e/data.js
+3 -3
View File
@@ -27,7 +27,7 @@ export const datasets = {
lasso: [
{
"coordinates-as-percent": { x1: 0.1, y1: 0.25, x2: 0.7, y2: 0.75 },
count: "1173",
count: "1131",
},
],
categorical: [
@@ -121,8 +121,8 @@ export const datasets = {
},
newCount: {
bySubsetConfig: {
false: "599",
true: "594",
false: "668",
true: "659",
},
},
},
client/__tests__/e2e/e2eAnnotations.test.js
+22
View File
@@ -297,6 +297,28 @@ describe.each([
expect(result).toMatchSnapshot();
});
test("truncate midpoint whitespace", async () => {
await setup(config);
const newLabelName = "123 456";
await renameLabel(perTestCategoryName, perTestLabelName, newLabelName);
const value = await waitByID(
`categorical-value-${perTestCategoryName}-${newLabelName}`
);
const result = await page.evaluate((elem) => elem.outerHTML, value);
expect(result).toMatchSnapshot();
});
test("truncate single character", async () => {
await setup(config);
const newLabelName = "T";
await renameLabel(perTestCategoryName, perTestLabelName, newLabelName);
const value = await waitByID(
`categorical-value-${perTestCategoryName}-${newLabelName}`
);
const result = await page.evaluate((elem) => elem.outerHTML, value);
expect(result).toMatchSnapshot();
});
async function assertCategoryExists(categoryName) {
const handle = await waitByID(`${categoryName}:category-label`);
client/configuration/babel/babel.dev.js
+8 -1
View File
@@ -2,7 +2,14 @@ module.exports = {
babelrc: false,
cacheDirectory: true,
presets: [
["modern-browsers", { loose: true, modules: false }],
[
"@babel/preset-env",
{
useBuiltIns: "entry",
corejs: 3,
modules: false,
},
],
"@babel/preset-react",
],
plugins: [
client/configuration/babel/babel.prod.js
+8 -1
View File
@@ -1,7 +1,14 @@
module.exports = {
babelrc: false,
presets: [
["modern-browsers", { loose: true, modules: false }],
[
"@babel/preset-env",
{
useBuiltIns: "entry",
corejs: 3,
modules: false,
},
],
"@babel/preset-react",
],
plugins: [
client/configuration/eslint/eslint.js
+12
View File
@@ -4,9 +4,21 @@ module.exports = {
extends: [
"airbnb",
"plugin:eslint-comments/recommended",
"plugin:compat/recommended",
"plugin:prettier/recommended",
"prettier/react",
],
settings: {
polyfills: [
"TextDecoder",
"TextEncoder",
"fetch",
"Request",
"Response",
"Headers",
"AbortController",
],
},
env: { browser: true, commonjs: true, es6: true },
globals: {
expect: true,
client/configuration/webpack/cspHashPlugin.js
+12 -20
View File
@@ -1,7 +1,12 @@
/* eslint-disable import/no-extraneous-dependencies -- this file is a devDependency*/
const cheerio = require("cheerio");
const crypto = require("crypto");
HtmlWebpackPlugin = require("html-webpack-plugin");
const HtmlWebpackPlugin = require("html-webpack-plugin");
const digest = (str) => {
const hash = crypto.createHash("sha256").update(str, "utf8").digest("base64");
return `sha256-${hash}`;
};
class CspHashPlugin {
constructor(opts) {
this.opts = { ...opts };
@@ -19,10 +24,7 @@ class CspHashPlugin {
if (filename) {
const results = {};
results["script-hashes"] = $("script:not([src]):not([no-csp-hash])")
.map((i, elmt) => this.digest($(elmt).html()))
.get();
results["style-hashes"] = $("style:not([href]):not([no-csp-hash])")
.map((i, elmt) => this.digest($(elmt).html()))
.map((i, elmt) => digest($(elmt).html()))
.get();
const json = JSON.stringify(results);
@@ -34,13 +36,10 @@ class CspHashPlugin {
// Remove no-csp-hash attributes. Cheerio does not parse Jinja templates
// correctly, so we brute force this with a regular expression.
data.html = data.html
.replace(/(<script .*)no-csp-hash(.*>)/, (match, p1, p2) =>
[p1, p2].join("")
)
.replace(/(<style .*)no-csp-hash(.*>)/, (match, p1, p2) =>
[p1, p2].join("")
);
data.html = data.html.replace(
/(<script .*)no-csp-hash(.*>)/,
(match, p1, p2) => [p1, p2].join("")
);
// Tell webpack to move on
cb(null, data);
@@ -48,14 +47,7 @@ class CspHashPlugin {
);
});
}
digest(str) {
const hash = crypto
.createHash("sha256")
.update(str, "utf8")
.digest("base64");
return `sha256-${hash}`;
}
}
module.exports = CspHashPlugin;
/* eslint-enable import/no-extraneous-dependencies -- enable*/
client/configuration/webpack/obsoleteHTMLTemplate.html
+82
View File
@@ -0,0 +1,82 @@
<script>
/** Reflect ANY changes to this script in the script hash in `server/eb/app.py` **/
var root = document.getElementById("root");
root.remove();
var portals = document.getElementsByClassName("bp3-portal");
for (var i = 0; i < portals.length; i += 1) {
portals[i].remove();
}
</script>
<div
style="
display: flex;
flex-direction: column;
width: 100vw;
height: 100vh;
text-align: center;
justify-content: center;
align-items: center;
background: #8080801a;
font-family: 'Roboto Condensed, sans serif';
"
>
<img
src="https://cellxgene.cziscience.com/s3/cellxgene/static/images/cellxgene-logo.png"
style="width: 320px;"
/>
<div
style="
margin-top: 16px;
background: white;
width: 40vw;
border-radius: 4px;
padding: 24px 64px;
-webkit-box-shadow: 0px 0px 3px 2px rgba(0, 0, 0, 0.38);
-moz-box-shadow: 0px 0px 3px 2px rgba(0, 0, 0, 0.38);
box-shadow: 0px 0px 3px 2px rgba(0, 0, 0, 0.38);
max-width: 550px;
"
>
<div style="margin-bottom: 0; font-weight: bolder; font-size: 1.2em;">
Unsupported Browser
</div>
<div style="margin-top: 0;">
cellxgene is currently supported on the following browsers
</div>
<div
style="display: flex; justify-content: space-around; margin-top: 16px;"
>
<a
href="https://www.google.com/chrome/?hl=en%22"
aria-label="Download Google Chrome"
>
<img
src="https://cellxgene.cziscience.com/s3/cellxgene/static/images/chrome.png"
style="width: 80px; height: 80px;"
/>
<div>Chrome &gt; 60</div>
</a>
<a href="https://www.apple.com/safari/" aria-label="Download Safari">
<img
src="https://cellxgene.cziscience.com/s3/cellxgene/static/images/safari.png"
style="width: 80px; height: 80px;"
/>
<div>Safari ≥ 10.1</div>
</a>
<a href="https://www.mozilla.com/firefox/" aria-label="Download Firefox">
<img
src="https://cellxgene.cziscience.com/s3/cellxgene/static/images/firefox.png"
style="width: 80px; height: 80px;"
/>
<div>Firefox ≥ 60</div>
</a>
<a href="//www.microsoft.com/edge" aria-label="Download Edge">
<img
src="https://cellxgene.cziscience.com/s3/cellxgene/static/images/edge.png"
style="width: 80px; height: 80px;"
/>
<div>Edge ≥ 15</div>
</a>
</div>
</div>
</div>
client/configuration/webpack/webpack.config.dev.js
+17 -42
View File
@@ -1,65 +1,32 @@
// jshint esversion: 6
const path = require("path");
const webpack = require("webpack");
const HtmlWebpackPlugin = require("html-webpack-plugin");
const FaviconsWebpackPlugin = require("favicons-webpack-plugin");
const ScriptExtHtmlWebpackPlugin = require("script-ext-html-webpack-plugin");
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
const { merge } = require("webpack-merge");
const sharedConfig = require("./webpack.config.shared");
const babelOptions = require("../babel/babel.dev");
const src = path.resolve("src");
const fonts = path.resolve("src/fonts");
const nodeModules = path.resolve("node_modules");
const babelOptions = require("../babel/babel.dev");
module.exports = {
const devConfig = {
mode: "development",
devtool: "eval",
entry: ["./src/index"],
output: {
path: path.resolve("build"),
pathinfo: true,
filename: "static/js/bundle.js",
publicPath: "/",
},
module: {
rules: [
{
test: /\.js$/,
include: src,
test: /\.jsx?$/,
loader: "babel-loader",
options: babelOptions,
},
{
test: /\.css$/,
include: src,
exclude: [path.resolve(src, "index.css")],
loader: [
{
loader: "style-loader",
},
{
loader: "css-loader",
options: {
modules: {
localIdentName: "[name]__[local]___[hash:base64:5]",
},
},
},
],
},
{
test: /index\.css$/,
include: [path.resolve(src, "index.css")],
loader: [
{
loader: "style-loader",
},
{
loader: "css-loader",
},
],
},
{ test: /\.json$/, include: [src, nodeModules], loader: "json-loader" },
{
test: /\.(jpg|png|gif|eot|svg|ttf|woff|woff2|otf)$/i,
loader: "file-loader",
@@ -88,6 +55,9 @@ module.exports = {
},
},
}),
new MiniCssExtractPlugin({
filename: "static/[name].css",
}),
new webpack.NoEmitOnErrorsPlugin(),
new webpack.DefinePlugin({
__REACT_DEVTOOLS_GLOBAL_HOOK__: "({ isDisabled: true })",
@@ -97,5 +67,10 @@ module.exports = {
process.env.CXG_SERVER_PORT
),
}),
new ScriptExtHtmlWebpackPlugin({
async: "obsolete",
}),
],
};
module.exports = merge(sharedConfig, devConfig);
client/configuration/webpack/webpack.config.prod.js
+12 -43
View File
@@ -1,32 +1,28 @@
// jshint esversion: 6
const path = require("path");
const HtmlWebpackPlugin = require("html-webpack-plugin");
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
const FaviconsWebpackPlugin = require("favicons-webpack-plugin");
const { CleanWebpackPlugin } = require("clean-webpack-plugin");
const TerserJSPlugin = require("terser-webpack-plugin");
const CleanCss = require("clean-css");
const OptimizeCSSAssetsPlugin = require("optimize-css-assets-webpack-plugin");
const FaviconsWebpackPlugin = require("favicons-webpack-plugin");
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
const CspHashPlugin = require("./cspHashPlugin");
const src = path.resolve("src");
const fonts = path.resolve("src/fonts");
const nodeModules = path.resolve("node_modules");
const { merge } = require("webpack-merge");
const babelOptions = require("../babel/babel.prod");
const publicPath = "/";
const CspHashPlugin = require("./cspHashPlugin");
const sharedConfig = require("./webpack.config.shared");
module.exports = {
const fonts = path.resolve("src/fonts");
const nodeModules = path.resolve("node_modules");
const prodConfig = {
mode: "production",
bail: true,
cache: false,
entry: ["./src/index.js"],
output: {
filename: "static/[name]-[contenthash].js",
path: path.resolve("build"),
publicPath,
},
optimization: {
minimize: true,
@@ -41,39 +37,10 @@ module.exports = {
module: {
rules: [
{
test: /\.js$/,
include: src,
test: /\.jsx?$/,
loader: "babel-loader",
options: babelOptions,
},
{
test: /\.css$/,
include: src,
exclude: [path.resolve(src, "index.css")],
use: [
MiniCssExtractPlugin.loader,
{
loader: "css-loader",
options: {
modules: {
localIdentName: "[name]__[local]___[hash:base64:5]",
},
importLoaders: 1,
},
},
],
},
{
test: /index\.css$/,
include: [path.resolve(src, "index.css")],
use: [MiniCssExtractPlugin.loader, "css-loader"],
},
{
test: /\.json$/,
include: [src, nodeModules],
loader: "json-loader",
exclude: /manifest.json$/,
},
{
test: /\.(jpg|png|gif|eot|svg|ttf|woff|woff2|otf)$/i,
loader: "file-loader",
@@ -121,3 +88,5 @@ module.exports = {
maxAssetSize: 2000000,
},
};
module.exports = merge(sharedConfig, prodConfig);
client/configuration/webpack/webpack.config.shared.js
+74
View File
@@ -0,0 +1,74 @@
const path = require("path");
const fs = require("fs");
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
const ObsoleteWebpackPlugin = require("obsolete-webpack-plugin");
const ScriptExtHtmlWebpackPlugin = require("script-ext-html-webpack-plugin");
const src = path.resolve("src");
const nodeModules = path.resolve("node_modules");
const publicPath = "/";
const rawObsoleteHTMLTemplate = fs.readFileSync(
`${__dirname}/obsoleteHTMLTemplate.html`,
"utf8"
);
const obsoleteHTMLTemplate = rawObsoleteHTMLTemplate.replace(/'/g, '"');
module.exports = {
entry: [
"core-js",
"regenerator-runtime/runtime",
"fastestsmallesttextencoderdecoder",
"whatwg-fetch",
"abort-controller/polyfill",
"./src/index",
],
output: {
path: path.resolve("build"),
publicPath,
},
module: {
rules: [
{
test: /\.css$/,
include: src,
exclude: [path.resolve(src, "index.css")],
use: [
MiniCssExtractPlugin.loader,
{
loader: "css-loader",
options: {
modules: {
localIdentName: "[name]__[local]___[hash:base64:5]",
},
importLoaders: 1,
},
},
],
},
{
test: /index\.css$/,
include: [path.resolve(src, "index.css")],
use: [MiniCssExtractPlugin.loader, "css-loader"],
},
{
test: /\.json$/,
include: [src, nodeModules],
loader: "json-loader",
exclude: /manifest.json$/,
},
],
},
plugins: [
new ObsoleteWebpackPlugin({
name: "obsolete",
template: obsoleteHTMLTemplate,
promptOnNonTargetBrowser: false,
}),
new ScriptExtHtmlWebpackPlugin({
async: "obsolete",
}),
],
};
client/index_template.html
+1 -1
View File
@@ -39,7 +39,7 @@
<script type="text/javascript">
window.CELLXGENE = {};
window.CELLXGENE.API = {
prefix: window.location.href + "api/",
prefix: `${location.origin}${location.pathname}api/`,
version: "v0.2/",
};
</script>
client/package-lock.json
Generated
+233 -14
View File
@@ -1,6 +1,6 @@
{
"name": "cellxgene",
"version": "0.15.0",
"version": "0.16.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
@@ -4069,6 +4069,24 @@
"regenerator-runtime": "^0.13.4"
}
},
"@babel/runtime-corejs2": {
"version": "7.10.5",
"resolved": "https://registry.npmjs.org/@babel/runtime-corejs2/-/runtime-corejs2-7.10.5.tgz",
"integrity": "sha512-LJwyb1ac//Jr2zrGTTaNJhrP1wYCgVw9rzHbQPogKXCTLQ60EEWgeNtuqs6cLsq64O557SYzziCrOxNp0rRi8w==",
"dev": true,
"requires": {
"core-js": "^2.6.5",
"regenerator-runtime": "^0.13.4"
},
"dependencies": {
"core-js": {
"version": "2.6.11",
"resolved": "https://registry.npmjs.org/core-js/-/core-js-2.6.11.tgz",
"integrity": "sha512-5wjnpaT/3dV+XB4borEsnAYQchn00XSgTAWKDkEqv+K8KevjbzmofK6hfJ9TZIlpj2N0xQpazy7PiRQiWHqzWg==",
"dev": true
}
}
},
"@babel/runtime-corejs3": {
"version": "7.10.5",
"resolved": "https://registry.npmjs.org/@babel/runtime-corejs3/-/runtime-corejs3-7.10.5.tgz",
@@ -5686,6 +5704,14 @@
"resolved": "https://registry.npmjs.org/abab/-/abab-2.0.3.tgz",
"integrity": "sha512-tsFzPpcttalNjFBCFMqsKYQcWxxen1pgJR56by//QwvJc4/OUS3kPOOttx2tSIfjsylB0pYu7f5D3K1RCxUnUg=="
},
"abort-controller": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz",
"integrity": "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==",
"requires": {
"event-target-shim": "^5.0.0"
}
},
"accepts": {
"version": "1.3.7",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.7.tgz",
@@ -6057,6 +6083,12 @@
"resolved": "https://registry.npmjs.org/assign-symbols/-/assign-symbols-1.0.0.tgz",
"integrity": "sha1-WWZ/QfrdTyDMvCu5a41Pf3jsA2c="
},
"ast-metadata-inferer": {
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/ast-metadata-inferer/-/ast-metadata-inferer-0.4.0.tgz",
"integrity": "sha512-tKHdBe8N/Vq2nLAm4YPBVREVZjMux6KrqyPfNQgIbDl0t7HaNSmy8w4OyVHYg/cvyn5BW7o7pVwpjPte89Zhcg==",
"dev": true
},
"ast-types-flow": {
"version": "0.0.7",
"resolved": "https://registry.npmjs.org/ast-types-flow/-/ast-types-flow-0.0.7.tgz",
@@ -6944,6 +6976,12 @@
"lodash.uniq": "^4.5.0"
}
},
"caniuse-db": {
"version": "1.0.30001107",
"resolved": "https://registry.npmjs.org/caniuse-db/-/caniuse-db-1.0.30001107.tgz",
"integrity": "sha512-ffbV17yvEamsNm4N4dDDHdj147tWwdKw+mGyeOmvQcnu+gu455xUg8degvUOCB+fIAm7Rv3gXVn7XlTiYKymMQ==",
"dev": true
},
"caniuse-lite": {
"version": "1.0.30001039",
"resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001039.tgz",
@@ -7469,9 +7507,9 @@
"dev": true
},
"codecov": {
"version": "3.7.0",
"resolved": "https://registry.npmjs.org/codecov/-/codecov-3.7.0.tgz",
"integrity": "sha512-uIixKofG099NbUDyzRk1HdGtaG8O+PBUAg3wfmjwXw2+ek+PZp+puRvbTohqrVfuudaezivJHFgTtSC3M8MXww==",
"version": "3.7.1",
"resolved": "https://registry.npmjs.org/codecov/-/codecov-3.7.1.tgz",
"integrity": "sha512-JHWxyPTkMLLJn9SmKJnwAnvY09kg2Os2+Ux+GG7LwZ9g8gzDDISpIN5wAsH1UBaafA/yGcd3KofMaorE8qd6Lw==",
"dev": true,
"requires": {
"argv": "0.0.2",
@@ -7725,10 +7763,9 @@
"integrity": "sha1-Z29us8OZl8LuGsOpJP1hJHSPV40="
},
"core-js": {
"version": "2.6.11",
"resolved": "https://registry.npmjs.org/core-js/-/core-js-2.6.11.tgz",
"integrity": "sha512-5wjnpaT/3dV+XB4borEsnAYQchn00XSgTAWKDkEqv+K8KevjbzmofK6hfJ9TZIlpj2N0xQpazy7PiRQiWHqzWg==",
"dev": true
"version": "3.6.5",
"resolved": "https://registry.npmjs.org/core-js/-/core-js-3.6.5.tgz",
"integrity": "sha512-vZVEEwZoIsI+vPEuoF9Iqf5H7/M3eeQqWlQnYa8FSKKePuYTf5MWnxb5SDAzCa60b3JBRS5g9b+Dq7b1y/RCrA=="
},
"core-js-compat": {
"version": "3.6.5",
@@ -8946,9 +8983,9 @@
"dev": true
},
"elliptic": {
"version": "6.5.2",
"resolved": "https://registry.npmjs.org/elliptic/-/elliptic-6.5.2.tgz",
"integrity": "sha512-f4x70okzZbIQl/NSRLkI/+tteV/9WqL98zx+SQ69KbXxmVrmjwsNUPn/gYJJ0sHvEak24cZgHIPegRePAtA/xw==",
"version": "6.5.3",
"resolved": "https://registry.npmjs.org/elliptic/-/elliptic-6.5.3.tgz",
"integrity": "sha512-IMqzv5wNQf+E6aHeIqATs0tOLeOTwj1QKbRcS3jBbYkl5oLAserA8yJTT7/VyHUYG91PRmPyeQDObKLPpeS4dw==",
"dev": true,
"requires": {
"bn.js": "^4.4.0",
@@ -9448,6 +9485,94 @@
}
}
},
"eslint-plugin-compat": {
"version": "3.8.0",
"resolved": "https://registry.npmjs.org/eslint-plugin-compat/-/eslint-plugin-compat-3.8.0.tgz",
"integrity": "sha512-5CuWUSZXZkXLCQJBriEpndn/YWrvggDSHTpRJq++kR8GVcsWbTdp8Eh+nBA7JlrNi7ZJ/+kniOVXmn3bpnxuRA==",
"dev": true,
"requires": {
"ast-metadata-inferer": "^0.4.0",
"browserslist": "^4.12.2",
"caniuse-db": "^1.0.30001090",
"core-js": "^3.6.5",
"find-up": "^4.1.0",
"lodash.memoize": "4.1.2",
"mdn-browser-compat-data": "^1.0.28",
"semver": "7.3.2"
},
"dependencies": {
"browserslist": {
"version": "4.13.0",
"resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.13.0.tgz",
"integrity": "sha512-MINatJ5ZNrLnQ6blGvePd/QOz9Xtu+Ne+x29iQSCHfkU5BugKVJwZKn/iiL8UbpIpa3JhviKjz+XxMo0m2caFQ==",
"dev": true,
"requires": {
"caniuse-lite": "^1.0.30001093",
"electron-to-chromium": "^1.3.488",
"escalade": "^3.0.1",
"node-releases": "^1.1.58"
}
},
"caniuse-lite": {
"version": "1.0.30001107",
"resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001107.tgz",
"integrity": "sha512-86rCH+G8onCmdN4VZzJet5uPELII59cUzDphko3thQFgAQG1RNa+sVLDoALIhRYmflo5iSIzWY3vu1XTWtNMQQ==",
"dev": true
},
"electron-to-chromium": {
"version": "1.3.510",
"resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.3.510.tgz",
"integrity": "sha512-sLtGB0znXdmo6lM8hy5wTVo+fLqvIuO8hEpgc0DvPmFZqvBu/WB7AarEwhxVKjf3rVbws/rC8Xf+AlsOb36lJQ==",
"dev": true
},
"find-up": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/find-up/-/find-up-4.1.0.tgz",
"integrity": "sha512-PpOwAdQ/YlXQ2vj8a3h8IipDuYRi3wceVQQGYWxNINccq40Anw7BlsEXCMbt1Zt+OLA6Fq9suIpIWD0OsnISlw==",
"dev": true,
"requires": {
"locate-path": "^5.0.0",
"path-exists": "^4.0.0"
}
},
"locate-path": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/locate-path/-/locate-path-5.0.0.tgz",
"integrity": "sha512-t7hw9pI+WvuwNJXwk5zVHpyhIqzg2qTlklJOf0mVxGSbe3Fp2VieZcduNYjaLDoy6p9uGpQEGWG87WpMKlNq8g==",
"dev": true,
"requires": {
"p-locate": "^4.1.0"
}
},
"node-releases": {
"version": "1.1.60",
"resolved": "https://registry.npmjs.org/node-releases/-/node-releases-1.1.60.tgz",
"integrity": "sha512-gsO4vjEdQaTusZAEebUWp2a5d7dF5DYoIpDG7WySnk7BuZDW+GPpHXoXXuYawRBr/9t5q54tirPz79kFIWg4dA==",
"dev": true
},
"p-locate": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/p-locate/-/p-locate-4.1.0.tgz",
"integrity": "sha512-R79ZZ/0wAxKGu3oYMlz8jy/kbhsNrS7SKZ7PxEHBgJ5+F2mtFW2fK2cOtBh1cHYkQsbzFV7I+EoRKe6Yt0oK7A==",
"dev": true,
"requires": {
"p-limit": "^2.2.0"
}
},
"path-exists": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/path-exists/-/path-exists-4.0.0.tgz",
"integrity": "sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w==",
"dev": true
},
"semver": {
"version": "7.3.2",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.3.2.tgz",
"integrity": "sha512-OrOb32TeeambH6UrhtShmF7CRDqhL6/5XpPNp2DuRH6+9QLw/orhp72j87v8Qa1ScDkvrrBNpZcDejAirJmfXQ==",
"dev": true
}
}
},
"eslint-plugin-eslint-comments": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/eslint-plugin-eslint-comments/-/eslint-plugin-eslint-comments-3.2.0.tgz",
@@ -9716,6 +9841,11 @@
"integrity": "sha1-Qa4u62XvpiJorr/qg6x9eSmbCIc=",
"dev": true
},
"event-target-shim": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/event-target-shim/-/event-target-shim-5.0.1.tgz",
"integrity": "sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ=="
},
"events": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/events/-/events-3.1.0.tgz",
@@ -10067,6 +10197,11 @@
"resolved": "https://registry.npmjs.org/fast-levenshtein/-/fast-levenshtein-2.0.6.tgz",
"integrity": "sha1-PYpcZog6FqMMqGQ+hR8Zuqd5eRc="
},
"fastestsmallesttextencoderdecoder": {
"version": "1.0.22",
"resolved": "https://registry.npmjs.org/fastestsmallesttextencoderdecoder/-/fastestsmallesttextencoderdecoder-1.0.22.tgz",
"integrity": "sha512-Pb8d48e+oIuY4MaM64Cd7OW1gt4nxCHs7/ddPPZ/Ic3sg8yVGM7O9wDvZ7us6ScaUupzM+pfBolwtYhN1IxBIw=="
},
"favicons": {
"version": "5.5.0",
"resolved": "https://registry.npmjs.org/favicons/-/favicons-5.5.0.tgz",
@@ -13886,6 +14021,15 @@
"safe-buffer": "^5.1.2"
}
},
"mdn-browser-compat-data": {
"version": "1.0.32",
"resolved": "https://registry.npmjs.org/mdn-browser-compat-data/-/mdn-browser-compat-data-1.0.32.tgz",
"integrity": "sha512-dqIstpk2ysqa6XcI8/fz1yB6bOKrIs61RIEE00Dj7+WHReXlGrCIiol1NBPsLUNE+HC/4y2f8va8vy1WsiCkAQ==",
"dev": true,
"requires": {
"extend": "3.0.2"
}
},
"mdn-data": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/mdn-data/-/mdn-data-2.0.4.tgz",
@@ -14769,6 +14913,26 @@
"has": "^1.0.3"
}
},
"obsolete-web": {
"version": "0.5.6",
"resolved": "https://registry.npmjs.org/obsolete-web/-/obsolete-web-0.5.6.tgz",
"integrity": "sha512-rrs7kSJxOVFvvY7wuCfjg/ngXbO6q1m7Llq30RaN9WYIDH1zAoA1xWLIY39D3e5967g/NeCasI/o8ojhMMkaaA==",
"dev": true,
"requires": {
"@babel/runtime-corejs2": "^7.0.0"
}
},
"obsolete-webpack-plugin": {
"version": "0.5.6",
"resolved": "https://registry.npmjs.org/obsolete-webpack-plugin/-/obsolete-webpack-plugin-0.5.6.tgz",
"integrity": "sha512-oKlRW4ycxJfF/mojtpGuQwaP+J4JwIgjFuFnMgURB6AaKxAVaRwiO0oWhqYjwwJ5LxhVybOl+CnGAlhHBHBdEQ==",
"dev": true,
"requires": {
"browserslist": "^4.0.0",
"obsolete-web": "^0.5.6",
"webpack-sources": "^1.0.0"
}
},
"omggif": {
"version": "1.0.10",
"resolved": "https://registry.npmjs.org/omggif/-/omggif-1.0.10.tgz",
@@ -16491,9 +16655,9 @@
}
},
"regenerator-runtime": {
"version": "0.13.5",
"resolved": "https://registry.npmjs.org/regenerator-runtime/-/regenerator-runtime-0.13.5.tgz",
"integrity": "sha512-ZS5w8CpKFinUzOwW3c83oPeVXoNsrLsaCoLtJvAClH135j/R77RuymhiSErhm2lKcwSCIpmvIWSbDkIfAqKQlA=="
"version": "0.13.7",
"resolved": "https://registry.npmjs.org/regenerator-runtime/-/regenerator-runtime-0.13.7.tgz",
"integrity": "sha512-a54FxoJDIr27pgf7IgeQGxmqUNYrcV338lf/6gH456HZ/PhX+5BcwHXG9ajESmwe6WRO0tAzRUrRmNONWgkrew=="
},
"regenerator-transform": {
"version": "0.14.5",
@@ -17147,6 +17311,14 @@
"ajv-keywords": "^3.1.0"
}
},
"script-ext-html-webpack-plugin": {
"version": "2.1.4",
"resolved": "https://registry.npmjs.org/script-ext-html-webpack-plugin/-/script-ext-html-webpack-plugin-2.1.4.tgz",
"integrity": "sha512-7MAv3paAMfh9y2Rg+yQKp9jEGC5cEcmdge4EomRqri10qoczmliYEVPVNz0/5e9QQ202e05qDll9B8zZlY9N1g==",
"requires": {
"debug": "^4.1.1"
}
},
"semver": {
"version": "5.7.1",
"resolved": "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz",
@@ -19513,6 +19685,14 @@
"minimist": "^1.2.0",
"request": "^2.88.0",
"rx": "^4.1.0"
},
"dependencies": {
"core-js": {
"version": "2.6.11",
"resolved": "https://registry.npmjs.org/core-js/-/core-js-2.6.11.tgz",
"integrity": "sha512-5wjnpaT/3dV+XB4borEsnAYQchn00XSgTAWKDkEqv+K8KevjbzmofK6hfJ9TZIlpj2N0xQpazy7PiRQiWHqzWg==",
"dev": true
}
}
},
"wait-port": {
@@ -20168,6 +20348,35 @@
"uuid": "^3.3.2"
}
},
"webpack-merge": {
"version": "5.0.9",
"resolved": "https://registry.npmjs.org/webpack-merge/-/webpack-merge-5.0.9.tgz",
"integrity": "sha512-P4teh6O26xIDPugOGX61wPxaeP918QOMjmzhu54zTVcLtOS28ffPWtnv+ilt3wscwBUCL2WNMnh97XkrKqt9Fw==",
"requires": {
"clone-deep": "^4.0.1",
"wildcard": "^2.0.0"
},
"dependencies": {
"clone-deep": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/clone-deep/-/clone-deep-4.0.1.tgz",
"integrity": "sha512-neHB9xuzh/wk0dIHweyAXv2aPGZIVk3pLMe+/RNzINf17fe0OG96QroktYAUm7SM1PBnzTabaLboqqxDyMU+SQ==",
"requires": {
"is-plain-object": "^2.0.4",
"kind-of": "^6.0.2",
"shallow-clone": "^3.0.0"
}
},
"shallow-clone": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/shallow-clone/-/shallow-clone-3.0.1.tgz",
"integrity": "sha512-/6KqX+GVUdqPuPPd2LxDDxzX6CAbjJehAAOKlNpqqUpAqPM6HeL8f+o3a+JsyGjn2lv0WY8UsTgUJjU9Ok55NA==",
"requires": {
"kind-of": "^6.0.2"
}
}
}
},
"webpack-sources": {
"version": "1.4.3",
"resolved": "https://registry.npmjs.org/webpack-sources/-/webpack-sources-1.4.3.tgz",
@@ -20194,6 +20403,11 @@
"iconv-lite": "0.4.24"
}
},
"whatwg-fetch": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/whatwg-fetch/-/whatwg-fetch-3.2.0.tgz",
"integrity": "sha512-SdGPoQMMnzVYThUbSrEvqTlkvC1Ux27NehaJ/GUHBfNrh5Mjg+1/uRyFMwVnxO2MrikMWvWAqUGgQOfVU4hT7w=="
},
"whatwg-mimetype": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/whatwg-mimetype/-/whatwg-mimetype-2.3.0.tgz",
@@ -20319,6 +20533,11 @@
}
}
},
"wildcard": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/wildcard/-/wildcard-2.0.0.tgz",
"integrity": "sha512-JcKqAHLPxcdb9KM49dufGXn2x3ssnfjbcaQdLlfZsL9rH9wgDQjUtDxbo8NE0F6SFvydeu1VhZe7hZuHsB2/pw=="
},
"word-wrap": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/word-wrap/-/word-wrap-1.2.3.tgz",
client/package.json
+20 -3
View File
@@ -1,6 +1,6 @@
{
"name": "cellxgene",
"version": "0.15.0",
"version": "0.16.0",
"license": "MIT",
"description": "cellxgene is a web application for the interactive exploration of single cell sequence data.",
"repository": "https://github.com/chanzuckerberg/cellxgene",
@@ -29,12 +29,23 @@
"resolutions": {
"eslint-scope": "3.7.1"
},
"browserslist": [
"Chrome > 60",
"Safari >= 10.1",
"iOS >= 10.3",
"Firefox >= 60",
"Edge >= 15",
"not Explorer > 0"
],
"dependencies": {
"@blueprintjs/core": "^3.30.0",
"@blueprintjs/icons": "^3.19.0",
"@blueprintjs/select": "^3.13.5",
"abort-controller": "^3.0.0",
"core-js": "^3.6.5",
"d3": "^4.10.0",
"d3-scale-chromatic": "^1.5.0",
"fastestsmallesttextencoderdecoder": "^1.0.22",
"flatbuffers": "^1.11.0",
"fuzzysort": "^1.1.4",
"gl-mat4": "^1.2.0",
@@ -52,8 +63,12 @@
"react-redux": "^7.2.0",
"redux": "^4.0.5",
"redux-thunk": "^2.3.0",
"regenerator-runtime": "^0.13.7",
"regl": "^1.6.1",
"tinyqueue": "^2.0.3"
"script-ext-html-webpack-plugin": "^2.1.4",
"tinyqueue": "^2.0.3",
"webpack-merge": "^5.0.9",
"whatwg-fetch": "^3.2.0"
},
"devDependencies": {
"@babel/core": "^7.10.5",
@@ -78,13 +93,14 @@
"cheerio": "^1.0.0-rc.3",
"clean-css": "^4.2.3",
"clean-webpack-plugin": "^3.0.0",
"codecov": "^3.7.0",
"codecov": "^3.7.1",
"connect-history-api-fallback": "^1.6.0",
"css-loader": "^3.6.0",
"eslint": "^7.4.0",
"eslint-config-airbnb": "^18.2.0",
"eslint-config-prettier": "^6.11.0",
"eslint-loader": "^3.0.4",
"eslint-plugin-compat": "^3.8.0",
"eslint-plugin-eslint-comments": "^3.2.0",
"eslint-plugin-filenames": "^1.3.2",
"eslint-plugin-import": "^2.22.0",
@@ -107,6 +123,7 @@
"json-loader": "^0.5.7",
"lint-staged": "^10.2.11",
"mini-css-extract-plugin": "^0.9.0",
"obsolete-webpack-plugin": "^0.5.6",
"optimize-css-assets-webpack-plugin": "^5.0.3",
"prettier": "^2.0.5",
"puppeteer": "^3.3.0",
client/src/actions/annotation.js
+19 -10
View File
@@ -34,7 +34,8 @@ export const annotationCreateCategoryAction = (
throw new Error("name collision on annotation category create");
let initialValue;
let categories;
let newSchema;
let ctor;
if (categoryToDuplicate) {
/* if we are duplicating a category, retrieve it */
const catDupSchema = schema.annotations.obsByName[categoryToDuplicate];
@@ -47,25 +48,33 @@ export const annotationCreateCategoryAction = (
.fetch("obs", categoryToDuplicate);
const col = catToDupDf.col(categoryToDuplicate);
initialValue = col.asArray();
({ categories } = col.summarize());
const { categories } = col.summarizeCategorical();
// all user-created annotations must have the unassigned category
if (!categories.includes(globals.unassignedCategoryLabel)) {
categories.push(globals.unassignedCategoryLabel);
}
ctor = initialValue.constructor;
newSchema = {
...catDupSchema,
name: newCategoryName,
categories,
writable: true,
};
} else {
/* else assign to the standard default value */
initialValue = globals.unassignedCategoryLabel;
categories = [globals.unassignedCategoryLabel];
ctor = Array;
newSchema = {
name: newCategoryName,
type: "categorical",
categories: [globals.unassignedCategoryLabel],
writable: true,
};
}
const obsCrossfilter = prevObsCrossfilter.addObsColumn(
{
name: newCategoryName,
type: "categorical",
categories,
writable: true,
},
Array,
newSchema,
ctor,
initialValue
);
client/src/actions/embedding.js
+44
View File
@@ -0,0 +1,44 @@
/*
action creators related to embeddings choice
*/
import { AnnoMatrixObsCrossfilter } from "../annoMatrix";
import { _setEmbeddingSubset } from "../util/stateManager/viewStackHelpers";
export async function _switchEmbedding(prevAnnoMatrix, newEmbeddingName) {
/*
DRY helper used by this and reembedding action creators
*/
const base = prevAnnoMatrix.base();
const embeddingDf = await base.fetch("emb", newEmbeddingName);
const annoMatrix = _setEmbeddingSubset(prevAnnoMatrix, embeddingDf);
const obsCrossfilter = await new AnnoMatrixObsCrossfilter(annoMatrix).select(
"emb",
newEmbeddingName,
{
mode: "all",
}
);
return [annoMatrix, obsCrossfilter];
}
export const layoutChoiceAction = (newLayoutChoice) => async (
dispatch,
getState
) => {
/*
On layout choice, make sure we have selected all on the previous layout, AND the new
layout.
*/
const { annoMatrix: prevAnnoMatrix } = getState();
const [annoMatrix, obsCrossfilter] = await _switchEmbedding(
prevAnnoMatrix,
newLayoutChoice
);
dispatch({
type: "set layout choice",
layoutChoice: newLayoutChoice,
obsCrossfilter,
annoMatrix,
});
};
client/src/actions/index.js
+23 -2
View File
@@ -12,6 +12,7 @@ import { loadUserColorConfig } from "../util/stateManager/colorHelpers";
import * as selnActions from "./selection";
import * as annoActions from "./annotation";
import * as viewActions from "./viewStack";
import * as embActions from "./embedding";
/*
return promise fetching user-configured colors
@@ -40,6 +41,15 @@ async function configFetch(dispatch) {
});
}
function prefetchEmbeddings(annoMatrix) {
/*
prefetch requests for all embeddings
*/
const { schema } = annoMatrix;
const available = schema.layout.obs.map((v) => v.name);
available.forEach((embName) => annoMatrix.prefetch("emb", embName));
}
/*
Application bootstrap
*/
@@ -48,7 +58,7 @@ const doInitialDataLoad = () =>
dispatch({ type: "initial data load start" });
try {
const [, schema] = await Promise.all([
const [config, schema] = await Promise.all([
configFetch(dispatch),
schemaFetch(dispatch),
userColorsFetchAndLoad(dispatch),
@@ -57,12 +67,23 @@ const doInitialDataLoad = () =>
const baseDataUrl = `${globals.API.prefix}${globals.API.version}`;
const annoMatrix = new AnnoMatrixLoader(baseDataUrl, schema.schema);
const obsCrossfilter = new AnnoMatrixObsCrossfilter(annoMatrix);
prefetchEmbeddings(annoMatrix);
dispatch({
type: "annoMatrix: init complete",
annoMatrix,
obsCrossfilter,
});
dispatch({ type: "initial data load complete" });
const defaultEmbedding = config?.parameters?.["default_embedding"];
const layoutSchema = schema?.schema?.layout?.obs ?? [];
if (
defaultEmbedding &&
layoutSchema.some((s) => s.name === defaultEmbedding)
) {
dispatch(embActions.layoutChoiceAction(defaultEmbedding));
}
} catch (error) {
dispatch({ type: "initial data load error", error });
}
@@ -210,6 +231,6 @@ export default {
annotationLabelCurrentSelection: annoActions.annotationLabelCurrentSelection,
saveObsAnnotationsAction: annoActions.saveObsAnnotationsAction,
needToSaveObsAnnotations: annoActions.needToSaveObsAnnotations,
layoutChoiceAction: selnActions.layoutChoiceAction,
layoutChoiceAction: embActions.layoutChoiceAction,
setCellSetFromSelection: selnActions.setCellSetFromSelection,
};
client/src/actions/reembed.js
+14 -21
View File
@@ -1,10 +1,10 @@
import { API } from "../globals";
import { MatrixFBS } from "../util/stateManager";
import {
postNetworkErrorToast,
postAsyncSuccessToast,
postAsyncFailureToast,
} from "../components/framework/toasters";
import { _switchEmbedding } from "./embedding";
function abortableFetch(request, opts, timeout = 0) {
const controller = new AbortController();
@@ -24,7 +24,7 @@ function abortableFetch(request, opts, timeout = 0) {
async function doReembedFetch(dispatch, getState) {
const state = getState();
let cells = state.world.obsAnnotations.rowIndex.labels();
let cells = state.annoMatrix.rowIndex.labels();
// These lines ensure that we convert any TypedArray to an Array.
// This is necessary because JSON.stringify() does some very strange
@@ -54,10 +54,7 @@ async function doReembedFetch(dispatch, getState) {
});
const res = await af.ready();
if (
res.ok &&
res.headers.get("Content-Type").includes("application/octet-stream")
) {
if (res.ok && res.headers.get("Content-Type").includes("application/json")) {
return res;
}
@@ -67,7 +64,6 @@ async function doReembedFetch(dispatch, getState) {
if (body && body.length > 0) {
msg = `${msg} -- ${body}`;
}
postNetworkErrorToast(msg);
throw new Error(msg);
}
@@ -78,17 +74,24 @@ export function requestReembed() {
return async (dispatch, getState) => {
try {
const res = await doReembedFetch(dispatch, getState);
const schema = JSON.parse(res.headers.get("CxG-Schema"));
const buffer = await res.arrayBuffer();
const df = MatrixFBS.matrixFBSToDataframe(buffer);
const schema = await res.json();
dispatch({
type: "reembed: request completed",
});
const { annoMatrix: prevAnnoMatrix } = getState();
const base = prevAnnoMatrix.base().addEmbedding(schema);
const [annoMatrix, obsCrossfilter] = await _switchEmbedding(
base,
schema.name
);
dispatch({
type: "reembed: add reembedding",
embedding: df,
schema,
annoMatrix,
obsCrossfilter,
});
postAsyncSuccessToast("Re-embedding has completed.");
} catch (error) {
dispatch({
@@ -103,13 +106,3 @@ export function requestReembed() {
}
};
}
/* disabled until reimplementation occurs
export function reembedResetWorldToUniverse(dispatch, getState) {
const { reembedController } = getState();
if (reembedController.pendingFetch) reembedController.pendingFetch.abort();
dispatch({
type: "reembed: clear all reembeddings",
});
}
*/
client/src/actions/selection.js
-25
View File
@@ -185,31 +185,6 @@ export const graphLassoEndAction = (embName, polygon) => async (
});
};
export const layoutChoiceAction = (newLayoutChoice) => async (
dispatch,
getState
) => {
/*
On layout choice, make sure we have selected all on the previous layout, AND the new
layout.
*/
const { obsCrossfilter: prevObsCrossfilter, layoutChoice } = getState();
let obsCrossfilter = await prevObsCrossfilter.select(
"emb",
layoutChoice.current,
{ mode: "all" }
);
obsCrossfilter = await obsCrossfilter.select("emb", newLayoutChoice, {
mode: "all",
});
dispatch({
type: "set layout choice",
layoutChoice: newLayoutChoice,
obsCrossfilter,
});
};
/*
Differential expression set selection
*/
client/src/actions/viewStack.js
+12 -36
View File
@@ -11,7 +11,12 @@ stack multiple subsets.
If these conventions change, code elsewhere (eg. menubar/clip.js) will need to
change as well.
*/
import { AnnoMatrixObsCrossfilter, clip, isubsetMask } from "../annoMatrix";
import { AnnoMatrixObsCrossfilter } from "../annoMatrix";
import {
_clipAnnoMatrix,
_userSubsetAnnoMatrix,
_userResetSubsetAnnoMatrix,
} from "../util/stateManager/viewStackHelpers";
export const clipAction = (min, max) => (dispatch, getState) => {
/*
@@ -19,9 +24,7 @@ export const clipAction = (min, max) => (dispatch, getState) => {
view is ALWAYS the top view.
*/
const { annoMatrix: prevAnnoMatrix } = getState();
const annoMatrix = prevAnnoMatrix.isClipped
? clip(prevAnnoMatrix.viewOf, min, max)
: clip(prevAnnoMatrix, min, max);
const annoMatrix = _clipAnnoMatrix(prevAnnoMatrix, min, max);
const obsCrossfilter = new AnnoMatrixObsCrossfilter(annoMatrix);
dispatch({
type: "set clip quantiles",
@@ -43,24 +46,10 @@ export const subsetAction = () => (dispatch, getState) => {
annoMatrix: prevAnnoMatrix,
obsCrossfilter: prevObsCrossfilter,
} = getState();
let annoMatrix;
if (prevAnnoMatrix.isClipped) {
// if there is a clip view, pop it and reapply after we subset
const { clipRange } = prevAnnoMatrix;
annoMatrix = isubsetMask(
prevAnnoMatrix.viewOf,
prevObsCrossfilter.allSelectedMask()
);
annoMatrix = clip(annoMatrix, ...clipRange);
} else {
// else just push a subset view.
annoMatrix = isubsetMask(
prevAnnoMatrix,
prevObsCrossfilter.allSelectedMask()
);
}
const annoMatrix = _userSubsetAnnoMatrix(
prevAnnoMatrix,
prevObsCrossfilter.allSelectedMask()
);
const obsCrossfilter = new AnnoMatrixObsCrossfilter(annoMatrix);
dispatch({
type: "subset to selection",
@@ -77,20 +66,7 @@ export const resetSubsetAction = () => (dispatch, getState) => {
*/
const { annoMatrix: prevAnnoMatrix } = getState();
const clipRange = prevAnnoMatrix.isClipped ? prevAnnoMatrix.clipRange : null;
/* pop all views */
let annoMatrix = prevAnnoMatrix;
while (annoMatrix.isView) {
annoMatrix = annoMatrix.viewOf;
}
/* re-apply the clip, if any */
if (clipRange !== null) {
annoMatrix = clip(annoMatrix, ...clipRange);
}
const annoMatrix = _userResetSubsetAnnoMatrix(prevAnnoMatrix);
const obsCrossfilter = new AnnoMatrixObsCrossfilter(annoMatrix);
dispatch({
type: "reset subset",
client/src/annoMatrix/annoMatrix.js
+16
View File
@@ -70,6 +70,8 @@ export default class AnnoMatrix {
The row index labels are as defined by the base dataset from the server.
* isView - true if this is a view, false if not.
* viewOf - pointer to parent annomatrix if a view, undefined/null if not a view.
* userFlags - container for any additional state a user of this API wants to hang
off of an annoMatrix, and have propagated by the (shallow) cloning protocol.
*/
this.schema = indexEntireSchema(schema);
this.nObs = nObs;
@@ -77,6 +79,7 @@ export default class AnnoMatrix {
this.rowIndex = rowIndex || new IdentityInt32Index(nObs);
this.isView = false;
this.viewOf = undefined;
this.userFlags = {};
/*
Private instance variables.
@@ -394,6 +397,19 @@ export default class AnnoMatrix {
_subclassResponsibility();
}
// eslint-disable-next-line class-methods-use-this, no-unused-vars -- make sure subclass implements
addEmbedding(colSchema) {
/*
Add a new obs embedding to the AnnoMatrix, with provided schema.
Returns a new annomatrix.
Typical use will be to add a re-embedding that the server has calculated.
Will throw if the column schema is invalid (eg, duplicate name).
*/
_subclassResponsibility();
}
/**
** Private interfaces below.
**/
client/src/annoMatrix/crossfilter.js
+5
View File
@@ -118,6 +118,11 @@ export default class AnnoMatrixObsCrossfilter {
return new AnnoMatrixObsCrossfilter(annoMatrix, obsCrossfilter);
}
addEmbedding(colSchema) {
const annoMatrix = this.annoMatrix.addEmbedding(colSchema);
return new AnnoMatrixObsCrossfilter(annoMatrix, this.obsCrossfilter);
}
/**
Selection state - API is identical to ImmutableTypedCrossfilter, as these
are just wrappers to lazy create indices.
client/src/annoMatrix/loader.js
+48 -28
View File
@@ -6,6 +6,7 @@ import {
removeObsAnnoColumn,
addObsAnnoCategory,
removeObsAnnoCategory,
addObsLayout,
} from "../util/stateManager/schemaHelpers";
import { isArrayOrTypedArray } from "../util/typeHelpers";
import { _whereCacheCreate } from "./whereCache";
@@ -47,9 +48,9 @@ export default class AnnoMatrixLoader extends AnnoMatrix {
const colSchema = _getColumnSchema(this.schema, "obs", col);
_writableCategoryTypeCheck(colSchema); // throws on error
const o = this._clone();
o.schema = addObsAnnoCategory(this.schema, col, category);
return o;
const newAnnoMatrix = this._clone();
newAnnoMatrix.schema = addObsAnnoCategory(this.schema, col, category);
return newAnnoMatrix;
}
async removeObsAnnoCategory(col, category, unassignedCategory) {
@@ -59,13 +60,17 @@ export default class AnnoMatrixLoader extends AnnoMatrix {
const colSchema = _getColumnSchema(this.schema, "obs", col);
_writableCategoryTypeCheck(colSchema); // throws on error
const o = await this.resetObsColumnValues(
const newAnnoMatrix = await this.resetObsColumnValues(
col,
category,
unassignedCategory
);
o.schema = removeObsAnnoCategory(o.schema, col, category);
return o;
newAnnoMatrix.schema = removeObsAnnoCategory(
newAnnoMatrix.schema,
col,
category
);
return newAnnoMatrix;
}
dropObsColumn(col) {
@@ -75,10 +80,10 @@ export default class AnnoMatrixLoader extends AnnoMatrix {
const colSchema = _getColumnSchema(this.schema, "obs", col);
_writableCheck(colSchema); // throws on error
const o = this._clone();
o._cache.obs = this._cache.obs.dropCol(col);
o.schema = removeObsAnnoColumn(this.schema, col);
return o;
const newAnnoMatrix = this._clone();
newAnnoMatrix._cache.obs = this._cache.obs.dropCol(col);
newAnnoMatrix.schema = removeObsAnnoColumn(this.schema, col);
return newAnnoMatrix;
}
addObsColumn(colSchema, Ctor, value) {
@@ -90,15 +95,15 @@ export default class AnnoMatrixLoader extends AnnoMatrix {
If an array, it must be of same size as nObs and same type as Ctor
*/
colSchema.writable = true;
const col = colSchema.name;
const colName = colSchema.name;
if (
_getColumnSchema(this.schema, "obs", col) ||
this._cache.obs.hasCol(col)
_getColumnSchema(this.schema, "obs", colName) ||
this._cache.obs.hasCol(colName)
) {
throw new Error("column already exists");
}
const o = this._clone();
const newAnnoMatrix = this._clone();
let data;
if (isArrayOrTypedArray(value)) {
if (value.constructor !== Ctor)
@@ -109,12 +114,13 @@ export default class AnnoMatrixLoader extends AnnoMatrix {
} else {
data = new Ctor(this.nObs).fill(value);
}
o._cache.obs = this._cache.obs.withCol(col, data);
o.schema = addObsAnnoColumn(this.schema, col, {
...colSchema,
writable: true,
});
return o;
newAnnoMatrix._cache.obs = this._cache.obs.withCol(colName, data);
_normalizeCategoricalSchema(
colSchema,
newAnnoMatrix._cache.obs.col(colName)
);
newAnnoMatrix.schema = addObsAnnoColumn(this.schema, colName, colSchema);
return newAnnoMatrix;
}
renameObsColumn(oldCol, newCol) {
@@ -157,13 +163,13 @@ export default class AnnoMatrixLoader extends AnnoMatrix {
data[idx] = value;
}
const o = this._clone();
o._cache.obs = this._cache.obs.replaceColData(col, data);
const newAnnoMatrix = this._clone();
newAnnoMatrix._cache.obs = this._cache.obs.replaceColData(col, data);
const { categories } = colSchema;
if (!categories?.includes(value)) {
o.schema = addObsAnnoCategory(this.schema, col, value);
newAnnoMatrix.schema = addObsAnnoCategory(this.schema, col, value);
}
return o;
return newAnnoMatrix;
}
async resetObsColumnValues(col, oldValue, newValue) {
@@ -187,13 +193,27 @@ export default class AnnoMatrixLoader extends AnnoMatrix {
if (data[i] === oldValue) data[i] = newValue;
}
const o = this._clone();
o._cache.obs = this._cache.obs.replaceColData(col, data);
const newAnnoMatrix = this._clone();
newAnnoMatrix._cache.obs = this._cache.obs.replaceColData(col, data);
const { categories } = colSchema;
if (!categories?.includes(newValue)) {
o.schema = addObsAnnoCategory(this.schema, col, newValue);
newAnnoMatrix.schema = addObsAnnoCategory(this.schema, col, newValue);
}
return o;
return newAnnoMatrix;
}
addEmbedding(colSchema) {
/*
add new layout to the obs embeddings
*/
const { name: colName } = colSchema;
if (_getColumnSchema(this.schema, "emb", colName)) {
throw new Error("column already exists");
}
const newAnnoMatrix = this._clone();
newAnnoMatrix.schema = addObsLayout(this.schema, colSchema);
return newAnnoMatrix;
}
/**
client/src/annoMatrix/schema.js
+8 -2
View File
@@ -66,9 +66,14 @@ export function _isContinuousType(schema) {
export function _normalizeCategoricalSchema(colSchema, col) {
const { type, writable } = colSchema;
if (type === "string" || type === "boolean" || type === "categorical") {
if (
type === "string" ||
type === "boolean" ||
type === "categorical" ||
writable
) {
const categorySet = new Set(
col.summarize().categories.concat(colSchema.categories ?? [])
col.summarizeCategorical().categories.concat(colSchema.categories ?? [])
);
if (writable && !categorySet.has(unassignedCategoryLabel)) {
categorySet.add(unassignedCategoryLabel);
@@ -79,4 +84,5 @@ export function _normalizeCategoricalSchema(colSchema, col) {
if (colSchema.categories) {
colSchema.categories = catLabelSort(writable, colSchema.categories);
}
return colSchema;
}
client/src/annoMatrix/viewCreators.js
+8 -1
View File
@@ -33,6 +33,13 @@ export function subset(annoMatrix, obsLabels) {
return new AnnoMatrixRowSubsetView(annoMatrix, obsIndex);
}
export function subsetByIndex(annoMatrix, obsIndex) {
/*
subset based upon the new obs index.
*/
return new AnnoMatrixRowSubsetView(annoMatrix, obsIndex);
}
export function clip(annoMatrix, qmin, qmax) {
/*
Create a view that clips all continuous data to the [min, max] range.
@@ -59,5 +66,5 @@ function _maskToList(mask) {
elems += 1;
}
}
return new Int32Array(list.buffer, 0, elems);
return list.subarray(0, elems);
}
client/src/annoMatrix/views.js
+46 -31
View File
@@ -17,59 +17,74 @@ class AnnoMatrixView extends AnnoMatrix {
}
addObsAnnoCategory(col, category) {
const o = this._clone();
o.viewOf = this.viewOf.addObsAnnoCategory(col, category);
o.schema = o.viewOf.schema;
return o;
const newAnnoMatrix = this._clone();
newAnnoMatrix.viewOf = this.viewOf.addObsAnnoCategory(col, category);
newAnnoMatrix.schema = newAnnoMatrix.viewOf.schema;
return newAnnoMatrix;
}
async removeObsAnnoCategory(col, category, unassignedCategory) {
const o = this._clone();
o.viewOf = await this.viewOf.removeObsAnnoCategory(
const newAnnoMatrix = this._clone();
newAnnoMatrix.viewOf = await this.viewOf.removeObsAnnoCategory(
col,
category,
unassignedCategory
);
o.schema = o.viewOf.schema;
return o;
newAnnoMatrix.schema = newAnnoMatrix.viewOf.schema;
return newAnnoMatrix;
}
dropObsColumn(col) {
const o = this._clone();
o.viewOf = this.viewOf.dropObsColumn(col);
o._cache.obs = this._cache.obs.dropCol(col);
o.schema = o.viewOf.schema;
return o;
const newAnnoMatrix = this._clone();
newAnnoMatrix.viewOf = this.viewOf.dropObsColumn(col);
newAnnoMatrix._cache.obs = this._cache.obs.dropCol(col);
newAnnoMatrix.schema = newAnnoMatrix.viewOf.schema;
return newAnnoMatrix;
}
addObsColumn(colSchema, Ctor, value) {
const o = this._clone();
o.viewOf = this.viewOf.addObsColumn(colSchema, Ctor, value);
o.schema = o.viewOf.schema;
return o;
const newAnnoMatrix = this._clone();
newAnnoMatrix.viewOf = this.viewOf.addObsColumn(colSchema, Ctor, value);
newAnnoMatrix.schema = newAnnoMatrix.viewOf.schema;
return newAnnoMatrix;
}
renameObsColumn(oldCol, newCol) {
const o = this._clone();
o.viewOf = this.viewOf.renameObsColumn(oldCol, newCol);
o.schema = o.viewOf.schema;
return o;
const newAnnoMatrix = this._clone();
newAnnoMatrix.viewOf = this.viewOf.renameObsColumn(oldCol, newCol);
newAnnoMatrix.schema = newAnnoMatrix.viewOf.schema;
return newAnnoMatrix;
}
async setObsColumnValues(col, rowLabels, value) {
const o = this._clone();
o.viewOf = await this.viewOf.setObsColumnValues(col, rowLabels, value);
o._cache.obs = this._cache.obs.dropCol(col);
o.schema = o.viewOf.schema;
return o;
const newAnnoMatrix = this._clone();
newAnnoMatrix.viewOf = await this.viewOf.setObsColumnValues(
col,
rowLabels,
value
);
newAnnoMatrix._cache.obs = this._cache.obs.dropCol(col);
newAnnoMatrix.schema = newAnnoMatrix.viewOf.schema;
return newAnnoMatrix;
}
async resetObsColumnValues(col, oldValue, newValue) {
const o = this._clone();
o.viewOf = await this.viewOf.resetObsColumnValues(col, oldValue, newValue);
o._cache.obs = this._cache.obs.dropCol(col);
o.schema = o.viewOf.schema;
return o;
const newAnnoMatrix = this._clone();
newAnnoMatrix.viewOf = await this.viewOf.resetObsColumnValues(
col,
oldValue,
newValue
);
newAnnoMatrix._cache.obs = this._cache.obs.dropCol(col);
newAnnoMatrix.schema = newAnnoMatrix.viewOf.schema;
return newAnnoMatrix;
}
addEmbedding(colSchema) {
const newAnnoMatrix = this._clone();
newAnnoMatrix.viewOf = this.viewOf.addEmbedding(colSchema);
newAnnoMatrix.schema = newAnnoMatrix.viewOf.schema;
return newAnnoMatrix;
}
}
client/src/components/app.js
+2
View File
@@ -11,6 +11,7 @@ import Legend from "./continuousLegend";
import Graph from "./graph/graph";
import MenuBar from "./menubar";
import Autosave from "./autosave";
import Embedding from "./embedding";
import TermsOfServicePrompt from "./termsPrompt";
import actions from "../actions";
@@ -73,6 +74,7 @@ class App extends React.Component {
{(viewportRef) => (
<>
<MenuBar />
<Embedding />
<Autosave />
<TermsOfServicePrompt />
<Legend viewportRef={viewportRef} />
client/src/components/autosave/filenameDialog.js
+4 -2
View File
@@ -13,6 +13,7 @@ import {
@connect((state) => ({
idhash: state.config?.parameters?.["annotations-user-data-idhash"] ?? null,
annotations: state.annotations,
auth: state.config?.authentication,
writableCategoriesEnabled: state.config?.parameters?.annotations ?? false,
}))
class FilenameDialog extends React.Component {
@@ -90,12 +91,13 @@ class FilenameDialog extends React.Component {
};
render() {
const { writableCategoriesEnabled, annotations, idhash } = this.props;
const { writableCategoriesEnabled, annotations, idhash, auth } = this.props;
const { filenameText } = this.state;
return writableCategoriesEnabled &&
!annotations.dataCollectionNameIsReadOnly &&
!annotations.dataCollectionName ? (
!annotations.dataCollectionName &&
auth.is_authenticated ? (
<Dialog
icon="tag"
title="Annotations Collection"
client/src/components/categorical/category/index.js
+7 -3
View File
@@ -1,7 +1,7 @@
import React, { useRef, useEffect } from "react";
import { connect, shallowEqual } from "react-redux";
import { FaChevronRight, FaChevronDown } from "react-icons/fa";
import { AnchorButton, Button, Tooltip } from "@blueprintjs/core";
import { AnchorButton, Button, Tooltip, Position } from "@blueprintjs/core";
import { Flipper, Flipped } from "react-flip-toolkit";
import Async from "react-async";
import memoize from "memoize-one";
@@ -438,9 +438,13 @@ const CategoryHeader = React.memo(
? `Coloring by ${metadataField} is disabled, as it exceeds the limit of ${globals.maxCategoricalOptionsToDisplay} labels`
: "Use as color scale"
}
position="bottom"
usePortal={false}
position={Position.LEFT}
usePortal
hoverOpenDelay={globals.tooltipHoverOpenDelay}
modifiers={{
preventOverflow: { enabled: false },
hide: { enabled: false },
}}
>
<AnchorButton
data-testclass="colorby"
client/src/components/categorical/value/index.js
+3 -1
View File
@@ -453,7 +453,9 @@ class CategoryValue extends React.Component {
label,
CHART_WIDTH,
VALUE_HEIGHT
) ?? {};
) ?? {}; // if createHistogramBins returns empty object assign null to deconstructed
if (!xScale || !yScale || !bins) return null;
return (
<MiniHistogram
client/src/components/continuous/continuous.js
+1
View File
@@ -16,6 +16,7 @@ class Continuous extends React.PureComponent {
const allContinuousNames = schema.annotations.obs.columns
.filter((col) => col.type === "int32" || col.type === "float32")
.filter((col) => col.name !== obsIndex)
.filter((col) => !col.writable) // skip user annotations - they will be treated as categorical
.map((col) => col.name);
return (
client/src/components/embedding/index.js
+153
View File
@@ -0,0 +1,153 @@
import React from "react";
import { connect } from "react-redux";
import { useAsync } from "react-async";
import {
ButtonGroup,
Popover,
Button,
Radio,
RadioGroup,
Tooltip,
Position,
} from "@blueprintjs/core";
import * as globals from "../../globals";
import actions from "../../actions";
import { getDiscreteCellEmbeddingRowIndex } from "../../util/stateManager/viewStackHelpers";
@connect((state) => {
return {
layoutChoice: state.layoutChoice, // TODO: really should clean up naming, s/layout/embedding/g
schema: state.annoMatrix?.schema,
crossfilter: state.obsCrossfilter,
};
})
class Embedding extends React.PureComponent {
constructor(props) {
super(props);
this.state = {};
}
handleLayoutChoiceChange = (e) => {
const { dispatch } = this.props;
dispatch(actions.layoutChoiceAction(e.currentTarget.value));
};
render() {
const { layoutChoice, schema, crossfilter } = this.props;
const { annoMatrix } = crossfilter;
return (
<ButtonGroup
style={{
position: "absolute",
display: "inherit",
left: 8,
bottom: 8,
zIndex: 9999,
}}
>
<Popover
target={
<Tooltip
content="Select embedding for visualization"
position="top"
hoverOpenDelay={globals.tooltipHoverOpenDelay}
>
<Button
type="button"
data-testid="layout-choice"
icon="heatmap"
// minimal
id="embedding"
style={{
cursor: "pointer",
}}
>
{layoutChoice?.current}: {crossfilter.countSelected()} out of{" "}
{crossfilter.size()} cells
</Button>
</Tooltip>
}
// minimal /* removes arrow */
position={Position.TOP_LEFT}
content={
<div
style={{
display: "flex",
justifyContent: "flex-start",
alignItems: "flex-start",
flexDirection: "column",
padding: 10,
width: 400,
}}
>
<h1>Embedding Choice</h1>
<p style={{ fontStyle: "italic" }}>
There are {schema?.dataframe?.nObs} cells in the entire dataset.
</p>
<EmbeddingChoices
onChange={this.handleLayoutChoiceChange}
annoMatrix={annoMatrix}
layoutChoice={layoutChoice}
/>
</div>
}
/>
</ButtonGroup>
);
}
}
export default Embedding;
const loadAllEmbeddingCounts = async ({ annoMatrix, available }) => {
const embeddings = await Promise.all(
available.map((name) => annoMatrix.base().fetch("emb", name))
);
return available.map((name, idx) => ({
embeddingName: name,
embedding: embeddings[idx],
discreteCellIndex: getDiscreteCellEmbeddingRowIndex(embeddings[idx]),
}));
};
const EmbeddingChoices = ({ onChange, annoMatrix, layoutChoice }) => {
const { available } = layoutChoice;
const { data, error, isPending } = useAsync({
promiseFn: loadAllEmbeddingCounts,
annoMatrix,
available,
});
if (error) {
/* log, as this is unexpected */
console.error(error);
}
if (error || isPending) {
/* still loading, or errored out - just omit counts (TODO: spinner?) */
return (
<RadioGroup onChange={onChange} selectedValue={layoutChoice.current}>
{layoutChoice.available.map((name) => (
<Radio label={`${name}`} value={name} key={name} />
))}
</RadioGroup>
);
}
if (data) {
return (
<RadioGroup onChange={onChange} selectedValue={layoutChoice.current}>
{data.map((summary) => {
const { discreteCellIndex, embeddingName } = summary;
const sizeHint = `${discreteCellIndex.size()} cells`;
return (
<Radio
label={`${embeddingName}: ${sizeHint}`}
value={embeddingName}
key={embeddingName}
/>
);
})}
</RadioGroup>
);
}
return null;
};
client/src/components/graph/graph.js
+5 -3
View File
@@ -34,8 +34,10 @@ function createProjectionTF(viewportWidth, viewportHeight) {
the projection transform accounts for the screen size & other layout
*/
const fractionToUse = 0.95; // fraction of min dimension to use
const topGutterSizePx = 32; // toolbar box height
const heightMinusGutter = viewportHeight - topGutterSizePx;
const topGutterSizePx = 32; // top gutter for tools
const bottomGutterSizePx = 32; // bottom gutter for tools
const heightMinusGutter =
viewportHeight - topGutterSizePx - bottomGutterSizePx;
const minDim = Math.min(viewportWidth, heightMinusGutter);
const aspectScale = [
(fractionToUse * minDim) / viewportWidth,
@@ -44,7 +46,7 @@ function createProjectionTF(viewportWidth, viewportHeight) {
const m = mat3.create();
mat3.fromTranslation(m, [
0,
-topGutterSizePx / viewportHeight / aspectScale[1],
(bottomGutterSizePx - topGutterSizePx) / viewportHeight / aspectScale[1],
]);
mat3.scale(m, m, aspectScale);
return m;
client/src/components/menubar/authButtons.js
+32
View File
@@ -0,0 +1,32 @@
import React from "react";
import { AnchorButton, Tooltip } from "@blueprintjs/core";
import * as globals from "../../globals";
import styles from "./menubar.css";
const Auth = React.memo((props) => {
const { auth } = props;
if (!auth || (auth && !auth.requires_client_login)) return null;
return (
<div className={`bp3-button-group ${styles.menubarButton}`}>
<Tooltip
content="Log in or log out of cellxgene"
position="bottom"
hoverOpenDelay={globals.tooltipHoverOpenDelay}
>
<AnchorButton
type="button"
data-testid="auth-button"
disabled={false}
icon={!auth.is_authenticated ? "log-in" : "log-out"}
href={!auth.is_authenticated ? auth.login : auth.logout}
>
{!auth.is_authenticated ? "Log In" : "Log Out"}
</AnchorButton>
</Tooltip>
</div>
);
});
export default Auth;
client/src/components/menubar/embedding.js
-118
View File
@@ -1,118 +0,0 @@
import React from "react";
import {
ButtonGroup,
Popover,
Button,
Radio,
RadioGroup,
Tooltip,
Position,
} from "@blueprintjs/core";
import { connect } from "react-redux";
import * as globals from "../../globals";
import styles from "./menubar.css";
import actions from "../../actions";
@connect((state) => ({
layoutChoice: state.layoutChoice,
// disabled temporarily. TODO - issue #1606
// reembedController: state.reembedController,
// enableReembedding: state.config?.parameters?.["enable-reembedding"] ?? false,
enableReembedding: false,
}))
class Embedding extends React.PureComponent {
handleLayoutChoiceChange = (e) => {
const { dispatch } = this.props;
dispatch(actions.layoutChoiceAction(e.currentTarget.value));
};
// eslint-disable-next-line class-methods-use-this -- temporary disable
renderReembedding() {
return null;
/* disabled pending rewrite. TODO - issue #1606
const {
enableReembedding,
world,
universe,
dispatch,
reembedController,
} = this.props;
if (!enableReembedding) return null;
const loading = !!reembedController?.pendingFetch;
const disabled = World.worldEqUniverse(world, universe);
const tipContent = disabled
? "Subset cells first, then click to recompute UMAP embedding."
: "Click to recompute UMAP embedding on the current cell subset.";
return (
<Tooltip
content={tipContent}
position="bottom"
hoverOpenDelay={globals.tooltipHoverOpenDelay}
>
<AnchorButton
icon="new-object"
style={{ marginRight: 10 }}
disabled={disabled}
onClick={() => dispatch(actions.requestReembed())}
loading={loading}
/>
</Tooltip>
);
*/
}
render() {
const { layoutChoice } = this.props;
return (
<ButtonGroup className={styles.menubarButton}>
<Popover
target={
<Tooltip
content="Select embedding for visualization"
position="bottom"
hoverOpenDelay={globals.tooltipHoverOpenDelay}
>
<Button
type="button"
data-testid="layout-choice"
icon="heatmap"
style={{
cursor: "pointer",
}}
/>
</Tooltip>
}
position={Position.BOTTOM_RIGHT}
content={
<div
style={{
display: "flex",
justifyContent: "flex-start",
alignItems: "flex-start",
flexDirection: "column",
padding: 10,
}}
>
<RadioGroup
label="Embedding Choice"
onChange={this.handleLayoutChoiceChange}
selectedValue={layoutChoice.current}
>
{layoutChoice.available.map((name) => (
<Radio label={name} value={name} key={name} />
))}
</RadioGroup>
</div>
}
/>
{this.renderReembedding()}
</ButtonGroup>
);
}
}
export default Embedding;
client/src/components/menubar/index.js
+15 -5
View File
@@ -6,11 +6,13 @@ import * as globals from "../../globals";
import styles from "./menubar.css";
import actions from "../../actions";
import Clip from "./clip";
import Embedding from "./embedding";
import AuthButtons from "./authButtons";
import InformationMenu from "./infoMenu";
import Subset from "./subset";
import UndoRedoReset from "./undoRedo";
import DiffexpButtons from "./diffexpButtons";
import Reembedding from "./reembedding";
import { getEmbSubsetView } from "../../util/stateManager/viewStackHelpers";
@connect((state) => {
const { annoMatrix } = state;
@@ -18,9 +20,11 @@ import DiffexpButtons from "./diffexpButtons";
const selectedCount = crossfilter.countSelected();
const subsetPossible =
selectedCount !== 0 && selectedCount !== crossfilter.size(); // ie, not all are selected
const subsetResetPossible =
annoMatrix.nObs !== annoMatrix.schema.dataframe.nObs;
selectedCount !== 0 && selectedCount !== crossfilter.size(); // ie, not all and not none are selected
const embSubsetView = getEmbSubsetView(annoMatrix);
const subsetResetPossible = !embSubsetView
? annoMatrix.nObs !== annoMatrix.schema.dataframe.nObs
: annoMatrix.nObs !== embSubsetView.nObs;
return {
subsetPossible,
@@ -37,6 +41,7 @@ import DiffexpButtons from "./diffexpButtons";
celllist1: state.differential.celllist1,
celllist2: state.differential.celllist2,
libraryVersions: state.config?.["library_versions"],
auth: state.config?.authentication,
undoDisabled: state["@@undoable/past"].length === 0,
redoDisabled: state["@@undoable/future"].length === 0,
aboutLink: state.config?.links?.["about-dataset"],
@@ -47,6 +52,8 @@ import DiffexpButtons from "./diffexpButtons";
tosURL: state.config?.parameters?.["about_legal_tos"],
privacyURL: state.config?.parameters?.["about_legal_privacy"],
categoricalSelection: state.categoricalSelection,
enableReembedding:
state.config?.parameters?.["enable-reembedding"] ?? false,
};
})
class MenuBar extends React.PureComponent {
@@ -212,6 +219,8 @@ class MenuBar extends React.PureComponent {
colorAccessor,
subsetPossible,
subsetResetPossible,
enableReembedding,
auth,
} = this.props;
const { pendingClipPercentiles } = this.state;
@@ -237,6 +246,7 @@ class MenuBar extends React.PureComponent {
zIndex: 3,
}}
>
<AuthButtons auth={auth} />
<InformationMenu
libraryVersions={libraryVersions}
aboutLink={aboutLink}
@@ -264,7 +274,7 @@ class MenuBar extends React.PureComponent {
this.handleClipPercentileMinValueChange
}
/>
<Embedding />
{enableReembedding ? <Reembedding /> : null}
<Tooltip
content="When a category is colored by, show labels on the graph"
position="bottom"
client/src/components/menubar/reembedding.js
+40
View File
@@ -0,0 +1,40 @@
import React from "react";
import { connect } from "react-redux";
import { AnchorButton, ButtonGroup, Tooltip } from "@blueprintjs/core";
import * as globals from "../../globals";
import actions from "../../actions";
import styles from "./menubar.css";
@connect((state) => ({
reembedController: state.reembedController,
annoMatrix: state.annoMatrix,
}))
class Reembedding extends React.PureComponent {
render() {
const { dispatch, annoMatrix, reembedController } = this.props;
const loading = !!reembedController?.pendingFetch;
const disabled = annoMatrix.nObs === annoMatrix.schema.dataframe.nObs;
const tipContent = disabled
? "Subset cells first, then click to recompute UMAP embedding."
: "Click to recompute UMAP embedding on the current cell subset.";
return (
<ButtonGroup className={styles.menubarButton}>
<Tooltip
content={tipContent}
position="bottom"
hoverOpenDelay={globals.tooltipHoverOpenDelay}
>
<AnchorButton
icon="new-object"
disabled={disabled}
onClick={() => dispatch(actions.requestReembed())}
loading={loading}
/>
</Tooltip>
</ButtonGroup>
);
}
}
export default Reembedding;
client/src/components/miniHistogram/index.js
+3 -2
View File
@@ -40,10 +40,11 @@ export default class MiniHistogram extends React.PureComponent {
};
componentDidUpdate = (prevProps) => {
const { obsOrVarContinuousFieldDisplayName } = this.props;
const { obsOrVarContinuousFieldDisplayName, bins } = this.props;
if (
prevProps.obsOrVarContinuousFieldDisplayName !==
obsOrVarContinuousFieldDisplayName
obsOrVarContinuousFieldDisplayName ||
prevProps.bins !== bins
)
this.drawHistogram();
};
client/src/components/scatterplot/scatterplot.js
+19 -8
View File
@@ -438,18 +438,19 @@ class Scatterplot extends React.PureComponent {
pointDilation,
} = this.props;
const { minimized, regl, viewport } = this.state;
const bottomToolbarGutter = 48; // gutter for bottom tool bar
return (
<div
style={{
position: "fixed",
bottom: minimized ? -height + -margin.top - 2 : 0,
bottom: bottomToolbarGutter,
borderRadius: "3px 3px 0px 0px",
left: globals.leftSidebarWidth + globals.scatterplotMarginLeft,
padding: "0px 20px 20px 0px",
background: "white",
/* x y blur spread color */
boxShadow: "0px 0px 6px 2px rgba(153,153,153,0.4)",
boxShadow: "0px 0px 3px 2px rgba(153,153,153,0.2)",
zIndex: 2,
}}
id="scatterplot_wrapper"
@@ -488,7 +489,9 @@ class Scatterplot extends React.PureComponent {
id="scatterplot"
style={{
width: `${width + margin.left + margin.right}px`,
height: `${height + margin.top + margin.bottom}px`,
height: `${
(minimized ? 0 : height + margin.top) + margin.bottom
}px`,
}}
>
<canvas
@@ -498,6 +501,7 @@ class Scatterplot extends React.PureComponent {
style={{
marginLeft: margin.left,
marginTop: margin.top,
display: minimized ? "none" : null,
}}
ref={this.setReglCanvas}
/>
@@ -523,9 +527,7 @@ class Scatterplot extends React.PureComponent {
}
return (
<ScatterplotAxis
width={width}
height={height}
margin={margin}
minimized={minimized}
scatterplotYYaccessor={scatterplotXXaccessor}
scatterplotXXaccessor={scatterplotYYaccessor}
xScale={asyncProps.xScale}
@@ -544,7 +546,13 @@ class Scatterplot extends React.PureComponent {
export default Scatterplot;
const ScatterplotAxis = React.memo(
({ scatterplotYYaccessor, scatterplotXXaccessor, xScale, yScale }) => {
({
minimized,
scatterplotYYaccessor,
scatterplotXXaccessor,
xScale,
yScale,
}) => {
/*
Axis for the scatterplot, rendered with SVG/D3. Props:
* scatterplotXXaccessor - name of X axis
@@ -559,7 +567,7 @@ const ScatterplotAxis = React.memo(
const svgRef = useRef(null);
useEffect(() => {
if (!svgRef.current) return;
if (!svgRef.current || minimized) return;
const svg = d3.select(svgRef.current);
svg.selectAll("*").remove();
@@ -608,6 +616,9 @@ const ScatterplotAxis = React.memo(
width={width + margin.left + margin.right}
height={height + margin.top + margin.bottom}
data-testid="scatterplot-svg"
style={{
display: minimized ? "none" : null,
}}
>
<g ref={svgRef} transform={`translate(${margin.left},${margin.top})`} />
</svg>
client/src/globals.js
+1 -1
View File
@@ -61,7 +61,7 @@ export const maxControlsWidth = 800;
export const graphMargin = { top: 20, right: 10, bottom: 30, left: 40 };
export const graphWidth = 700;
export const graphHeight = 700;
export const scatterplotMarginLeft = 25;
export const scatterplotMarginLeft = 11;
export const rightSidebarWidth = 365;
export const leftSidebarWidth = 365;
client/src/index.js
-1
View File
@@ -1,4 +1,3 @@
// jshint esversion: 6
import React from "react";
import ReactDOM from "react-dom";
import { Provider } from "react-redux";
client/src/reducers/continuousSelection.js
+1
View File
@@ -3,6 +3,7 @@ import { makeContinuousDimensionName } from "../util/nameCreators";
const ContinuousSelection = (state = {}, action) => {
switch (action.type) {
case "reset subset":
case "subset to selection":
case "set clip quantiles": {
return {};
}
client/src/reducers/graphSelection.js
+1
View File
@@ -7,6 +7,7 @@ const GraphSelection = (
) => {
switch (action.type) {
case "set clip quantiles":
case "subset to selection":
case "reset subset":
case "set layout choice": {
return {
client/src/reducers/index.js
+1 -3
View File
@@ -18,7 +18,7 @@ import autosave from "./autosave";
import ontology from "./ontology";
import centroidLabels from "./centroidLabels";
import pointDialation from "./pointDilation";
import { reembedController, reembedding } from "./reembed";
import { reembedController } from "./reembed";
import { gcMiddleware as annoMatrixGC } from "../annoMatrix";
import undoableConfig from "./undoableConfig";
@@ -30,7 +30,6 @@ const Reducer = undoable(
["obsCrossfilter", obsCrossfilter],
["ontology", ontology],
["annotations", annotations],
["reembedding", reembedding],
["layoutChoice", layoutChoice],
["categoricalSelection", categoricalSelection],
["continuousSelection", continuousSelection],
@@ -55,7 +54,6 @@ const Reducer = undoable(
"layoutChoice",
"centroidLabels",
"annotations",
"reembedding",
],
undoableConfig
);
client/src/reducers/layoutChoice.js
+4 -16
View File
@@ -48,27 +48,15 @@ const LayoutChoice = (
}
case "reembed: add reembedding": {
const { schema } = nextSharedState.annoMatrix;
const { name } = action.schema;
const available = Array.from(new Set(state.available).add(name));
const currentDimNames = schema.layout.obsByName[name].dims;
return {
...state,
available,
};
}
case "reembed: clear all reembeddings": {
const { annoMatrix } = nextSharedState;
const { current } = state;
const dflt = setToDefaultLayout(annoMatrix.schema);
if (dflt.available.includes(current)) {
return {
...state,
available: dflt.available,
};
}
return {
...state,
...dflt,
current: name,
currentDimNames,
};
}
client/src/reducers/reembed.js
-35
View File
@@ -27,38 +27,3 @@ export const reembedController = (
}
}
};
/*
actual reembedding data is part of the undo/redo history
*/
export const reembedding = (
state = {
reembeddings: new Map(),
},
action
) => {
switch (action.type) {
case "reembed: add reembedding": {
const { schema, embedding } = action;
const { name } = schema.name;
const { reembeddings } = state;
return {
...state,
reembeddings: new Map(reembeddings).set(name, {
name,
schema,
embedding,
}),
};
}
case "reembed: clear all reembeddings": {
return {
...state,
reembeddings: new Map(),
};
}
default: {
return state;
}
}
};
client/src/util/stateManager/controlsHelpers.js
+6 -1
View File
@@ -64,7 +64,12 @@ function topNCategories(colSchema, summary, N) {
export function isSelectableCategoryName(schema, name) {
const { index } = schema.annotations.obs;
return name && name !== index && isCategoricalAnnotation(schema, name);
const colSchema = schema.annotations.obsByName[name];
return (
name &&
name !== index &&
(isCategoricalAnnotation(schema, name) || colSchema.writable)
);
}
export function selectableCategoryNames(schema, names) {
client/src/util/stateManager/matrix.js
+2 -2
View File
@@ -170,11 +170,11 @@ export function encodeMatrixFBS(df) {
function promoteTypedArray(o) {
/*
Decide what internal data type to use for the data returned from
Decide what internal data type to use for the data returned from
the server.
TODO - future optimization: not all int32/uint32 data series require
promotion to float64. We COULD simply look at the data to decide.
promotion to float64. We COULD simply look at the data to decide.
*/
if (isFpTypedArray(o) || Array.isArray(o)) return o;
client/src/util/stateManager/viewStackHelpers.js
+175
View File
@@ -0,0 +1,175 @@
/*
The annoMatrix view stack has a set of conventions which are assumed elsewhere in the
application. These helper functions make it simple for action creators to manage
the stack.
The annoMatrix module does not care about this order, but we maintain it as
a convention to make it simpler to manipulate the views.
Terminology:
- clip view: AnnoMatrixClipView
- subset view: AnnoMatrixRowSubsetView
- user subset view: create by the user explicitly subsetting by selection
- embedding subset view: implicitly created by switching the current embedding
- loader, or base annoMatrix: the root, which loads data
Rules:
1. there will be zero or one clip view
2. there will be zero or more subset views
3. there will be zero or one embedding view
4. there will be one loader/base, which is always the bottom view
5. the view ordering MUST be (top to bottom):
[clip] -> [user subset] -> [embedding subset] -> loader
There is code elsewhere in the app (eg, menubar/clip.js) which assumes this order.
Views can be interogated for their type with the following:
* is a view: annoMatrix.isView
* is the loader: !anonMatrix.isView (or annoMatrix === annoMatrix.base())
* is a clip view: annoMatrix.isClipped (or annoMatrix.clipRange)
* is a subset view: (annoMatrix.isView && !annoMatrix.isClipped)
* is a user subset view: annoMatrix.userFlags?.isUserSubsetView
* is an embedding subset view: annomatrix.userFlags?.isEmbSubsetView
*/
import { clip, isubsetMask, isubset } from "../../annoMatrix";
import { memoize } from "../dataframe/util";
export function _clipAnnoMatrix(annoMatrix, min, max) {
/*
clip the annoMatrix.
*/
return annoMatrix.isClipped
? clip(annoMatrix.viewOf, min, max)
: clip(annoMatrix, min, max);
}
export function _userSubsetAnnoMatrix(annoMatrix, mask) {
/*
user-requested row subset of annoMatrix, to be added on top of any
other previous row subsets.
*/
const { clipRange } = annoMatrix;
if (clipRange) {
annoMatrix = annoMatrix.viewOf;
}
annoMatrix = isubsetMask(annoMatrix, mask);
annoMatrix.userFlags.isUserSubsetView = true;
if (clipRange) {
annoMatrix = clip(annoMatrix, ...clipRange);
}
return annoMatrix;
}
export function _userResetSubsetAnnoMatrix(annoMatrix) {
/*
Reset/remove all user-requested subsets. Do not remove clip or embedding subset.
*/
/* stash clipping info, if any */
const { clipRange } = annoMatrix;
if (clipRange) {
annoMatrix = annoMatrix.viewOf;
}
/* pop all views except embedding subset and loader */
while (annoMatrix.isView && annoMatrix.userFlags.isUserSubsetView) {
annoMatrix = annoMatrix.viewOf;
}
/* re-apply the clip, if any */
if (clipRange) {
annoMatrix = clip(annoMatrix, ...clipRange);
}
return annoMatrix;
}
export function _setEmbeddingSubset(annoMatrix, embeddingDf) {
/*
Set the embedding subset view. Only create a subset view for the embedding
when it is needed, ie, there are NaN values in the embeddings.
*/
const embRowOffsets = _getEmbeddingRowOffsets(
annoMatrix.rowIndex,
embeddingDf
);
const curEmbSubsetView = getEmbSubsetView(annoMatrix);
/* if no current embedding subset, and no new embedding subset, just noop */
if (!embRowOffsets && !curEmbSubsetView) return annoMatrix;
// ... otherwise, do the work
/* stash clipping info, if any */
const clipRange = annoMatrix.isClipped ? annoMatrix.clipRange : null;
/* pop all subsets, user or embedding */
while (annoMatrix.isView) {
annoMatrix = annoMatrix.viewOf;
}
/* apply new embedding row index, if needed */
if (embRowOffsets) {
annoMatrix = isubset(annoMatrix, embRowOffsets);
annoMatrix.userFlags.isEmbSubsetView = true;
}
/* re-apply clip, if needed */
if (clipRange) {
annoMatrix = clip(annoMatrix, ...clipRange);
}
return annoMatrix;
}
function _getEmbeddingRowOffsets(baseRowIndex, embeddingDf) {
/*
given a dataframe containing an embedding:
- if the embedding contains no NaN coordinates, return null
- if the embedding contains NaN coordinates, return a rowIndex
that contains only the rows with discrete valued coordinates.
Currently assumes that there will be onl two dimensions in the embedding.
*/
const X = embeddingDf.icol(0).asArray();
const Y = embeddingDf.icol(1).asArray();
const offsets = new Int32Array(X.length);
let numOffsets = 0;
for (let i = 0, l = X.length; i < l; i += 1) {
if (!Number.isNaN(X[i]) && !Number.isNaN(Y[i])) {
offsets[numOffsets] = i;
numOffsets += 1;
}
}
if (numOffsets === X.length) return null;
return offsets.subarray(0, numOffsets);
}
export function _getDiscreteCellEmbeddingRowIndex(embeddingDf) {
const idx = _getEmbeddingRowOffsets(embeddingDf.rowIndex, embeddingDf);
if (idx === null) return embeddingDf.rowIndex;
return embeddingDf.rowIndex.isubset(idx);
}
export const getDiscreteCellEmbeddingRowIndex = memoize(
_getDiscreteCellEmbeddingRowIndex,
(df) => df.__id
);
export function getEmbSubsetView(annoMatrix) {
/* if there is an embedding subset in the view stack, return it. Falsish if not. */
while (annoMatrix.isView) {
if (annoMatrix.userFlags.isEmbSubsetView) return annoMatrix;
annoMatrix = annoMatrix.viewOf;
}
return undefined;
}
dev_docs/cxg.md
+162
View File
@@ -0,0 +1,162 @@
# CXG Data Format Specification
Document Status: _draft_
Version: 0.2.0 (_DRAFT, not yet approved_)
Date Last Modified: 2020-07-23
## Introduction
CXG is a cellxgene-private data format, used for at-rest storage of annotated matrix data. It is similar to [AnnData](https://anndata.readthedocs.io/en/stable/), but with performance and access characteristics amenable to a multi-dataset, multi-user serving environment.
CXG is built upon the [TileDB](https://tiledb.com/) embedded database. Each CXG is a TileDB [group](https://docs.tiledb.com/main/api-usage/object-management), which in turn includes one or more TileDB multi-dimensional arrays.
This document presumes familiarity with [TileDB terminology and concepts](https://docs.tiledb.com/main/), the [Corpora schema](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema.md) and its [H5AD encoding](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema_h5ad_implementation.md), and the AnnData/H5AD data model.
This document also leverages the current cellxgene schema, which is documented in the [REST API spec](./REST_API.md).
### Terminology
Unless explicitly noted, the AnnData conventions and terminology are adopted when referring to general annotated matrix characteristics (eg, `n_obs` is the number of observations/rows/cells in the annotated matrix). Where implied by context, eg, "TileDB array attribute", domain-specific terms are used.
Where capitalized, [IETF RFC 2119](https://www.ietf.org/rfc/rfc2119.txt) conventions are followed (ie, conventions MUST be followed).
_Author's note:_ if you see any ambiguous terms, please call them out for clarification.
### Reserved
The `cxg` prefix is used for CXG-specific names.
### Encoding Data With TileDB Arrays
The TileDB array schema authoritatively defines the characteristics of each array (eg, the type of `X` is defined by [`X.schema`](https://tiledb-inc-tiledb-py.readthedocs-hosted.com/en/stable/python-api.html#tiledb.libtiledb.Array.schema)). In some cases, additional metadata is required for the CXG, and is attched to the array using the TileDB [array metadata](https://docs.tiledb.com/main/basic-concepts/array-metadata) capability.
All TileDB arrays MUST have a uint32 domain, zero based. All X counts and embedding coordinates SHOULD be coerced to float32, which is ample precision for visualization purposes, and MUST be a numeric type. Dataframe (metadata) types are generally preserved, or where that is not possible, converted to something with equal representative value in the cellxgene application (eg, categorical types are converted to string, bools to uint8, etc).
CXG consumers (readers) MUST be prepared to handle any legal TileDB compression, global layout and tile size. CXG writers SHOULD attempt to encode data using best-effort heuristics for time and space considerations (eg, dense/sparse encoding tradeoffs).
## Entities
### CXG
The CXG is a TileDB group containing all data and metadata for a single annotated matrix. The following objects MUST be present in a CXG, except where noted as optional:
* __obs__: a TileDB array, of shape (n_obs,), containing obs annotations, each annotation stored in a separate TileDB array attribute.
* __var__: a TileDB array, of shape (n_var,), containing var annotations, each annotation stored in a separate TileDB array attribute.
* __X__: a TileDB array, of shape (n_obs, n_var), with a single TileDB attribute of numeric type.
* __X_col_shift__: (optional) TilebDB Array used in column shift encoding, shape (n_var,), dtype = X.dtype. Single unnamed numeric attribute.
* __emb__: a TileDB group, which in turn contains all (zero or more) embeddings.
* __emb__/\<embedding_name\>__: a TileDB array, with a single anonymous attribute, of numeric type, and shape (n_obs, N>=2).
* __cxg_group_metadata__: an empty TileDB array, used to store CXG-wide metadata
### obs and var
All per-observation (obs) and per-feature (var) data is encoded in a TileDB array named `obs` and `var` respectively, with shape (n_obs,) and (n_var,). Each TileDB array has an array attribute for each obs/var column. All TileDB array attributes will have the same type and value as the original data, eg, float32, with the following exceptions:
* bool is encoded as uint8 (1/0)
* categorical is encoded as string
* Numeric types are cast to 32-bit equivalents
In addition to the obs/var data, both TileDB arrays contain an optional 'cxg_schema' metadata field that is a JSON string containing per-column (attribute) schema hinting. This is used where the TileDB native typing information is insufficient to reconstruct useful information such as categorical typing from Pandas DataFrames, and to communicate which column is the preferred human-readable index for obs & var.
The `cxg_schema` JSON string is attached to the TileDB array metadata, and is a dictionary containing the following top-level names:
* "index": string, containing the name of the index column
* \<column-name\>: optional, a JSON dict, contain a schema definition using the same format as the cellxgene REST API /schema route
For example:
```
{
"index": "obs_index",
"louvain": { "type": "categorical", "categories": [ "0", "1", "2", "3", "4" ]}
"is_useful": { "type": "boolean" }
}
```
### X
TileDB array, with a single anonymous attribute, shape (n_obs, n_var), containing the count matrix (equivalent to the AnnData `X` array). MUST have numeric type, and SHOULD be float32. The TileDB schema defines type and sparsity, and both dense and sparse encoding are supported.
### X_col_shift
Optional TileDB array, used to encode-per column offsets for column-shift sparse encoding. The TileDB array will have a single anonymous attribute, of the same type as the X array, and shape (n_var,).
If the X array is sparse, and X_col_shift exists, then all values in the i'th column were subtracted by X_col_shift[i].
### emb and embedding arrays
A CXG must have a group named `emb`, which will contain all embeddings. Embeddings are encoded as TileDB arrays, of numeric type and shape (n_obs, >=2). The arrays SHOULD be coerced to float32, and MUST be a numeric type. The TileDB array name will be assumed to be the embedding name (conventionally, embedding names in CXG are _not_ prefixed with an `X_` as they are in AnnData).
CXG supports zero or more embeddings. Note that cellxgene currently _requires_ at least one embedding.
### cxg_group_metadata
Required, but empty TileDB array, used to store CXG-wide metadata. The following fields are defined:
* __cxg_version__: (required) a semver string identifying the specification version used to encode the CXG.
* __cxg_properties__: (optional) a dictionary containing dataset wide properties, defined below.
* __cxg_category_colors__: (optional) a categorical color table, defined below.
#### cxg_properties
The properties metadata dictionary contains dataset-wide properties, encoded as a JSON dictionary. Currently, the following fields are defined:
* title: string, dataset human name (eg, "Lung Tissue")
* about: string, fully-qualified http/https URL, linking to more information on the dataset.
All implementions MUST ignore unrecognized fields.
#### cxg_category_colors
This optional field contains a copy of the category color table, which MAY be used to display category-specific color labels. This is a JSON dictionary, containing a per-category color-table. Each color table is named `{category_name}_colors`, and is itself a dictionary mapping label name to RGB color. For example:
```
{
"louvain_colors": {
"0": "#FFFFFF",
"1": "#000000"
}
}
```
## Corpora Schema Encoding
The [Corpora schema](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema.md) and [Corpora AnnData encoding](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema_h5ad_implementation.md) define a set of metadata and encoding conventions for annotated matrices. When a Corpora dataset is encoded as a CXG, the following shall apply.
### Corpora metadata property
A CXG containing a Corpora dataset will contain a property in the __cxg_group_metadata__ field named `corpora`. The value will be a JSON encoded string, which in turn contains all properties defined in the [Corpora AnnData uns](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema_h5ad_implementation.md#uns) container. For example:
```
{
"corpora": {
"version": {
"corpora_schema_version": "1.0.0",
"corpora_encoding_version": "0.1.0",
}
}
}
```
The `corpora` metadata, if present, MUST contain the version information. Optionality of other values in this object will follow the specifications set forth in the relevant Corpora schema specification (ie, optional fields are optional, required are present, etc), with the following changes:
* the contents of `corpora_encoding_version` MUST be identical to the `cxg_version`, as this field is defined as the current object encoding version, *NOT* the source data encoding version.
* the entire encoding will be JSON, rather than a hybrid Python/JSON encoding, but will otherwise follow the data structure defined by the AnnData Corpora encoding.
* the `<obs_column>_colors` will be omitted in favor of `cxg_category_colors`
### Other Corpora fields
All other Corpora schema fields will be encoded into a CXG using the conventions defined in the [Corpora Schema AnnData Implementation](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema_h5ad_implementation.md). For example, fields in `AnnData.obs` will be encoded in the CXG `obs`array as defined [above](#obs-and-var).
### Compatibility with CXG 0.1.0
For backwards compatibility and continuity with CXG version 0.1.0, the following MUST be implemented.
#### Presentation Hints
* The [Corpora `title`](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema.md#presentation-metadata) value MUST be saved in the `cxg_properties.title` field.
* The [Corpora `color_map`](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema.md#presentation-hints), when present in the dataset, MUST be saved in the `cxg_category_colors` field and NOT in the `corpora` field.
* The [Corpora SUMMARY `project_link`](https://github.com/chanzuckerberg/corpora-data-portal/blob/main/backend/schema/corpora_schema.md#presentation-hints), if present, MUST be saved in the `cxg_properties.about` field.
Where these values differ in the final CXG, the `cxg_properties` values WILL take precedence.
## CXG Version History
There were several ad hoc version of CXG created prior to this spec. This describes the _proposed_ next version of CXG, which incoporates support for Corpora schema semantics. Prior verisons:
* _unnamed_ - an unnamed development version. Did not include explicit versioning support in the data model, but can be detected by the absence of __cxg_group_metadata__ and any version property. Created in early 2020, and not actively used in production
* 0.1 - the first and current version, defined to support the capabilities of the mid-2020 cellxgene. Created in early 2020, and in active use. Includes everything in this spec, excluding Corpora schema support. __NOTE:__ this version is encoded with a short-hand (malformed) semver version number.
* 0.2.0 - this specification.
dev_docs/developer_guidelines.md
+15
View File
@@ -151,3 +151,18 @@ If you would like to run the smoke tests against a hot-reloaded version of the c
- You can also install/launch the server side code from npm scrips (requires python3.6 with virtualenv) with the `scripts/backend_dev` script.
- Check out [e2e Tests](e2e_tests.md) for more details
## Doc changes
To evaluate changes to documentation in `cellxgene/docs`, you must install [jekyll](https://jekyllrb.com/docs/installation/), then run:
```bash
cd docs
bundle install
bundle exec jekyll serve
```
If changes look good, run the following, then commit and push:
```bash
bundle exec jekyll build
```
docs/Gemfile
+1
View File
@@ -1,2 +1,3 @@
source 'https://rubygems.org'
gem "github-pages", group: :jekyll_plugins
gem 'jekyll-relative-links'
docs/_config.yml
+3
View File
@@ -3,6 +3,9 @@ show_downloads: false
url: "https://chanzuckerberg.github.io"
baseurl: "/cellxgene"
plugins:
- jekyll-relative-links
logo: cellxgene-logo.png
nav:
docs/_site/index.html
+7 -7
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>Index | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="Index" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebSite","headline":"Index","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","url":"https://chanzuckerberg.github.io/cellxgene/","name":"cellxgene","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","@type":"WebSite","headline":"Index","url":"https://chanzuckerberg.github.io/cellxgene/","name":"cellxgene","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
@@ -99,7 +99,7 @@
<p>Whether you need to visualize one thousand cells or one million, cellxgene helps you gain insight into your single-cell data.</p>
<p>To install cellxgene you need Python 3.6+. We recommend <a href="install">installing cellxgene into a conda or virtual environment.</a></p>
<p>To install cellxgene you need Python 3.6+. We recommend <a href="posts/install">installing cellxgene into a conda or virtual environment.</a></p>
<p>Install the package.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip <span class="nb">install </span>cellxgene
@@ -110,8 +110,8 @@
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cellxgene launch https://cellxgene-example-data.czi.technology/pbmc3k.h5ad
</code></pre></div></div>
<p>To explore more datasets already formatted for cellxgene, check out the <a href="demo-data">Demo data</a> or
see <a href="prepare">Preparing your data</a> to learn more about formatting your own
<p>To explore more datasets already formatted for cellxgene, check out the <a href="posts/demo-data">Demo data</a> or
see <a href="posts/prepare">Preparing your data</a> to learn more about formatting your own
data for cellxgene.</p>
<h1 id="getting-help">Getting help</h1>
docs/_site/posts/annotations.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>annotations | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="annotations" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Creating annotations" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/annotations.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"annotations","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Creating annotations","url":"https://chanzuckerberg.github.io/cellxgene/posts/annotations.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Creating annotations","@type":"WebPage","headline":"annotations","url":"https://chanzuckerberg.github.io/cellxgene/posts/annotations.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/contact.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>Contact | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="Contact" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Contact" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/contact.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"Contact","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Contact","url":"https://chanzuckerberg.github.io/cellxgene/posts/contact.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Contact","@type":"WebPage","headline":"Contact","url":"https://chanzuckerberg.github.io/cellxgene/posts/contact.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/contribute.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>Code of conduct | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="Code of conduct" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/contribute.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"Code of conduct","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","url":"https://chanzuckerberg.github.io/cellxgene/posts/contribute.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","@type":"WebPage","headline":"Code of conduct","url":"https://chanzuckerberg.github.io/cellxgene/posts/contribute.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/demo-data.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>demo-data | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="demo-data" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Demo datasets" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/demo-data.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"demo-data","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Demo datasets","url":"https://chanzuckerberg.github.io/cellxgene/posts/demo-data.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Demo datasets","@type":"WebPage","headline":"demo-data","url":"https://chanzuckerberg.github.io/cellxgene/posts/demo-data.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/gallery.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>Gallery | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="Gallery" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/gallery.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"Gallery","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","url":"https://chanzuckerberg.github.io/cellxgene/posts/gallery.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","@type":"WebPage","headline":"Gallery","url":"https://chanzuckerberg.github.io/cellxgene/posts/gallery.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/hosted.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>Hosting cellxgene on the web | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="Hosting cellxgene on the web" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/hosted.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"Hosting cellxgene on the web","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","url":"https://chanzuckerberg.github.io/cellxgene/posts/hosted.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","@type":"WebPage","headline":"Hosting cellxgene on the web","url":"https://chanzuckerberg.github.io/cellxgene/posts/hosted.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/index.html
-109
View File
@@ -1,109 +0,0 @@
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<title>Index | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta property="og:title" content="Index" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
<meta property="og:description" content="An interactive explorer for single-cell transcriptomics data" />
<link rel="canonical" href="http://localhost:4000/" />
<meta property="og:url" content="http://localhost:4000/" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"description":"An interactive explorer for single-cell transcriptomics data","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"http://localhost:4000/cellxgene-logo.svg"}},"@type":"WebSite","url":"http://localhost:4000/","name":"cellxgene","headline":"Index","@context":"http://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/assets/css/style.css?v=d12c1b6de7e247cdbabc4ce83364f9ea50978892">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
</head>
<body>
<div class="wrapper">
<header>
<img src="/cellxgene-logo.svg" alt="cellxgene" />
<p>An interactive explorer for single-cell transcriptomics data</p>
<p>
<a href="/" class="btn">Quick start</a><br>
<a href="/posts/gallery" class="btn">Gallery</a><br>
<a href="/posts/install" class="btn">Installation</a><br>
<a href="/posts/demo-data" class="btn">Demo datasets</a><br>
<a href="/posts/prepare" class="btn">Preparing your data</a><br>
<a href="/posts/launch" class="btn">Launching cellxgene</a><br>
<a href="/posts/hosted" class="btn">Hosting cellxgene</a><br>
<a href="/posts/contribute" class="btn">Contributing (ideas or code)</a><br>
<a href="/posts/methods" class="btn">Methods</a><br>
<a href="/posts/faq" class="btn">FAQ</a><br>
<a href="/posts/roadmap" class="btn">Roadmap</a><br>
<a href="/posts/contact" class="btn">Contact & finding help</a><br>
<a href="https://github.com/chanzuckerberg/cellxgene" class="btn" target="_blank">Code</a>
</p>
</header>
<section>
<h1 id="quick-start">Quick start</h1>
<p>Whether you need to visualize one thousand cells or one million, cellxgene helps you gain insight into your single-cell data.</p>
<p>To install cellxgene you need Python 3.6+. We recommend <a href="/faq.html#how-do-i-create-a-python-environment-for-cellxgene">installing cellxgene into a conda or virtual environment.</a></p>
<p>Install the package.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install cellxgene
</code></pre></div></div>
<p>Download an example <a href="https://anndata.readthedocs.io/en/latest/">anndata</a> file</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-o</span> tabula-muris.h5ad https://cellxgene-example-data.czi.technology/tabula-muris.h5ad.zip
unzip tabula-muris.h5ad.zip
</code></pre></div></div>
<p>Launch cellxgene</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cellxgene launch tabula-muris.h5ad <span class="nt">--open</span>
</code></pre></div></div>
<p>To explore more datasets already formatted for cellxgene, check out the <a href="demo-data">Demo data</a> or
see <a href="prepare">Preparing your data</a> to learn more about formatting your own
data for cellxgene.</p>
<h1 id="getting-help">Getting help</h1>
<p>We’d love to hear from you!</p>
<p>For questions, suggestions, or accolades, <a href="https://join-cellxgene-users.herokuapp.com/">join the <code class="highlighter-rouge">#cellxgene-users</code> channel on the CZI Science Slack</a> and say “hi!”.</p>
<p>For any errors, <a href="https://github.com/chanzuckerberg/cellxgene/issues">report bugs on Github</a>.</p>
</section>
<footer>
<p>This project is maintained by <a href="https://github.com/chanzuckerberg">chanzuckerberg</a></p>
</footer>
</div>
<script src="/assets/js/scale.fix.js"></script>
</body>
</html>
docs/_site/posts/install.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>Install | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="Install" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/install.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"Install","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","url":"https://chanzuckerberg.github.io/cellxgene/posts/install.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","@type":"WebPage","headline":"Install","url":"https://chanzuckerberg.github.io/cellxgene/posts/install.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/launch.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>demo-data | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="demo-data" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Demo datasets" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/launch.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"demo-data","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Demo datasets","url":"https://chanzuckerberg.github.io/cellxgene/posts/launch.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Demo datasets","@type":"WebPage","headline":"demo-data","url":"https://chanzuckerberg.github.io/cellxgene/posts/launch.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/methods.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>Methods | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="Methods" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/methods.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"Methods","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","url":"https://chanzuckerberg.github.io/cellxgene/posts/methods.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"An interactive explorer for single-cell transcriptomics data","@type":"WebPage","headline":"Methods","url":"https://chanzuckerberg.github.io/cellxgene/posts/methods.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/prepare.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>prepare | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="prepare" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Preparing your data" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/prepare.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"prepare","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Preparing your data","url":"https://chanzuckerberg.github.io/cellxgene/posts/prepare.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Preparing your data","@type":"WebPage","headline":"prepare","url":"https://chanzuckerberg.github.io/cellxgene/posts/prepare.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/roadmap.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>roadmap | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="roadmap" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Roadmap" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/roadmap.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"roadmap","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Roadmap","url":"https://chanzuckerberg.github.io/cellxgene/posts/roadmap.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Roadmap","@type":"WebPage","headline":"roadmap","url":"https://chanzuckerberg.github.io/cellxgene/posts/roadmap.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/_site/posts/troubleshooting.html
+4 -4
View File
@@ -5,9 +5,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<!-- Begin Jekyll SEO tag v2.6.1 -->
<title>Troubleshooting | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta name="generator" content="Jekyll v3.8.7" />
<meta property="og:title" content="Troubleshooting" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Troubleshooting" />
@@ -16,10 +16,10 @@
<meta property="og:url" content="https://chanzuckerberg.github.io/cellxgene/posts/troubleshooting.html" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"@type":"WebPage","headline":"Troubleshooting","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Troubleshooting","url":"https://chanzuckerberg.github.io/cellxgene/posts/troubleshooting.html","@context":"http://schema.org"}</script>
{"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://chanzuckerberg.github.io/cellxgene/cellxgene-logo.png"}},"description":"Troubleshooting","@type":"WebPage","headline":"Troubleshooting","url":"https://chanzuckerberg.github.io/cellxgene/posts/troubleshooting.html","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=b5ba39592c9370d8c06398ca9ca12e7dc89afa50">
<link rel="stylesheet" href="/cellxgene/assets/css/style.css?v=6bda27f5542fb7f469425e1cd99f2f37268b095f">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
docs/index.md
+3 -3
View File
@@ -7,7 +7,7 @@ layout: default
Whether you need to visualize one thousand cells or one million, cellxgene helps you gain insight into your single-cell data.
To install cellxgene you need Python 3.6+. We recommend [installing cellxgene into a conda or virtual environment.](install)
To install cellxgene you need Python 3.6+. We recommend [installing cellxgene into a conda or virtual environment.](posts/install)
Install the package.
``` bash
@@ -20,8 +20,8 @@ Launch cellxgene with an example [anndata](https://anndata.readthedocs.io/en/lat
cellxgene launch https://cellxgene-example-data.czi.technology/pbmc3k.h5ad
```
To explore more datasets already formatted for cellxgene, check out the [Demo data](demo-data) or
see [Preparing your data](prepare) to learn more about formatting your own
To explore more datasets already formatted for cellxgene, check out the [Demo data](posts/demo-data) or
see [Preparing your data](posts/prepare) to learn more about formatting your own
data for cellxgene.
# Getting help
docs/posts/cellxgene_cziscience_com.md
+24
View File
@@ -311,5 +311,29 @@ with a link to embed on your own site, please drop us a note at <mailto:cellxgen
<a href="https://www.nature.com/articles/nature25739">Nature</a>
</td>
</tr>
<tr>
<td><a href="https://cellxgene.cziscience.com/d/Single_soma_transcriptomics_AT8-45.cxg/" target="_blank">Single Soma Transcriptomics - AT8</a></td>
<td>
<a href="https://www.biorxiv.org/content/10.1101/2020.05.11.088591v1">bioRxiv preprint</a>
</td>
</tr>
<tr>
<td><a href="https://cellxgene.cziscience.com/d/Single_soma_transcriptomics_MAP2-46.cxg/" target="_blank">Single Soma Transcriptomics - MAP2</a></td>
<td>
<a href="https://www.biorxiv.org/content/10.1101/2020.05.11.088591v1">bioRxiv preprint</a>
</td>
</tr>
<tr>
<td><a href="https://cellxgene.cziscience.com/d/Single_soma_transcriptomics_MAP2AT8-47.cxg/" target="_blank">Single Soma Transcriptomics - MAP2AT8</a></td>
<td>
<a href="https://www.biorxiv.org/content/10.1101/2020.05.11.088591v1">bioRxiv preprint</a>
</td>
</tr>
<tr>
<td><a href="https://cellxgene.cziscience.com/d/Single_cell_longitudinal_analysis_of_SARS_CoV_2_infection_in_human_bronchial_epithelial_cells-29.cxg/" target="_blank">Single-cell longitudinal analysis of SARS-CoV-2 infection in human bronchial epithelial cells</a></td>
<td>
<a href="https://www.biorxiv.org/content/10.1101/2020.05.06.081695v2">bioRxiv preprint</a>
</td>
</tr>
</tbody>
</table>
docs/posts/hosted.md
+23 -16
View File
@@ -38,31 +38,38 @@ If you know of other solutions, drop us a note and we'll add to this list.
# Deploying cellxgene with Heroku
## Quickstart
## Heroku Support
Clicking on the following button will forward you to Heroku to begin the deployment process:
The cellxgene team has decided to end our support for our experimental deploy to Heroku button as we move towards providing a supported method of hosted cellxgene.
<a href="https://heroku.com/deploy?template=https://github.com/chanzuckerberg/cellxgene">
<img src="https://www.herokucdn.com/deploy/button.svg" alt="Deploy">
</a>
While we no longer directly support Heroku, it is still possible to create a Heroku app via [our provided Dockerfile here](https://github.com/chanzuckerberg/cellxgene/blob/main/Dockerfile) and [Heroku's documentation](https://devcenter.heroku.com/articles/build-docker-images-heroku-yml).
If not already logged in to Heroku, there you will be prompted to log in or sign up for an account.
You may have to tweak the `Dockerfile` like so:
Once logged in you will be sent to the setup page. Here you can set some of the basic settings for the app:
```Dockerfile
FROM ubuntu:bionic
### Default settings
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
- `App name`: the unique name for your deployment
- This will also serve as the default URL (e.g. https://cellxgene.herokapp.com/)
- `App owner`: Who will own this app. Either you personally or an organization/team
- `Region`: Location of the server where the app will be deployed (EU or US)
RUN apt-get update && \
apt-get install -y build-essential libxml2-dev python3-dev python3-pip zlib1g-dev python3-requests && \
pip3 install cellxgene
### Configuration
# ENTRYPOINT ["cellxgene"] # Heroku doesn't work well with ENTRYPOINT
```
- `DATASET`: A _publicly_ accessible URL pointing to a .h5ad file to view
- This defaults to pbm3k.h5ad
and provide a `heroku.yml` file similar to this:
After filling out the settings and pressing the `Deploy app` button Heroku will begin building your deployment. This process will take a few minutes, but once completed you will have a personal free hosted version of cellxgene!
```yml
build:
docker:
web: Dockerfile
run:
web:
command:
- cellxgene launch --host 0.0.0.0 --port $PORT $DATASET # the DATATSET config var must be defined in your dashboard settings.
```
## What is Heroku?
docs/posts/index.html
-109
View File
@@ -1,109 +0,0 @@
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Begin Jekyll SEO tag v2.5.0 -->
<title>Index | cellxgene</title>
<meta name="generator" content="Jekyll v3.8.5" />
<meta property="og:title" content="Index" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="An interactive explorer for single-cell transcriptomics data" />
<meta property="og:description" content="An interactive explorer for single-cell transcriptomics data" />
<link rel="canonical" href="http://localhost:4000/" />
<meta property="og:url" content="http://localhost:4000/" />
<meta property="og:site_name" content="cellxgene" />
<script type="application/ld+json">
{"description":"An interactive explorer for single-cell transcriptomics data","publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"http://localhost:4000/cellxgene-logo.svg"}},"@type":"WebSite","url":"http://localhost:4000/","name":"cellxgene","headline":"Index","@context":"http://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/assets/css/style.css?v=d12c1b6de7e247cdbabc4ce83364f9ea50978892">
<!--[if lt IE 9]>
<script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
<![endif]-->
</head>
<body>
<div class="wrapper">
<header>
<img src="/cellxgene-logo.svg" alt="cellxgene" />
<p>An interactive explorer for single-cell transcriptomics data</p>
<p>
<a href="/" class="btn">Quick start</a><br>
<a href="/posts/gallery" class="btn">Gallery</a><br>
<a href="/posts/install" class="btn">Installation</a><br>
<a href="/posts/demo-data" class="btn">Demo datasets</a><br>
<a href="/posts/prepare" class="btn">Preparing your data</a><br>
<a href="/posts/launch" class="btn">Launching cellxgene</a><br>
<a href="/posts/hosted" class="btn">Hosting cellxgene</a><br>
<a href="/posts/contribute" class="btn">Contributing (ideas or code)</a><br>
<a href="/posts/methods" class="btn">Methods</a><br>
<a href="/posts/faq" class="btn">FAQ</a><br>
<a href="/posts/roadmap" class="btn">Roadmap</a><br>
<a href="/posts/contact" class="btn">Contact & finding help</a><br>
<a href="https://github.com/chanzuckerberg/cellxgene" class="btn" target="_blank">Code</a>
</p>
</header>
<section>
<h1 id="quick-start">Quick start</h1>
<p>Whether you need to visualize one thousand cells or one million, cellxgene helps you gain insight into your single-cell data.</p>
<p>To install cellxgene you need Python 3.6+. We recommend <a href="/faq.html#how-do-i-create-a-python-environment-for-cellxgene">installing cellxgene into a conda or virtual environment.</a></p>
<p>Install the package.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install cellxgene
</code></pre></div></div>
<p>Download an example <a href="https://anndata.readthedocs.io/en/latest/">anndata</a> file</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-o</span> tabula-muris.h5ad https://cellxgene-example-data.czi.technology/tabula-muris.h5ad.zip
unzip tabula-muris.h5ad.zip
</code></pre></div></div>
<p>Launch cellxgene</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cellxgene launch tabula-muris.h5ad <span class="nt">--open</span>
</code></pre></div></div>
<p>To explore more datasets already formatted for cellxgene, check out the <a href="demo-data">Demo data</a> or
see <a href="prepare">Preparing your data</a> to learn more about formatting your own
data for cellxgene.</p>
<h1 id="getting-help">Getting help</h1>
<p>We’d love to hear from you!</p>
<p>For questions, suggestions, or accolades, <a href="https://join-cellxgene-users.herokuapp.com/">join the <code class="highlighter-rouge">#cellxgene-users</code> channel on the CZI Science Slack</a> and say “hi!”.</p>
<p>For any errors, <a href="https://github.com/chanzuckerberg/cellxgene/issues">report bugs on Github</a>.</p>
</section>
<footer>
<p>This project is maintained by <a href="https://github.com/chanzuckerberg">chanzuckerberg</a></p>
</footer>
</div>
<script src="/assets/js/scale.fix.js"></script>
</body>
</html>
experiments/heroku/Dockerfile
-7
View File
@@ -1,7 +0,0 @@
FROM python:3.7
WORKDIR /usr/src/app
RUN pip3 install cellxgene
expose 5005
experiments/heroku/README.md
-58
View File
@@ -1,58 +0,0 @@
# cellxgene cloud deployment with Heroku
## Quickstart
Clicking on the following button will forward you to Heroku to begin the deployment process:
<a href="https://heroku.com/deploy?template=https://github.com/chanzuckerberg/cellxgene/tree/main">
<img src="https://www.herokucdn.com/deploy/button.svg" alt="Deploy">
</a>
If not already logged in to Heroku, there you will be prompted to log in or sign up for an account.
Once logged in you will be sent to the setup page. Here you can set some of the basic settings for the app:
#### Default settings
- `App name`: the unique name for your deployment
- This will also serve as the default URL (e.g. https://cellxgene.herokapp.com/)
- `App owner`: Who will own this app. Either you personally or an organization/team
- `Region`: Location of the server where the app will be deployed (EU or US)
#### Configuration
- `DATASET`: A _publicly_ accessible URL pointing to a .h5ad file to view
- This defaults to pbm3k.h5ad
After filling out the settings and pressing the `Deploy app` button Heroku will begin building your deployment. This process will take a few minutes, but once completed you will have a personal free hosted version of cellxgene!
## What is Heroku?
Heroku is a quick and easy way to host applications on the cloud.
A Heroku deployment of cellxgene means that the app is not running on your local machine. Instead, the app is installed, configured, and ran on the Heroku servers (read: cloud).
On Heroku's servers, applications run on a [dyno](https://www.heroku.com/dynos) which are Heroku's implementation and abstraction of containers.
Heroku is one of many options available for hosting instances of cellxgene on the web.
Some other options include: Amazon Web Services, Google Cloud Platform, Digital Ocean, and Microsoft Azure.
## Why use Heroku to deploy cellxgene?
What Heroku enables is a quick, non-technical method of setting up a cellxgene instance. No command line knowledge needed. This also allows machines to access the instance via the internet, so sharing a visualized dataset is as simple as sharing a link.
Because cellxgene currently heavily relies on its Python backend for providing the viewer with the necessary data and tooling, it is currently not possible to host cellxgene as a static webpage.
This is a good option if you want to quickly deploy an instance of cellxgene to the web. Heroku deployments are free for small datasets up to around 250MBs in size. See below regarding larger datasets.
## When should I not deploy with Heroku?
- The default free dyno offered by Heroku is limited in memory to 512 MBs
- The amount of memory needed for the dyno is roughly the same size as the h5ad file
- Heroku offers tiered paid dynos. More can be found [here](https://www.heroku.com/pricing)
- Note that this can get _very_ expensive for larger datasets (\$25+ a month)
- On the free dyno, after 30 minutes of inactivity, Heroku will put your app into a hibernation mode. On the next access, Heroku will need time to boot the dyno back online.
- Having multiple simultaneous users requires more memory. This means that the free container size is easily overwhelmed by multiple users, even with small datasets; this can be addressed by purchasing a larger container size
- For this facilitated Heroku deployment to work, your dataset must be hosted on a publicly accessible URL
- By default, Heroku publically shares your instance to anyone with the URL.
- There are many ways of securing your instance. One quick and simple way is by installing [wwwhisper](https://elements.heroku.com/addons/wwwhisper), a Heroku addon
heroku.yml
-5
View File
@@ -1,5 +0,0 @@
build:
docker:
web: experiments/heroku/Dockerfile
run:
web: cellxgene launch $DATASET --host 0.0.0.0 --port $PORT
server/Makefile
+26 -2
View File
@@ -7,11 +7,35 @@ clean:
rm -f common/web/csp-hashes.json
.PHONY: unit-test
unit-test:
unit-test: create-test-db
PYTHONWARNINGS=ignore:ResourceWarning coverage run \
--source=app,cli,common,compute,converters,data_anndata,data_common,data_cxg \
--omit=.coverage,data_common/fbs/NetEncoding,venv \
-m unittest discover \
--start-directory test/ \
--top-level-directory ../ \
--verbose
--verbose; test_result=$$?; \
$(MAKE) clean-test-db; \
exit $$test_result \
.PHONY: test-db
test-db: create-test-db
PYTHONWARNINGS=ignore:ResourceWarning coverage run \
--source=app,cli,common,compute,converters,data_anndata,data_common,data_cxg \
--omit=.coverage,data_common/fbs/NetEncoding,venv \
-m unittest discover \
--start-directory test/test_database \
--top-level-directory ../ \
--verbose; test_result=$$?; \
$(MAKE) clean-test-db; \
exit $$test_result
.PHONY: create-test-db
create-test-db:
-docker run -d -p 5432:5432 --name test_db -e POSTGRES_PASSWORD=test_pw postgres
.PHONY: clean-test-db
clean-test-db:
-docker stop test_db
-docker rm test_db
server/__init__.py
+3 -2
View File
@@ -1,8 +1,9 @@
from server.common.utils import import_plugins
import logging
import sys
__version__ = "0.15.0"
from server.common.utils.utils import import_plugins
__version__ = "0.16.0"
display_version = "cellxgene v" + __version__
try:
server/app/app.py
+39 -9
View File
@@ -1,22 +1,19 @@
import datetime
import logging
from functools import wraps
from http import HTTPStatus
from flask import Flask, redirect, current_app, make_response, render_template, abort
from flask import Blueprint, request
from flask import Flask, redirect, current_app, make_response, render_template, abort, Blueprint, request
from flask_restful import Api, Resource
from server_timing import Timing as ServerTiming
from http import HTTPStatus
import server.common.rest as common_rest
from server.common.errors import DatasetAccessError, RequestException
from server.common.utils import path_join, Float32JSONEncoder
from server.common.data_locator import DataLocator
from server.common.errors import DatasetAccessError, RequestException
from server.common.health import health_check
from server.common.utils.utils import path_join, Float32JSONEncoder
from server.data_common.matrix_loader import MatrixDataLoader
from functools import wraps
webbp = Blueprint("webapp", "server.common.web", template_folder="templates")
ONE_WEEK = 7 * 24 * 60 * 60
@@ -85,6 +82,7 @@ def dataset_index(url_dataroot=None, dataset=None):
try:
cache_manager = current_app.matrix_data_cache_manager
with cache_manager.data_adaptor(url_dataroot, location, app_config) as data_adaptor:
data_adaptor.set_uri_path(f"{url_dataroot}/{dataset}")
dataset_title = app_config.get_title(data_adaptor)
return render_template(
"index.html", datasetTitle=dataset_title, SCRIPTS=scripts, INLINE_SCRIPTS=inline_scripts
@@ -129,7 +127,7 @@ def get_data_adaptor(url_dataroot=None, dataset=None):
# sufficient to check that the datapath starts with the
# dataroot to determine that the datapath is under the dataroot.
if not datapath.startswith(dataroot):
raise DatasetAccessError("Invalid dataset {url_dataroot}/{dataset}")
raise DatasetAccessError(f"Invalid dataset {url_dataroot}/{dataset}")
if datapath is None:
return common_rest.abort_and_log(HTTPStatus.BAD_REQUEST, "Invalid dataset NONE", loglevel=logging.INFO)
@@ -138,11 +136,24 @@ def get_data_adaptor(url_dataroot=None, dataset=None):
return cache_manager.data_adaptor(dataset_key, datapath, config)
def requires_authentication(func):
@wraps(func)
def wrapped_function(self, *args, **kwargs):
auth = current_app.auth
if auth.is_user_authenticated():
return func(self, *args, **kwargs)
else:
return make_response("not authenticated", HTTPStatus.UNAUTHORIZED)
return wrapped_function
def rest_get_data_adaptor(func):
@wraps(func)
def wrapped_function(self, dataset=None):
try:
with get_data_adaptor(self.url_dataroot, dataset) as data_adaptor:
data_adaptor.set_uri_path(f"{self.url_dataroot}/{dataset}")
return func(self, data_adaptor)
except DatasetAccessError as e:
return common_rest.abort_and_log(
@@ -160,6 +171,17 @@ def dataroot_test_index():
config = current_app.app_config
server_config = config.server_config
auth = server_config.auth
if auth.is_valid_authentication_type():
if server_config.auth.is_user_authenticated():
data += f"<p>Logged in as {auth.get_user_id()} / {auth.get_user_name()} / {auth.get_user_email()}</p>"
if auth.requires_client_login():
if server_config.auth.is_user_authenticated():
data += "<p><a href='/logout'>Logout</a></p>"
else:
data += "<p><a href='/login'>Login</a></p>"
datasets = []
for dataroot_dict in server_config.multi_dataset__dataroot.values():
dataroot = dataroot_dict["dataroot"]
@@ -224,6 +246,7 @@ class AnnotationsObsAPI(DatasetResource):
def get(self, data_adaptor):
return common_rest.annotations_obs_get(request, data_adaptor)
@requires_authentication
@cache_control(no_store=True)
@rest_get_data_adaptor
def put(self, data_adaptor):
@@ -338,6 +361,7 @@ class Server:
lambda dataset, url_dataroot=url_dataroot: dataset_index(url_dataroot, dataset),
methods=["GET"],
)
else:
bp_api = Blueprint("api", __name__, url_prefix=api_version)
resources = get_api_resources(bp_api)
@@ -345,3 +369,9 @@ class Server:
self.app.matrix_data_cache_manager = server_config.matrix_data_cache_manager
self.app.app_config = app_config
auth = server_config.auth
self.app.auth = auth
if auth.requires_client_login():
auth.add_url_rules(self.app)
auth.complete_setup(self.app)
server/auth/__init__.py
+7
View File
@@ -0,0 +1,7 @@
# import the built in auth types so they can be registered
import server.auth.auth_none # noqa: F401
import server.auth.auth_test # noqa: F401
import server.auth.auth_session # noqa: F401
import server.auth.auth_oauth # noqa: F401
server/auth/auth.py
+87
View File
@@ -0,0 +1,87 @@
from abc import ABC, abstractmethod
class AuthTypeBase(ABC):
"""Base type for all authentication types."""
def __init__(self):
super().__init__()
@abstractmethod
def is_valid_authentication_type(self):
"""Return True if the auth type is valid, e.g. it can return userinfo and username.
(AuthTypeNone is the only one type that returns False)"""
pass
def requires_client_login(self):
"""Return True if the user needs to login from the client (e.g. Login button is shown)"""
return False
@abstractmethod
def complete_setup(self, app):
"""complete any setup that may be needed by this auth type. The Flask app is passed in.
This is the last auth function called before the server starts to run."""
pass
@abstractmethod
def is_user_authenticated(self):
"""Return True if the user is authenticated"""
pass
@abstractmethod
def get_user_id(self):
"""Return the id for this user (string)"""
pass
@abstractmethod
def get_user_name(self):
"""Return the name of the user (string)"""
pass
@abstractmethod
def get_user_email(self):
"""Return the name of the user (string)"""
pass
class AuthTypeClientBase(AuthTypeBase):
"""Base type for all authentication types that require the client to login"""
def __init__(self):
super().__init__()
def requires_client_login(self):
return True
@abstractmethod
def add_url_rules(self, selfapp):
"""Add url rules to the app (like /login, /logout, etc)"""
pass
@abstractmethod
def get_login_url(self, data_adaptor):
"""Return the url for the login route"""
pass
@abstractmethod
def get_logout_url(self, data_adaptor):
"""Return the url for the logout route"""
pass
class AuthTypeFactory:
"""Factory class to create an authentication type"""
auth_types = {}
@staticmethod
def register(name, auth_type):
assert(issubclass(auth_type, AuthTypeBase))
AuthTypeFactory.auth_types[name] = auth_type
@staticmethod
def create(name, app_config):
auth_type = AuthTypeFactory.auth_types.get(name)
if auth_type is None:
return None
return auth_type(app_config)
server/auth/auth_none.py
+28
View File
@@ -0,0 +1,28 @@
from server.auth.auth import AuthTypeBase, AuthTypeFactory
class AuthTypeNone(AuthTypeBase):
def __init__(self, app_config):
super().__init__()
def is_valid_authentication_type(self):
return False
def complete_setup(self, app):
pass
def is_user_authenticated(self):
return True
def get_user_id(self):
return None
def get_user_name(self):
return None
def get_user_email(self):
return None
AuthTypeFactory.register(None, AuthTypeNone)
server/auth/auth_oauth.py
+240
View File
@@ -0,0 +1,240 @@
from flask import session, request, redirect, current_app, after_this_request, has_request_context, g
from server.auth.auth import AuthTypeClientBase, AuthTypeFactory
from server.common.errors import AuthenticationError, ConfigurationError
from urllib.parse import urlencode
from urllib.request import urlopen
import json
# It is not required to have authlib or jose.
# However, it is a configuration error to use this auth type if they are not installed.
missingimport = []
try:
from authlib.integrations.flask_client import OAuth
except ModuleNotFoundError:
missingimport.append("authlib")
try:
from jose import jwt
except ModuleNotFoundError:
missingimport.append("jose")
class AuthTypeOAuth(AuthTypeClientBase):
"""An authentication type for oauth2 logins."""
CXG_ID_TOKEN = "id_token"
def __init__(self, server_config):
super().__init__()
if missingimport:
raise ConfigurationError(f"oauth requires these modules: {', '.join(missingimport)}")
self.algorithms = ["RS256"]
self.api_base_url = server_config.authentication__params_oauth__api_base_url
self.client_id = server_config.authentication__params_oauth__client_id
self.client_secret = server_config.authentication__params_oauth__client_secret
self.callback_base_url = server_config.authentication__params_oauth__callback_base_url
self.session_cookie = server_config.authentication__params_oauth__session_cookie
self.cookie_params = server_config.authentication__params_oauth__cookie
self._validate_cookie_params()
# set the audience
self.audience = self.client_id
# load the jwks (JSON Web Key Set).
# The JSON Web Key Set (JWKS) is a set of keys which contains the public keys used to verify
# any JSON Web Token (JWT) issued by the authorization server and signed using the RS256
try:
jwksloc = f"{self.api_base_url}/.well-known/jwks.json"
jwksurl = urlopen(jwksloc)
self.jwks = json.loads(jwksurl.read())
except Exception:
raise ConfigurationError(f"error in oauth, api_url_base: {self.api_base_url}, cannot access {jwksloc}")
def _validate_cookie_params(self):
"""check the cookie_params, and raise a ConfigurationError if there is something wrong"""
if self.session_cookie:
return
if not isinstance(self.cookie_params, dict):
raise ConfigurationError("either session_cookie or cookie must be set")
valid_keys = {"key", "max_age", "expires", "path", "domain", "secure", "httponly", "samesite"}
keys = set(self.cookie_params.keys())
unknown = keys - valid_keys
if unknown:
raise ConfigurationError(f"unexpected key in cookie params: {', '.join(unknown)}")
if "key" not in keys:
raise ConfigurationError("must have a key (name) in the cookie params")
def is_valid_authentication_type(self):
return True
def requires_client_login(self):
return True
def add_url_rules(self, app):
app.add_url_rule("/login", "login", self.login, methods=["GET"])
app.add_url_rule("/logout", "logout", self.logout, methods=["GET"])
app.add_url_rule("/oauth2/callback", "callback", self.callback, methods=["GET"])
def complete_setup(self, flask_app):
self.oauth = OAuth(flask_app)
if self.callback_base_url is None:
# In this case, assume the server is running on the same host as the client,
# and the oauth provider has been configured
# with a callback that understands a localhost callback (e.g. A http://localhost:5005).
server_config = flask_app.app_config.server_config
self.callback_base_url = f"http://{server_config.app__host}:{server_config.app__port}"
self.client = self.oauth.register(
"oauth",
client_id=self.client_id,
client_secret=self.client_secret,
api_base_url=self.api_base_url,
access_token_url=f"{self.api_base_url}/oauth/token",
authorize_url=f"{self.api_base_url}/authorize",
client_kwargs={
"scope" : "openid profile email",
}
)
def is_user_authenticated(self):
try:
payload = self.get_jwt_payload()
return payload is not None
except AuthenticationError:
return False
def get_user_id(self):
payload = self.get_jwt_payload()
if payload and payload.get("sub"):
return payload.get("sub")
return None
def get_user_name(self):
payload = self.get_jwt_payload()
if payload and payload.get("name"):
return payload.get("name")
return None
def get_user_email(self):
payload = self.get_jwt_payload()
if payload and payload.get("email"):
return payload.get("email")
return None
def update_response(self, response):
response.cache_control.update(
dict(public=True, max_age=0, no_store=True, no_cache=True, must_revalidate=True))
def login(self):
callbackurl = f'{self.callback_base_url}/oauth2/callback'
return_path = request.args.get("dataset", "")
return_to = f"{self.callback_base_url}/{return_path}"
# save the return path in the session cookie, accessed in the callback function
session["oauth_callback_redirect"] = return_to
response = self.client.authorize_redirect(redirect_uri=callbackurl)
self.update_response(response)
return response
def logout(self):
if self.session_cookie:
if self.CXG_ID_TOKEN in session:
del session[self.CXG_ID_TOKEN]
else:
@after_this_request
def remove_cookie(response):
response.set_cookie(self.cookie_params["key"], "", expires=0)
self.update_response(response)
return response
params = {'returnTo' : self.callback_base_url, 'client_id' : self.client_id}
response = redirect(self.client.api_base_url + '/v2/logout?' + urlencode(params))
self.update_response(response)
return response
def callback(self):
token = self.client.authorize_access_token()
id_token = token.get("id_token")
oauth_callback_redirect = session.pop("oauth_callback_redirect", "/")
resp = redirect(oauth_callback_redirect)
if self.session_cookie:
session[self.CXG_ID_TOKEN] = id_token
else:
args = self.cookie_params.copy()
del args["key"]
try:
resp.set_cookie(
self.cookie_params["key"],
id_token,
**args)
g.token = id_token
except Exception as e:
raise AuthenticationError(f"unable to set_cookie {self.cookie_params}") from e
self.update_response(resp)
return resp
def get_login_url(self, data_adaptor):
"""Return the url for the login route"""
if current_app.app_config.is_multi_dataset():
return f"/login?dataset={data_adaptor.uri_path}"
else:
return "/login"
def get_logout_url(self, data_adaptor):
"""Return the url for the logout route"""
return "/logout"
def get_token(self):
"""Function to return the token"""
if "token" in g:
return g.token
if self.session_cookie:
g.token = session.get(self.CXG_ID_TOKEN)
else:
g.token = request.cookies.get(self.cookie_params["key"])
return g.token
def get_jwt_payload(self):
if not has_request_context():
return None
token = self.get_token()
if token is None:
return None
unverified_header = jwt.get_unverified_header(token)
rsa_key = {}
for key in self.jwks['keys']:
if key['kid'] == unverified_header['kid']:
rsa_key = {
'kty': key['kty'],
'kid': key['kid'],
'use': key['use'],
'n': key['n'],
'e': key['e']
}
if rsa_key:
try:
payload = jwt.decode(
token,
rsa_key,
algorithms=self.algorithms,
audience=self.audience,
issuer=self.api_base_url + "/"
)
return payload
except jwt.JWTError as e:
raise AuthenticationError(f"invalid signature: {str(e)}")
except jwt.ExpiredSignatureError as e:
raise AuthenticationError(f"token expired: {str(e)}")
except jwt.JWTClaimsError as e:
raise AuthenticationError(f"invalid claims {str(e)}")
raise AuthenticationError("Unable to find the appropriate key")
AuthTypeFactory.register("oauth", AuthTypeOAuth)
server/auth/auth_session.py
+39
View File
@@ -0,0 +1,39 @@
from server.auth.auth import AuthTypeBase, AuthTypeFactory
from flask import session
from uuid import uuid4
class AuthTypeSession(AuthTypeBase):
"""Session based authentication. The user is always logged. The user id is a random number
associated with the session. This is a good choice for desktop servers."""
# key in the session token for userid
CXGUID = "cxguid"
def __init__(self, app_config):
super().__init__()
def is_valid_authentication_type(self):
return True
def complete_setup(self, app):
pass
def is_user_authenticated(self):
# always authenticated
return True
def get_user_id(self):
if self.CXGUID not in session:
session[self.CXGUID] = uuid4().hex
session.permanent = True
return session[self.CXGUID]
def get_user_name(self):
return "anonymous"
def get_user_email(self):
return None
AuthTypeFactory.register("session", AuthTypeSession)
server/auth/auth_test.py
+72
View File
@@ -0,0 +1,72 @@
from server.auth.auth import AuthTypeClientBase, AuthTypeFactory
from flask import session, request, redirect, current_app
class AuthTypeTest(AuthTypeClientBase):
"""An authentication type for testing client based logins. When the login route is accessed
the user is automatically logged in with a default or configured username"""
# key in session token with userid and username
CXGUID = "cxguid_test"
CXGUNAME = "cxguname_test"
CXGUEMAIL = "cxguemail_test"
def __init__(self, app_config):
super().__init__()
self.user_name = "test_account"
self.user_id = "id0001"
self.user_email = "test_account@test.com"
def is_valid_authentication_type(self):
return True
def requires_client_login(self):
return True
def add_url_rules(self, app):
app.add_url_rule("/login", "login", self.login, methods=["GET"])
app.add_url_rule("/logout", "logout", self.logout, methods=["GET"])
def complete_setup(self, app):
pass
def is_user_authenticated(self):
return self.CXGUID in session
def get_user_id(self):
return session.get(self.CXGUID)
def get_user_name(self):
return session.get(self.CXGUNAME)
def get_user_email(self):
return session.get(self.CXGUEMAIL)
def login(self):
args = request.args
return_to = args.get("dataset", "/")
session[self.CXGUID] = args.get("userid", self.user_id)
session[self.CXGUNAME] = args.get("username", self.user_name)
return redirect(return_to)
def logout(self):
session.clear()
return_to = request.args.get("dataset", "/")
return redirect(return_to)
def get_login_url(self, data_adaptor):
"""Return the url for the login route"""
if current_app.app_config.is_multi_dataset():
return f"/login?dataset={data_adaptor.uri_path}"
else:
return "/login"
def get_logout_url(self, data_adaptor):
"""Return the url for the logout route"""
if current_app.app_config.is_multi_dataset():
return f"/logout?dataset={data_adaptor.uri_path}"
else:
return "/logout"
AuthTypeFactory.register("test", AuthTypeTest)
server/cli/launch.py
+35 -35
View File
@@ -1,19 +1,19 @@
import errno
import functools
import logging
from os import devnull
import sys
import webbrowser
from os import devnull
import click
from flask_compress import Compress
from flask_cors import CORS
from server.common.utils import sort_options
from server.common.errors import DatasetAccessError, ConfigurationError
from server.app.app import Server
from server.common.app_config import AppConfig
from server.common.default_config import default_config
from server.app.app import Server
from server.common.errors import DatasetAccessError, ConfigurationError
from server.common.utils.utils import sort_options
DEFAULT_CONFIG = AppConfig()
@@ -33,7 +33,7 @@ def annotation_args(func):
multiple=False,
metavar="<path>",
help="CSV file to initialize editing of existing annotations; will be altered in-place. "
"Incompatible with --annotations-dir.",
"Incompatible with --annotations-dir.",
)
@click.option(
"--annotations-dir",
@@ -42,7 +42,7 @@ def annotation_args(func):
multiple=False,
metavar="<directory path>",
help="Directory of where to save output annotations; filename will be specified in the application. "
"Incompatible with --annotations-file.",
"Incompatible with --annotations-file.",
)
@click.option(
"--experimental-annotations-ontology",
@@ -170,7 +170,7 @@ def server_args(func):
default=DEFAULT_CONFIG.server_config.app__debug,
show_default=True,
help="Run in debug mode. This is helpful for cellxgene developers, "
"or when you want more information about an error condition.",
"or when you want more information about an error condition.",
)
@click.option(
"--verbose",
@@ -203,7 +203,7 @@ def server_args(func):
multiple=True,
metavar="<text>",
help="Additional script files to include in HTML page. If not specified, "
"no additional script files will be included.",
"no additional script files will be included.",
show_default=False,
)
@functools.wraps(func)
@@ -223,7 +223,7 @@ def launch_args(func):
default=DEFAULT_CONFIG.server_config.multi_dataset__dataroot,
metavar="<data directory>",
help="Enable cellxgene to serve multiple files. Supply path (local directory or URL)"
" to folder containing H5AD and/or CXG datasets.",
" to folder containing H5AD and/or CXG datasets.",
hidden=True,
) # TODO, unhide when dataroot is supported)
@click.argument("datapath", required=False, metavar="<path to data file>")
@@ -307,32 +307,32 @@ class CliLaunchServer(Server):
)
@launch_args
def launch(
datapath,
dataroot,
verbose,
debug,
open_browser,
port,
host,
embedding,
obs_names,
var_names,
max_category_items,
disable_custom_colors,
diffexp_lfc_cutoff,
title,
scripts,
about,
disable_annotations,
annotations_file,
annotations_dir,
backed,
disable_diffexp,
experimental_annotations_ontology,
experimental_annotations_ontology_obo,
experimental_enable_reembedding,
config_file,
dump_default_config,
datapath,
dataroot,
verbose,
debug,
open_browser,
port,
host,
embedding,
obs_names,
var_names,
max_category_items,
disable_custom_colors,
diffexp_lfc_cutoff,
title,
scripts,
about,
disable_annotations,
annotations_file,
annotations_dir,
backed,
disable_diffexp,
experimental_annotations_ontology,
experimental_annotations_ontology_obo,
experimental_enable_reembedding,
config_file,
dump_default_config,
):
"""Launch the cellxgene data viewer.
This web app lets you explore single-cell expression data.
server/cli/prepare.py
+14 -14
View File
@@ -5,7 +5,7 @@ import pandas as pd
from numpy import ndarray, unique
from scipy.sparse.csc import csc_matrix
from server.common.utils import sort_options
from server.common.utils.utils import sort_options
@sort_options
@@ -37,7 +37,7 @@ from server.common.utils import sort_options
default=False,
is_flag=True,
help="Do not run quality control metrics. By default cellxgene runs them "
"(saved to adata.obs and adata.var; see scanpy.pp.calculate_qc_metrics for details).",
"(saved to adata.obs and adata.var; see scanpy.pp.calculate_qc_metrics for details).",
)
@click.option(
"--make-obs-names-unique/--no-make-obs-names-unique",
@@ -53,18 +53,18 @@ from server.common.utils import sort_options
)
@click.help_option("--help", "-h", help="Show this message and exit.")
def prepare(
data,
embedding,
recipe,
output,
plotting,
sparse,
overwrite,
set_obs_names,
set_var_names,
skip_qc,
make_obs_names_unique,
make_var_names_unique,
data,
embedding,
recipe,
output,
plotting,
sparse,
overwrite,
set_obs_names,
set_var_names,
skip_qc,
make_obs_names_unique,
make_var_names_unique,
):
"""
Preprocess data for use with cellxgene.
server/test/test_datasets/pbmc3k.cxg/X/__lock.tdb → server/common/annotations/__init__.py
View File
server/common/annotations/annotations.py
+78
View File
@@ -0,0 +1,78 @@
from abc import ABCMeta, abstractmethod
import fastobo
import fsspec
from server.common.errors import OntologyLoadFailure
from server.common.utils.type_conversion_utils import get_schema_type_hint_of_array
class Annotations(metaclass=ABCMeta):
""" baseclass for annotations, including ontologies"""
""" our default ontology is the PURL for the Cell Ontology.
See http://www.obofoundry.org/ontology/cl.html """
DefaultOnotology = "http://purl.obolibrary.org/obo/cl.obo"
def __init__(self):
self.ontology_data = None
def load_ontology(self, path):
"""Load and parse ontologies - currently support OBO files only."""
if path is None:
path = self.DefaultOnotology
try:
with fsspec.open(path) as f:
obo = fastobo.iter(f)
terms = filter(lambda stanza: type(stanza) is fastobo.term.TermFrame, obo)
names = [tag.name for term in terms for tag in term if type(tag) is fastobo.term.NameClause]
self.ontology_data = names
except FileNotFoundError as e:
raise OntologyLoadFailure("Unable to find OBO ontology path") from e
except SyntaxError as e:
raise OntologyLoadFailure("Syntax error loading OBO ontology") from e
except Exception as e:
raise OntologyLoadFailure("Error loading OBO file") from e
def get_schema(self, data_adaptor):
schema = []
labels = self.read_labels(data_adaptor)
if labels is not None and not labels.empty:
for col in labels.columns:
col_schema = dict(name=col, writable=True)
col_schema.update(get_schema_type_hint_of_array(labels[col]))
schema.append(col_schema)
return schema
@abstractmethod
def set_collection(self, name):
"""set or create a new annotation collection"""
pass
@abstractmethod
def read_labels(self, data_adaptor):
"""Return the labels as a pandas.DataFrame"""
pass
@abstractmethod
def write_labels(self, df, data_adaptor):
"""Write the labels (df) to a persistent storage such that it can later be read"""
pass
def update_parameters(self, parameters, data_adaptor):
"""Update configuration parameters that describe information about the annotations feature"""
params = {}
params["annotations"] = True
if self.ontology_data:
params["annotations_cell_ontology_enabled"] = True
params["annotations_cell_ontology_terms"] = self.ontology_data
else:
params["annotations_cell_ontology_enabled"] = False
parameters.update(params)

Some files were not shown because too many files have changed in this diff Show More