bump version

* bump version to 0.11.0

* manually fix version number

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.10.1
+current_version = 0.11.1
 
 [bumpversion:file:setup.py]
 search = version="{current_version}"
@@ -13,3 +13,7 @@ replace = version="{new_version}"
 search = "version": "{current_version}"
 replace = "version": "{new_version}"
 
+[bumpversion:file:server/__init__.py]
+search = "__version__ = "{current_version}"
+replace = "__version__ = "{new_version}"
+

CONTRIBUTING.md

@@ -0,0 +1,49 @@
+# contributing to cellxgene
+
+We warmly welcome contributions from the community! Please submit any bug reports and feature requests through [Github issues](https://github.com/chanzuckerberg/cellxgene/issues). Please submit any direct contributions by forking the repository, creating a branch, and submitting a Pull Request. It'd be great for PRs to include test cases and documentation updates where relevant, though we know the core test suite is itself still a work in progress.
+
+All code contributions and dependencies must be compatible with the project's [open-source license (MIT)](LICENSE.txt).
+
+This project adheres to the Contributor Covenant
+[code of conduct](https://github.com/chanzuckerberg/.github/tree/master/CODE_OF_CONDUCT.md).
+By participating, you are expected to uphold this code. Please report unacceptable behavior
+to opensource@chanzuckerberg.com.
+
+And finally, if you have any questions about any of this stuff, just ask! :)
+
+## developer guide
+
+This project has made a few key design choices
+
+- The front-end is built with [`regl`](https://github.com/regl-project/regl) (a webgl library), [`react`](https://reactjs.org/), [`redux`](https://redux.js.org/), [`d3`](https://github.com/d3/d3), and [`blueprint`](https://blueprintjs.com/docs/#core) to handle rendering large numbers of cells with lots of complex interactivity
+- The app is designed with a client-server model that can support a range of existing analysis packages for backend computational tasks (currently built for [scanpy](https://github.com/theislab/scanpy))
+- The client uses fast cross-filtering to handle selections and comparisons across subsets of data
+
+Depending on your background and interests, you might want to contribute to the frontend, or backend, or both!
+
+If you are interested in working on `cellxgene` development, we recommend cloning the project from Gitub. First you'll need the following installed on your machine
+
+- python 3.6+
+- node and npm (we recommend using [nvm](https://github.com/creationix/nvm) if this is your first time with node)
+
+Then clone the project
+
+```
+git clone https://github.com/chanzuckerberg/cellxgene.git
+```
+
+Build the client web assets by calling `make` from inside the `cellxgene` folder
+
+```
+make
+```
+
+Install all requirements (we recommend doing this inside a virtual environment)
+
+```
+pip install -e .
+```
+
+You can start the app while developing either by calling `cellxgene` or by calling `python -m server`. We recommend using the `--debug` flag to see more output, which you can include when reporting bugs.
+
+If you have any questions about developing or contributing, come hang out with us by joining the [CZI Science Slack](https://join-cellxgene-users.herokuapp.com/) and posting in the `#cellxgene-dev` channel.

README.md

@@ -10,7 +10,7 @@ _cellxgene_ (pronounced "sell-by-jean") is an interactive data explorer for sing
 
 - Want to install and use cellxgene? Visit the [cellxgene docs](https://chanzuckerberg.github.io/cellxgene/).
 - Want to see where we are going? Check out [our roadmap](ROADMAP.md).
-- Want to contribute? See our [contributors guide](#Contributing)
+- Want to contribute? See our [contributors guide](CONTRIBUTING.md).
 
 ## quick start
 
@@ -32,11 +32,17 @@ Launch _cellxgene_
 cellxgene launch pbmc3k.h5ad --open
 ```
 
-To learn more about what you can do with _cellxgene_, see the [Getting Started](https://chanzuckerberg.github.io/cellxgene/getting-stared/) guide.
+To learn more about what you can do with _cellxgene_, see the [Getting Started](https://chanzuckerberg.github.io/cellxgene/getting-started.html) guide.
 
 ## get in touch
 
-Have questions, suggestions, or comments? You can come hang out with us by joining the [CZI Science Slack](https://join-cellxgene-users.herokuapp.com/) and posting in the `#cellxgene-users` channel. As mentioned above, please submit any feature requests or bugs as [Github issues](https://github.com/chanzuckerberg/cellxgene/issues). We'd love to hear from you!
+Have questions, suggestions, or comments? You can come hang out with us by joining the [CZI Science Slack](https://join-cellxgene-users.herokuapp.com/) and posting in the `#cellxgene-users` channel. Have feature requests or bugs? Please submit these as [Github issues](https://github.com/chanzuckerberg/cellxgene/issues). We'd love to hear from you!
+
+## contributing
+
+We warmly welcome contributions from the community! Please see our [contributing guide](CONTRIBUTING.md) and don't hesitate to open an issue or send a pull request to improve cellxgene.
+
+This project adheres to the Contributor Covenant [code of conduct](https://github.com/chanzuckerberg/.github/blob/master/CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to opensource@chanzuckerberg.com.
 
 ## where we are going
 
@@ -52,46 +58,19 @@ There are 4 key features we plan to implement in the near term.
 
 For more detail on these features and where we are going, see [our roadmap](ROADMAP.md).
 
-## contributing
+## risks of hosting cellxgene
 
-We warmly welcome contributions from the community! Please submit any bug reports and feature requests through [Github issues](https://github.com/chanzuckerberg/cellxgene/issues). Please submit any direct contributions by forking the repository, creating a branch, and submitting a Pull Request. It'd be great for PRs to include test cases and documentation updates where relevant, though we know the core test suite is itself still a work in progress. And all code contributions and dependencies must be compatible with the project's open-source license (MIT). If you have any questions about this stuff, just ask!
+_cellxgene_ is built on standard web technologies, but is currently designed as a single-user desktop application.
+We've done this so we can prioritize the features on [our roadmap](ROADMAP.md).
 
-### developer guide
+Some of our users have experimented with hosting _cellxgene_ either for their lab or for public use, but please note that the _cellxgene_ team does not officially support, troubleshoot, or maintain any web deployments at this time.
 
-This project has made a few key design choices
+If do you choose setup _cellxgene_ as a hosted service, you should be aware of the following risks:
 
-- The front-end is built with [`regl`](https://github.com/regl-project/regl) (a webgl library), [`react`](https://reactjs.org/), [`redux`](https://redux.js.org/), [`d3`](https://github.com/d3/d3), and [`blueprint`](https://blueprintjs.com/docs/#core) to handle rendering large numbers of cells with lots of complex interactivity
-- The app is designed with a client-server model that can support a range of existing analysis packages for backend computational tasks (currently built for [scanpy](https://github.com/theislab/scanpy))
-- The client uses fast cross-filtering to handle selections and comparisons across subsets of data
-
-Depending on your background and interests, you might want to contribute to the frontend, or backend, or both!
-
-If you are interested in working on `cellxgene` development, we recommend cloning the project from Gitub. First you'll need the following installed on your machine
-
-- python 3.6+
-- node and npm (we recommend using [nvm](https://github.com/creationix/nvm) if this is your first time with node)
-
-Then clone the project
-
-```
-git clone https://github.com/chanzuckerberg/cellxgene.git
-```
-
-Build the client web assets by calling `make` from inside the `cellxgene` folder
-
-```
-make
-```
-
-Install all requirements (we recommend doing this inside a virtual environment)
-
-```
-pip install -e .
-```
-
-You can start the app while developing either by calling `cellxgene` or by calling `python -m server`. We recommend using the `--debug` flag to see more output, which you can include when reporting bugs.
-
-If you have any questions about developing or contributing, come hang out with us by joining the [CZI Science Slack](https://join-cellxgene-users.herokuapp.com/) and posting in the `#cellxgene-dev` channel.
+- `$ cellxgene launch` uses Flask's development server, which is not recommended for hosted deployment (see the [Flask documentation](http://flask.pocoo.org/docs/1.0/tutorial/deploy/#run-with-a-production-server))
+- We have no testing or official support for deployments where multiple users are accessing the same _cellxgene_ instance.
+- Your _cellxgene_ instance is likely to hang or crash if too many people access it at the same time, especially if they using functions that call the Python backend (such as differential expression, updating the layout, or coloring by gene).
+- _cellxgene_ only supports one instance per dataset
 
 ## inspiration
 

ROADMAP.md

@@ -1,54 +1,46 @@
 # cellxgene roadmap
 
-We are very exited for _cellxgene_ to become a valuable tool in collaborations
-between computational biologists and experimental biologists working on
-single-cell transcriptomics data. _cellxgene_ is in active development, and we
-would love to include the community as we plan new features to work on. If you
-have questions of feedback about this roadmap, please submit an issue on
-GitHub.
-
+cellxgene makes it easier for biologists to collaboratively explore and understand their single-cell RNA-seq data. 
+In the near term, we are focused on continuing to enable fast, interactive exploration of single-cell data, supporting collaborative workflows in single-cell analysis, and improving user support. 
+If you have questions or feedback about this roadmap, please submit an issue on GitHub.
 Please note: this roadmap is subject to change.
 
-*Last updated: April 11, 2019*
+Last updated: June 25, 2019
 
-## what we are building now
+## Fast, interactive exploration of single-cell data
 
-In the near term, our goal is to enable teams of computational and experimental
-biologists to collaboratively explore and annotate their single-cell RNA-seq data.
+### Exposing Relationships Between Metadata and Data
+Biologists need to understand how variables (stored in metadata) are associated with one another and how they relate to changes in gene expression. 
+Building upon visualization features that reveal categorical metadata relationships (cluster occupancy) and gene expression relationships (scatterplot), we plan to add exploratory visualization components that enable investigation of relationships between metadata and gene expression. 
+See [issue #616](https://github.com/chanzuckerberg/cellxgene/issues/616) for more details.
 
-There are 4 key features we plan to implement in the near term.
+### Contextualizing Genes
+While exploring a transcriptomics dataset, scientists need to understand the biological context of genes.
+This context may be provided by user-defined gene metadata or publicly available gene databases. 
+We plan to support augmenting gene names with additional information that is useful to biologists. 
+See [issue #96](https://github.com/chanzuckerberg/cellxgene/issues/96) for more detail.
 
-- Click install and launch
-- Manual annotation workflows
-- Toggle embeddings
-- Gene information
+## Support collaborative workflows in single-cell analysis
 
-### simple install and launch
+### Manual Annotations
+cellxgene offers exploratory visualizations that are critical for manual annotation workflows, especially in collaborative environments. 
+We plan to support manually annotating cells with labels (i.e., cell type or QC flags), and their easy export for downstream analysis. 
+See [issue #524](https://github.com/chanzuckerberg/cellxgene/issues/524) for more details.
 
-The command line interface for installing and launching cellxgene is a barrier
-for users who are not used to Python or using the command line. We plan to
-support installation and launch of cellxgene on Mac and Windows. See
-[Issue #687](https://github.com/chanzuckerberg/cellxgene/issues/687) for more details.
+### Simple Click to Launch
 
-### manual annotation workflows
+Many biologists prefer not to interact with the command line and need an OS-native experience when using cellxgene. 
+We plan to implement a point-and-click installation and launch experience so that users can easily load data into cellxgene. 
+See [issue #687](https://github.com/chanzuckerberg/cellxgene/issues/687) for details.
 
-The exploratory visualization that cellxgene offers is critical for manual
-annotation workflows, especially in collaborative environments. We plan to
-support manually annotate cells with labels (i.e., cell type or QC flags) for
-downstream analysis. See [Issue #524](https://github.com/chanzuckerberg/cellxgene/issues/524)
-for more details.
+### Python API
+For computational biologists, saving h5ad files then loading them into cellxgene is a point of friction.  
+We plan to support importing cellxgene as a Python package so that users can launch cellxgene directly from an interactive environment (such as Jupyter, IPython, or Spyder), and pass data to and from the cellxgene UI. 
 
-### toggle embeddings
+## Improving user support
 
-While a single dataset may have multiple embeddings calculated (tSNE, umap, in
-situ coordinates, trajectories, etc), cellxgene currently requires the user to select the
-embedding to use in the main layout at launch. We plan to support letting users
-toggle between any embedding present in a file from the cellxgene interface.
-See [Issue #594](https://github.com/chanzuckerberg/cellxgene/issues/594) for details.
-
-### gene information
-
-Differential expression returns only the names of genes, but no additional information
-about gene metadata, function, or known associations. We plan to help users learn
-more about genes they discover by exposing additional gene metadata. See
-[Issue #96](https://github.com/chanzuckerberg/cellxgene/issues/96) for details.
+### Improved documentation
+cellxgene has some specific expectations about how data is stored. 
+We want to ensure that new users can get started easily and learn how to use cellxgene with their own data. 
+We plan to improve documentation on getting started, installation, data, and contributing. 
+See [issue #533](https://github.com/chanzuckerberg/cellxgene/issues/533) for more details.

cellxgene-osx.spec

@@ -0,0 +1,41 @@
+# -*- mode: python ; coding: utf-8 -*-
+
+block_cipher = None
+
+
+a = Analysis(['server/gui/main.py'],
+             pathex=['/Users/charlotteweaver/Documents/Git/cellxgene'],
+             binaries=[('/System/Library/Frameworks/Tk.framework/Tk', 'tk'), ('/System/Library/Frameworks/Tcl.framework/Tcl', 'tcl')],
+             datas=[('server/app/web/templates/', 'server/app/web/templates/'), ('server/app/web/static/', 'server/app/web/static/')],
+             hiddenimports=['sklearn', 'sklearn.utils._cython_blas', 'sklearn.neighbors.typedefs', 'sklearn.neighbors.quad_tree', 'sklearn.tree', 'sklearn.tree._utils'],
+             hookspath=['server/gui/'],
+             runtime_hooks=[],
+             excludes=[],
+             win_no_prefer_redirects=False,
+             win_private_assemblies=False,
+             cipher=block_cipher,
+             noarchive=False)
+pyz = PYZ(a.pure, a.zipped_data,
+             cipher=block_cipher)
+exe = EXE(pyz,
+          a.scripts,
+          [],
+          exclude_binaries=True,
+          name='cellxgene',
+          debug=False,
+          bootloader_ignore_signals=False,
+          strip=False,
+          upx=True,
+          console=False , icon='server/gui/images/cxg_icons.icns')
+coll = COLLECT(exe,
+               a.binaries,
+               a.zipfiles,
+               a.datas,
+               strip=False,
+               upx=True,
+               upx_exclude=[],
+               name='cellxgene')
+app = BUNDLE(coll,
+             name='cellxgene.app',
+             icon='server/gui/images/cxg_icons.icns',
+             bundle_identifier=None)

cellxgene-windows.spec

@@ -0,0 +1,36 @@
+# -*- mode: python -*-
+
+block_cipher = None
+
+
+a = Analysis(['server\\gui\\main.py'],
+             pathex=['C:\\Users\\Charlotte\\Documents\\git\\cellxgene'],
+             binaries=[],
+             datas=[('server/app/web/templates/', 'server/app/web/templates'), ('server/app/web/static/', 'server/app/web/static')],
+             hiddenimports=[],
+             hookspath=['server/gui/'],
+             runtime_hooks=[],
+             excludes=[],
+             win_no_prefer_redirects=False,
+             win_private_assemblies=False,
+             cipher=block_cipher,
+             noarchive=False)
+pyz = PYZ(a.pure, a.zipped_data,
+             cipher=block_cipher)
+exe = EXE(pyz,
+          a.scripts,
+          [],
+          exclude_binaries=True,
+          name='cellxgene',
+          debug=False,
+          bootloader_ignore_signals=False,
+          strip=False,
+          upx=True,
+          console=False , icon='server\\gui\\images\\icon.ico')
+coll = COLLECT(exe,
+               a.binaries,
+               a.zipfiles,
+               a.datas,
+               strip=False,
+               upx=True,
+               name='cellxgene')

client/__tests__/e2e/data.js

@@ -27,7 +27,7 @@ export const datasets = {
       lasso: [
         {
           "coordinates-as-percent": { x1: 0.05, y1: 0.25, x2: 0.15, y2: 0.35 },
-          count: "101"
+          count: "71"
         }
       ],
       categorical: [
@@ -92,7 +92,7 @@ export const datasets = {
       },
       lasso: {
         "coordinates-as-percent": { x1: 0.45, y1: 0.05, x2: 0.65, y2: 0.15 },
-        count: "46"
+        count: "36"
       }
     },
     scatter: {

client/__tests__/e2e/e2e.test.js

@@ -54,14 +54,14 @@ afterAll(() => {
   }
 });
 
-describe("did launch", async () => {
+describe("did launch", () => {
   test("page launched", async () => {
     let el = await utils.getOneElementInnerHTML("[data-testid='header']");
     expect(el).toBe(data.title);
   });
 });
 
-describe("metadata loads", async () => {
+describe("metadata loads", () => {
   test("categories and values from dataset appear", async () => {
     for (const label in data.categorical) {
       await utils.waitByID(`category-${label}`);
@@ -87,7 +87,7 @@ describe("metadata loads", async () => {
   });
 });
 
-describe("cell selection", async () => {
+describe("cell selection", () => {
   test("selects all cells cellset 1", async () => {
     const cellCount = await cxgActions.cellSet(1);
     expect(cellCount).toBe(data.dataframe.nObs);
@@ -138,7 +138,7 @@ describe("cell selection", async () => {
   });
 });
 
-describe("gene entry", async () => {
+describe("gene entry", () => {
   test("search for single gene", async () => {
     // blueprint's  typeahead is treating typing weird, clicking & waiting first solves this
     await utils.typeInto("gene-search", data.genes.search);
@@ -154,14 +154,17 @@ describe("gene entry", async () => {
     await utils.clickOn("section-bulk-add");
     await utils.typeInto("input-bulk-add", testGenes.join(","));
     await page.keyboard.press("Enter");
-    const userGeneHist = await cxgActions.getAllHistograms(
-      "histogram-user-gene"
+
+    const allHistograms = await cxgActions.getAllHistograms(
+      "histogram-user-gene",
+      testGenes
     );
-    expect(userGeneHist).toEqual(expect.arrayContaining(testGenes));
+    expect(allHistograms).toEqual(expect.arrayContaining(testGenes));
+    expect(allHistograms.length).toEqual(testGenes.length);
   });
 });
 
-describe("diffexp", async () => {
+describe("diffexp", () => {
   test("selects cells, saves them and performs diffexp", async () => {
     for (const select of data.diffexp.cellset1) {
       if (select.kind === "categorical") {
@@ -176,14 +179,18 @@ describe("diffexp", async () => {
     }
     await cxgActions.cellSet(2);
     await utils.clickOn("diffexp-button");
-    const diffExpHists = await cxgActions.getAllHistograms("histogram-diffexp");
-    expect(diffExpHists).toEqual(
+    const allHistograms = await cxgActions.getAllHistograms(
+      "histogram-diffexp",
+      data.diffexp["gene-results"]
+    );
+    expect(allHistograms).toEqual(
       expect.arrayContaining(data.diffexp["gene-results"])
     );
+    expect(allHistograms.length).toEqual(data.diffexp["gene-results"].length);
   });
 });
 
-describe("subset/reset", async () => {
+describe("subset/reset", () => {
   test("subset - cell count matches", async () => {
     for (const select of data.subset.cellset1) {
       if (select.kind === "categorical") {
@@ -257,7 +264,7 @@ describe("subset/reset", async () => {
   });
 });
 
-describe("scatter plot", async () => {
+describe("scatter plot", () => {
   test("scatter plot appears", async () => {
     await cxgActions.reset();
     const testGenes = data.scatter.genes;
@@ -270,7 +277,7 @@ describe("scatter plot", async () => {
   });
 });
 
-describe("clipping", async () => {
+describe("clipping", () => {
   test("clip continuous", async () => {
     await cxgActions.clip(data.clip.min, data.clip.max);
     const histId = `histogram-${data.clip.metadata}-plot-brush`;
@@ -300,7 +307,7 @@ describe("clipping", async () => {
 });
 
 // interact with UI elements just that they do not break
-describe("ui elements don't error", async () => {
+describe("ui elements don't error", () => {
   test("color by", async () => {
     for (const label in data.categorical) {
       await utils.clickOn(`colorby-${label}`);

client/__tests__/e2e/puppeteerUtils.js

@@ -13,6 +13,26 @@ export const puppeteerUtils = puppeteerPage => ({
     );
   },
 
+  async waitForAllByIds(testids, props = {}) {
+    await Promise.all(
+      testids.map(testid =>
+        puppeteerPage.waitForSelector(`[data-testid='${testid}']`)
+      )
+    );
+  },
+
+  async getAllByClass(testclass, props = {}) {
+    const elements = await puppeteerPage.$$eval(
+      `[data-testclass=${testclass}]`,
+      els => {
+        return els.map(el => {
+          return el.dataset.testid;
+        });
+      }
+    );
+    return elements;
+  },
+
   async typeInto(testid, text) {
     // only works for text without special characters
     await this.waitByID(testid);
@@ -32,8 +52,8 @@ export const puppeteerUtils = puppeteerPage => ({
     await puppeteerPage.waitFor(200);
     // select all
 
-    await puppeteerPage.click(selector, {clickCount: 3})
-    await puppeteerPage.keyboard.type("Backspace")
+    await puppeteerPage.click(selector, { clickCount: 3 });
+    await puppeteerPage.keyboard.type("Backspace");
     await puppeteerPage.type(selector, text);
   },
 
@@ -77,20 +97,16 @@ export const cellxgeneActions = puppeteerPage => ({
     await puppeteerPage.mouse.up();
   },
 
-  async getAllHistograms(testclass) {
-    await puppeteerUtils(puppeteerPage).waitByClass(testclass);
-    const histograms = await puppeteerPage.$$eval(
-      `[data-testclass=${testclass}]`,
-      els => {
-        return els.map(el => {
-          return el.dataset.testid.substring(
-            "histogram_".length,
-            el.dataset.testid.length
-          );
-        });
-      }
+  async getAllHistograms(testclass, testids) {
+    const histTestIds = testids.map(tid => `histogram-${tid}`);
+    // these load asynchronously, so we need to wait for each histogram individually
+    await puppeteerUtils(puppeteerPage).waitForAllByIds(histTestIds);
+    const allHistograms = await puppeteerUtils(puppeteerPage).getAllByClass(
+      testclass
+    );
+    return allHistograms.map(hist =>
+      hist.substr("histogram_".length, hist.length)
     );
-    return histograms;
   },
 
   async getAllCategoriesAndCounts(category) {
@@ -180,11 +196,16 @@ export const cellxgeneActions = puppeteerPage => ({
     await page.waitFor(200);
   },
 
-  async clip(min = 0, max = 100) { 
+  async clip(min = 0, max = 100) {
     await puppeteerUtils(puppeteerPage).clickOn("visualization-settings");
-    await puppeteerUtils(puppeteerPage).clearInputAndTypeInto("clip-min-input", min);
-    await puppeteerUtils(puppeteerPage).clearInputAndTypeInto("clip-max-input", max);
+    await puppeteerUtils(puppeteerPage).clearInputAndTypeInto(
+      "clip-min-input",
+      min
+    );
+    await puppeteerUtils(puppeteerPage).clearInputAndTypeInto(
+      "clip-max-input",
+      max
+    );
     await puppeteerUtils(puppeteerPage).clickOn("clip-commit");
   }
-
 });

client/__tests__/util/dataframe/histogram.test.js

@@ -0,0 +1,69 @@
+import * as Dataframe from "../../../src/util/dataframe";
+
+describe("Dataframe column histogram", () => {
+  test("categorical by categorical", () => {
+    const df = new Dataframe.Dataframe(
+      [3, 3],
+      [["n1", "n2", "n3"], ["c1", "c2", "c3"], new Int32Array([0, 1, 2])],
+      null,
+      new Dataframe.KeyIndex(["name", "cat", "value"])
+    );
+
+    const h1 = df.col("cat").histogram(df.col("name"));
+    expect(h1).toMatchObject(
+      new Map([
+        ["n1", new Map([["c1", 1]])],
+        ["n2", new Map([["c2", 1]])],
+        ["n3", new Map([["c3", 1]])]
+      ])
+    );
+    // memoized?
+    expect(df.col("cat").histogram(df.col("name"))).toMatchObject(h1);
+  });
+
+  test("continuous by categorical", () => {
+    const df = new Dataframe.Dataframe(
+      [3, 3],
+      [["n1", "n2", "n3"], ["c1", "c2", "c3"], new Int32Array([0, 1, 2])],
+      null,
+      new Dataframe.KeyIndex(["name", "cat", "value"])
+    );
+
+    const h1 = df.col("value").histogram(3, [0, 2], df.col("name"));
+    expect(h1).toMatchObject(
+      new Map([["n1", [1, 0, 0]], ["n2", [0, 1, 0]], ["n3", [0, 0, 1]]])
+    );
+    // memoized?
+    expect(df.col("value").histogram(3, [0, 2], df.col("name"))).toMatchObject(
+      h1
+    );
+  });
+
+  test("categorical", () => {
+    const df = new Dataframe.Dataframe(
+      [3, 3],
+      [["n1", "n2", "n3"], ["c1", "c2", "c3"], new Int32Array([0, 1, 2])],
+      null,
+      new Dataframe.KeyIndex(["name", "cat", "value"])
+    );
+
+    const h1 = df.col("cat").histogram();
+    expect(h1).toMatchObject(new Map([["c1", 1], ["c2", 1], ["c3", 1]]));
+    // memoized?
+    expect(df.col("value").histogram(3, [0, 2])).toMatchObject(h1);
+  });
+
+  test("continuous", () => {
+    const df = new Dataframe.Dataframe(
+      [3, 3],
+      [["n1", "n2", "n3"], ["c1", "c2", "c3"], new Int32Array([0, 1, 2])],
+      null,
+      new Dataframe.KeyIndex(["name", "cat", "value"])
+    );
+
+    const h1 = df.col("value").histogram(3, [0, 2]);
+    expect(h1).toMatchObject([1, 1, 1]);
+    // memoized?
+    expect(df.col("value").histogram(3, [0, 2])).toMatchObject(h1);
+  });
+});

client/index.html

@@ -3,7 +3,7 @@
   <head>
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width, initial-scale=1">
-    <title>cellxgene</title>
+    <title>cell&times;gene</title>
     <link href="https://fonts.googleapis.com/css?family=Roboto+Condensed:400,400i,700" rel="stylesheet">
     <style>
       html, body, p, h1, h2, h3, h4, h5, h6, span, button, input, label, text, div {

client/index_template.html

@@ -3,7 +3,7 @@
 <head>
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width, initial-scale=1">
-    <title>cellxgene</title>
+    <title>cell&times;gene</title>
     <link href="https://fonts.googleapis.com/css?family=Roboto+Condensed:400,400i,700" rel="stylesheet">
     <style>
         html, body, p, h1, h2, h3, h4, h5, h6, span, button, input, label, text, div {

client/package-lock.json

@@ -1,6 +1,6 @@
 {
   "name": "cellxgene",
-  "version": "0.10.1",
+  "version": "0.11.1",
   "lockfileVersion": 1,
   "requires": true,
   "dependencies": {
@@ -11848,9 +11848,9 @@
       }
     },
     "lodash": {
-      "version": "4.17.11",
-      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.11.tgz",
-      "integrity": "sha512-cQKh8igo5QUhZ7lg38DYWAxMvjSAKG0A8wGSVimP07SIUEK2UO+arSRKbRZWtelMtN5V0Hkwh5ryOto/SshYIg=="
+      "version": "4.17.14",
+      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.14.tgz",
+      "integrity": "sha512-mmKYbW3GLuJeX+iGP+Y7Gp1AiGHGbXHCOh/jZmrawMmsE7MS4znI3RL2FsjbqOyMayHInjOeykW7PEajUk1/xw=="
     },
     "lodash._reinterpolate": {
       "version": "3.0.0",

client/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cellxgene",
-  "version": "0.10.1",
+  "version": "0.11.1",
   "license": "MIT",
   "description": "cellxgene is a web application for the interactive exploration of single cell sequence data.",
   "repository": "https://github.com/chanzuckerberg/cellxgene",
@@ -41,11 +41,11 @@
     "font-color-contrast": "^1.0.3",
     "fuzzysort": "^1.1.4",
     "gl-mat4": "^1.1.4",
-    "gl-vec3": "^1.1.3",
     "gl-matrix": "^3.0.0",
+    "gl-vec3": "^1.1.3",
     "is-number": "^7.0.0",
     "key-pressed": "0.0.1",
-    "lodash": "^4.17.4",
+    "lodash": "^4.17.14",
     "memoize-one": "^5.0.4",
     "mouse-position": "^2.0.1",
     "mouse-pressed": "^1.0.0",

client/src/components/app.js

@@ -4,9 +4,11 @@ import Helmet from "react-helmet";
 import { connect } from "react-redux";
 
 import Container from "./framework/container";
-import LeftSideBar from "./leftsidebar";
+import LeftSideBar from "./leftSidebar";
 import Legend from "./continuousLegend";
 import Graph from "./graph/graph";
+import MenuBar from "./menubar";
+
 import actions from "../actions";
 
 @connect(state => ({
@@ -71,18 +73,23 @@ class App extends React.Component {
             loading cellxgene
           </div>
         ) : null}
-        <div>
-          {loading ? null : <LeftSideBar />}
+        {error ? (
           <div
             style={{
-              padding: 15,
-              width: 1440 - 410 /* but responsive */,
-              marginLeft: 350 /* but responsive */
+              position: "fixed",
+              fontWeight: 500,
+              top: window.innerHeight / 2,
+              left: window.innerWidth / 2 - 50
             }}
           >
-            {loading ? null : <Graph key={graphRenderCounter} />}
-            <Legend />
+            error loading
           </div>
+        ) : null}
+        <div>
+          {loading ? null : <LeftSideBar />}
+          {loading ? null : <MenuBar />}
+          {loading ? null : <Graph key={graphRenderCounter} />}
+          <Legend />
         </div>
       </Container>
     );

client/src/components/categorical/categorical.css

@@ -0,0 +1,3 @@
+:local(.value):hover {
+  background: rgba(167, 182, 194, 0.3);
+}

client/src/components/categorical/categorical.js

@@ -19,13 +19,6 @@ class Categories extends React.Component {
           padding: globals.leftSidebarSectionPadding
         }}
       >
-        <p
-          style={Object.assign({}, globals.leftSidebarSectionHeading, {
-            marginTop: 4
-          })}
-        >
-          Categorical Metadata
-        </p>
         {_.map(categoricalSelection, (catState, catName) => (
           <Category key={catName} metadataField={catName} />
         ))}

client/src/components/categorical/occupancy.js

@@ -2,60 +2,191 @@
 import React from "react";
 import { connect } from "react-redux";
 import * as d3 from "d3";
+import {
+  Popover,
+  PopoverInteractionKind,
+  Position,
+  Classes
+} from "@blueprintjs/core";
 
 @connect()
 class Occupancy extends React.Component {
-  render() {
-    const { occupancy, colorScale, colorAccessor, schema, world } = this.props;
-    const width = 100;
-    const height = 11;
+  _WIDTH = 100;
 
-    const categories = schema.annotations.obsByName[colorAccessor]?.categories;
+  _HEIGHT = 11;
+
+  createHistogram = () => {
+    /* 
+      Knowing that colorScale is based off continous data, 
+      createHistogram fetches the continous data in relation to the cells releveant to the catagory value.
+      It then seperates that data into 50 bins for drawing the mini-histogram
+    */
+    const {
+      world,
+      metadataField,
+      colorAccessor,
+      category,
+      categoryIndex
+    } = this.props;
+
+    if (!this.canvas) return;
+
+    const groupBy = world.obsAnnotations.col(metadataField);
+
+    const col =
+      world.obsAnnotations.col(colorAccessor) ||
+      world.varData.col(colorAccessor);
+
+    const range = col.summarize();
+
+    const histogramMap = col.histogram(
+      50,
+      [range.min, range.max],
+      groupBy
+    ); /* Because the signature changes we really need different names for histogram to differentiate signatures  */
+
+    const bins = histogramMap.get(category.categoryValues[categoryIndex]);
+
+    const xScale = d3
+      .scaleLinear()
+      .domain([0, bins.length])
+      .range([0, this._WIDTH]);
+
+    const largestBin = Math.max(...bins);
+
+    const yScale = d3
+      .scaleLinear()
+      .domain([0, largestBin])
+      .range([0, this._HEIGHT]);
+
+    const ctx = this.canvas.getContext("2d");
+
+    ctx.fillStyle = "#000";
+
+    let x;
+    let y;
+
+    const rectWidth = this._WIDTH / bins.length;
+
+    for (let i = 0, { length } = bins; i < length; i += 1) {
+      x = xScale(i);
+      y = yScale(bins[i]);
+      ctx.fillRect(x, this._HEIGHT - y, rectWidth, y);
+    }
+  };
+
+  createOccupancyStack = () => {
+    /* 
+      Knowing that the color scale is based off of catagorical data, 
+      createOccupancyStack obtains a map showing the number if cells per colored value
+      Using the colorScale a stack of colored bars is drawn representing the map
+     */
+    const {
+      world,
+      metadataField,
+      colorAccessor,
+      category,
+      categoryIndex,
+      schema,
+      colorScale
+    } = this.props;
+
+    const ctx = this.canvas?.getContext("2d");
+
+    if (!ctx) return;
+
+    const groupBy = world.obsAnnotations.col(metadataField);
+    const occupancyMap = world.obsAnnotations
+      .col(colorAccessor)
+      .histogram(groupBy);
+
+    const occupancy = occupancyMap.get(category.categoryValues[categoryIndex]);
 
     const x = d3
       .scaleLinear()
       /* get all the keys d[1] as an array, then find the sum */
-      .domain([0, d3.sum(Array.from(occupancy, d => d[1]))])
-      .range([0, width]);
+      .domain([0, d3.sum(Array.from(occupancy.values()))])
+      .range([0, this._WIDTH]);
+    const categories = schema.annotations.obsByName[colorAccessor]?.categories;
 
     let currentOffset = 0;
     const dfColumn = world.obsAnnotations.col(colorAccessor);
     const categoryValues = dfColumn.summarize().categories;
-    const stacks = categoryValues.map(d => {
-      const o = occupancy.get(d);
 
-      const scaledValue = x(o);
+    let o;
+    let scaledValue;
+    let value;
 
-      const stackItem = {
-        key: d,
-        value: o || 0,
-        rectWidth: o ? scaledValue : 0,
-        offset: currentOffset,
-        fill: o ? colorScale(categories.indexOf(d)) : "rgb(255,255,255)"
-      };
+    for (let i = 0, { length } = categoryValues; i < length; i += 1) {
+      value = categoryValues[i];
+      o = occupancy.get(value);
+      scaledValue = x(o);
+      ctx.fillStyle = o
+        ? colorScale(categories.indexOf(value))
+        : "rgb(255,255,255)";
+      ctx.fillRect(currentOffset, 0, o ? scaledValue : 0, this._HEIGHT);
       currentOffset += o ? scaledValue : 0;
-      return stackItem;
-    });
+    }
+  };
+
+  render() {
+    const {
+      colorAccessor,
+      categoricalSelection,
+      category,
+      categoryIndex
+    } = this.props;
+
+    this.canvas?.getContext("2d").clearRect(0, 0, this._WIDTH, this._HEIGHT);
+
+    const colorByIsCatagoricalData = !!categoricalSelection[colorAccessor];
 
     return (
-      <svg
-        style={{
-          marginRight: 5,
-          width,
-          height
+      <Popover
+        interactionKind={PopoverInteractionKind.HOVER_TARGET_ONLY}
+        hoverOpenDelay={1500}
+        hoverCloseDelay={200}
+        position={Position.LEFT}
+        modifiers={{
+          preventOverflow: { enabled: false },
+          hide: { enabled: false }
         }}
+        lazy
+        usePortal
+        disabled={colorByIsCatagoricalData}
+        popoverClassName={Classes.POPOVER_CONTENT_SIZING}
       >
-        {stacks.map(d => (
-          <rect
-            key={d.key}
-            width={d.rectWidth}
-            height={height}
-            x={d.offset}
-            title={d.metadataField}
-            fill={d.fill}
-          />
-        ))}
-      </svg>
+        <canvas
+          className="bp3-popover-targer"
+          style={{
+            marginRight: 5,
+            width: this._WIDTH,
+            height: this._HEIGHT,
+            borderBottom: colorByIsCatagoricalData
+              ? ""
+              : "solid rgb(230, 230, 230) 0.25px"
+          }}
+          width={this._WIDTH}
+          height={this._HEIGHT}
+          ref={ref => {
+            this.canvas = ref;
+            if (colorByIsCatagoricalData) this.createOccupancyStack();
+            else this.createHistogram();
+          }}
+        />
+        <div key="text" style={{ fontFamily: "Roboto", fontSize: "14px" }}>
+          <p style={{ margin: "0" }}>
+            This histograms shows the distribution of{" "}
+            <strong>{colorAccessor}</strong> within{" "}
+            <strong>{category.categoryValues[categoryIndex]}</strong>.
+            <br />
+            <br />
+            The x axis is the same for each histogram, while the y axis is
+            scaled to the largest bin within this histogram instead of the
+            largest bin within the whole category.
+          </p>
+        </div>
+      </Popover>
     );
   }
 }

client/src/components/categorical/value.js

@@ -2,8 +2,8 @@
 import { connect } from "react-redux";
 import React from "react";
 import Occupancy from "./occupancy";
-import { countCategoryValues2D } from "../../util/stateManager/worldUtil";
 import * as globals from "../../globals";
+import styles from "./categorical.css";
 
 @connect(state => ({
   categoricalSelection: state.categoricalSelection,
@@ -13,23 +13,68 @@ import * as globals from "../../globals";
   world: state.world
 }))
 class CategoryValue extends React.Component {
-  toggleOff() {
+  toggleOff = () => {
     const { dispatch, metadataField, categoryIndex } = this.props;
     dispatch({
       type: "categorical metadata filter deselect",
       metadataField,
       categoryIndex
     });
-  }
+  };
 
-  toggleOn() {
+  shouldComponentUpdate = nextProps => {
+    /* 
+    Checks to see if at least one of the following changed:
+    * world state
+    * the color accessor (what is currently being colored by)
+    * if this catagorical value's selection status has changed
+    
+    If and only if true, update the component
+    */
+    const { props } = this;
+    const { metadataField, categoryIndex, categoricalSelection } = props;
+    const { categoricalSelection: newCategoricalSelection } = nextProps;
+
+    const valueSelectionChange =
+      categoricalSelection[metadataField].categoryValueSelected[
+        categoryIndex
+      ] !==
+      newCategoricalSelection[metadataField].categoryValueSelected[
+        categoryIndex
+      ];
+
+    const worldChange = props.world !== nextProps.world;
+    const colorAccessorChange = props.colorAccessor !== nextProps.colorAccessor;
+
+    return valueSelectionChange || worldChange || colorAccessorChange;
+  };
+
+  toggleOn = () => {
     const { dispatch, metadataField, categoryIndex } = this.props;
     dispatch({
       type: "categorical metadata filter select",
       metadataField,
       categoryIndex
     });
-  }
+  };
+
+  handleMouseEnter = () => {
+    const { dispatch, metadataField, categoryIndex } = this.props;
+    dispatch({
+      type: "category value mouse hover start",
+      metadataField,
+      categoryIndex
+    });
+  };
+
+  handleMouseExit = () => {
+    const { dispatch, metadataField, categoryIndex } = this.props;
+    dispatch({
+      type: "category value mouse hover end",
+      metadataField,
+      categoryIndex
+    });
+  };
 
   render() {
     const {
@@ -39,8 +84,7 @@ class CategoryValue extends React.Component {
       colorAccessor,
       colorScale,
       i,
-      schema,
-      world
+      schema
     } = this.props;
 
     if (!categoricalSelection) return null;
@@ -56,29 +100,26 @@ class CategoryValue extends React.Component {
     /* this is the color scale, so add swatches below */
     const isColorBy = metadataField === colorAccessor;
     let categories = null;
-    let occupancy = null;
 
     if (isColorBy && schema) {
       categories = schema.annotations.obsByName[colorAccessor]?.categories;
     }
 
-    if (colorAccessor && !isColorBy && categoricalSelection[colorAccessor]) {
-      occupancy = countCategoryValues2D(
-        metadataField,
-        colorAccessor,
-        world.obsAnnotations
-      );
-    }
-
     return (
       <div
         key={i}
+        className={styles.value}
+        data-testclass="categorical-row"
         style={{
+          padding: "4px 7px",
           display: "flex",
           alignItems: "baseline",
-          justifyContent: "space-between"
+          justifyContent: "space-between",
+          marginBottom: "2px",
+          borderRadius: "2px"
         }}
-        data-testclass="categorical-row"
+        onMouseEnter={this.handleMouseEnter}
+        onMouseLeave={this.handleMouseExit}
       >
         <div
           style={{
@@ -90,34 +131,32 @@ class CategoryValue extends React.Component {
             justifyContent: "space-between"
           }}
         >
-          <label className="bp3-control bp3-checkbox">
-            <input
-              onChange={
-                selected ? this.toggleOff.bind(this) : this.toggleOn.bind(this)
-              }
-              data-testclass="categorical-value-select"
-              data-testid={`categorical-value-select-${metadataField}-${displayString}`}
-              checked={selected}
-              type="checkbox"
-            />
-            <span className="bp3-control-indicator" />
+          <div style={{ display: "flex" }}>
+            <label className="bp3-control bp3-checkbox" style={{ margin: 0 }}>
+              <input
+                onChange={selected ? this.toggleOff : this.toggleOn}
+                data-testclass="categorical-value-select"
+                data-testid={`categorical-value-select-${metadataField}-${displayString}`}
+                checked={selected}
+                type="checkbox"
+              />
+              <span
+                className="bp3-control-indicator"
+                onMouseEnter={this.handleMouseExit}
+                onMouseLeave={this.handleMouseEnter}
+              />
+            </label>
             <span
               data-testid={`categorical-value-${metadataField}-${displayString}`}
               data-testclass="categorical-value"
+              style={{ wordBreak: "break-all" }}
             >
               {displayString}
             </span>
-          </label>
+          </div>
           <span style={{ flexShrink: 0 }}>
-            {colorAccessor &&
-            !isColorBy &&
-            categoricalSelection[colorAccessor] ? (
-              <Occupancy
-                occupancy={occupancy.get(
-                  category.categoryValues[categoryIndex]
-                )}
-                {...this.props}
-              />
+            {colorAccessor && !isColorBy ? (
+              <Occupancy category={category} {...this.props} />
             ) : null}
           </span>
         </div>

client/src/components/continuous/continuous.js

@@ -48,7 +48,7 @@ class Continuous extends React.Component {
     }
 
     /* initial value for iterator to simulate index, ranges is an object */
-    let zebra = -1;
+    let zebra = 0;
 
     return (
       <div>
@@ -71,7 +71,10 @@ class Continuous extends React.Component {
 
               const summary = obsAnnotations.col(key).summarize();
               const nonFiniteExtent =
-                summary.min === undefined || summary.max === undefined;
+                summary.min === undefined ||
+                summary.max === undefined ||
+                Number.isNaN(summary.min) ||
+                Number.isNaN(summary.max);
               if (!summary.categorical && !nonFiniteExtent) {
                 zebra += 1;
                 return (

client/src/components/continuousLegend/index.js

@@ -2,7 +2,7 @@
 import React from "react";
 import { connect } from "react-redux";
 import * as d3 from "d3";
-import { interpolateViridis, interpolateCool } from "d3-scale-chromatic";
+import { interpolateCool } from "d3-scale-chromatic";
 
 // create continuous color legend
 // http://bl.ocks.org/syntagmatic/e8ccca52559796be775553b467593a9f

client/src/components/framework/logo.js

@@ -4,7 +4,7 @@ import * as globals from "../../globals";
 const Logo = props => {
   const { size } = props;
   return (
-    <svg width={size} height={size} viewBox={`0 0 48 48`} fill="none">
+    <svg width={size} height={size} viewBox="0 0 48 48" fill="none">
       <rect width="48" height="48" fill="white" />
       <rect width="48" height="48" fill={globals.logoColor} />
       <rect x="19" y="19" width="22" height="22" fill="white" />

client/src/components/geneExpression/expressionButtons.js

@@ -1,101 +0,0 @@
-// jshint esversion: 6
-import React from "react";
-import _ from "lodash";
-import { Button, AnchorButton, Tooltip } from "@blueprintjs/core";
-import { connect } from "react-redux";
-import * as globals from "../../globals";
-import actions from "../../actions";
-import CellSetButton from "./cellSetButtons";
-
-@connect(state => ({
-  differential: state.differential,
-  world: state.world,
-  crossfilter: state.crossfilter
-}))
-class Expression extends React.Component {
-  constructor(props) {
-    super(props);
-    this.state = {};
-  }
-
-  computeDiffExp() {
-    const { dispatch, differential } = this.props;
-    if (differential.celllist1 && differential.celllist2) {
-      dispatch(
-        actions.requestDifferentialExpression(
-          differential.celllist1,
-          differential.celllist2
-        )
-      );
-    }
-  }
-
-  clearDifferentialExpression() {
-    const { dispatch, differential } = this.props;
-    dispatch({
-      type: "clear differential expression",
-      diffExp: differential.diffExp
-    });
-    dispatch({
-      type: "clear scatterplot"
-    });
-  }
-
-  render() {
-    const { differential } = this.props;
-    if (!differential) {
-      return null;
-    }
-    const haveBothCellSets =
-      !!differential.celllist1 && !!differential.celllist2;
-    return (
-      <div
-        style={{
-          marginRight: 10,
-          marginBottom: 10,
-          paddingLeft: globals.leftSidebarSectionPadding
-        }}
-      >
-        <CellSetButton {...this.props} eitherCellSetOneOrTwo={1} />
-        <CellSetButton {...this.props} eitherCellSetOneOrTwo={2} />
-        {!differential.diffExp ? (
-          <Tooltip
-            content="Add two cells selections, see the top 15 differentially expressed genes between them"
-            position="bottom"
-          >
-            <AnchorButton
-              style={{ marginTop: 10 }}
-              disabled={!haveBothCellSets}
-              intent="primary"
-              data-testid="diffexp-button"
-              loading={differential.loading}
-              fill
-              type="button"
-              onClick={this.computeDiffExp.bind(this)}
-            >
-              Compute Differential Expression
-            </AnchorButton>
-          </Tooltip>
-        ) : null}
-        {differential.diffExp ? (
-          <Tooltip
-            content="Remove differentially expressed gene list and clear cell selections"
-            position="bottom"
-          >
-            <Button
-              type="button"
-              fill
-              style={{ marginTop: 10 }}
-              intent="warning"
-              onClick={this.clearDifferentialExpression.bind(this)}
-            >
-              Clear Differential Expression
-            </Button>
-          </Tooltip>
-        ) : null}
-      </div>
-    );
-  }
-}
-
-export default Expression;

client/src/components/geneExpression/index.js

@@ -21,7 +21,6 @@ import {
   postUserErrorToast,
   keepAroundErrorToast
 } from "../framework/toasters";
-import ExpressionButtons from "./expressionButtons";
 
 const renderGene = (fuzzySortResult, { handleClick, modifiers, query }) => {
   if (!modifiers.matchesPredicate) {
@@ -180,19 +179,7 @@ class GeneExpression extends React.Component {
 
     return (
       <div>
-        <div
-          style={{
-            marginTop: 30
-          }}
-        >
-          <p
-            style={Object.assign({}, globals.leftSidebarSectionHeading, {
-              paddingLeft: globals.leftSidebarSectionPadding,
-              margin: 0
-            })}
-          >
-            Selected Genes
-          </p>
+        <div>
           <div
             style={{
               padding: globals.leftSidebarSectionPadding
@@ -208,7 +195,7 @@ class GeneExpression extends React.Component {
                 this.setState({ tab: "autosuggest" });
               }}
             >
-              Autosuggest
+              Autosuggest genes
             </Button>
             <Button
               active={tab === "bulkadd"}
@@ -260,7 +247,7 @@ class GeneExpression extends React.Component {
                 data-testid={"add-gene"}
                 loading={userDefinedGenesLoading}
               >
-                Add
+                Add gene
               </Button>
             </ControlGroup>
           ) : null}
@@ -291,7 +278,7 @@ class GeneExpression extends React.Component {
                       onClick={this.handleBulkAddClick.bind(this)}
                       loading={userDefinedGenesLoading}
                     >
-                      Add
+                      Add genes
                     </Button>
                   </ControlGroup>
                 </FormGroup>
@@ -318,15 +305,6 @@ class GeneExpression extends React.Component {
             : null}
         </div>
         <div>
-          <p
-            style={Object.assign({}, globals.leftSidebarSectionHeading, {
-              marginTop: 40,
-              paddingLeft: globals.leftSidebarSectionPadding
-            })}
-          >
-            Differentially Expressed Genes
-          </p>
-          <ExpressionButtons />
           {differential.diffExp
             ? _.map(differential.diffExp, (value, index) => {
                 const name = world.varAnnotations.at(value[0], varIndexName);

client/src/components/graph/graph.js

@@ -3,82 +3,32 @@ import React from "react";
 import * as d3 from "d3";
 import { connect } from "react-redux";
 import mat4 from "gl-mat4";
-import vec3 from "gl-vec3";
 import _regl from "regl";
 import memoize from "memoize-one";
-import {
-  Button,
-  AnchorButton,
-  Tooltip,
-  Popover,
-  Menu,
-  MenuItem,
-  Position,
-  NumericInput,
-  Icon,
-  RadioGroup,
-  Radio
-} from "@blueprintjs/core";
 
 import * as globals from "../../globals";
 import setupSVGandBrushElements from "./setupSVGandBrush";
+import setupCentroidSVG from "./setupCentroidSVG";
 import actions from "../../actions";
 import _camera from "../../util/camera";
 import _drawPoints from "./drawPointsRegl";
 import scaleLinear from "../../util/scaleLinear";
-import { World } from "../../util/stateManager";
 
 /* https://bl.ocks.org/mbostock/9078690 - quadtree for onClick / hover selections */
 
 @connect(state => ({
   world: state.world,
-  universe: state.universe,
   crossfilter: state.crossfilter,
-  clipPercentileMin: Math.round(100 * (state.world?.clipQuantiles?.min ?? 0)),
-  clipPercentileMax: Math.round(100 * (state.world?.clipQuantiles?.max ?? 1)),
   responsive: state.responsive,
   colorRGB: state.colors.rgb,
-  opacityForDeselectedCells: state.controls.opacityForDeselectedCells,
-  resettingInterface: state.controls.resettingInterface,
-  userDefinedGenes: state.controls.userDefinedGenes,
-  diffexpGenes: state.controls.diffexpGenes,
-  colorAccessor: state.colors.colorAccessor,
-  scatterplotXXaccessor: state.controls.scatterplotXXaccessor,
-  scatterplotYYaccessor: state.controls.scatterplotYYaccessor,
-  celllist1: state.differential.celllist1,
-  celllist2: state.differential.celllist2,
-  libraryVersions: state.config?.library_versions, // eslint-disable-line camelcase
-  undoDisabled: state["@@undoable/past"].length === 0,
-  redoDisabled: state["@@undoable/future"].length === 0,
   selectionTool: state.graphSelection.tool,
   currentSelection: state.graphSelection.selection,
-  layoutChoice: state.layoutChoice
+  layoutChoice: state.layoutChoice,
+  centroidLabel: state.centroidLabel,
+  graphInteractionMode: state.controls.graphInteractionMode,
+  colorAccessor: state.colors.colorAccessor
 }))
 class Graph extends React.Component {
-  static isValidDigitKeyEvent(e) {
-    /*
-    Return true if this event is necessary to enter a percent number input.
-    Return false if not.
-
-    Returns true for events with keys: backspace, control, alt, meta, [0-9],
-    or events that don't have a key.
-    */
-    if (e.key === null) return true;
-    if (e.ctrlKey || e.altKey || e.metaKey) return true;
-
-    // concept borrowed from blueprint's numericInputUtils:
-    // keys that print a single character when pressed have a `key` name of
-    // length 1. every other key has a longer `key` name (e.g. "Backspace",
-    // "ArrowUp", "Shift"). since none of those keys can print a character
-    // to the field--and since they may have important native behaviors
-    // beyond printing a character--we don't want to disable their effects.
-    const isSingleCharKey = e.key.length === 1;
-    if (!isSingleCharKey) return true;
-
-    const key = e.key.charCodeAt(0) - 48; /* "0" */
-    return key >= 0 && key <= 9;
-  }
-
   computePointPositions = memoize((X, Y, scaleX, scaleY) => {
     /*
     compute webgl coordinate buffer for each point
@@ -102,20 +52,45 @@ class Graph extends React.Component {
     return colors;
   });
 
-  computePointSizes = memoize((len, crossfilter) => {
-    /*
-    compute webgl dot size for each point
-    */
+  computePointSizesFromCrossfilter = memoize((len, crossfilter) => {
     const sizes = new Float32Array(len);
     crossfilter.fillByIsSelected(sizes, 4, 0.2);
+
     return sizes;
   });
 
+  computePointSizes = memoize(
+    (len, crossfilter, metadataField, categoryField) => {
+      /*
+    compute webgl dot size for each point
+    */
+
+      const selectionSizes = this.computePointSizesFromCrossfilter(
+        len,
+        crossfilter
+      );
+      let sizes;
+
+      if (metadataField && categoryField) {
+        sizes = selectionSizes.slice();
+        const valuesArr = crossfilter.data.col(metadataField).asArray();
+
+        for (let i = 0; i < len; i += 1) {
+          if (valuesArr[i] === categoryField) {
+            sizes[i] = 10;
+          }
+        }
+      } else {
+        sizes = selectionSizes;
+      }
+      return sizes;
+    }
+  );
+
   constructor(props) {
     super(props);
     this.count = 0;
     this.graphPaddingTop = 0;
-    this.graphPaddingBottom = 45;
     this.graphPaddingRight = globals.leftSidebarWidth;
     this.renderCache = {
       X: null,
@@ -125,11 +100,10 @@ class Graph extends React.Component {
       sizes: null
     };
     this.state = {
-      svg: null,
+      toolSVG: null,
+      centroidSVG: null,
       tool: null,
-      container: null,
-      mode: "select",
-      pendingClipPercentiles: null
+      container: null
     };
   }
 
@@ -146,10 +120,17 @@ class Graph extends React.Component {
     const sizeBuffer = regl.buffer();
 
     // preallocate coordinate system transformation between data and gl
-    const fractionToUse = 0.98; // fraction of dimension to use
+    const fractionToUse = 0.93; // fraction of dimension to use
+    const shiftForMenuBar = 0.05;
     const transform = {
       glScaleX: scaleLinear([0, 1], [-1 * fractionToUse, 1 * fractionToUse]),
-      glScaleY: scaleLinear([0, 1], [1 * fractionToUse, -1 * fractionToUse])
+      glScaleY: scaleLinear(
+        [0, 1],
+        [
+          (1 + shiftForMenuBar) * fractionToUse,
+          (-1 + shiftForMenuBar) * fractionToUse
+        ]
+      )
     };
 
     /* first time, but this duplicates above function, should be possile to avoid this */
@@ -179,7 +160,7 @@ class Graph extends React.Component {
     });
   }
 
-  componentDidUpdate(prevProps, prevState) {
+  componentDidUpdate(prevProps) {
     const { renderCache } = this;
     const {
       world,
@@ -188,14 +169,32 @@ class Graph extends React.Component {
       responsive,
       selectionTool,
       currentSelection,
-      layoutChoice
+      layoutChoice,
+      graphInteractionMode,
+      colorAccessor,
+      centroidLabel
     } = this.props;
-    const { reglRender, mode, regl, svg } = this.state;
+    const { reglRender, mode, regl, toolSVG, centroidSVG } = this.state;
     let stateChanges = {};
 
-    if (reglRender && this.reglRenderState === "rendering" && mode !== "zoom") {
-      reglRender.cancel();
-      this.reglRenderState = "paused";
+    if (reglRender) {
+      if (
+        // If it IS RENDERING and it is NOT IN ZOOM mode, stop rendering.
+        this.reglRenderState === "rendering" &&
+        graphInteractionMode !== "zoom"
+      ) {
+        reglRender.cancel();
+        this.reglRenderState = "paused";
+      }
+
+      if (
+        // If it is NOT RENDERING and it IS IN ZOOM mode, start rendering
+        this.reglRenderState !== "rendering" &&
+        graphInteractionMode === "zoom"
+      ) {
+        this.restartReglLoop();
+        this.reglRenderState = "rendering";
+      }
     }
 
     if (regl && world) {
@@ -230,7 +229,13 @@ class Graph extends React.Component {
       }
 
       /* sizes for each point */
-      const newSizes = this.computePointSizes(nObs, crossfilter);
+      const { metadataField, categoryField } = centroidLabel;
+      const newSizes = this.computePointSizes(
+        nObs,
+        crossfilter,
+        metadataField,
+        categoryField
+      );
       if (renderCache.sizes !== newSizes) {
         /* update our cache & GL if the buffer changes */
         renderCache.size = newSizes;
@@ -250,16 +255,10 @@ class Graph extends React.Component {
       );
     }
 
-    if (
-      prevProps.responsive.height !== responsive.height ||
-      prevProps.responsive.width !== responsive.width ||
-      /* first time */
-      (responsive.height && responsive.width && !svg) ||
-      selectionTool !== prevProps.selectionTool
-    ) {
+    const createToolSVG = () => {
       /* clear out whatever was on the div, even if nothing, but usually the brushes etc */
       d3.select("#graphAttachPoint")
-        .selectAll("svg")
+        .select("#tool")
         .remove();
 
       let handleStart;
@@ -275,16 +274,62 @@ class Graph extends React.Component {
         handleEnd = this.handleLassoEnd.bind(this);
         handleCancel = this.handleLassoCancel.bind(this);
       }
-      const { svg: newSvg, tool, container } = setupSVGandBrushElements(
+
+      const { svg: newToolSVG, tool, container } = setupSVGandBrushElements(
         selectionTool,
         handleStart,
         handleDrag,
         handleEnd,
         handleCancel,
         responsive,
-        this.graphPaddingRight
+        this.graphPaddingRight,
+        graphInteractionMode
       );
-      stateChanges = { ...stateChanges, svg: newSvg, tool, container };
+
+      stateChanges = { ...stateChanges, toolSVG: newToolSVG, tool, container };
+    };
+
+    const createCentroidSVG = () => {
+      d3.select("#graphAttachPoint")
+        .select("#centroid-container")
+        .remove();
+
+      if (centroidLabel.metadataField === "" || !centroidLabel.centroidXY) {
+        return;
+      }
+
+      const centroidScreen = this.mapPointToScreen(centroidLabel.centroidXY);
+
+      const newCentroidSVG = setupCentroidSVG(
+        responsive,
+        this.graphPaddingRight,
+        centroidScreen,
+        centroidLabel.categoryField,
+        colorAccessor
+      );
+
+      stateChanges = { ...stateChanges, centroidSVG: newCentroidSVG };
+    };
+
+    // Centroid SVG creation is disabled for now but should go into the first and third cases if enabled
+    if (
+      prevProps.responsive.height !== responsive.height ||
+      prevProps.responsive.width !== responsive.width
+    ) {
+      // If the window size has changed we want to recreate all SVGs
+      createToolSVG();
+    } else if (
+      (responsive.height && responsive.width && !toolSVG) ||
+      selectionTool !== prevProps.selectionTool ||
+      prevProps.graphInteractionMode !== graphInteractionMode
+    ) {
+      // first time or change of selection tool6
+      createToolSVG();
+    } else if (
+      centroidLabel !== prevProps.centroidLabel ||
+      (responsive.height && responsive.width && !centroidSVG)
+    ) {
+      // First time for centroid or label change
     }
 
     /*
@@ -293,8 +338,8 @@ class Graph extends React.Component {
     */
     if (
       currentSelection !== prevProps.currentSelection ||
-      mode !== prevState.mode ||
-      stateChanges.svg
+      graphInteractionMode !== prevProps.graphInteractionMode ||
+      stateChanges.toolSVG
     ) {
       const { tool, container } = this.state;
       this.selectionToolUpdate(
@@ -308,165 +353,6 @@ class Graph extends React.Component {
     }
   }
 
-  isResetDisabled = () => {
-    /*
-    Reset should be disabled when all of the following are true:
-      * nothing is selected in the crossfilter
-      * world EQ universe
-      * nothing is colored by
-      * there are no userDefinedGenes or diffexpGenes displayed
-      * scatterplot is not displayed
-      * nothing in cellset1 or cellset2
-      * clip percentiles are [0,100]
-    */
-    const {
-      crossfilter,
-      world,
-      universe,
-      userDefinedGenes,
-      diffexpGenes,
-      colorAccessor,
-      scatterplotXXaccessor,
-      scatterplotYYaccessor,
-      celllist1,
-      celllist2,
-      clipPercentileMin,
-      clipPercentileMax
-    } = this.props;
-
-    if (!crossfilter || !world || !universe) {
-      return false;
-    }
-    const nothingSelected = crossfilter.countSelected() === crossfilter.size();
-    const nothingColoredBy = !colorAccessor;
-    const noGenes = userDefinedGenes.length === 0 && diffexpGenes.length === 0;
-    const scatterNotDpl = !scatterplotXXaccessor || !scatterplotYYaccessor;
-    const nothingInCellsets = !celllist1 && !celllist2;
-
-    return (
-      nothingSelected &&
-      World.worldEqUniverse(world, universe) &&
-      nothingColoredBy &&
-      noGenes &&
-      scatterNotDpl &&
-      nothingInCellsets &&
-      clipPercentileMax === 100 &&
-      clipPercentileMin === 0
-    );
-  };
-
-  resetInterface = () => {
-    const { dispatch } = this.props;
-    dispatch({
-      type: "interface reset started"
-    });
-    dispatch(actions.resetInterface());
-  };
-
-  isClipDisabled = () => {
-    /*
-    return true if clip button should be disabled.
-    */
-    const { pendingClipPercentiles } = this.state;
-    const clipPercentileMin = pendingClipPercentiles?.clipPercentileMin;
-    const clipPercentileMax = pendingClipPercentiles?.clipPercentileMax;
-
-    const { world } = this.props;
-    const currentClipMin = 100 * world?.clipQuantiles?.min;
-    const currentClipMax = 100 * world?.clipQuantiles?.max;
-
-    // if you change this test, be careful with logic around
-    // comparisons between undefined / NaN handling.
-    const isDisabled =
-      !(clipPercentileMin < clipPercentileMax) ||
-      (clipPercentileMin === currentClipMin &&
-        clipPercentileMax === currentClipMax);
-
-    return isDisabled;
-  };
-
-  handleClipOnKeyPress = e => {
-    /*
-    allow only numbers, plus other critical keys which
-    may be required to make a number
-    */
-    if (!Graph.isValidDigitKeyEvent(e)) {
-      e.preventDefault();
-    }
-  };
-
-  handleClipPercentileMinValueChange = v => {
-    /*
-    Ignore anything that isn't a legit number
-    */
-    if (!Number.isFinite(v)) return;
-
-    const { pendingClipPercentiles } = this.state;
-    const clipPercentileMax = pendingClipPercentiles?.clipPercentileMax;
-
-    /*
-    clamp to [0, currentClipPercentileMax]
-    */
-    if (v <= 0) v = 0;
-    if (v > 100) v = 100;
-    const clipPercentileMin = Math.round(v); // paranoia
-    this.setState({
-      pendingClipPercentiles: { clipPercentileMin, clipPercentileMax }
-    });
-  };
-
-  handleClipPercentileMaxValueChange = v => {
-    /*
-    Ignore anything that isn't a legit number
-    */
-    if (!Number.isFinite(v)) return;
-
-    const { pendingClipPercentiles } = this.state;
-    const clipPercentileMin = pendingClipPercentiles?.clipPercentileMin;
-
-    /*
-    clamp to [0, 100]
-    */
-    if (v < 0) v = 0;
-    if (v > 100) v = 100;
-    const clipPercentileMax = Math.round(v); // paranoia
-
-    this.setState({
-      pendingClipPercentiles: { clipPercentileMin, clipPercentileMax }
-    });
-  };
-
-  handleClipCommit = () => {
-    const { dispatch } = this.props;
-    const { pendingClipPercentiles } = this.state;
-    const { clipPercentileMin, clipPercentileMax } = pendingClipPercentiles;
-    const min = clipPercentileMin / 100;
-    const max = clipPercentileMax / 100;
-    dispatch({
-      type: "set clip quantiles",
-      clipQuantiles: { min, max }
-    });
-  };
-
-  handleClipOpening = () => {
-    const { clipPercentileMin, clipPercentileMax } = this.props;
-    this.setState({
-      pendingClipPercentiles: { clipPercentileMin, clipPercentileMax }
-    });
-  };
-
-  handleClipClosing = () => {
-    this.setState({ pendingClipPercentiles: null });
-  };
-
-  handleLayoutChoiceChange = e => {
-    const { dispatch } = this.props;
-    dispatch({
-      type: "set layout choice",
-      layoutChoice: e.currentTarget.value
-    });
-  };
-
   brushToolUpdate(tool, container) {
     /*
     this is called from componentDidUpdate(), so be very careful using
@@ -509,7 +395,7 @@ class Graph extends React.Component {
     }
   }
 
-  lassoToolUpdate(tool, container) {
+  lassoToolUpdate(tool) {
     /*
     this is called from componentDidUpdate(), so be very careful using
     anything from this.state, which may be updated asynchronously.
@@ -636,7 +522,7 @@ class Graph extends React.Component {
     const scale = aspect < 1 ? 1 / aspect : 1;
 
     // compute inverse view matrix
-    let inverse = mat4.invert([], camera.view());
+    const inverse = mat4.invert([], camera.view());
 
     // variable names are choosen to reflect inverse of those used
     // in mapScreenToPoint().
@@ -769,370 +655,19 @@ class Graph extends React.Component {
   }
 
   render() {
-    const {
-      dispatch,
-      responsive,
-      crossfilter,
-      resettingInterface,
-      libraryVersions,
-      undoDisabled,
-      redoDisabled,
-      selectionTool,
-      clipPercentileMin,
-      clipPercentileMax,
-      layoutChoice
-    } = this.props;
-    const { mode, pendingClipPercentiles } = this.state;
-
-    const clipMin =
-      pendingClipPercentiles?.clipPercentileMin ?? clipPercentileMin;
-    const clipMax =
-      pendingClipPercentiles?.clipPercentileMax ?? clipPercentileMax;
-    const activeClipClass =
-      clipPercentileMin > 0 || clipPercentileMax < 100
-        ? " bp3-intent-warning"
-        : "";
-
-    // constants used to create selection tool button
-    let selectionTooltip;
-    let selectionButtonClass;
-    if (selectionTool === "brush") {
-      selectionTooltip = "Brush selection";
-      selectionButtonClass = "bp3-icon-select";
-    } else {
-      selectionTooltip = "Lasso selection";
-      selectionButtonClass = "bp3-icon-polygon-filter";
-    }
+    const { responsive, graphInteractionMode } = this.props;
 
     return (
       <div id="graphWrapper">
-        <div
-          style={{
-            position: "fixed",
-            right: 0,
-            top: 0
-          }}
-        >
-          <div
-            style={{
-              padding: 10,
-              display: "flex",
-              justifyContent: "flex-end",
-              alignItems: "baseline"
-            }}
-          >
-            <Tooltip
-              content="Show only metadata and cells which are currently selected"
-              position="left"
-            >
-              <AnchorButton
-                type="button"
-                data-testid="subset-button"
-                disabled={
-                  crossfilter &&
-                  (crossfilter.countSelected() === 0 ||
-                    crossfilter.countSelected() === crossfilter.size())
-                }
-                style={{ marginRight: 10 }}
-                onClick={() => {
-                  dispatch(actions.regraph());
-                  dispatch({ type: "increment graph render counter" });
-                }}
-              >
-                subset to current selection
-              </AnchorButton>
-            </Tooltip>
-            <Tooltip
-              content="Reset cellxgene, clearing all selections"
-              position="left"
-            >
-              <AnchorButton
-                disabled={this.isResetDisabled()}
-                type="button"
-                loading={resettingInterface}
-                intent="warning"
-                style={{ marginRight: 10 }}
-                onClick={this.resetInterface}
-                data-testid="reset"
-                data-testclass={`resetting-${resettingInterface}`}
-              >
-                reset
-              </AnchorButton>
-            </Tooltip>
-            <div className="bp3-button-group">
-              <Tooltip content={selectionTooltip} position="left">
-                <Button
-                  type="button"
-                  data-testid="mode-lasso"
-                  className={`bp3-button ${selectionButtonClass}`}
-                  active={mode === "select"}
-                  onClick={() => {
-                    this.setState({ mode: "select" });
-                  }}
-                  style={{
-                    cursor: "pointer"
-                  }}
-                />
-              </Tooltip>
-              <Tooltip content="Pan and zoom" position="left">
-                <Button
-                  type="button"
-                  data-testid="mode-pan-zoom"
-                  className="bp3-button bp3-icon-zoom-in"
-                  active={mode === "zoom"}
-                  onClick={() => {
-                    this.restartReglLoop();
-                    this.setState({ mode: "zoom" });
-                  }}
-                  style={{
-                    cursor: "pointer"
-                  }}
-                />
-              </Tooltip>
-            </div>
-            <div
-              className="bp3-button-group"
-              style={{
-                marginLeft: 10
-              }}
-            >
-              <Tooltip content="Undo" position="left">
-                <AnchorButton
-                  type="button"
-                  className="bp3-button bp3-icon-undo"
-                  disabled={undoDisabled}
-                  onClick={() => {
-                    dispatch({ type: "@@undoable/undo" });
-                  }}
-                  style={{
-                    cursor: "pointer"
-                  }}
-                />
-              </Tooltip>
-              <Tooltip content="Redo" position="left">
-                <AnchorButton
-                  type="button"
-                  className="bp3-button bp3-icon-redo"
-                  disabled={redoDisabled}
-                  onClick={() => {
-                    dispatch({ type: "@@undoable/redo" });
-                  }}
-                  style={{
-                    cursor: "pointer"
-                  }}
-                />
-              </Tooltip>
-            </div>
-
-            <div
-              className="bp3-button-group"
-              style={{
-                marginLeft: 10
-              }}
-            >
-              <Popover
-                target={
-                  <Button
-                    type="button"
-                    data-testid="layout-choice"
-                    className="bp3-button bp3-icon-heatmap"
-                    style={{
-                      cursor: "pointer"
-                    }}
-                  />
-                }
-                position={Position.BOTTOM_RIGHT}
-                content={
-                  <div
-                    style={{
-                      display: "flex",
-                      justifyContent: "flex-start",
-                      alignItems: "flex-start",
-                      flexDirection: "column",
-                      padding: 10
-                    }}
-                  >
-                    <RadioGroup
-                      label="Layout Choice"
-                      onChange={this.handleLayoutChoiceChange}
-                      selectedValue={layoutChoice.current}
-                    >
-                      {layoutChoice.available.map(name => (
-                        <Radio label={name} value={name} key={name} />
-                      ))}
-                    </RadioGroup>
-                  </div>
-                }
-              />
-            </div>
-
-            <div
-              className="bp3-button-group"
-              style={{
-                marginLeft: 10
-              }}
-            >
-              <Popover
-                target={
-                  <Button
-                    type="button"
-                    data-testid="visualization-settings"
-                    className={`bp3-button bp3-icon-timeline-bar-chart ${activeClipClass}`}
-                    style={{
-                      cursor: "pointer"
-                    }}
-                  />
-                }
-                position={Position.BOTTOM_RIGHT}
-                onOpening={this.handleClipOpening}
-                onClosing={this.handleClipClosing}
-                content={
-                  <div
-                    style={{
-                      display: "flex",
-                      justifyContent: "flex-start",
-                      alignItems: "flex-start",
-                      flexDirection: "column",
-                      padding: 10
-                    }}
-                  >
-                    <div>Clip all continuous values to percentile range</div>
-                    <div
-                      style={{
-                        display: "flex",
-                        justifyContent: "space-between",
-                        alignItems: "center",
-                        paddingTop: 5,
-                        paddingBottom: 5
-                      }}
-                    >
-                      <NumericInput
-                        style={{ width: 50 }}
-                        data-testid={"clip-min-input"}
-                        onValueChange={this.handleClipPercentileMinValueChange}
-                        onKeyPress={this.handleClipOnKeyPress}
-                        value={clipMin}
-                        min={0}
-                        max={100}
-                        fill={false}
-                        minorStepSize={null}
-                        rightElement={
-                          <div style={{ padding: "4px 2px" }}>
-                            <Icon
-                              icon="percentage"
-                              intent="primary"
-                              iconSize={14}
-                            />
-                          </div>
-                        }
-                      />
-                      <span style={{ marginRight: 5, marginLeft: 5 }}> - </span>
-                      <NumericInput
-                        style={{ width: 50 }}
-                        data-testid={"clip-max-input"}
-                        onValueChange={this.handleClipPercentileMaxValueChange}
-                        onKeyPress={this.handleClipOnKeyPress}
-                        value={clipMax}
-                        min={0}
-                        max={100}
-                        fill={false}
-                        minorStepSize={null}
-                        rightElement={
-                          <div style={{ padding: "4px 2px" }}>
-                            <Icon
-                              icon="percentage"
-                              intent="primary"
-                              iconSize={14}
-                            />
-                          </div>
-                        }
-                      />
-                      <Button
-                        type="button"
-                        data-testid="clip-commit"
-                        className="bp3-button"
-                        disabled={this.isClipDisabled()}
-                        style={{
-                          cursor: "pointer",
-                          marginRight: 5,
-                          marginLeft: 5
-                        }}
-                        onClick={this.handleClipCommit}
-                      >
-                        Clip
-                      </Button>
-                    </div>
-                  </div>
-                }
-              />
-            </div>
-
-            <div style={{ marginLeft: 10 }} className="bp3-button-group">
-              <Popover
-                content={
-                  <Menu>
-                    <MenuItem
-                      href="https://chanzuckerberg.github.io/cellxgene/faq.html"
-                      target="_blank"
-                      icon="help"
-                      text="FAQ"
-                    />
-                    <MenuItem
-                      href="https://join-cellxgene-users.herokuapp.com/"
-                      target="_blank"
-                      icon="chat"
-                      text="Chat"
-                    />
-                    <MenuItem
-                      href="https://chanzuckerberg.github.io/cellxgene/"
-                      target="_blank"
-                      icon="book"
-                      text="Docs"
-                    />
-                    <MenuItem
-                      href="https://github.com/chanzuckerberg/cellxgene"
-                      target="_blank"
-                      icon="git-branch"
-                      text="Github"
-                    />
-                    <MenuItem
-                      target="_blank"
-                      text={`cellxgene v${
-                        libraryVersions && libraryVersions.cellxgene
-                          ? libraryVersions.cellxgene
-                          : null
-                      }`}
-                    />
-                    <MenuItem text="MIT License" />
-                  </Menu>
-                }
-                position={Position.BOTTOM_RIGHT}
-              >
-                <Button
-                  type="button"
-                  className="bp3-button bp3-icon-info-sign"
-                  style={{
-                    cursor: "pointer"
-                  }}
-                />
-              </Popover>
-            </div>
-          </div>
-        </div>
         <div
           style={{
             zIndex: -9999,
             position: "fixed",
-            top: 0,
+            top: this.graphPaddingTop,
             right: 0
           }}
         >
-          <div
-            style={{
-              display: mode === "select" ? "inherit" : "none"
-            }}
-            id="graphAttachPoint"
-          />
+          <div id="graphAttachPoint" />
           <div style={{ padding: 0, margin: 0 }}>
             <canvas
               width={responsive.width - this.graphPaddingRight}

client/src/components/graph/setupCentroidSVG.js

@@ -0,0 +1,34 @@
+import * as d3 from "d3";
+import styles from "./graph.css";
+
+export default (responsive, graphPaddingRight, xy, text, colorBy) => {
+  const containerWidth = responsive.width - graphPaddingRight;
+
+  const svg = d3
+    .select("#graphAttachPoint")
+    .append("svg")
+    .attr("id", "centroid-container")
+    .attr("data-testid", "centroid-overlay")
+    .attr("width", containerWidth)
+    .attr("height", responsive.height)
+    .attr("class", `${styles.graphSVG}`)
+    .style("z-index", 998)
+    .style("pointer-events", "none");
+  //  TODO: Create own styles, ask Colin for an explanation on the css
+  // For now I'm going to put centroid z-index at 998 and lasso on 999
+
+  const label = svg
+    .append("g")
+    .attr("transform", `translate(${xy[0]}, ${xy[1]})`);
+
+  label
+    .append("text")
+    .attr("text-anchor", "middle")
+    .text(text)
+    .style("font-family", "Roboto Condensed")
+    .style("font-size", "18px")
+    .style("font-weight", "700")
+    .style("fill", colorBy ? "black" : "rgb(32, 178, 212)");
+
+  return svg;
+};

client/src/components/graph/setupSVGandBrush.js

@@ -16,15 +16,19 @@ export default (
   handleEndAction,
   handleCancelAction,
   responsive,
-  graphPaddingRight
+  graphPaddingRight,
+  graphInteractionMode
 ) => {
   const svg = d3
     .select("#graphAttachPoint")
     .append("svg")
+    .attr("id", "tool")
     .attr("data-testid", "layout-overlay")
     .attr("width", responsive.width - graphPaddingRight)
     .attr("height", responsive.height)
-    .attr("class", `${styles.graphSVG}`);
+    .attr("class", `${styles.graphSVG}`)
+    .style("z-index", 999)
+    .style("display", graphInteractionMode === "select" ? "inherit" : "none");
 
   if (selectionToolType === "brush") {
     const brush = d3

client/src/components/leftSidebar/index.js

@@ -0,0 +1,61 @@
+// jshint esversion: 6
+import React from "react";
+import { connect } from "react-redux";
+import Categorical from "../categorical/categorical";
+import Continuous from "../continuous/continuous";
+import GeneExpression from "../geneExpression";
+import * as globals from "../../globals";
+import DynamicScatterplot from "../scatterplot/scatterplot";
+import TopLeftLogoAndTitle from "./topLeftLogoAndTitle";
+
+@connect(state => ({
+  responsive: state.responsive,
+  scatterplotXXaccessor: state.controls.scatterplotXXaccessor,
+  scatterplotYYaccessor: state.controls.scatterplotYYaccessor
+}))
+class LeftSideBar extends React.Component {
+  render() {
+    const {
+      responsive,
+      scatterplotXXaccessor,
+      scatterplotYYaccessor
+    } = this.props;
+
+    /*
+    this magic number should be made less fragile,
+    if cellxgene logo or tabs change, this must as well
+    */
+    const logoRelatedPadding = 50;
+
+    return (
+      <div
+        style={{
+          position: "fixed",
+          backgroundColor: "white",
+          /* x y blur spread color */
+          boxShadow: "-3px 0px 6px 2px rgba(153,153,153,0.4)"
+        }}
+      >
+        <TopLeftLogoAndTitle />
+        <div
+          style={{
+            height: responsive.height - logoRelatedPadding,
+            marginTop: logoRelatedPadding,
+            width: globals.leftSidebarWidth,
+            overflowY: "auto",
+            overflowX: "hidden"
+          }}
+        >
+          <Categorical />
+          <GeneExpression />
+          <Continuous />
+        </div>
+        {scatterplotXXaccessor && scatterplotYYaccessor ? (
+          <DynamicScatterplot />
+        ) : null}
+      </div>
+    );
+  }
+}
+
+export default LeftSideBar;

client/src/components/leftSidebar/topLeftLogoAndTitle.js

@@ -0,0 +1,71 @@
+// jshint esversion: 6
+import React from "react";
+import { connect } from "react-redux";
+import * as globals from "../../globals";
+import Logo from "../framework/logo";
+
+@connect(state => ({
+  responsive: state.responsive,
+  datasetTitle: state.config?.displayNames?.dataset ?? "",
+  scatterplotXXaccessor: state.controls.scatterplotXXaccessor,
+  scatterplotYYaccessor: state.controls.scatterplotYYaccessor
+}))
+class LeftSideBar extends React.Component {
+  render() {
+    const { datasetTitle } = this.props;
+
+    const paddingToAvoidScrollBar = 15;
+
+    return (
+      <div
+        style={{
+          paddingLeft: 8,
+          paddingTop: 8,
+          width: globals.leftSidebarWidth - paddingToAvoidScrollBar,
+          position: "absolute",
+          backgroundColor: "white",
+          zIndex: 8888
+          /* x y blur spread color */
+          // boxShadow: "-5px -1px 4px 2px rgba(225,225,225,0.4)"
+        }}
+      >
+        <Logo size={30} />
+        <span
+          style={{
+            fontSize: 28,
+            position: "relative",
+            top: -6,
+            fontWeight: "bold",
+            marginLeft: 5,
+            color: globals.logoColor,
+            userSelect: "none"
+          }}
+        >
+          cell<span
+            style={{
+              position: "relative",
+              top: 1,
+              fontWeight: 300,
+              fontSize: 24
+            }}
+          >
+            ×
+          </span>gene
+        </span>
+        <span
+          data-testid="header"
+          style={{
+            fontSize: 14,
+            position: "relative",
+            marginLeft: 7,
+            top: -8
+          }}
+        >
+          {datasetTitle}
+        </span>
+      </div>
+    );
+  }
+}
+
+export default LeftSideBar;

client/src/components/leftsidebar.js

@@ -1,105 +0,0 @@
-// jshint esversion: 6
-import React from "react";
-import { connect } from "react-redux";
-import Categorical from "./categorical/categorical";
-import Continuous from "./continuous/continuous";
-import GeneExpression from "./geneExpression";
-import * as globals from "../globals";
-import DynamicScatterplot from "./scatterplot/scatterplot";
-import Logo from "./framework/logo.js";
-
-@connect(state => ({
-  responsive: state.responsive,
-  datasetTitle: state.config?.displayNames?.dataset,
-  scatterplotXXaccessor: state.controls.scatterplotXXaccessor,
-  scatterplotYYaccessor: state.controls.scatterplotYYaccessor
-}))
-class LeftSideBar extends React.Component {
-  render() {
-    const {
-      responsive,
-      datasetTitle,
-      scatterplotXXaccessor,
-      scatterplotYYaccessor
-    } = this.props;
-
-    /*
-    this magic number should be made less fragile,
-    if cellxgene logo or tabs change, this must as well
-    */
-    const metadataSectionPadding = 0;
-
-    return (
-      <div
-        style={{
-          position: "fixed",
-          backgroundColor: "white",
-          /* x y blur spread color */
-          boxShadow: "1px 0px 6px 2px rgba(153,153,153,0.4)"
-        }}
-      >
-        <p
-          style={{
-            position: "fixed",
-            top: globals.cellxgeneTitleTopPadding,
-            left: globals.leftSidebarWidth + globals.cellxgeneTitleLeftPadding,
-            margin: 0
-          }}
-        >
-          <Logo size={32} />
-          <span
-            style={{
-              fontSize: 28,
-              position: "relative",
-              top: -4,
-              fontWeight: "bold",
-              marginLeft: 5,
-              color: globals.logoColor,
-              userSelect: "none"
-            }}
-          >
-            cell<span
-              style={{
-                position: "relative",
-                top: 1,
-                fontWeight: 300,
-                fontSize: 24
-              }}
-            >
-              ×
-            </span>gene
-          </span>
-          <span
-            data-testid="header"
-            style={{
-              fontSize: 16,
-              display: "block",
-              position: "relative",
-              marginTop: 10,
-              top: -4
-            }}
-          >
-            {datasetTitle}
-          </span>
-        </p>
-        <div
-          style={{
-            height: responsive.height - metadataSectionPadding,
-            width: globals.leftSidebarWidth,
-            overflowY: "auto",
-            overflowX: "hidden"
-          }}
-        >
-          <Categorical />
-          <GeneExpression />
-          <Continuous />
-        </div>
-        {scatterplotXXaccessor && scatterplotYYaccessor ? (
-          <DynamicScatterplot />
-        ) : null}
-      </div>
-    );
-  }
-}
-
-export default LeftSideBar;

client/src/components/menubar/cellSetButtons.js

@@ -32,7 +32,7 @@ class CellSetButton extends React.Component {
   render() {
     const { differential, eitherCellSetOneOrTwo } = this.props;
     const cellListName = `celllist${eitherCellSetOneOrTwo}`;
-    let cells_selected = differential[cellListName]
+    const cellsSelected = differential[cellListName]
       ? differential[cellListName].length
       : 0;
     return (
@@ -41,7 +41,6 @@ class CellSetButton extends React.Component {
         position="top"
       >
         <AnchorButton
-          style={{ marginRight: 10 }}
           type="button"
           disabled={differential.diffExp}
           onClick={this.set.bind(this)}
@@ -50,7 +49,7 @@ class CellSetButton extends React.Component {
           {eitherCellSetOneOrTwo}
           {": "}
           <span data-testid={`cellset-count-${eitherCellSetOneOrTwo}`}>
-            {cells_selected}
+            {cellsSelected}
           </span>
           {" cells"}
         </AnchorButton>

client/src/components/menubar/clip.js

@@ -0,0 +1,130 @@
+// jshint esversion: 6
+import React from "react";
+import {
+  Position,
+  Button,
+  Popover,
+  NumericInput,
+  Icon
+} from "@blueprintjs/core";
+
+function Clip(props) {
+  const {
+    pendingClipPercentiles,
+    clipPercentileMin,
+    clipPercentileMax,
+    handleClipOpening,
+    handleClipClosing,
+    handleClipCommit,
+    isClipDisabled,
+    handleClipOnKeyPress,
+    handleClipPercentileMaxValueChange,
+    handleClipPercentileMinValueChange
+  } = props;
+
+  const clipMin =
+    pendingClipPercentiles?.clipPercentileMin ?? clipPercentileMin;
+  const clipMax =
+    pendingClipPercentiles?.clipPercentileMax ?? clipPercentileMax;
+  const activeClipClass =
+    clipPercentileMin > 0 || clipPercentileMax < 100
+      ? " bp3-intent-warning"
+      : "";
+
+  return (
+    <div
+      className="bp3-button-group"
+      style={{
+        marginLeft: 10
+      }}
+    >
+      <Popover
+        target={
+          <Button
+            type="button"
+            data-testid="visualization-settings"
+            className={`bp3-button bp3-icon-timeline-bar-chart ${activeClipClass}`}
+            style={{
+              cursor: "pointer"
+            }}
+          />
+        }
+        position={Position.BOTTOM_RIGHT}
+        onOpening={handleClipOpening}
+        onClosing={handleClipClosing}
+        content={
+          <div
+            style={{
+              display: "flex",
+              justifyContent: "flex-start",
+              alignItems: "flex-start",
+              flexDirection: "column",
+              padding: 10
+            }}
+          >
+            <div>Clip all continuous values to percentile range</div>
+            <div
+              style={{
+                display: "flex",
+                justifyContent: "space-between",
+                alignItems: "center",
+                paddingTop: 5,
+                paddingBottom: 5
+              }}
+            >
+              <NumericInput
+                style={{ width: 50 }}
+                data-testid="clip-min-input"
+                onValueChange={handleClipPercentileMinValueChange}
+                onKeyPress={handleClipOnKeyPress}
+                value={clipMin}
+                min={0}
+                max={100}
+                fill={false}
+                minorStepSize={null}
+                rightElement={
+                  <div style={{ padding: "4px 2px" }}>
+                    <Icon icon="percentage" intent="primary" iconSize={14} />
+                  </div>
+                }
+              />
+              <span style={{ marginRight: 5, marginLeft: 5 }}> - </span>
+              <NumericInput
+                style={{ width: 50 }}
+                data-testid="clip-max-input"
+                onValueChange={handleClipPercentileMaxValueChange}
+                onKeyPress={handleClipOnKeyPress}
+                value={clipMax}
+                min={0}
+                max={100}
+                fill={false}
+                minorStepSize={null}
+                rightElement={
+                  <div style={{ padding: "4px 2px" }}>
+                    <Icon icon="percentage" intent="primary" iconSize={14} />
+                  </div>
+                }
+              />
+              <Button
+                type="button"
+                data-testid="clip-commit"
+                className="bp3-button"
+                disabled={isClipDisabled()}
+                style={{
+                  cursor: "pointer",
+                  marginRight: 5,
+                  marginLeft: 5
+                }}
+                onClick={handleClipCommit}
+              >
+                Clip
+              </Button>
+            </div>
+          </div>
+        }
+      />
+    </div>
+  );
+}
+
+export default Clip;

client/src/components/menubar/index.js

@@ -0,0 +1,464 @@
+// jshint esversion: 6
+import React from "react";
+import { connect } from "react-redux";
+import {
+  Button,
+  AnchorButton,
+  Tooltip,
+  Popover,
+  Position,
+  RadioGroup,
+  Radio
+} from "@blueprintjs/core";
+import { World } from "../../util/stateManager";
+import actions from "../../actions";
+import CellSetButton from "./cellSetButtons";
+import InformationMenu from "./infoMenu";
+import UndoRedoReset from "./undoRedoReset";
+import Clip from "./clip";
+
+@connect(state => ({
+  universe: state.universe,
+  world: state.world,
+  loading: state.controls.loading,
+  crossfilter: state.crossfilter,
+  differential: state.differential,
+  resettingInterface: state.controls.resettingInterface,
+  layoutChoice: state.layoutChoice,
+  graphInteractionMode: state.controls.graphInteractionMode,
+  clipPercentileMin: Math.round(100 * (state.world?.clipQuantiles?.min ?? 0)),
+  clipPercentileMax: Math.round(100 * (state.world?.clipQuantiles?.max ?? 1)),
+  userDefinedGenes: state.controls.userDefinedGenes,
+  diffexpGenes: state.controls.diffexpGenes,
+  colorAccessor: state.colors.colorAccessor,
+  scatterplotXXaccessor: state.controls.scatterplotXXaccessor,
+  scatterplotYYaccessor: state.controls.scatterplotYYaccessor,
+  celllist1: state.differential.celllist1,
+  celllist2: state.differential.celllist2,
+  libraryVersions: state.config?.library_versions, // eslint-disable-line camelcase
+  undoDisabled: state["@@undoable/past"].length === 0,
+  redoDisabled: state["@@undoable/future"].length === 0
+}))
+class MenuBar extends React.Component {
+  static isValidDigitKeyEvent(e) {
+    /*
+    Return true if this event is necessary to enter a percent number input.
+    Return false if not.
+
+    Returns true for events with keys: backspace, control, alt, meta, [0-9],
+    or events that don't have a key.
+    */
+    if (e.key === null) return true;
+    if (e.ctrlKey || e.altKey || e.metaKey) return true;
+
+    // concept borrowed from blueprint's numericInputUtils:
+    // keys that print a single character when pressed have a `key` name of
+    // length 1. every other key has a longer `key` name (e.g. "Backspace",
+    // "ArrowUp", "Shift"). since none of those keys can print a character
+    // to the field--and since they may have important native behaviors
+    // beyond printing a character--we don't want to disable their effects.
+    const isSingleCharKey = e.key.length === 1;
+    if (!isSingleCharKey) return true;
+
+    const key = e.key.charCodeAt(0) - 48; /* "0" */
+    return key >= 0 && key <= 9;
+  }
+
+  constructor(props) {
+    super(props);
+    this.state = {
+      pendingClipPercentiles: null
+    };
+  }
+
+  isClipDisabled = () => {
+    /*
+    return true if clip button should be disabled.
+    */
+    const { pendingClipPercentiles } = this.state;
+    const clipPercentileMin = pendingClipPercentiles?.clipPercentileMin;
+    const clipPercentileMax = pendingClipPercentiles?.clipPercentileMax;
+
+    const { world } = this.props;
+    const currentClipMin = 100 * world?.clipQuantiles?.min;
+    const currentClipMax = 100 * world?.clipQuantiles?.max;
+
+    // if you change this test, be careful with logic around
+    // comparisons between undefined / NaN handling.
+    const isDisabled =
+      !(clipPercentileMin < clipPercentileMax) ||
+      (clipPercentileMin === currentClipMin &&
+        clipPercentileMax === currentClipMax);
+
+    return isDisabled;
+  };
+
+  isResetDisabled = () => {
+    /*
+    Reset should be disabled when all of the following are true:
+      * nothing is selected in the crossfilter
+      * world EQ universe
+      * nothing is colored by
+      * there are no userDefinedGenes or diffexpGenes displayed
+      * scatterplot is not displayed
+      * nothing in cellset1 or cellset2
+      * clip percentiles are [0,100]
+    */
+    const {
+      crossfilter,
+      world,
+      universe,
+      userDefinedGenes,
+      diffexpGenes,
+      colorAccessor,
+      scatterplotXXaccessor,
+      scatterplotYYaccessor,
+      celllist1,
+      celllist2,
+      clipPercentileMin,
+      clipPercentileMax
+    } = this.props;
+
+    if (!crossfilter || !world || !universe) {
+      return false;
+    }
+    const nothingSelected = crossfilter.countSelected() === crossfilter.size();
+    const nothingColoredBy = !colorAccessor;
+    const noGenes = userDefinedGenes.length === 0 && diffexpGenes.length === 0;
+    const scatterNotDpl = !scatterplotXXaccessor || !scatterplotYYaccessor;
+    const nothingInCellsets = !celllist1 && !celllist2;
+
+    return (
+      nothingSelected &&
+      World.worldEqUniverse(world, universe) &&
+      nothingColoredBy &&
+      noGenes &&
+      scatterNotDpl &&
+      nothingInCellsets &&
+      clipPercentileMax === 100 &&
+      clipPercentileMin === 0
+    );
+  };
+
+  resetInterface = () => {
+    const { dispatch } = this.props;
+    dispatch({
+      type: "interface reset started"
+    });
+    dispatch(actions.resetInterface());
+  };
+
+  handleClipOnKeyPress = e => {
+    /*
+    allow only numbers, plus other critical keys which
+    may be required to make a number
+    */
+    if (!MenuBar.isValidDigitKeyEvent(e)) {
+      e.preventDefault();
+    }
+  };
+
+  handleClipPercentileMinValueChange = v => {
+    /*
+    Ignore anything that isn't a legit number
+    */
+    if (!Number.isFinite(v)) return;
+
+    const { pendingClipPercentiles } = this.state;
+    const clipPercentileMax = pendingClipPercentiles?.clipPercentileMax;
+
+    /*
+    clamp to [0, currentClipPercentileMax]
+    */
+    if (v <= 0) v = 0;
+    if (v > 100) v = 100;
+    const clipPercentileMin = Math.round(v); // paranoia
+    this.setState({
+      pendingClipPercentiles: { clipPercentileMin, clipPercentileMax }
+    });
+  };
+
+  handleClipPercentileMaxValueChange = v => {
+    /*
+    Ignore anything that isn't a legit number
+    */
+    if (!Number.isFinite(v)) return;
+
+    const { pendingClipPercentiles } = this.state;
+    const clipPercentileMin = pendingClipPercentiles?.clipPercentileMin;
+
+    /*
+    clamp to [0, 100]
+    */
+    if (v < 0) v = 0;
+    if (v > 100) v = 100;
+    const clipPercentileMax = Math.round(v); // paranoia
+
+    this.setState({
+      pendingClipPercentiles: { clipPercentileMin, clipPercentileMax }
+    });
+  };
+
+  handleClipCommit = () => {
+    const { dispatch } = this.props;
+    const { pendingClipPercentiles } = this.state;
+    const { clipPercentileMin, clipPercentileMax } = pendingClipPercentiles;
+    const min = clipPercentileMin / 100;
+    const max = clipPercentileMax / 100;
+    dispatch({
+      type: "set clip quantiles",
+      clipQuantiles: { min, max }
+    });
+  };
+
+  handleClipOpening = () => {
+    const { clipPercentileMin, clipPercentileMax } = this.props;
+    this.setState({
+      pendingClipPercentiles: { clipPercentileMin, clipPercentileMax }
+    });
+  };
+
+  handleClipClosing = () => {
+    this.setState({ pendingClipPercentiles: null });
+  };
+
+  handleLayoutChoiceChange = e => {
+    const { dispatch } = this.props;
+    dispatch({
+      type: "set layout choice",
+      layoutChoice: e.currentTarget.value
+    });
+  };
+
+  computeDiffExp = () => {
+    const { dispatch, differential } = this.props;
+    if (differential.celllist1 && differential.celllist2) {
+      dispatch(
+        actions.requestDifferentialExpression(
+          differential.celllist1,
+          differential.celllist2
+        )
+      );
+    }
+  };
+
+  clearDifferentialExpression = () => {
+    const { dispatch, differential } = this.props;
+    dispatch({
+      type: "clear differential expression",
+      diffExp: differential.diffExp
+    });
+    dispatch({
+      type: "clear scatterplot"
+    });
+  };
+
+  render() {
+    const {
+      dispatch,
+      differential,
+      crossfilter,
+      resettingInterface,
+      libraryVersions,
+      undoDisabled,
+      redoDisabled,
+      selectionTool,
+      clipPercentileMin,
+      clipPercentileMax,
+      layoutChoice,
+      graphInteractionMode
+    } = this.props;
+    const { pendingClipPercentiles } = this.state;
+
+    const haveBothCellSets =
+      !!differential.celllist1 && !!differential.celllist2;
+
+    // constants used to create selection tool button
+    let selectionTooltip;
+    let selectionButtonClass;
+    if (selectionTool === "brush") {
+      selectionTooltip = "Brush selection";
+      selectionButtonClass = "bp3-icon-select";
+    } else {
+      selectionTooltip = "Lasso selection";
+      selectionButtonClass = "bp3-icon-polygon-filter";
+    }
+
+    return (
+      <div
+        style={{
+          position: "fixed",
+          right: 8,
+          top: 8
+        }}
+      >
+        <div className="bp3-button-group" style={{ marginRight: 10 }}>
+          <CellSetButton {...this.props} eitherCellSetOneOrTwo={1} />
+          <CellSetButton {...this.props} eitherCellSetOneOrTwo={2} />
+          {!differential.diffExp ? (
+            <Tooltip
+              content="Add two cells selections, see the top 15 differentially expressed genes between them"
+              position="bottom"
+            >
+              <AnchorButton
+                disabled={!haveBothCellSets}
+                intent="primary"
+                data-testid="diffexp-button"
+                loading={differential.loading}
+                icon="left-join"
+                fill
+                type="button"
+                onClick={this.computeDiffExp}
+              >
+                Compute Differential Expression
+              </AnchorButton>
+            </Tooltip>
+          ) : null}
+
+          {differential.diffExp ? (
+            <Tooltip
+              content="Remove differentially expressed gene list and clear cell selections"
+              position="bottom"
+            >
+              <Button
+                type="button"
+                fill
+                intent="warning"
+                onClick={this.clearDifferentialExpression}
+              >
+                Clear Differential Expression
+              </Button>
+            </Tooltip>
+          ) : null}
+        </div>
+        <Tooltip
+          content="Show only metadata and cells which are currently selected"
+          position="left"
+        >
+          <AnchorButton
+            type="button"
+            data-testid="subset-button"
+            disabled={
+              crossfilter &&
+              (crossfilter.countSelected() === 0 ||
+                crossfilter.countSelected() === crossfilter.size())
+            }
+            style={{
+              marginRight: 10
+            }}
+            onClick={() => {
+              dispatch(actions.regraph());
+              dispatch({ type: "increment graph render counter" });
+            }}
+          >
+            subset to current selection
+          </AnchorButton>
+        </Tooltip>
+        <div className="bp3-button-group">
+          <Tooltip content={selectionTooltip} position="left">
+            <Button
+              type="button"
+              data-testid="mode-lasso"
+              className={`bp3-button ${selectionButtonClass}`}
+              active={graphInteractionMode === "select"}
+              onClick={() => {
+                dispatch({
+                  type: "change graph interaction mode",
+                  data: "select"
+                });
+              }}
+              style={{
+                cursor: "pointer"
+              }}
+            />
+          </Tooltip>
+          <Tooltip content="Pan and zoom" position="left">
+            <Button
+              type="button"
+              data-testid="mode-pan-zoom"
+              className="bp3-button bp3-icon-zoom-in"
+              active={graphInteractionMode === "zoom"}
+              onClick={() => {
+                dispatch({
+                  type: "change graph interaction mode",
+                  data: "zoom"
+                });
+              }}
+              style={{
+                cursor: "pointer"
+              }}
+            />
+          </Tooltip>
+        </div>
+        <div
+          className="bp3-button-group"
+          style={{
+            marginLeft: 10
+          }}
+        >
+          <Popover
+            target={
+              <Button
+                type="button"
+                data-testid="layout-choice"
+                className="bp3-button bp3-icon-heatmap"
+                style={{
+                  cursor: "pointer"
+                }}
+              />
+            }
+            position={Position.BOTTOM_RIGHT}
+            content={
+              <div
+                style={{
+                  display: "flex",
+                  justifyContent: "flex-start",
+                  alignItems: "flex-start",
+                  flexDirection: "column",
+                  padding: 10
+                }}
+              >
+                <RadioGroup
+                  label="Layout Choice"
+                  onChange={this.handleLayoutChoiceChange}
+                  selectedValue={layoutChoice.current}
+                >
+                  {layoutChoice.available.map(name => (
+                    <Radio label={name} value={name} key={name} />
+                  ))}
+                </RadioGroup>
+              </div>
+            }
+          />
+        </div>
+        <Clip
+          pendingClipPercentiles={pendingClipPercentiles}
+          clipPercentileMin={clipPercentileMin}
+          clipPercentileMax={clipPercentileMax}
+          handleClipOpening={this.handleClipOpening}
+          handleClipClosing={this.handleClipClosing}
+          handleClipCommit={this.handleClipCommit}
+          isClipDisabled={this.isClipDisabled}
+          handleClipOnKeyPress={this.handleClipOnKeyPress}
+          handleClipPercentileMaxValueChange={
+            this.handleClipPercentileMaxValueChange
+          }
+          handleClipPercentileMinValueChange={
+            this.handleClipPercentileMinValueChange
+          }
+        />
+        <UndoRedoReset
+          dispatch={dispatch}
+          isResetDisabled={this.isResetDisabled}
+          resetInterface={this.resetInterface}
+          resettingInterface={resettingInterface}
+          undoDisabled={undoDisabled}
+          redoDisabled={redoDisabled}
+        />
+        <InformationMenu libraryVersions={libraryVersions} />
+      </div>
+    );
+  }
+}
+
+export default MenuBar;

client/src/components/menubar/infoMenu.js

@@ -0,0 +1,61 @@
+// jshint esversion: 6
+import React from "react";
+import { Button, Popover, Menu, MenuItem, Position } from "@blueprintjs/core";
+
+function InformationMenu(props) {
+  const { libraryVersions } = props;
+  return (
+    <div style={{ marginLeft: 10 }} className="bp3-button-group">
+      <Popover
+        content={
+          <Menu>
+            <MenuItem
+              href="https://chanzuckerberg.github.io/cellxgene/faq.html"
+              target="_blank"
+              icon="help"
+              text="FAQ"
+            />
+            <MenuItem
+              href="https://join-cellxgene-users.herokuapp.com/"
+              target="_blank"
+              icon="chat"
+              text="Chat"
+            />
+            <MenuItem
+              href="https://chanzuckerberg.github.io/cellxgene/"
+              target="_blank"
+              icon="book"
+              text="Docs"
+            />
+            <MenuItem
+              href="https://github.com/chanzuckerberg/cellxgene"
+              target="_blank"
+              icon="git-branch"
+              text="Github"
+            />
+            <MenuItem
+              target="_blank"
+              text={`cellxgene v${
+                libraryVersions && libraryVersions.cellxgene
+                  ? libraryVersions.cellxgene
+                  : null
+              }`}
+            />
+            <MenuItem text="MIT License" />
+          </Menu>
+        }
+        position={Position.BOTTOM_RIGHT}
+      >
+        <Button
+          type="button"
+          className="bp3-button bp3-icon-info-sign"
+          style={{
+            cursor: "pointer"
+          }}
+        />
+      </Popover>
+    </div>
+  );
+}
+
+export default InformationMenu;

client/src/components/menubar/undoRedoReset.js

@@ -0,0 +1,64 @@
+// jshint esversion: 6
+import React from "react";
+import { AnchorButton, Tooltip } from "@blueprintjs/core";
+
+function InformationMenu(props) {
+  const {
+    resettingInterface,
+    undoDisabled,
+    redoDisabled,
+    resetInterface,
+    isResetDisabled,
+    dispatch
+  } = props;
+  return (
+    <div style={{ marginLeft: 10 }} className="bp3-button-group">
+      <Tooltip content="Undo" position="left">
+        <AnchorButton
+          type="button"
+          className="bp3-button bp3-icon-undo"
+          disabled={undoDisabled}
+          onClick={() => {
+            dispatch({ type: "@@undoable/undo" });
+          }}
+          style={{
+            cursor: "pointer"
+          }}
+        />
+      </Tooltip>
+      <Tooltip content="Redo" position="left">
+        <AnchorButton
+          type="button"
+          className="bp3-button bp3-icon-redo"
+          disabled={redoDisabled}
+          onClick={() => {
+            dispatch({ type: "@@undoable/redo" });
+          }}
+          style={{
+            cursor: "pointer"
+          }}
+        />
+      </Tooltip>
+      <Tooltip
+        content="Reset cellxgene, clearing all selections"
+        position="left"
+      >
+        <AnchorButton
+          disabled={isResetDisabled()}
+          style={{ marginLeft: 10 }}
+          type="button"
+          loading={resettingInterface}
+          intent="none"
+          icon="refresh"
+          onClick={resetInterface}
+          data-testid="reset"
+          data-testclass={`resetting-${resettingInterface}`}
+        >
+          reset
+        </AnchorButton>
+      </Tooltip>
+    </div>
+  );
+}
+
+export default InformationMenu;

client/src/components/scatterplot/scatterplot.js

@@ -3,7 +3,6 @@
 // https://peterbeshai.com/scatterplot-in-d3-with-voronoi-interaction.html
 
 import React from "react";
-import _ from "lodash";
 import { connect } from "react-redux";
 import { Button, ButtonGroup } from "@blueprintjs/core";
 import _regl from "regl";

client/src/globals.js

@@ -73,114 +73,3 @@ let _API = {
 
 if (window.CELLXGENE && window.CELLXGENE.API) _API = window.CELLXGENE.API;
 export const API = _API;
-
-export const ordinalColors = [
-  "#0ac115",
-  "#c10ab6",
-  "#c1710a",
-  "#0a5ac1",
-  "#c1150a",
-  "#0ab6c1",
-  "#5ac10a",
-  "#710ac1",
-  "#0ac171",
-  "#c10a5a",
-  "#b6c10a",
-  "#150ac1",
-  "#b2ffb7",
-  "#ffb2fa",
-  "#ffddb2",
-  "#b2d4ff",
-  "#ffb7b2",
-  "#b2faff",
-  "#d4ffb2",
-  "#ddb2ff",
-  "#b2ffdd",
-  "#ffb2d4",
-  "#faffb2",
-  "#b7b2ff",
-  "#27a908",
-  "#8b08a9",
-  "#a93a08",
-  "#0877a9",
-  "#a90827",
-  "#08a98b",
-  "#77a908",
-  "#3a08a9",
-  "#08a93a",
-  "#a90877",
-  "#a98b08",
-  "#0827a9",
-  "#00ff0f",
-  "#ff00ef",
-  "#ff8e00",
-  "#0070ff",
-  "#ff0f00",
-  "#00efff",
-  "#70ff00",
-  "#8e00ff",
-  "#00ff8e",
-  "#ff0070",
-  "#efff00",
-  "#0f00ff",
-  "#006606",
-  "#66005f",
-  "#663900",
-  "#002c66",
-  "#660600",
-  "#005f66",
-  "#2c6600",
-  "#390066",
-  "#006639",
-  "#66002c",
-  "#5f6600",
-  "#060066",
-  "#83ff65",
-  "#e165ff",
-  "#ff9565",
-  "#65cfff",
-  "#ff6583",
-  "#65ffe1",
-  "#cfff65",
-  "#9565ff",
-  "#65ff95",
-  "#ff65cf",
-  "#ffe165",
-  "#6583ff",
-  "#009909",
-  "#99008f",
-  "#995500",
-  "#004399",
-  "#990900",
-  "#008f99",
-  "#439900",
-  "#550099",
-  "#009955",
-  "#990043",
-  "#8f9900",
-  "#090099",
-  "#d9fecc",
-  "#f1ccfe",
-  "#fed7cc",
-  "#ccf3fe",
-  "#feccd9",
-  "#ccfef1",
-  "#f3fecc",
-  "#d7ccfe",
-  "#ccfed7",
-  "#feccf3",
-  "#fef1cc",
-  "#ccd9fe",
-  "#47ea51",
-  "#ea47e0",
-  "#eaa247",
-  "#478fea",
-  "#ea5147",
-  "#47e0ea",
-  "#8fea47",
-  "#a247ea",
-  "#47eaa2",
-  "#ea478f",
-  "#e0ea47",
-  "#5147ea"
-];

client/src/reducers/centroidLabel.js

@@ -0,0 +1,44 @@
+import calcCentroid from "../util/centroid";
+
+const initialState = {
+  metadataField: "",
+  categoryIndex: -1,
+  categoryField: "",
+  centroidXY: [-1, -1]
+};
+
+const CentroidLabel = (state = initialState, action, sharedNextState) => {
+  const { categoricalSelection, world, layoutChoice } = sharedNextState;
+  const { metadataField, categoryIndex } = action;
+  const categoryField =
+    categoricalSelection?.[metadataField]?.categoryValues[categoryIndex];
+  switch (action.type) {
+    case "category value mouse hover start":
+      return {
+        ...state,
+        metadataField,
+        categoryIndex,
+        categoryField,
+        centroidXY: null /* calcCentroid(  This function call is computationally heavy and also leading to large GC. Before reimplementation, look into optimization and memoization
+          world,
+          metadataField,
+          categoryField,
+          layoutChoice.currentDimNames
+        ) */
+      };
+
+    case "category value mouse hover end":
+      if (
+        metadataField === state.metadataField &&
+        categoryIndex === state.categoryIndex
+      ) {
+        return initialState;
+      }
+      return state;
+
+    default:
+      return state;
+  }
+};
+
+export default CentroidLabel;

client/src/reducers/colors.js

@@ -1,4 +1,4 @@
-import { createColors } from "../util/stateManager";
+import { ColorHelpers } from "../util/stateManager";
 
 const ColorsReducer = (
   state = {
@@ -17,7 +17,7 @@ const ColorsReducer = (
       const { world } = nextSharedState;
       const colorMode = null;
       const colorAccessor = null;
-      const { rgb, scale } = createColors(world, colorMode);
+      const { rgb, scale } = ColorHelpers.createColors(world, colorMode);
       return {
         ...state,
         colorAccessor,
@@ -29,9 +29,23 @@ const ColorsReducer = (
 
     case "set clip quantiles":
     case "set World to current selection": {
+      const { world: prevWorld, controls: prevControls } = prevSharedState;
+      const resetColorState = ColorHelpers.checkIfColorByDiffexpAndResetColors(
+        prevControls,
+        state,
+        prevWorld
+      );
+      if (resetColorState) {
+        return resetColorState;
+      }
+
       const { colorMode, colorAccessor } = state;
       const { world } = nextSharedState;
-      const { rgb, scale } = createColors(world, colorMode, colorAccessor);
+      const { rgb, scale } = ColorHelpers.createColors(
+        world,
+        colorMode,
+        colorAccessor
+      );
       return {
         ...state,
         rgb,
@@ -40,14 +54,9 @@ const ColorsReducer = (
     }
 
     case "reset colorscale": {
-      const { world } = prevSharedState;
-      const { rgb, scale } = createColors(world);
       return {
         ...state,
-        colorMode: null,
-        colorAccessor: null,
-        rgb,
-        scale
+        ...ColorHelpers.resetColors(prevSharedState.world)
       };
     }
 
@@ -62,7 +71,11 @@ const ColorsReducer = (
       const colorMode = !resetCurrent ? action.type : null;
       const colorAccessor = !resetCurrent ? action.colorAccessor : null;
 
-      const { rgb, scale } = createColors(world, colorMode, colorAccessor);
+      const { rgb, scale } = ColorHelpers.createColors(
+        world,
+        colorMode,
+        colorAccessor
+      );
       return {
         ...state,
         colorMode,
@@ -81,7 +94,11 @@ const ColorsReducer = (
       const colorMode = !resetCurrent ? action.type : null;
       const colorAccessor = !resetCurrent ? action.gene : null;
 
-      const { rgb, scale } = createColors(world, colorMode, colorAccessor);
+      const { rgb, scale } = ColorHelpers.createColors(
+        world,
+        colorMode,
+        colorAccessor
+      );
       return {
         ...state,
         colorMode,
@@ -91,6 +108,19 @@ const ColorsReducer = (
       };
     }
 
+    case "clear differential expression": {
+      const { world: prevWorld, controls: prevControls } = prevSharedState;
+      const resetColorState = ColorHelpers.checkIfColorByDiffexpAndResetColors(
+        prevControls,
+        state,
+        prevWorld
+      );
+      if (resetColorState) {
+        return resetColorState;
+      }
+      return state;
+    }
+
     default: {
       return state;
     }

client/src/reducers/controls.js

@@ -16,7 +16,7 @@ const Controls = (
     diffexpGenes: [],
 
     resettingInterface: false,
-
+    graphInteractionMode: "select",
     opacityForDeselectedCells: 0.2,
     scatterplotXXaccessor: null, // just easier to read
     scatterplotYYaccessor: null,
@@ -138,6 +138,11 @@ const Controls = (
     /*******************************
              User Events
      *******************************/
+    case "change graph interaction mode":
+      return {
+        ...state,
+        graphInteractionMode: action.data
+      };
     case "change opacity deselected cells in 2d graph background":
       return {
         ...state,

client/src/reducers/index.js

@@ -16,37 +16,39 @@ import layoutChoice from "./layoutChoice";
 import responsive from "./responsive";
 import controls from "./controls";
 import resetCache from "./resetCache";
+import centroidLabel from "./centroidLabel";
 
 import undoableConfig from "./undoableConfig";
 
 const Reducer = undoable(
-    cascadeReducers([
-        ["config", config],
-        ["universe", universe],
-        ["world", world],
-        ["layoutChoice", layoutChoice],
-        ["categoricalSelection", categoricalSelection],
-        ["continuousSelection", continuousSelection],
-        ["graphSelection", graphSelection],
-        ["crossfilter", crossfilter],
-        ["colors", colors],
-        ["controls", controls],
-        ["differential", differential],
-        ["responsive", responsive],
-        ["resetCache", resetCache]
-    ]),
-    [
-        "world",
-        "categoricalSelection",
-        "continuousSelection",
-        "graphSelection",
-        "crossfilter",
-        "colors",
-        "controls",
-        "differential",
-        "layoutChoice"
-    ],
-    undoableConfig
+  cascadeReducers([
+    ["config", config],
+    ["universe", universe],
+    ["world", world],
+    ["layoutChoice", layoutChoice],
+    ["categoricalSelection", categoricalSelection],
+    ["continuousSelection", continuousSelection],
+    ["graphSelection", graphSelection],
+    ["crossfilter", crossfilter],
+    ["colors", colors],
+    ["controls", controls],
+    ["differential", differential],
+    ["responsive", responsive],
+    ["centroidLabel", centroidLabel],
+    ["resetCache", resetCache]
+  ]),
+  [
+    "world",
+    "categoricalSelection",
+    "continuousSelection",
+    "graphSelection",
+    "crossfilter",
+    "colors",
+    "controls",
+    "differential",
+    "layoutChoice"
+  ],
+  undoableConfig
 );
 
 const store = createStore(Reducer, applyMiddleware(thunk));

client/src/reducers/undoableConfig.js

@@ -29,7 +29,10 @@ const skipOnActions = new Set([
   "clear all user defined genes",
 
   "get single gene expression for coloring started",
-  "get single gene expression for coloring error"
+  "get single gene expression for coloring error",
+
+  "category value mouse hover start",
+  "category value mouse hover end"
 ]);
 
 /*
@@ -74,7 +77,8 @@ const saveOnActions = new Set([
   "set World to current selection",
   "set clip quantiles",
 
-  "set layout choice"
+  "set layout choice",
+  "change graph interaction mode"
 ]);
 
 /**

client/src/util/centroid.js

@@ -0,0 +1,58 @@
+import quantile from "./quantile";
+
+/*
+  Centroid coordinate calculation
+*/
+const calcMeanCentroid = (world, annoName, annoValue, layoutDimNames) => {
+  const centroid = { x: 0, y: 0, size: 0 };
+  const annoArray = world.obsAnnotations.col(annoName).asArray();
+  const layoutXArray = world.obsLayout.col(layoutDimNames[0]).asArray();
+  const layoutYArray = world.obsLayout.col(layoutDimNames[1]).asArray();
+
+  for (let i = 0, len = annoArray.length; i < len; i += 1) {
+    if (annoArray[i] === annoValue) {
+      centroid.x += layoutXArray[i];
+      centroid.y += layoutYArray[i];
+      centroid.size += 1;
+    }
+  }
+
+  if (centroid[2] !== 0) {
+    centroid.x /= centroid.size;
+    centroid.y /= centroid.size;
+  }
+
+  return [centroid.x, centroid.y];
+};
+
+const calcMedianCentroid = (world, annoName, annoValue, layoutDimNames) => {
+  const centroidX = [];
+  const centroidY = [];
+  let hasFinite = false;
+
+  const annoArray = world.obsAnnotations.col(annoName).asArray();
+  const layoutXArray = world.obsLayout.col(layoutDimNames[0]).asArray();
+  const layoutYArray = world.obsLayout.col(layoutDimNames[1]).asArray();
+
+  for (let i = 0, len = annoArray.length; i < len; i += 1) {
+    if (annoArray[i] === annoValue) {
+      hasFinite =
+        Number.isFinite(layoutXArray[i]) || Number.isFinite(layoutYArray[i])
+          ? true
+          : hasFinite;
+      centroidX.push(layoutXArray[i]);
+      centroidY.push(layoutYArray[i]);
+    }
+  }
+
+  if (hasFinite) {
+    const medianX = quantile([0.5], Float64Array.from(centroidX));
+    const medianY = quantile([0.5], Float64Array.from(centroidY));
+
+    return [medianX, medianY];
+  }
+
+  return null;
+};
+
+export default calcMedianCentroid;

client/src/util/dataframe/dataframe.js

@@ -1,8 +1,19 @@
 import { IdentityInt32Index, isLabelIndex } from "./labelIndex";
 // weird cross-dependency that we should clean up someday...
 import { sortArray } from "../typedCrossfilter/sort";
-import { isTypedArray, isArrayOrTypedArray, callOnceLazy } from "./util";
+import {
+  isTypedArray,
+  isArrayOrTypedArray,
+  callOnceLazy,
+  memoize
+} from "./util";
 import { summarizeContinuous, summarizeCategorical } from "./summarize";
+import {
+  histogramCategorical,
+  hashCategorical,
+  histogramContinuous,
+  hashContinuous
+} from "./histogram";
 
 /*
 Dataframe is an immutable 2D matrix similiar to Python Pandas Dataframe,
@@ -59,6 +70,17 @@ Dataframe
 **/
 
 class Dataframe {
+  /**
+  memoization helpers.
+  **/
+  static __DataframeId__ = 0;
+
+  static __getId() {
+    const id = Dataframe.__DataframeId__;
+    Dataframe.__DataframeId__ += 1;
+    return id;
+  }
+
   /**
   Constructors & factories
   **/
@@ -102,6 +124,7 @@ class Dataframe {
     this.length = nRows; // convenience accessor for row dimension
     this.rowIndex = rowIndex;
     this.colIndex = colIndex;
+    this.__id = Dataframe.__getId();
 
     this.__compile(__columnsAccessor);
   }
@@ -144,7 +167,7 @@ class Dataframe {
     }
   }
 
-  static __compileColumn(column, getOffset, getLabel) {
+  static __compileColumn(column, getRowByOffset, getRowByLabel) {
     /*
       Each column accessor is a function which will lookup data by
       index (ie, is equivalent to dataframe.get(row, col), where 'col'
@@ -172,12 +195,15 @@ class Dataframe {
 
       iget(offset) -- return the value at 'offset'
 
+      ... and more ...
+
     */
     const { length } = column;
+    const __id = Dataframe.__getId();
 
     /* get value by row label */
     const get = function get(rlabel) {
-      return column[getOffset(rlabel)];
+      return column[getRowByOffset(rlabel)];
     };
 
     /* get value by row offset */
@@ -192,7 +218,7 @@ class Dataframe {
 
     /* test for row label inclusion in column */
     const has = function has(rlabel) {
-      const offset = getOffset(rlabel);
+      const offset = getRowByOffset(rlabel);
       return offset >= 0 && offset < length;
     };
 
@@ -212,7 +238,7 @@ class Dataframe {
       if (offset === -1) {
         return undefined;
       }
-      return getLabel(offset);
+      return getRowByLabel(offset);
     };
 
     /*
@@ -224,12 +250,25 @@ class Dataframe {
         : summarizeCategorical(column)
     );
 
+    /*
+    Create histogram bins for this column.  Memoized.
+    */
+    if (isTypedArray(column)) {
+      const mFn = memoize(histogramContinuous, hashContinuous);
+      get.histogram = (bins, domain, by) => mFn(get, bins, domain, by);
+    } else {
+      const mFn = memoize(histogramCategorical, hashCategorical);
+      get.histogram = by => mFn(get, by);
+    }
+
     get.summarize = summarize;
     get.asArray = asArray;
     get.has = has;
     get.ihas = ihas;
     get.indexOf = indexOf;
     get.iget = iget;
+    get.__id = __id;
+
     return get;
   }
 
@@ -239,12 +278,15 @@ class Dataframe {
 
     Use an existing accessor if provided, else compile a new one.
     */
-    const { getOffset, getLabel } = this.rowIndex;
+    const {
+      getOffset: getRowByOffset,
+      getLabel: getRowByLabel
+    } = this.rowIndex;
     this.__columnsAccessor = this.__columns.map((column, idx) => {
       if (accessors[idx]) {
         return accessors[idx];
       }
-      return Dataframe.__compileColumn(column, getOffset, getLabel);
+      return Dataframe.__compileColumn(column, getRowByOffset, getRowByLabel);
     });
   }
 

client/src/util/dataframe/histogram.js

@@ -0,0 +1,135 @@
+/*
+Dataframe histogram
+*/
+import { isTypedArray } from "./util";
+
+function _histogramContinuous(column, bins, min, max) {
+  const valBins = new Array(bins).fill(0);
+  if (!column) {
+    return valBins;
+  }
+  const binWidth = (max - min) / (bins - 1);
+  const colArray = column.asArray();
+  for (let r = 0, len = colArray.length; r < len; r += 1) {
+    const val = colArray[r];
+    if (val <= max && val >= min) {
+      // ensure test excludes NaN values
+      const valBin = (val - min) / binWidth;
+      valBins[valBin] += 1;
+    }
+  }
+  return valBins;
+}
+
+function _histogramContinuousBy(column, bins, min, max, by) {
+  const byMap = new Map();
+  if (!column || !by) {
+    return byMap;
+  }
+  const binWidth = (max - min) / (bins - 1);
+  const byArray = by.asArray();
+  const colArray = column.asArray();
+  for (let r = 0, len = colArray.length; r < len; r += 1) {
+    const byBin = byArray[r];
+    let valBins = byMap.get(byBin);
+    if (valBins === undefined) {
+      valBins = new Array(bins).fill(0);
+      byMap.set(byBin, valBins);
+    }
+    const val = colArray[r];
+    if (val <= max && val >= min) {
+      // ensure test excludes NaN values
+      const valBin = (val - min) / binWidth;
+      valBins[Math.floor(valBin)] += 1;
+    }
+  }
+  return byMap;
+}
+
+function _histogramCategorical(column) {
+  const valMap = new Map();
+  if (!column) {
+    return valMap;
+  }
+  const colArray = column.asArray();
+  for (let r = 0, len = colArray.length; r < len; r += 1) {
+    const valBin = colArray[r];
+    let curCount = valMap.get(valBin);
+    if (curCount === undefined) {
+      curCount = 0;
+    }
+    valMap.set(valBin, curCount + 1);
+  }
+  return valMap;
+}
+
+function _histogramCategoricalBy(column, by) {
+  const byMap = new Map();
+  if (!column || !by) {
+    return byMap;
+  }
+  const byArray = by.asArray();
+  const colArray = column.asArray();
+  for (let r = 0, len = colArray.length; r < len; r += 1) {
+    const byBin = byArray[r];
+    let valMap = byMap.get(byBin);
+    if (valMap === undefined) {
+      valMap = new Map();
+      byMap.set(byBin, valMap);
+    }
+    const valBin = colArray[r];
+    let curCount = valMap.get(valBin);
+    if (curCount === undefined) {
+      curCount = 0;
+    }
+    valMap.set(valBin, curCount + 1);
+  }
+  return byMap;
+}
+
+/*
+Count category occupancy.  Optional group-by category.
+*/
+export function histogramCategorical(column, by) {
+  if (by && isTypedArray(by)) {
+    throw new Error("Group by column must be categorical");
+  }
+  return by
+    ? _histogramCategoricalBy(column, by)
+    : _histogramCategorical(column);
+}
+
+/*
+Memoization hash for histogramCategorical()
+*/
+export function hashCategorical(column, by) {
+  if (by) {
+    return `${column.__id}:${by.__id}`;
+  }
+  return `${column.__id}:`;
+}
+
+/*
+Bin counts for continuous/scalar values, with optional group-by category.
+Values outside domain are ignored.
+*/
+export function histogramContinuous(column, bins = 40, domain = [0, 1], by) {
+  if (by && isTypedArray(by)) {
+    throw new Error("Group by column must be categorical");
+  }
+  const [min, max] = domain;
+  return by
+    ? _histogramContinuousBy(column, bins, min, max, by)
+    : _histogramContinuous(column, bins, min, max);
+}
+
+/*
+Memoization hash for histogramContinuous
+*/
+export function hashContinuous(column, bins = "", domain = [0, 0], by) {
+  const [min, max] = domain;
+  if (by) {
+    return `${column.__id}:${bins}:${min}:${max}:${by.__id}`;
+  }
+  return `${column.__id}::${bins}:${min}:${max}`;
+}

client/src/util/dataframe/util.js

@@ -5,6 +5,10 @@ Private utility code for dataframe
 export { isTypedArray, isArrayOrTypedArray } from "../typeHelpers";
 
 export function callOnceLazy(f) {
+  /*
+  call function once, and save the result, regardless of arguments (this is not
+  the same as typical memoization).
+  */
   let value;
   let calledOnce = false;
   const result = function result(...args) {
@@ -14,6 +18,25 @@ export function callOnceLazy(f) {
     }
     return value;
   };
-
   return result;
 }
+
+export function memoize(fn, hashFn) {
+  /* 
+  function memoization, with user-provided hash.  hashFn must return a
+  key which will be unique as a Map key (ie, obeys "sameValueZero" algorithm
+  as defined in the JS spec).  For more info on hash key, see:
+  https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map#Key_equality
+  */
+  const cache = new Map();
+  const wrap = function wrap(...args) {
+    const key = hashFn(...args);
+    if (cache.has(key)) {
+      return cache.get(key);
+    }
+    const result = fn(...args);
+    cache.set(key, result);
+    return result;
+  };
+  return wrap;
+}

client/src/util/stateManager/colorHelpers.js

@@ -15,7 +15,7 @@ create new colors state object.   Paramters:
     "color by continuous metadata", "color by categorical metadata"
   -
 */
-function createColors(world, colorMode = null, colorAccessor = null) {
+export function createColors(world, colorMode = null, colorAccessor = null) {
   switch (colorMode) {
     case "color by categorical metadata": {
       return createColorsByCategoricalMetadata(world, colorAccessor);
@@ -117,4 +117,26 @@ function createColorsByExpression(world, accessor) {
   return { rgb, scale };
 }
 
-export default createColors;
+export const resetColors = world => {
+  const { rgb, scale } = createColors(world);
+  return {
+    colorMode: null,
+    colorAccessor: null,
+    rgb,
+    scale
+  };
+};
+
+export const checkIfColorByDiffexpAndResetColors = (
+  prevControls,
+  state,
+  prevWorld
+) => {
+  if (prevControls.diffexpGenes.includes(state.colorAccessor)) {
+    return {
+      ...state,
+      ...resetColors(prevWorld)
+    };
+  }
+  return null;
+};

client/src/util/stateManager/index.js

@@ -14,7 +14,7 @@ This is all VERY tightly integrated with reducers and actions, and
 exists to support those concepts.
 */
 
-export { default as createColors } from "./colorHelpers";
+export * as ColorHelpers from "./colorHelpers";
 export * as Universe from "./universe";
 export * as World from "./world";
 export * as WorldUtil from "./worldUtil";

docs/data.md

@@ -1,29 +1,94 @@
----
-layout: default
-title: data
-description: Data
----
+---
+layout: default
+title: data
+description: Data
+---
+
+# Using `cellxgene prepare`
+
+#### What is `cellxgene prepare`?
+
+`prepare` offers an easy command line interface (CLI) to preliminarily wrangle your data into the required format for previewing it with `cellxgene`.
+
+#### What is `cellxgene prepare` _not_?
+
+`cellxgene prepare` is not meant as a way to formally process or analyze your data. It's simply a utility for quickly wrangling your data into cellxgene-compatible format and computing a "vanilla" embedding so you can try out `cellxgene` and get a general sense of a dataset.
+
+#### What input formats does it accept?
+
+Currently, we accept `h5ad` and `loom` files, as well as `10x` directories, and are hoping to accept more formats in the future.
+
+While we'd like to support quick conversion from seurat and bioconductor, these packages don't currently output a python-parseable intermediate file type. In the meantime, you might check out the [converters](https://satijalab.org/seurat/v3.0/conversion_vignette.html) that are under early development.
+
+#### What can `cellxgene prepare` do?
+
+`prepare` uses scanpy to:
+
+- Handle simple data normalization (from a [recipe](https://www.pydoc.io/pypi/scanpy-0.2.3/autoapi/preprocessing/recipes/index.html))
+- Do basic preprocessing to run PCA and compute the neighbor graph
+- Infer clusters
+- Reduce dimensionality to generate embeddings.  
+  You can control which steps to run and their methods (when applicable), via the CLI. The CLI also includes options for computing QC metrics, enforcing matrix sparcity, specifying index names, and plotting output.
+
+**To see a full list of available arguments and options, run `cellxgene prepare --help`.**
+
+#### How do I use `cellxgene prepare`?
+
+As a quick example, let's construct a command to use `prepare` to take a raw expression matrix and generate a processed `h5ad` ready to visualize with cellxgene.
+
+We'll start off using the raw data from the pbmc3k dataset. This dataset is described [here](https://icb-scanpy.readthedocs-hosted.com/en/stable/api/scanpy.datasets.pbmc3k.html), and is available as part of the scanpy API. For this example, we'll assume this raw data is stored in a file called `pbmc3k-raw.h5ad`.
+
+Our `prepare` compose our command looks like this:  
+<img src="prepare-cmd-example.jpg" width="700" />
+
+Let's look at what `prepare` is doing to our data, and how each step relates to the command above. You can see a walkthrough of what's going on under the hood for this example in [this notebook](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/pbmc3k-prepare-example.ipynb).
+
+**1 - Compute quality control metrics and store this in our `AnnData` object for later inspection (A)**  
+**2 - Normalize the expression matrix using a basic preprocessing recipe (B)**  
+**3 - Do some preprocessing to run PCA and compute the neighbor graph (auto)**  
+**4 - Infer clusters with the Louvain algorithm and store these labels to visualize later (auto)**  
+**5 - Compute and store umap and tsne embeddings (C)**  
+**6 - Write results to file (D)**
+
+# Example datasets to use with cellxgene
+
 
-
-# data vignette: how to use cellxgene prepare
-
-#### coming soon!
-
-# example datasets to use with cellxgene
-
-### Examination of single cells from primary human pancreas tissue
-cells: 2,544  
-tissue(s): pancreas  
-data: [Human Cell Atlas Data Portal](https://prod.data.humancellatlas.org/explore/projects?filter=%5B%7B%22facetName%22%3A%22organ%22%2C%22terms%22%3A%5B%22pancreas%22%5D%7D%2C%7B%22facetName%22%3A%22project%22%2C%22terms%22%3A%5B%22Single+cell+transcriptome+analysis+of+human+pancreas%22%5D%7D%5D)  
-paper: [Enge, Martin, et al.](https://www.cell.com/cell/fulltext/S0092-8674(17)31053-X?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS009286741731053X%3Fshowall%3Dtrue)
-
-### Tabula Muris
-cells: 53,800  
-tissue(s): muscle, pancreas, bone, large intestine, heart, brain, fat, mammary gland, tongue , diaphragm, bladder, spleen, thymus, lung , skin, liver, trachea, kidney  
-data: [Tabula Muris Data](https://github.com/czbiohub/tabula-muris-vignettes/tree/master/data)  
-paper: [Tabula Muris Consortium](https://www.nature.com/articles/s41586-018-0590-4)  
-
-### Transcriptional profiling of 1.3 million brain cells
-cells: 1,330,000  
-tissue(s): brain  
-data: [10x Genomics](https://community.10xgenomics.com/t5/10x-Blog/Our-1-3-million-single-cell-dataset-is-ready-to-download/ba-p/276)  
+**To download and use these datasets, run:**  
+`curl -O [URL]`  
+`unzip [filename.zip]`  
+`cellxgene launch [filename.h5ad] --open`
+
+### Peripheral blood mononuclear cells
+
+Healthy human PBMCs (10X).
+
+- Source: [10X genomics](https://support.10xgenomics.com/single-cell-gene-expression/datasets/1.1.0/pbmc3k)
+- Cells: 2,638
+- File size: 19MB
+- [Raw data](http://cf.10xgenomics.com/samples/cell-exp/1.1.0/pbmc3k/pbmc3k_filtered_gene_bc_matrices.tar.gz)
+- [Processing](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/pbmc3k-processing.ipynb)
+- Download: `curl -O https://cellxgene-example-data.czi.technology/pbmc3k.h5ad.zip`
+
+### Tabula muris
+
+20 organs and tissues from healthy mice (Smart-Seq2).  
+Rich metadata and annotations.
+
+- Source: [bioRxiv, CZBiohub](https://www.biorxiv.org/content/10.1101/237446v2)
+- Cells: 45,423
+- File size: 174MB
+- [Raw data](https://figshare.com/projects/Tabula_Muris_Transcriptomic_characterization_of_20_organs_and_tissues_from_Mus_musculus_at_single_cell_resolution/27733)
+- [Processing](https://github.com/chanzuckerberg/cellxgene-vignettes/blob/master/dataset-processing/tabula-muris-processing.ipynb)
+- Download: `curl -O https://cellxgene-example-data.czi.technology/tabula-muris.h5ad.zip`
+
+### Tabula muris senis
+
+22 organs and tissues from healthy mice at ages 3mo, 18mo, 21mo, and 24mo (Smart-Seq2).  
+Rich metadata and annotations.
+
+- Source: [bioRxiv, CZBiohub](https://www.biorxiv.org/content/10.1101/661728v1)
+- Cells: 81,478
+- File size: 3.9GB
+- Raw data [geo link coming soon!]
+- [Processing](https://www.biorxiv.org/content/10.1101/661728v1)
+- Download: `curl -O https://cellxgene-example-data.czi.technology/tabula-muris-senis.h5ad.zip`

makefile

@@ -1,7 +1,7 @@
 BUILDDIR := build
 CLIENTBUILD := $(BUILDDIR)/client
 SERVERBUILD := $(BUILDDIR)/server
-CLEANFILES :=  $(BUILDDIR)/ client/build dist cellxgene.egg-info
+CLEANFILES :=  $(BUILDDIR)/ client/build build dist cellxgene.egg-info
 
 PART ?= patch
 
@@ -84,6 +84,9 @@ release-directly-to-prod : dev-env pydist twine-prod
 dev-env :
 	pip install -r server/requirements-dev.txt
 
+gui-env : dev-env
+	pip install -r server/requirements-gui.txt
+
 # give PART=[major, minor, part] as param to make bump
 bump :
 	bumpversion --config-file .bumpversion.cfg $(PART)
@@ -130,3 +133,27 @@ uninstall :
 	pip uninstall -y cellxgene || :
 
 .PHONY : install install-dev install-release-test install-release uninstall
+
+# GUI
+
+build-assets :
+	pyside2-rcc server/gui/cellxgene.qrc -o server/gui/cellxgene_rc.py
+
+gui-spec-osx : clean-lite gui-env
+	pip install -e .[gui]
+	pyi-makespec -D -w --additional-hooks-dir server/gui/ -n cellxgene  --add-binary='/System/Library/Frameworks/Tk.framework/Tk':'tk' --add-binary='/System/Library/Frameworks/Tcl.framework/Tcl':'tcl'  --add-data server/app/web/templates/:server/app/web/templates/ --add-data server/app/web/static/:server/app/web/static/ --icon server/gui/images/cxg_icons.icns server/gui/main.py
+	mv cellxgene.spec cellxgene-osx.spec
+
+gui-spec-windows : clean-lite dev-env
+	pip install -e .[gui]
+	pyi-makespec -D -w --additional-hooks-dir server/gui/ -n cellxgene --add-data server/app/web/templates;server/app/web/templates --add-data server/app/web/static;server/app/web/static --icon server/gui/images/icon.ico server/gui/main.py
+	mv cellxgene.spec cellxgene-windows.spec
+
+gui-build-osx : clean-lite
+	pyinstaller --clean cellxgene-osx.spec
+
+gui-build-windows : clean-lite
+	pyinstaller --clean cellxgene-windows.spec
+
+.PHONY : build-assets gui-build-osx gui-build-windows gui-build-osx gui-build-windows
+

server/__init__.py

@@ -0,0 +1 @@
+__version__ = ""__version__ = "0.11.1""

server/app/driver/driver.py

@@ -28,7 +28,6 @@ class CXGDriver(metaclass=ABCMeta):
     def _get_default_config():
         return {
             "layout": None,
-            "diffexp": None,
             "max_category_items": None,
             "diffexp_lfc_cutoff": None
         }
@@ -38,14 +37,12 @@ class CXGDriver(metaclass=ABCMeta):
         features = {
             "cluster": {"available": False},
             "layout": {"obs": {"available": False}, "var": {"available": False}},
-            "diffexp": {"available": False},
+            "diffexp": {"available": True, "interactiveLimit": 50000}
         }
         # TODO - Interactive limit should be generated from the actual available methods see GH issue #94
         if self.config["layout"]:
             # TODO handle "var" when gene layout becomes available
             features["layout"]["obs"] = {"available": True, "interactiveLimit": 50000}
-        if self.config["diffexp"]:
-            features["diffexp"] = {"available": True, "interactiveLimit": 50000}
         return features
 
     @abstractmethod

server/app/rest_api/rest.py

@@ -1,9 +1,10 @@
 from http import HTTPStatus
-import pkg_resources
 import warnings
 
 from flask import Blueprint, current_app, jsonify, make_response, request
 from flask_restful import Api, Resource
+from server import __version__ as cellxgene_version
+from anndata import __version__ as anndata_version
 
 from server.app.util.constants import (
     Axis,
@@ -59,16 +60,15 @@ class ConfigAPI(Resource):
                     },
                 ],
                 "displayNames": {
-                    "engine": f"cellxgene Scanpy engine version {pkg_resources.get_distribution('cellxgene').version}",
+                    "engine": f"cellxgene Scanpy engine version ",
                     "dataset": current_app.config["DATASET_TITLE"],
                 },
                 "parameters": {
-                    "max_category_items": current_app.data.config["max_category_items"]
+                    "max-category-items": current_app.data.config["max_category_items"]
                 },
                 "library_versions": {
-                    "scanpy": pkg_resources.get_distribution("scanpy").version,
-                    "cellxgene": pkg_resources.get_distribution("cellxgene").version,
-                    "anndata": pkg_resources.get_distribution("cellxgene").version
+                    "cellxgene": cellxgene_version,
+                    "anndata": anndata_version
                 }
             }
         }

server/app/scanpy_engine/scanpy_engine.py

@@ -43,7 +43,6 @@ class ScanpyEngine(CXGDriver):
     def _get_default_config():
         return {
             "layout": [],
-            "diffexp": "ttest",
             "max_category_items": 100,
             "obs_names": None,
             "var_names": None,

server/cli/cli.py

@@ -5,7 +5,7 @@ from .prepare import prepare
 
 
 @click.group(name="cellxgene", context_settings=dict(max_content_width=85))
-@click.version_option(version="0.10.1", prog_name="cellxgene", message="[%(prog)s] Version %(version)s")
+@click.version_option(version="0.11.1", prog_name="cellxgene", message="[%(prog)s] Version %(version)s")
 def cli():
     pass
 

server/cli/launch.py

@@ -1,4 +1,5 @@
 import errno
+import functools
 import logging
 from os import devnull
 from os.path import splitext, basename, getsize
@@ -13,30 +14,56 @@ from server.app.util.errors import ScanpyFileError
 from server.app.util.utils import custom_format_warning
 from server.utils.utils import find_available_port, is_port_available
 
-
 # anything bigger than this will generate a special message
-BIG_FILE_SIZE_THRESHOLD = 100 * 2**20  # 100MB
+BIG_FILE_SIZE_THRESHOLD = 100 * 2 ** 20  # 100MB
+
+
+def common_args(func):
+    """
+    Decorator to contain CLI args that will be common to both CLI and GUI: title and engine args.
+    """
+    @click.option("--title", "-t", help="Title to display (if omitted will use file name).")
+    @click.option(
+        "--layout",
+        "-l",
+        default=[],
+        multiple=True,
+        show_default=True,
+        help="Layout name, eg, 'umap'."
+    )
+    @click.option("--obs-names", default=None, metavar="", help="Name of annotation field to use for observations.")
+    @click.option("--var-names", default=None, metavar="", help="Name of annotation to use for variables.")
+    @click.option(
+        "--max-category-items",
+        default=1000,
+        metavar="",
+        show_default=True,
+        help="Categories with more distinct values than this will not be displayed.",
+    )
+    @click.option(
+        "--diffexp-lfc-cutoff",
+        default=0.01,
+        show_default=True,
+        help="Relative expression cutoff used when selecting top N differentially expressed genes",
+    )
+    @functools.wraps(func)
+    def wrapper(*args, **kwargs):
+        return func(*args, **kwargs)
+    return wrapper
+
+
+def parse_engine_args(layout, obs_names, var_names, max_category_items, diffexp_lfc_cutoff):
+    return {
+        "layout": layout,
+        "max_category_items": max_category_items,
+        "diffexp_lfc_cutoff": diffexp_lfc_cutoff,
+        "obs_names": obs_names,
+        "var_names": var_names,
+    }
 
 
 @click.command()
 @click.argument("data", metavar="<data file>", type=click.Path(exists=True, file_okay=True, dir_okay=False))
-@click.option(
-    "--layout",
-    "-l",
-    default=[],
-    multiple=True,
-    show_default=True,
-    help="Layout name, eg, 'umap'."
-)
-@click.option(
-    "--diffexp",
-    "-d",
-    type=click.Choice(["ttest"]),
-    default="ttest",
-    show_default=True,
-    help="Method for differential expression.",
-)
-@click.option("--title", "-t", help="Title to display (if omitted will use file name).", metavar="")
 @click.option(
     "--verbose",
     "-v",
@@ -57,22 +84,7 @@ BIG_FILE_SIZE_THRESHOLD = 100 * 2**20  # 100MB
 )
 @click.option("--port", "-p", help="Port to run server on, if not specified cellxgene will find an available port.",
               metavar="", show_default=True)
-@click.option("--obs-names", default=None, metavar="", help="Name of annotation field to use for observations.")
-@click.option("--var-names", default=None, metavar="", help="Name of annotation to use for variables.")
 @click.option("--host", default="127.0.0.1", help="Host IP address")
-@click.option(
-    "--max-category-items",
-    default=100,
-    metavar="",
-    show_default=True,
-    help="Limits the number of categorical annotation items displayed.",
-)
-@click.option(
-    "--diffexp-lfc-cutoff",
-    default=0.01,
-    show_default=True,
-    help="Relative expression cutoff used when selecting top N differentially expressed genes",
-)
 @click.option(
     "--scripts",
     default=[],
@@ -80,21 +92,21 @@ BIG_FILE_SIZE_THRESHOLD = 100 * 2**20  # 100MB
     help="Additional script files to include in html page",
     show_default=True,
 )
+@common_args
 def launch(
         data,
-        layout,
-        diffexp,
-        title,
         verbose,
         debug,
-        obs_names,
-        var_names,
         open_browser,
         port,
         host,
+        layout,
+        obs_names,
+        var_names,
         max_category_items,
         diffexp_lfc_cutoff,
-        scripts,
+        title,
+        scripts
 ):
     """Launch the cellxgene data viewer.
     This web app lets you explore single-cell expression data.
@@ -107,6 +119,7 @@ def launch(
 
     > cellxgene launch <your data file> --title <your title>"""
 
+    e_args = parse_engine_args(layout, obs_names, var_names, max_category_items, diffexp_lfc_cutoff)
     # Startup message
     click.echo("[cellxgene] Starting the CLI...")
 
@@ -121,22 +134,22 @@ def launch(
     else:
         warnings.formatwarning = custom_format_warning
 
-    if scripts:
-        click.echo(r"""
-/ / /\ \ \__ _ _ __ _ __ (_)_ __   __ _
-\ \/  \/ / _` | '__| '_ \| | '_ \ / _` |
- \  /\  / (_| | |  | | | | | | | | (_| |
-  \/  \/ \__,_|_|  |_| |_|_|_| |_|\__, |
-                                  |___/
-The --scripts flag is intended for developers to include google analytics etc. You could be opening yourself to a
-security risk by including the --scripts flag. Make sure you trust the scripts that you are including.
-        """)
-        scripts_pretty = ", ".join(scripts)
-        click.confirm(f"Are you sure you want to inject these scripts: {scripts_pretty}?", abort=True)
-
     if not verbose:
         sys.tracebacklimit = 0
 
+    if scripts:
+        click.echo(r"""
+    / / /\ \ \__ _ _ __ _ __ (_)_ __   __ _
+    \ \/  \/ / _` | '__| '_ \| | '_ \ / _` |
+     \  /\  / (_| | |  | | | | | | | | (_| |
+      \/  \/ \__,_|_|  |_| |_|_|_| |_|\__, |
+                                      |___/
+    The --scripts flag is intended for developers to include google analytics etc. You could be opening yourself to a
+    security risk by including the --scripts flag. Make sure you trust the scripts that you are including.
+            """)
+        scripts_pretty = ", ".join(scripts)
+        click.confirm(f"Are you sure you want to inject these scripts: {scripts_pretty}?", abort=True)
+
     if not title:
         file_parts = splitext(basename(data))
         title = file_parts[0]
@@ -179,17 +192,8 @@ security risk by including the --scripts flag. Make sure you trust the scripts t
     mpl.use("TkAgg")
     from server.app.scanpy_engine.scanpy_engine import ScanpyEngine
 
-    args = {
-        "layout": layout,
-        "diffexp": diffexp,
-        "max_category_items": max_category_items,
-        "diffexp_lfc_cutoff": diffexp_lfc_cutoff,
-        "obs_names": obs_names,
-        "var_names": var_names,
-    }
-
     try:
-        server.attach_data(ScanpyEngine(data, args), title=title)
+        server.attach_data(ScanpyEngine(data, e_args), title=title)
     except ScanpyFileError as e:
         raise click.ClickException(f"{e}")
 

server/gui/browser.py

@@ -11,10 +11,6 @@ WindowUtils = cef.WindowUtils()
 # OS differences
 # noinspection PyUnresolvedReferences
 CefWidgetParent = QWidget
-if LINUX:
-    # noinspection PyUnresolvedReferences
-    CefWidgetParent = QX11EmbedContainer
-
 
 class CefWidget(CefWidgetParent):
     def __init__(self, parent=None):

server/gui/cellxgene.qrc

@@ -0,0 +1,8 @@
+<!DOCTYPE RCC><RCC version="1.0">
+<qresource>
+    <file alias="logo.png">images/cellxgene_logo.png</file>
+    <file alias="collapsed.svg">images/properties_contract.svg</file>
+    <file alias="expanded.svg">images/properties_expand.svg</file>
+    <file alias="icon.png">images/properties_expand.svg</file>
+</qresource>
+</RCC>

server/gui/cellxgene.spec

@@ -0,0 +1,29 @@
+# -*- mode: python -*-
+
+block_cipher = None
+
+
+a = Analysis(['main.py'],
+             pathex=['/Users/charlotteweaver/Documents/Git/cellxgene/server/gui'],
+             hookspath=["/Users/charlotteweaver/Documents/Git/cellxgene/server/gui/"],
+             win_no_prefer_redirects=False,
+             win_private_assemblies=False,
+             cipher=block_cipher,
+             noarchive=False)
+
+pyz = PYZ(a.pure, a.zipped_data,
+             cipher=block_cipher)
+
+exe = EXE(pyz,
+          a.scripts,
+          a.binaries,
+          a.zipfiles,
+          a.datas,
+          [],
+          name='cellxgene',
+          debug=False,
+          bootloader_ignore_signals=False,
+          strip=False,
+          upx=True,
+          runtime_tmpdir=None,
+          console=True )

server/gui/cellxgene_rc.py

@@ -0,0 +1,440 @@
+# -*- coding: utf-8 -*-
+
+# Resource object code
+#
+# Created: Wed Jun 19 15:02:04 2019
+#      by: The Resource Compiler for PySide2 (Qt v5.12.3)
+#
+# WARNING! All changes made in this file will be lost!
+
+from PySide2 import QtCore
+
+qt_resource_data = b"\
+\x00\x00\x0d\xde\
+\x89\
+PNG\x0d\x0a\x1a\x0a\x00\x00\x00\x0dIHDR\x00\
+\x00\x01;\x00\x00\x00j\x08\x06\x00\x00\x00\xc3y\xf6!\
+\x00\x00\x00\x09pHYs\x00\x00\x0b\x13\x00\x00\x0b\x13\
+\x01\x00\x9a\x9c\x18\x00\x00\x00\x01sRGB\x00\xae\xce\
+\x1c\xe9\x00\x00\x00\x04gAMA\x00\x00\xb1\x8f\x0b\xfc\
+a\x05\x00\x00\x0dsIDATx\x01\xed\xdd\x7fn\
+\x1b\xc7\x15\x07\xf07C\xdbQ\x93\x00f\xfe\x88e7\
+\x05L\x9d J/PJ9@\xe4^@\x12\x90H\
+(\x0aT\xce\x09,\x9d\xc0\xf2\x1fE \xb9\x80\xd9\x0b\
+\xd4\xf2\x01,2'\xb0|\x023@\x0bG\xea\x1fQ\
+\xff\x09\x04\xd1;\xaf\xef-\xc9X\xa48\xb3?\xb8\xfc\
+e~?\x80b\x85\xbb\xab%g\xdf\xbe\x9d\x9d\x99\x1d\
+\x12\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
+\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
+\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
+\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
+\x00\x00\xc0\x9c24\x22\xa7\xab\x87L3l\xf1xk\
+de\x03\x00\xe3g\x09\x00`\x0e \xd9\x01\xc0\x5c@\
+\xb2\x03\x80\xb9\x80d\x07\x00s\x01\xc9\x0e\x00\xe6\xc2\x0d\
+\x02\x18\xe0m\xf5i\xd5Z\xae\xfb\x96KW{\xed\xee\
+\xf1\xd6f\xef6?T\xac-\xbd!\xbf\xa6\xf4r/\
+\x11\xc0\x04\xccD\xb2\xbb\xf3\xf2;\x1a\x85\xb3\xaf\x9f\x12\
+\x00\xcc\x07\xdc\xc6\x02\xc0\x5c@\xb2\x03\x80\xb9\x80d\x07\
+\x00s\x01\xc9\x0e\x00\xe6\x02\x92\x1d\x00\xcc\x05$;\x00\
+\x98\x0b31\xf4\x04CD\x00`X\xa8\xd9\x01\xc0\x5c\
+\xc0\x13\x14c\xf0\xa6\xfa\xac\xfcI\xa9\xb5\xee\x98\x96\xad\
+\xa1\x0a3W\x88\xcc\xb91t.\xbf\xcb\xbf\xa6\x11E\
+\xd1\x8b{\x8d\xbf4i\x08\xfa\xd4\x831\xbc&?\xf7\
+\xe5\xefW\x88\xb8,\x7f\xbb\xa9\xcb\x9c\xa3\xd7\xcc\xe6\xe8\
+^\xe3\xbb\x06\xc1o\xbaef-}\xa9\xc7E\xcbk\
+\xd019\xad\x1e\xac9k\xca\x83\xfeF\xc9\xb5N\xee\
+4\xfez\x92f\x7f\x1a\x0b\xb7JQ\xb5\xe4\x5c\xb5\xff\
+8\xe9~\x1d\xd1kr\xb6\x91\xf68\xbd]=\xdc\xf0\
+.tQ\xe3jL\xf5\xc6\x07-wc0b\xfeQ\
+\x02\xa46L\xfc%\xc5^dm\xe3\x8b\x97\xdf\x1e\xd1\
+\x04a\xf2N\x8f\x22&\xef\x8c\x1f\x9f2v\x97\x8cY\
+O\xb3\xbe\x04\x9e\x9e`\x9bY\x82NO\x9eO\xed\xbb\
+\x1d&~(\x05^N\xb1I\xd3\x92\xa9}~\xfc\xdd\
+^h\xa5\x0f\xfdq\xb1\xff|}\xb0v\x83\xe9q\xfb\
+\xc4\xf4\x93\xb2\xda\xd5\xb2\x92x~Eq\x82\xb8\xce\x11\
+\xef\xdd;\xde\xde\x0d\xfd\x9d<\xc7\x89$v\x16_~\
+\xfb\xcf\xd0J\xa1\xf3L\xf6\xf5\xfd\xdd\xe3\xed\xfd\xf6\xb1\
+t\xcf\x92?+\xef\x7f~\xbc\xfd=e\xf0\xdf\xd5\xa7\
+\x8f\x8a\x8e\xbdQ\xc1m\xec\x88\x9c}}\xb8S\xb2\xa5\
+Wi\x13\x9db\xa6\xaa&\x0b\x0d\xa04\xebkr\xf9\
+\xd8^\xbe\x92\x93m7e\xb0\xa9\x8a\xae/'\xc9\x1b\
+\xdd\x9e\xa6\x88&\x04\xca)\xcb\xb6g\xabO\x1f\xdf`\
+\xf3<\xe9\xe4WZVg\xab\x87\xcf\x0cS\xe2\xba>\
+\x9al>\xb1\xad7Y\x8f\x13\xb1\xab\xe9\xbe\xf3\x96\x8b\
+\xeeK\xe3\xb0}\xd12\x95\xa4\xf5\x1d\x99\x87\xb2~\x9d\
+R\xd0\xd8\xd1\x0b\xc0,\xc5\xdeDncC\xcf\xba\x0e\
+\xea\x8c\x98\xb5gcO\xf5j\xc7\xbcK9i@H\
+\xc2\xa3\xd0\x15\xf0\xac\xfa\xf7e\xb2\xa5z\x86@\xeb\xa7\
+\xb5\xb0\xba\x04\xdd\xca\xb0\xb7\xcfE\xd0\x93\x92\xb9\xb5{\
+V=\x5c\xb9\xd3\xd8JuK\xd8\xd5\xaeQ^\xd6%\
+14\xee\xf4\xd56\xfb\x9dvj\x22\x94\x81\x94\xf1F\
+\xde{\xa0\xd3\xea?\xd6\xc9\xbaZ\xde\xdb\x1c\xdd\xf7\xc7\
+\xb6\xb5,\x09oe\xa9\xb1y\x9ee[\xa9E\xads\
+\xc6$\xad\x17\xdc\x9fW\x0f\x1ej\x8d\xd0\xb7N\xbb\xbc\
+\xad&\xc5\x0a\xe53\x91\xd8C\xcd\xae`oW\x7f\xd8\
+\x90\x90\xd9\xa5!i\xc2\x93`\xa8\x0e\xdc\x87\x04\x1b\xdb\
+\x1b\xcf\x87Ht]\x1at\xcfi\xc24!\xc8I\xa6\
+'W\x99-\xd5%\xe1-\xa7\xdd\xf6\xfd\x89g*\x9a\
+\x18B\xb5\xe2\xa2\x8eM\xfa\xf7\xf6\xb4\xaa\x89\x8e\x86\xb7\
+\xfc\xa9\xbdLU\xdb\xefS\xa1\x1c\x0c\x19\x7f\x19^)\
+o\x1a\x8e\xc6\xde\xf10\xb5\xf9\xac\x90\xec\x0a\x14\x07\x02\
+\xd9<A9P\xa9T\x1a\xf8\xb7\xe2v@2\x15*\
+\xc6\xf2\xe9\xea\xc1.MH\xbb\x86\xda\x93\x10R'\xbc\
+A'\x9e^$\xb4=n\xd0\xfaE\x1e\x9b4\xda\xed\
+d\xc5\xd0[L\xdf\xc5o\x04\xca\xbe}\x15\x1c{K\
+\xd2\x0c\x93\xa9\x96=\x0c$\xbb\x22Y\xaa\x86\x02A\xda\
+}\xce\xb51\x9b\x1c?\xd0\x1f\xc7\x1cl|\xd6[\x8a\
+\xfe\xa0\x8bk\x0b\x09\xed\x80\xb2]\xc3Q\xb4\xe9\x5c\xb4\
+\x92f?\xf2\xcev\xc6y\x85\xbdJ{1\xe32\xe9\
+\x95\x98\xf0\xbc5\x0c\xf9\xac_\xbc\xdc\xbe\xd6\xeb\xd7\xae\
+\xd5\xf5\xad\xdb\xe7\xea\xf1\xd1\xb2\x8b\x7fgnR\x0ei\
+\xf6w\xf58\xc9\xcf\xa6\xdc\xb3\x06o\xdf}\x17\xbf$\
+\xda\x99\x94-\x1e\xe4\x9d[{\xad\xec\xe3v\xb6\xa46\
+h\xf9\x0c\xd2\xd1\xf60\xed\xbe\x98\xcc\xdf\xc6\x15{\x18\
+zR\xa0\xc4\x9a\x03\xb7V\xee\xd5{\x86(\x1c\xbd]\
+=\xd0\x1e*\xefvrBW\xe5\x9f\xc6o\xffo\xdc\
+F\xb8\x13\x9d\x8f\xee\xd6\xb7\x1f\xf4\xbd\x98\xb4\x9f\xf2\x82\
+mim\xa8F\x13\xa0=\x99\xf2\xfe\xa8\xef\xfdu\x13\
+\xde\xb56\xbcP\xa2[\xacoo\x0c\xdaG\x89K\xeb\
+\x1c,6jF\x1c\xc9\xf1\xe9iCj\xc8\xbejr\
+\x5c\xebr\x92W(\x83\xe4Z$\xef\xc9q\xda\xed{\
+\xb1\xf6\xf3\xca\xe1\xbe$\x8b\x9d\x81[\xb4/~\x95,\
+\xed\x5c\xce\x91\xf4\xeeo\xd5\xfa^>\x92\xda\xfcm)\
+\xbf5\xff\x96\xe6\xcb\xfeW\xda\xb5\xba\x10>Z\xcc\x18\
+{rH\xca\xbf\xb3\x97\x1b\xf2\xab\xb7\x8d\xb0(3[\
+\xb3\xd3\xce\x85,?\xa3\xf6\xef\xb8\x16b*\xbe\xe5\xcc\
+\xee\xc5\xa0\xb1Xz\xa2k\x8d\xc2\xbf\x1d\xf7^a\x13\
+\xae\xac\xce\xb9\x81C\x07\xdaC#\xfc\xb5\x14\x09\xba?\
+\xd1\x04\xe9\xfbKS\xc3\xd3Z\xc0\xe0DG'\xbeD\
+\xf7\x8bl#\x89\xaeJ!lw\x07%\x11}\xcd\x19\
+J\xac\x09]\x95\x14\x0b\x9aX\x17=CU>\xe2\x9b\
+\xbb\x14 5\xae5J\xaf9 \xd1\xc5$N\x9e\x84\
+6\xd4\xf1\xa0\xfd\xafI[\xde7\xa1m\xc2\xb1\xe7\xad\
+\xb5\x1a\x09\xf2\xb1\xc4\x1enc\x0br\xc3r5\xb8\x82\
+1\x0d\xdf\xa2\x88\xf8\x85o\x99\x9c\xd8\xb7\xbb\xbf\xc7\xb7\
+\xb0\x01\x9aPCW}\xc7\xf4\xa3o\x99$\xbb*M\
+X(\xe1i\x8dF\x13\x9d\xf4L\x0eLt\xbf\xf2\xcd\
+\x15\xdf\xdf\xbd\xa0w\xe1\xf6?M>\x8do3%\xb4\
+\x90\xa4X`\xc3\xdeD\xf3Y\xdc\xe3\x1a\xbau\xbe^\
+\xe3\xca\xa9\x99ee\x8d=\xb9`xo7u\x8ch\
+8\xf6\xf8\xb5\x7f[[\xd4g\x0a\xc2mlA\xe4\x16\
+c\xd9\x04n\x93\xd8\xb9\x93\xc0\xc2\x9a\xd4\x1e\x1a\x83\x16\
+9\xb2\xbf\xd5\xfa\x8cu\xcb\xa1[Xc\xccIh\xfc\
+\x925\xe66\xf9Uh\x0a\xf8niu\xa8\xc2\xc7\xd4\
+\xd2\xb2\xe8M\x5c\x9dD\x17\x1a\x96\x91TnL\xce{\
+\x22\xe6!5\xa0`ME\xda\xa9\xce\xf3\x8e3\xb3L\
+\xa9{\xaa\x0be\xdfU\xa41\xc0\xbbX\x9f\x92\x18&\
+\xf6\xf4B\x96uhMVHv\x05)\x91\xb9\xcf\xc1\
+\xe5%\xef\x81\x94+b\x83\xd2`S\x09\x8f\xf72\x8f\
+$)\xe4\xeeq\x1cG\xc0\xa5\xe1Ix\x95k+\xa6\
+Ht\xed\xd5\xa8\x1c.6\x7f\xad;\x0fi\x96(\x87\
+\xda\x07\xe5v\xea\x19\xd9\x12\xe5\x11\xaa]\x8d\x96\xad\x84\
+\x96j;\xa31\xa5\x1d\xcai\x81.\xf4s\x8d4\xf6\
+p\x1b;&\x11EC\x1fH&\x1ei\xa0w\x02n\
+*xni\xdfK\x99\xe8:\xebV\xc2\x8bM\xa1'\
+\xd9\xe4\x12\xd2\xe8\x1863\xff\x99\x90\xec`j]\xb8\
+[\xfb\xbe\xce\x1by}s\x1aj\xa1\x83\xf1\x07\x97\xec\
+\xf8\x03\xf8LHv0\xb5>\xb5\x97\xcf|\xb5$\xe9\
+\xb4x\x9e\xb6\xdd\xcb\x14\x5cs\x83\xd9\x846\xbb\x82H\
+\x8f\xeaO6\xd0\xa0&mvC_\x19\x13OZ\xe6\
+\xa6\xac\xd3\xa4\x9c.haj\x92\x82>\x00\xef\x88B\
+\xc3,R?_)\xbd\x9f\xe7&\xd4\xb1Sp\xadE\
+n\xf9\x9al\x02\xb7\xce:\xf06g\xfb\x94td5\
+i\x12\x92\xf6;T\xec1\x8f#\xf6\x90\xec\x0a\x92\x94\
+\x88$\x11\xde'\x0a\x8f\x90O\xde\x89\x0eI\x08\xf5*\
+\x9a\x17\x8b\xf5\xad\x874\xe34\xd1\xc5\x0f\xdf_\xc5:\
+\xe5\x11\x9f\xf4\x0d\x84M\x99\xf0\x5c3\xd4\x93X\xe0p\
+\x8eX\xd2\x85O\x12\xd6\x93{\xc7\x83\xc7\xbfM\xafp\
+\x19\xb2t\xf2,&L\xc20i\xb8\x8d-\x08\x9b(\
+\x98\xc8\x22\xeb\xbe\xf2-{\xbbrP\xd3y\xc9\x06\xfd\
+\x5c\x9dr'8|\x85\xe2\x1e\xb1\xf5\xb4\x8f\xdeL\xea\
+\xf1\xb0$\xbeD\xe78ZY<\xde~@\xd7\x1f?\
+\xea&\xbc\x8a\xefoF\xae\x14.7\xa9A\x16Y\x1e\
+r\x1c\xc2\x8f}\xc9q\xa2Y\xe3\x06\x0f\x8d\xea\xcaR\
+\x86\x93\x8a=$\xbb\x82,\xb8\x85\xa3\xd0rc\xfcc\
+\xaf\xacI\xf7\xf4\x82\x0eQ\x09=m!\xcaif\xc7\
+h\xcf\x83\xd7\xaa\xa7\x9d7o\x5cB\x89\xae[s\x8b\
+\x9f\x92\xc8\x98\xf0\xfe\xd0\xd8:I*\xb7\xd0\x03\xe9Y\
+{\x22?r\xb7j\xa1\xe5\xfa\xd8W\x9a\xb2o\xcf\x19\
+w\xf0f\x8c\x13\x00x\xb5\xcb?\xf8\x9cp\xf9\x13\xdb\
+zL\x09\xba\x03\xc3'\x11{Hv\x05\xd1\x91\xefr\
+B5|\xcb}\x01~\x1a\xbff*\xbe\xed\xe2)\xb3\
+\xaf\xfe\x7f`\xf4\xbd\x8a'`\x94\xa41\xe8\xc4\xd7@\
+\xd3\x07\xd4;s\x91-\xc7\x13)\xea|kS M\
+\xa2\xeb\xca\x93\xf0\xa4\xfd/i2\x84G:\x8f[\xff\
+\xab:\x83\x8a\xcdX\x13K\x8a\x85\xf6\xfb\x89\xe7,|\
+4\xa8\x96\xa3\xaf\xe9\xb2x\xf2W2\xf14\x5c\xd30\
+\xd1jR\x19\xea\xf1\x93\xe4\xfc<\x14{:\xd9,M\
+(\xf6f\xb6\xcdnT\x13z\x0e#\xe2h\xcf\x9aR\
+\xd5\xb7\xbc3K\xeb\x86\x84\xc5\x89\x8e\xed\x92+\xcdZ\
+b\x97\xbe\xeb\x9d\x0fm\xc1\xdd\xdao\x99\xd6Nh,\
+\x97\x06\x9d\x9c \x1br\x0b\xdcp\xdcnX\xb6:\xd6\
+\xcc\xb4\x96\xb9\xbf\xa3\xc4\xba\xfd\xb3\xea\xe1\xeb\xac\x13f\
+\x16)K\xa2\xeb\xd2\x84w\xbar\xd0\xff\xac\xb0\xb7\x0d\
+\x8f9:J\x1a\xf4*\xed\xae\x8f\xe5\xf8\xecpg\xe2\
+\x05\xe9\xb8\xa8\xea@\xee\xd0`q\x9f\xa4XP\x1a\x0f\
+R\x1b\xd2\xd9\x81O\xba\xc7\xc9\xc4O\x88\xb4$\x19\xf4\
+(O\xc3D\xab\x1a{\x97\xe6r=<)\x82Y\x93\
+\xf7\xba\x962\xf6\x1e\x8f3\xf6P\xb3+P\xe76\xb3\
+\x91\xb0ZE\x03B\x82z#i\xf2M\x9d\x9a\xa7?\
+\xb8\xb5\xd6\xc0\xcc\xa9\x1a\x82\xb56\xa9\xfb\x89\xf7e\xa8\
+\xea\xd9_9\xcb0\x8e\xa2\xc5\xfb\xe5\xbe^\xd7\x84D\
+\xd7\xe5\xab\xe1\x19\xba\xfe\xb0|\xcac\xd3\xd9\xbe]f\
+\xc1\x87\xf9\x13\xe8\xfe\xa4\xfc\x9f$\xad\xa7\xc7\xe4\xeaq\
+\x22\xf2>\x0e6\xf1\x89V5\xf6\x1c\xbb\x22c\xef3\
+\x89\xbd\x7f\x8d+\xf6\x90\xec\x0a&W\xf4\xcd\xbcs\xa0\
+\xf5\x90\x13\x9e]4\xf0\x09\x82\xc5\xc6\xf6Q\xf0\xe9\x82\
+\xec\xfb\x9a\x988\xa1qk\xe5\xca{I\x95\xe8\xba4\
+\xe1\xf5&1\xde\xbb\xdb\x18<\xa5\xb8\x1e\x9b\x84\xb6\xbb\
+B\xdd\xd5\x9e\xf1\x14s\xc7\xa5\xc6T\x9et\xc7\x92&\
+\xf1Y\x8d=$\xbb\x82\xc5S\x02\xb1[\x19*\xe1\xc9\
+\x09o\xb8\xf5 t\xc2'>N\x95a_Y\x92\xcb\
+(\xc4\x13x\xb6'\xaf\xcc\xf5^n\xf2\xcd\x07\xed\x89\
+/yo1\xf0-_\xdd\xc4:\xce\x84\xa7\xc98M\
+\x0d/\x91|>-\x9biyv\xb9\xa0\xd8{#q\
+\xfe\xe7q\xc5\x1e\x92\xdd\x08t\x13^\xca\xdb\xa6\x1e\xf1\
+\xec\xb5\x12\xd4i\xbe\x874\x0e:\x9d\x116gb\xd5\
+\x93\xf0W\xbe\xf9\xd54|\xe1\x8e\xbc\x87\xdab}k\
+)\xcf{\xd1\xdb+\xd9\xf6\xab\xc5\x84\xaf3Tq\xb9\
+jM2m\x99i\x0d{\xc8IM\xb5\x86\xd7N\xe6\
+\xd9\x8fSw\xf6d\xfd|\xd3p\x9c\xba:\xb1\xb7\x94\
+;\xf6\xa4L%\xf6\xfe\x98\xf6\xfbv\x8b0\x91\x0e\x8a\
+\xac\x93i\x8ec\xf2\xcd\xa2u\x02sE\xda#6J\
+&\x9e%\xb7\x1aZ_\x93\x5cd\xf9\xc9\x17\xc7\xdbG\
+\x94Ag\xc6\x94\xa5\xb4\xfb\xd1\x93\xe7\x9d\xf4\xaa\x19\x8e\
+\xf6\xa7\xe9\xe4\x19'=\xc1\xb4\xb1_\xae\xf5\xbbV'\
+-\x1d\xd0\xe0\xae\xe5\xa4=\xdf\x17|k\x7f\xc1\x5c>\
+4y\xbf^\xacC\x93\xb9\xfcSK{\x9ct\x98\x87\
+\xf6~\xea\xfe\x97\xea\xd3\xf9\x0cp'~\xf2\xc4\xdeQ\
+\xea\x99~\x0a4\xdc\x11\x0c\xc0\x97d\xf7\xd2\xb6\x96\x05\
+\xbaX\x96\xbat\xe5\xfdt9\xaei\x9d=\xff\x95n\
+5\x8a\xbc=\x89\xc7e\xc9~$\x00oG\xcc\x9f\x95\
+\x8c\xf9E\xda\xab\xfe\xa7\x03C\xe75\xc1\x85t\xcbK\
+\x8f\x8b\x96\x15G\xfc\xd3\xd5c\x22\xb1\xac\x1d\x03\xdeG\
+\xd7$)mf}\x22\x22\x14\x0f\x8e\xdc\xc9\xac\x1e'\
+-KK\xb6lJ\xe6\xfe\xb4\xc5\x1e\x92\x9dG\xd1\xc9\
+\x0e\xa6\x8b~\xabY\xda[(\x1d\xd8\x1b\xea\x99\xd5\xa6\
+\x84I\xd4T \x1b\xb4\xd9\xc1\xdc\xd1D\xc7\xf6\xe6\xab\
+4\xa3\xf8\x93\x06}\x8bs$\xba\xd9\x80\x89\x00`\xae\
+\xb4\xbf`\xdc\xc6\xe3\xd5tP\xef\xd9\xea\xe1Z\xcb\xd8\
+\xbd\xcb\xa8\xd4\xd3\x94\xa0\xdf\xb9\xd0\xfe&7\x0e\x7fm\
+%Q\xa66V\x98\x1c$;\x98+\xfd_\xf2,\xc9\
+j\xf9\x06\xbb\xe77\xac\xd3\xdb\xd5\xa6\xbe&\x9d\x11\xe5\
+\xf6\x93-\xc9-\x19\xbe\xb1\x900}\x90\xec`nt\
+\xbe\xb8:PS3\x15\xfdo\xfa\xc6f\xdeC\x87\xcf\
+\xec\x18Y\xb2C\x03?L\x1bv\xb6\x5c\x5c+ux\
+\x003L\x1ftP\xc0\xdc\xf8}c{\x7f\xd8\x91\xff\
+:VLnq\xbfG\xa2\x9b=\xa8}\xc1\xdc\x89\x1f\
+<\x97\xb6;kL\xea)\x860\x18{\xf6!\xd9\xc1\
+\xdc\xea\xcc\xb6Q\xb5\xd6~\xa3S9\xc9\xc9Py?\
+u\x167\xe5\xb5\xf3\x88\xe8Gc\xa2\x93\x0b\xb7p4\
+\xbd\xdff\x06\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
+\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
+\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
+\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
+\x00\x00\x00\x00\x00\x00\x01\xff\x07\xb4<2M\x93'\x06\
+\xcb\x00\x00\x00\x00IEND\xaeB`\x82\
+\x00\x00\x03X\
+<\
+?xml version=\x221.\
+0\x22 encoding=\x22utf\
+-8\x22?>\x0a<!-- Gener\
+ator: Adobe Illu\
+strator 23.0.1, \
+SVG Export Plug-\
+In . SVG Version\
+: 6.00 Build 0) \
+ -->\x0a<svg versio\
+n=\x221.1\x22 id=\x22Laye\
+r_1\x22 xmlns=\x22http\
+://www.w3.org/20\
+00/svg\x22 xmlns:xl\
+ink=\x22http://www.\
+w3.org/1999/xlin\
+k\x22 x=\x220px\x22 y=\x220p\
+x\x22\x0a\x09 viewBox=\x220 \
+0 100 100\x22 style\
+=\x22enable-backgro\
+und:new 0 0 100 \
+100\x22 xml:space=\x22\
+preserve\x22>\x0a<styl\
+e type=\x22text/css\
+\x22>\x0a    path{fill\
+:rgb(150, 146, 1\
+44)}\x0a    polygon\
+{fill:rgb(150, 1\
+46, 144)}\x0a    ci\
+rcle{fill:rgb(15\
+0, 146, 144)}\x0a  \
+  rect{fill:rgb(\
+150, 146, 144)}\x0a\
+</style><path d=\
+\x22M51.3,75.9c-1.9\
+,0-3.8-0.8-5-2.4\
+L18.1,38.4c-2.2-\
+2.8-1.8-6.9,1-9.\
+1c2.8-2.2,6.9-1.\
+8,9.1,1l28.2,35.\
+1c2.2,2.8,1.8,6.\
+9-1,9.1\x0a\x09C54.2,7\
+5.5,52.7,75.9,51\
+.3,75.9z\x22/>\x0a<pat\
+h d=\x22M51.3,75.9c\
+-1.4,0-2.9-0.5-4\
+-1.4c-2.8-2.2-3.\
+2-6.3-1-9.1l28.2\
+-35.1c2.2-2.8,6.\
+3-3.2,9.1-1c2.8,\
+2.2,3.2,6.3,1,9.\
+1L56.4,73.5\x0a\x09C55\
+.1,75.1,53.2,75.\
+9,51.3,75.9z\x22/>\x0a\
+</svg>\x0a\
+\x00\x00\x03Y\
+<\
+?xml version=\x221.\
+0\x22 encoding=\x22utf\
+-8\x22?>\x0a<!-- Gener\
+ator: Adobe Illu\
+strator 23.0.1, \
+SVG Export Plug-\
+In . SVG Version\
+: 6.00 Build 0) \
+ -->\x0a<svg versio\
+n=\x221.1\x22 id=\x22Laye\
+r_1\x22 xmlns=\x22http\
+://www.w3.org/20\
+00/svg\x22 xmlns:xl\
+ink=\x22http://www.\
+w3.org/1999/xlin\
+k\x22 x=\x220px\x22 y=\x220p\
+x\x22\x0a\x09 viewBox=\x220 \
+0 100 100\x22 style\
+=\x22enable-backgro\
+und:new 0 0 100 \
+100;\x22 xml:space=\
+\x22preserve\x22>\x0a<sty\
+le type=\x22text/cs\
+s\x22>\x0a    path{fil\
+l:rgb(150, 146, \
+144)}\x0a    polygo\
+n{fill:rgb(150, \
+146, 144)}\x0a    c\
+ircle{fill:rgb(1\
+50, 146, 144)}\x0a \
+   rect{fill:rgb\
+(150, 146, 144)}\
+\x0a</style><path d\
+=\x22M31.8,56.4c-1.\
+9,0-3.8-0.8-5-2.\
+4c-2.2-2.8-1.8-6\
+.9,1-9.1l35.1-28\
+.2c2.8-2.2,6.9-1\
+.8,9.1,1c2.2,2.8\
+,1.8,6.9-1,9.1L3\
+5.8,54.9\x0a\x09C34.6,\
+55.9,33.2,56.4,3\
+1.8,56.4z\x22/>\x0a<pa\
+th d=\x22M66.9,84.6\
+c-1.4,0-2.9-0.5-\
+4-1.4L27.7,54.9c\
+-2.8-2.2-3.2-6.3\
+-1-9.1c2.2-2.8,6\
+.3-3.2,9.1-1l35.\
+1,28.2c2.8,2.2,3\
+.2,6.3,1,9.1\x0a\x09C7\
+0.6,83.8,68.8,84\
+.6,66.9,84.6z\x22/>\
+\x0a</svg>\x0a\
+\x00\x00\x03X\
+<\
+?xml version=\x221.\
+0\x22 encoding=\x22utf\
+-8\x22?>\x0a<!-- Gener\
+ator: Adobe Illu\
+strator 23.0.1, \
+SVG Export Plug-\
+In . SVG Version\
+: 6.00 Build 0) \
+ -->\x0a<svg versio\
+n=\x221.1\x22 id=\x22Laye\
+r_1\x22 xmlns=\x22http\
+://www.w3.org/20\
+00/svg\x22 xmlns:xl\
+ink=\x22http://www.\
+w3.org/1999/xlin\
+k\x22 x=\x220px\x22 y=\x220p\
+x\x22\x0a\x09 viewBox=\x220 \
+0 100 100\x22 style\
+=\x22enable-backgro\
+und:new 0 0 100 \
+100\x22 xml:space=\x22\
+preserve\x22>\x0a<styl\
+e type=\x22text/css\
+\x22>\x0a    path{fill\
+:rgb(150, 146, 1\
+44)}\x0a    polygon\
+{fill:rgb(150, 1\
+46, 144)}\x0a    ci\
+rcle{fill:rgb(15\
+0, 146, 144)}\x0a  \
+  rect{fill:rgb(\
+150, 146, 144)}\x0a\
+</style><path d=\
+\x22M51.3,75.9c-1.9\
+,0-3.8-0.8-5-2.4\
+L18.1,38.4c-2.2-\
+2.8-1.8-6.9,1-9.\
+1c2.8-2.2,6.9-1.\
+8,9.1,1l28.2,35.\
+1c2.2,2.8,1.8,6.\
+9-1,9.1\x0a\x09C54.2,7\
+5.5,52.7,75.9,51\
+.3,75.9z\x22/>\x0a<pat\
+h d=\x22M51.3,75.9c\
+-1.4,0-2.9-0.5-4\
+-1.4c-2.8-2.2-3.\
+2-6.3-1-9.1l28.2\
+-35.1c2.2-2.8,6.\
+3-3.2,9.1-1c2.8,\
+2.2,3.2,6.3,1,9.\
+1L56.4,73.5\x0a\x09C55\
+.1,75.1,53.2,75.\
+9,51.3,75.9z\x22/>\x0a\
+</svg>\x0a\
+"
+
+qt_resource_name = b"\
+\x00\x08\
+\x05\xe2Y'\
+\x00l\
+\x00o\x00g\x00o\x00.\x00p\x00n\x00g\
+\x00\x08\
+\x0aaZ\xa7\
+\x00i\
+\x00c\x00o\x00n\x00.\x00p\x00n\x00g\
+\x00\x0d\
+\x0dq\x0b\x87\
+\x00c\
+\x00o\x00l\x00l\x00a\x00p\x00s\x00e\x00d\x00.\x00s\x00v\x00g\
+\x00\x0c\
+\x07)\x8aG\
+\x00e\
+\x00x\x00p\x00a\x00n\x00d\x00e\x00d\x00.\x00s\x00v\x00g\
+"
+
+qt_resource_struct = b"\
+\x00\x00\x00\x00\x00\x02\x00\x00\x00\x04\x00\x00\x00\x01\
+\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\
+\x00\x00\x00L\x00\x00\x00\x00\x00\x01\x00\x00\x14\x9b\
+\x00\x00\x00\x16\x00\x00\x00\x00\x00\x01\x00\x00\x0d\xe2\
+\x00\x00\x00,\x00\x00\x00\x00\x00\x01\x00\x00\x11>\
+"
+
+def qInitResources():
+    QtCore.qRegisterResourceData(0x01, qt_resource_struct, qt_resource_name, qt_resource_data)
+
+def qCleanupResources():
+    QtCore.qUnregisterResourceData(0x01, qt_resource_struct, qt_resource_name, qt_resource_data)
+
+qInitResources()

server/gui/hook-cefpython3.py

@@ -0,0 +1,237 @@
+"""
+This is PyInstaller hook file for CEF Python. This file
+helps PyInstaller find CEF Python dependencies that are
+required to run final executable.
+
+See PyInstaller docs for hooks:
+https://pyinstaller.readthedocs.io/en/stable/hooks.html
+"""
+
+import glob
+import os
+import platform
+import re
+import sys
+import PyInstaller
+from PyInstaller.utils.hooks import is_module_satisfies, get_package_paths
+from PyInstaller.compat import is_win, is_darwin, is_linux, is_py2
+from PyInstaller import log as logging
+
+# Constants
+CEFPYTHON_MIN_VERSION = "57.0"
+PYINSTALLER_MIN_VERSION = "3.2.1"
+
+# Makes assumption that using "python.exe" and not "pyinstaller.exe"
+# TODO: use this code to work cross-platform:
+# from PyInstaller.utils.hooks import get_package_paths
+# get_package_paths("cefpython3")
+
+CEFPYTHON3_DIR = get_package_paths("cefpython3")[1]
+
+CYTHON_MODULE_EXT = ".pyd" if is_win else ".so"
+
+# Globals
+logger = logging.getLogger(__name__)
+
+
+# Functions
+def check_platforms():
+    if not is_win and not is_darwin and not is_linux:
+        raise SystemExit("Error: Currently only Windows, Linux and Darwin "
+                         "platforms are  supported, see Issue #135.")
+
+
+def check_pyinstaller_version():
+    """Using is_module_satisfies() for pyinstaller fails when
+    installed using 'pip install develop.zip' command
+    (PyInstaller Issue #2802)."""
+    # Example version string for dev version of pyinstaller:
+    # > 3.3.dev0+g5dc9557c
+    version = PyInstaller.__version__
+    match = re.search(r"^\d+\.\d+(\.\d+)?", version)
+    if not (match.group(0) >= PYINSTALLER_MIN_VERSION):
+        raise SystemExit("Error: pyinstaller %s or higher is required"
+                         % PYINSTALLER_MIN_VERSION)
+
+
+def check_cefpython3_version():
+    if not is_module_satisfies("cefpython3 >= %s" % CEFPYTHON_MIN_VERSION):
+        raise SystemExit("Error: cefpython3 %s or higher is required"
+                         % CEFPYTHON_MIN_VERSION)
+
+
+def get_cefpython_modules():
+    """Get all cefpython Cython modules in the cefpython3 package.
+    It returns a list of names without file extension. Eg.
+    'cefpython_py27'. """
+    pyds = glob.glob(os.path.join(CEFPYTHON3_DIR,
+                                  "cefpython_py*" + CYTHON_MODULE_EXT))
+    assert len(pyds) > 1, "Missing cefpython3 Cython modules"
+    modules = []
+    for path in pyds:
+        filename = os.path.basename(path)
+        mod = filename.replace(CYTHON_MODULE_EXT, "")
+        modules.append(mod)
+    return modules
+
+
+def get_excluded_cefpython_modules():
+    """CEF Python package includes Cython modules for various Python
+       versions. When using Python 2.7 pyinstaller should not
+       bundle modules for eg. Python 3.6, otherwise it will
+       cause to include Python 3 dll dependencies. Returns a list
+       of fully qualified names eg. 'cefpython3.cefpython_py27'."""
+    pyver = "".join(map(str, sys.version_info[:2]))
+    pyver_string = "py%s" % pyver
+    modules = get_cefpython_modules()
+    excluded = []
+    for mod in modules:
+        if pyver_string in mod:
+            continue
+        excluded.append("cefpython3.%s" % mod)
+        logger.info("Exclude cefpython3 module: %s" % excluded[-1])
+    return excluded
+
+
+def get_cefpython3_datas():
+    """Returning almost all of cefpython binaries as DATAS (see exception
+    below), because pyinstaller does strange things and fails if these are
+    returned as BINARIES. It first updates manifest in .dll files:
+    >> Updating manifest in chrome_elf.dll
+
+    And then because of that it fails to load the library:
+    >> hsrc = win32api.LoadLibraryEx(filename, 0, LOAD_LIBRARY_AS_DATAFILE)
+    >> pywintypes.error: (5, 'LoadLibraryEx', 'Access is denied.')
+
+    It is not required for pyinstaller to modify in any way
+    CEF binaries or to look for its dependencies. CEF binaries
+    does not have any external dependencies like MSVCR or similar.
+
+    The .pak .dat and .bin files cannot be marked as BINARIES
+    as pyinstaller would fail to find binary depdendencies on
+    these files.
+
+    One exception is subprocess (subprocess.exe on Windows) executable
+    file, which is passed to pyinstaller as BINARIES in order to collect
+    its dependecies.
+
+    DATAS are in format: tuple(full_path, dest_subdir).
+    """
+    ret = list()
+
+    if is_win:
+        cefdatadir = "."
+    elif is_darwin or is_linux:
+        cefdatadir = "."
+    else:
+        assert False, "Unsupported system {}".format(platform.system())
+
+    # Binaries, licenses and readmes in the cefpython3/ directory
+    for filename in os.listdir(CEFPYTHON3_DIR):
+        # Ignore Cython modules which are already handled by
+        # pyinstaller automatically.
+        if filename[:-len(CYTHON_MODULE_EXT)] in get_cefpython_modules():
+            continue
+
+        # CEF binaries and datas
+        extension = os.path.splitext(filename)[1]
+        if extension in \
+            [".exe", ".dll", ".pak", ".dat", ".bin", ".txt", ".so", ".plist"] \
+                or filename.lower().startswith("license"):
+            logger.info("Include cefpython3 data: {}".format(filename))
+            ret.append((os.path.join(CEFPYTHON3_DIR, filename), cefdatadir))
+
+    if is_darwin:
+        # "Chromium Embedded Framework.framework/Resources" with subdirectories
+        # is required. Contain .pak files and locales (each locale in separate
+        # subdirectory).
+        resources_subdir = \
+            os.path.join("Chromium Embedded Framework.framework", "Resources")
+        base_path = os.path.join(CEFPYTHON3_DIR, resources_subdir)
+        assert os.path.exists(base_path), \
+            "{} dir not found in cefpython3".format(resources_subdir)
+        for path, dirs, files in os.walk(base_path):
+            for file in files:
+                absolute_file_path = os.path.join(path, file)
+                dest_path = os.path.relpath(path, CEFPYTHON3_DIR)
+                ret.append((absolute_file_path, dest_path))
+                logger.info("Include cefpython3 data: {}/{}".format(dest_path, file))
+    elif is_win or is_linux:
+        # The .pak files in cefpython3/locales/ directory
+        locales_dir = os.path.join(CEFPYTHON3_DIR, "locales")
+        assert os.path.exists(locales_dir), \
+            "locales/ dir not found in cefpython3"
+        for filename in os.listdir(locales_dir):
+            logger.info("Include cefpython3 data: {}/{}".format(
+                os.path.basename(locales_dir), filename))
+            ret.append((os.path.join(locales_dir, filename),
+                        os.path.join(cefdatadir, "locales")))
+
+        # Optional .so/.dll files in cefpython3/swiftshader/ directory
+        swiftshader_dir = os.path.join(CEFPYTHON3_DIR, "swiftshader")
+        if os.path.isdir(swiftshader_dir):
+            for filename in os.listdir(swiftshader_dir):
+                logger.info("Include cefpython3 data: {}/{}".format(
+                    os.path.basename(swiftshader_dir), filename))
+                ret.append((os.path.join(swiftshader_dir, filename),
+                            os.path.join(cefdatadir, "swiftshader")))
+    return ret
+
+
+# ----------------------------------------------------------------------------
+# Main
+# ----------------------------------------------------------------------------
+
+# Checks
+check_platforms()
+check_pyinstaller_version()
+check_cefpython3_version()
+
+# Info
+logger.info("CEF Python package directory: %s" % CEFPYTHON3_DIR)
+
+# Hidden imports.
+# PyInstaller has no way on detecting imports made by Cython
+# modules, so all pure Python imports made in cefpython .pyx
+# files need to be manually entered here.
+# TODO: Write a tool script that would find such imports in
+#       .pyx files automatically.
+hiddenimports = [
+    "codecs",
+    "copy",
+    "datetime",
+    "inspect",
+    "json",
+    "os",
+    "platform",
+    "random",
+    "re",
+    "sys",
+    "time",
+    "traceback",
+    "types",
+    "urllib",
+    "weakref",
+]
+if is_py2:
+    hiddenimports += [
+        "urlparse",
+    ]
+
+# Excluded modules
+excludedimports = get_excluded_cefpython_modules()
+
+# Include binaries requiring to collect its dependencies
+if is_darwin or is_linux:
+    binaries = [(os.path.join(CEFPYTHON3_DIR, "subprocess"), ".")]
+elif is_win:
+    binaries = [(os.path.join(CEFPYTHON3_DIR, "subprocess.exe"), ".")]
+else:
+    binaries = []
+
+# Include datas
+datas = get_cefpython3_datas()
+
+# Notify pyinstaller.spec code that this hook was executed
+# and that it succeeded.
+os.environ["PYINSTALLER_CEFPYTHON3_HOOK_SUCCEEDED"] = "1"

server/gui/images/properties_contract.svg

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 23.0.1, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 100 100" style="enable-background:new 0 0 100 100;" xml:space="preserve">
+<style type="text/css">
+    path{fill:rgb(150, 146, 144)}
+    polygon{fill:rgb(150, 146, 144)}
+    circle{fill:rgb(150, 146, 144)}
+    rect{fill:rgb(150, 146, 144)}
+</style><path d="M31.8,56.4c-1.9,0-3.8-0.8-5-2.4c-2.2-2.8-1.8-6.9,1-9.1l35.1-28.2c2.8-2.2,6.9-1.8,9.1,1c2.2,2.8,1.8,6.9-1,9.1L35.8,54.9
+	C34.6,55.9,33.2,56.4,31.8,56.4z"/>
+<path d="M66.9,84.6c-1.4,0-2.9-0.5-4-1.4L27.7,54.9c-2.8-2.2-3.2-6.3-1-9.1c2.2-2.8,6.3-3.2,9.1-1l35.1,28.2c2.8,2.2,3.2,6.3,1,9.1
+	C70.6,83.8,68.8,84.6,66.9,84.6z"/>
+</svg>

server/gui/images/properties_expand.svg

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 23.0.1, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 100 100" style="enable-background:new 0 0 100 100" xml:space="preserve">
+<style type="text/css">
+    path{fill:rgb(150, 146, 144)}
+    polygon{fill:rgb(150, 146, 144)}
+    circle{fill:rgb(150, 146, 144)}
+    rect{fill:rgb(150, 146, 144)}
+</style><path d="M51.3,75.9c-1.9,0-3.8-0.8-5-2.4L18.1,38.4c-2.2-2.8-1.8-6.9,1-9.1c2.8-2.2,6.9-1.8,9.1,1l28.2,35.1c2.2,2.8,1.8,6.9-1,9.1
+	C54.2,75.5,52.7,75.9,51.3,75.9z"/>
+<path d="M51.3,75.9c-1.4,0-2.9-0.5-4-1.4c-2.8-2.2-3.2-6.3-1-9.1l28.2-35.1c2.2-2.8,6.3-3.2,9.1-1c2.8,2.2,3.2,6.3,1,9.1L56.4,73.5
+	C55.1,75.1,53.2,75.9,51.3,75.9z"/>
+</svg>

server/gui/main.py

@@ -1,33 +1,49 @@
 # flake8: noqa F403, F405
-from os.path import splitext, basename
+from functools import partialmethod
+from multiprocessing import Pipe, Process, freeze_support
+from os import environ
+from os.path import splitext, basename, dirname, join
 import sys
 import threading
 
 from cefpython3 import cefpython as cef
+import PySide2
+from PySide2.QtGui import *
 from PySide2.QtCore import *
 from PySide2.QtWidgets import *
 
-from server.app.app import Server
+import server.gui.cellxgene_rc
 from server.gui.browser import CefWidget, CefApplication
-from server.gui.workers import DataLoadWorker, ServerRunWorker
-from server.gui.utils import WINDOWS, LINUX, MAC, FileLoadSignals
-from server.utils.constants import MODES
+from server.gui.workers import Worker, SiteReadyWorker
+from server.gui.utils import WINDOWS, LINUX, MAC, FileLoadSignals, Emitter, WorkerSignals, FileChanged
+from server.utils.utils import find_available_port
 
+if WINDOWS or LINUX:
+    dirname = dirname(PySide2.__file__)
+    plugin_path = join(dirname, 'plugins', 'platforms')
+    environ['QT_QPA_PLATFORM_PLUGIN_PATH'] = plugin_path
 
 # Configuration
 # TODO remember this or calculate it?
-WIDTH = 1024
-HEIGHT = 768
+WIDTH = 1300
+HEIGHT = 800
+MAX_CONTENT_WIDTH = 700
+GUI_PORT = find_available_port("localhost")
+BROWSER_INDEX = 0
+LOAD_INDEX = 1
+
 
-# noinspection PyUnresolvedReferences
 class MainWindow(QMainWindow):
     def __init__(self):
         super(MainWindow, self).__init__(None)
         self.cef_widget = None
         self.data_widget = None
-        self.server = Server()
-        self.server.create_app()
-        self.runServer()
+        self.stacked_layout = None
+        self.parent_conn, self.child_conn = None, None
+        self.load_emitter = None
+        self.emitter_thread = None
+        self.worker = None
+        self.url = f"http://localhost:{GUI_PORT}/"
         self.setWindowTitle("cellxgene")
 
         # Strong focus - accepts focus by tab & click
@@ -35,13 +51,26 @@ class MainWindow(QMainWindow):
         self.setupLayout()
         self.setupMenu()
 
+    def showBrowser(self):
+        self.stacked_layout.setCurrentIndex(BROWSER_INDEX)
+
+    def restartOnError(self):
+        self.window().shutdownServer()
+        # close emitter on error/finished
+        self.parent_conn, self.child_conn = Pipe()
+        self.load_emitter = Emitter(self.parent_conn, WorkerSignals)
+        self.emitter_thread = threading.Thread(target=self.load_emitter.run, daemon=True)
+        self.emitter_thread.start()
+        # send to load with error message?
+
     def setupLayout(self):
         self.resize(WIDTH, HEIGHT)
         self.cef_widget = CefWidget(self)
+        self.cef_widget.setSizePolicy(QSizePolicy(QSizePolicy.MinimumExpanding, QSizePolicy.MinimumExpanding))
         self.data_widget = LoadWidget(self)
         self.stacked_layout = QStackedLayout()
-        self.stacked_layout.addWidget(self.data_widget)
         self.stacked_layout.addWidget(self.cef_widget)
+        self.stacked_layout.addWidget(self.data_widget)
         main_layout = QVBoxLayout()
         main_layout.setContentsMargins(0, 0, 0, 0)
         main_layout.setSpacing(0)
@@ -67,9 +96,26 @@ class MainWindow(QMainWindow):
             # cef widget in the layout with the container.
             self.container = QWidget.createWindowContainer(
                 self.cef_widget.hidden_window, parent=self)
-            stacked_layout.addWidget(self.container, 1, 0)
+            self.stacked_layout.replaceWidget(self.cef_widget, self.container)
+        self.stacked_layout.setCurrentIndex(LOAD_INDEX)
+
+    def setupServer(self):
+        self.shutdownServer()
+        # close emitter on error/finished
+        self.parent_conn, self.child_conn = Pipe()
+        self.load_emitter = Emitter(self.parent_conn, WorkerSignals)
+        self.emitter_thread = threading.Thread(target=self.load_emitter.run, daemon=True)
+        self.emitter_thread.start()
+        # send to load with error message?
+
+    def shutdownServer(self):
+        if self.worker:
+            self.worker.terminate()
+        if self.parent_conn:
+            self.parent_conn.close()
 
     def setupMenu(self):
+        # TODO add communication to subprocess on reload
         main_menu = self.menuBar()
         file_menu = main_menu.addMenu('File')
         load_action = QAction("Load file...", self)
@@ -79,7 +125,11 @@ class MainWindow(QMainWindow):
         file_menu.addAction(load_action)
 
     def showLoad(self):
-        self.stacked_layout.setCurrentIndex(0)
+        self.clearMessages()
+        self.stacked_layout.setCurrentIndex(LOAD_INDEX)
+
+    def clearMessages(self):
+        self.data_widget.reset()
 
     def closeEvent(self, event):
         # Close browser (force=True) and free CEF reference
@@ -87,11 +137,6 @@ class MainWindow(QMainWindow):
             self.cef_widget.browser.CloseBrowser(True)
             self.clearBrowserReferences()
 
-    def runServer(self):
-        worker = ServerRunWorker(self.server.app, host="127.0.0.1", port=8000)
-        self.httpd = threading.Thread(target=worker.run, daemon=True)
-        self.httpd.start()
-
     def clearBrowserReferences(self):
         # Clear browser references that you keep anywhere in your
         # code. All references must be cleared for CEF to shutdown cleanly.
@@ -102,93 +147,217 @@ class LoadWidget(QFrame):
     def __init__(self, parent):
         super(LoadWidget, self).__init__(parent=parent)
         # Init layout
-        self.MAX_CONTENT_WIDTH = 500
         load_ui_layout = QVBoxLayout()
-        h_margin = (WIDTH - self.MAX_CONTENT_WIDTH) // 2
+        h_margin = (WIDTH - MAX_CONTENT_WIDTH) // 2
         if h_margin < 10:
             h_margin = 10
         load_ui_layout.setContentsMargins(h_margin, 20, h_margin, 20)
         logo_layout = QHBoxLayout()
         logo_layout.setContentsMargins(0, 0, 0, 20)
 
-        load_layout = QGridLayout()
-        load_layout.setContentsMargins(0, 0, 0, 0)
-        load_layout.setSpacing(0)
+        file_layout = QVBoxLayout()
+
         message_layout = QHBoxLayout()
         message_layout.setContentsMargins(0, 0, 0, 0)
 
-        self.title = ""
-        self.label = QLabel("cellxgene")
+        self.serverError = False
+        self.file_name = FilePath()
+
+        self.label = QLabel()
+        logo = QPixmap(":/logo.png")
+        self.label.setPixmap(logo)
+        self.label.setContentsMargins(100, 0, 100, 0)
         logo_layout.addWidget(self.label)
 
         # UI section
-        # TODO add load spinner
         # TODO add cancel button to send back to browser (if available)
-        self.embedding_label = QLabel("embedding: ")
-        load_layout.addWidget(self.embedding_label, 0, 0)
-        self.file_label = QLabel("file: ")
-        load_layout.addWidget(self.file_label, 0, 1)
-        self.embeddings = QComboBox(self)
-        self.embeddings.currentIndexChanged.connect(self.updateEmbedding)
-        self.embeddings.addItems(MODES)
-        self.embedding_selection = MODES[0]
-        load_layout.addWidget(self.embeddings, 1, 0)
 
-        self.load = QPushButton("Open...")
-        self.load.clicked.connect(self.onLoad)
-        load_layout.addWidget(self.load, 1, 1)
+        self.file_area = FileArea(self)
+        self.file_name.signals.changed.connect(self.updatePath)
+
+        self.launch_widget = QLabel("Select a file to launch cellxgene")
+        # self.launch_widget.setEnabled(False)
+        # self.launch_widget.clicked.connect(self.onLoad)
+
+        self.progress = QProgressBar()
+        self.progress.setTextVisible(False)
+
+        file_layout.addWidget(self.file_area)
+        self.loading_layout = QStackedLayout()
+        self.loading_layout.addWidget(self.launch_widget)
+        self.loading_layout.addWidget(self.progress)
+        file_layout.addLayout(self.loading_layout)
+        file_layout.setStretch(0, 10)
 
         # Error section
         self.error_label = QLabel("")
         self.error_label.setWordWrap(True)
-        self.error_label.setFixedWidth(self.MAX_CONTENT_WIDTH)
-        message_layout.addWidget(self.error_label, alignment=Qt.AlignTop)
+        self.error_label.setFixedWidth(MAX_CONTENT_WIDTH)
+        message_layout.addWidget(self.error_label)
+
+        # Options Form
 
         # Layout
-        for l in [logo_layout, load_layout, message_layout ]:
+        for l in [logo_layout, file_layout, message_layout]:
             load_ui_layout.addLayout(l)
 
-        load_ui_layout.setStretch(2, 10)
+        #TODO remove magic number
+        load_ui_layout.setStretch(1, 10)
         self.setLayout(load_ui_layout)
 
+        self.timer = QTimer()
+        self.timer.setInterval(100)
+        self.timer.timeout.connect(self.updateProgress)
+
         self.signals = FileLoadSignals()
         self.signals.selectedFile.connect(self.createScanpyEngine)
+        self.signals.error.connect(self.onError)
 
-    def updateEmbedding(self, idx):
-        self.embedding_selection = MODES[idx]
+    def updatePath(self):
+        file_name = self.file_name.value
+        if file_name:
+            self.file_area.label.setText("File: " + file_name)
+        else:
+            self.file_area.label.setText("")
+        # self.launch_widget.setEnabled(bool(file_name))
+
+    def reset(self):
+        self.loading_layout.setCurrentIndex(0)
+        self.timer.stop()
+        self.error_label.setText("")
+        self.file_name.updateValue(None)
+
+    def updateProgress(self):
+        curr_val = self.progress.value()
+        next_val = (curr_val + 1) % 100
+        self.progress.setValue(next_val)
+
+    def resetProgress(self):
+        self.progress.setValue(0)
+        self.loading_layout.setCurrentIndex(0)
+        self.timer.stop()
 
     def createScanpyEngine(self, file_name):
-        worker = DataLoadWorker(file_name, self.embedding_selection)
-        worker.signals.result.connect(self.onDataSuccess)
-        worker.signals.error.connect(self.onDataError)
-        self.load_worker = threading.Thread(target=worker.run, daemon=True)
-        self.load_worker.start()
+        title = splitext(basename(file_name))[0]
+        self.window().setupServer()
+        worker = Worker(self.window().parent_conn, self.window().child_conn, file_name, host="127.0.0.1",
+                        port=GUI_PORT, title=title, engine_options={})
+        self.window().load_emitter.signals.ready.connect(self.onDataReady)
+        self.window().load_emitter.signals.engine_error.connect(self.onServerError)
+        self.window().load_emitter.signals.server_error.connect(self.onServerError)
+        # Error is generic error from emitter
+        self.window().load_emitter.signals.error.connect(self.onServerError)
+        self.window().worker = Process(target=worker.run, daemon=True)
+        self.window().worker.start()
+        self.window().child_conn.close()
 
     def onLoad(self):
+        if self.file_name.value:
+            # Reset error on reload
+            self.serverError = False
+            self.loading_layout.setCurrentIndex(1)
+            self.timer.start()
+            self.signals.selectedFile.emit(self.file_name.value)
+        else:
+            self.signals.error.emit("Please select a file before launching.")
+
+    def onDataReady(self):
+        self.site_ready_worker = SiteReadyWorker(self.window().url)
+        self.site_ready_worker.signals.ready.connect(self.onServerReady)
+        self.site_ready_worker.signals.error.connect(self.onServerError)
+
+        srw_thread = threading.Thread(target=self.site_ready_worker.run, daemon=True)
+        srw_thread.start()
+
+    def onServerReady(self):
+        if not self.serverError:
+            self.resetProgress()
+            self.window().cef_widget.browser.Navigate(self.window().url)
+            self.window().showBrowser()
+
+    def onError(self, err, server_error=False):
+        # Restart worker
+        if server_error:
+            self.serverError = True
+            # Report error and switch to load screen
+        self.window().shutdownServer()
+        self.resetProgress()
+        self.window().stacked_layout.setCurrentIndex(LOAD_INDEX)
+        self.error_label.setText(f"Error: {err}")
+        self.error_label.resize(MAX_CONTENT_WIDTH, self.error_label.height())
+        self.window().repaint()
+
+    onServerError = partialmethod(onError, server_error=True)
+
+class FilePath(QObject):
+    def __init__(self):
+        super(FilePath, self).__init__()
+        self.value = ""
+        self.signals = FileChanged()
+
+    def updateValue(self, path=None):
+        self.value = path
+        self.signals.changed.emit(self.value != path)
+
+
+class FileArea(QFrame):
+    def __init__(self, parent):
+        super(FileArea, self).__init__()
+        self.setFrameShape(QFrame.Box)
+        self.setMinimumHeight(100)
+        self.setFixedWidth(MAX_CONTENT_WIDTH)
+        self.setAcceptDrops(True)
+        self.instructions = QLabel(self)
+        self.instructions.setText("Drag & Drop a h5ad file to load or open")
+        self.instructions.setGeometry(10, 10, MAX_CONTENT_WIDTH, self.instructions.height())
+        self.loadButton = QPushButton("Open...", parent=self)
+        x_pos = (MAX_CONTENT_WIDTH - self.loadButton.width()) / 2
+        self.loadButton.setGeometry(x_pos, 50, self.loadButton.width(), self.loadButton.height())
+        self.loadButton.clicked.connect(self.fileBrowse)
+        self.label = QLabel(self)
+        self.label.setGeometry(10, 75, MAX_CONTENT_WIDTH, self.label.height())
+
+    def fileBrowse(self):
         options = QFileDialog.Options()
         # options |= QFileDialog.DontUseNativeDialog
         file_name, _ = QFileDialog.getOpenFileName(self,
-                                                  "Open H5AD File", "", "H5AD Files (*.h5ad)", options=options)
-        self.title = splitext(basename(file_name))[0]
+                                                   "Open H5AD File", "", "H5AD Files (*.h5ad)", options=options)
         if file_name:
-            self.signals.selectedFile.emit(file_name)
+            self.parent().file_name.updateValue(file_name)
+            self.parent().onLoad()
 
+    def dragEnterEvent(self, e):
+        if e.mimeData().hasUrls:
+            e.accept()
+        else:
+            e.ignore()
 
-    def onDataSuccess(self, data):
-        self.window().server.attach_data(data, self.title)
-        self.navigateToLocation()
-        # Reveal browser
-        self.window().stacked_layout.setCurrentIndex(1)
+    def dragMoveEvent(self, e):
+        if e.mimeData().hasUrls:
+            e.accept()
+        else:
+            e.ignore()
 
-    def onDataError(self, err):
-        self.error_label.setText(f"Error: {err}")
-        self.error_label.resize(self.MAX_CONTENT_WIDTH, self.error_label.height())
-
-    def navigateToLocation(self, location="http://localhost:8000/"):
-        self.window().cef_widget.browser.Navigate(location)
+    def dropEvent(self, e):
+        """
+        Drop files directly onto the widget
+        File locations are stored in fname
+        :param e:
+        :return:
+        """
+        if e.mimeData().hasUrls:
+            e.setDropAction(Qt.CopyAction)
+            e.accept()
+            for url in e.mimeData().urls():
+                file_name = str(url.toLocalFile())
+            self.parent().file_name.updateValue(file_name)
+            self.parent().onLoad()
+        else:
+            e.ignore()
 
 
 def main():
+    freeze_support()
     # This generates an error.log file on error
     sys.excepthook = cef.ExceptHook  # To shutdown all CEF processes on error
     settings = {}
@@ -200,19 +369,26 @@ def main():
     cef.Initialize(settings)
     app = CefApplication(sys.argv)
     main_window = MainWindow()
+    main_window.setWindowTitle("cellxgene")
+    main_window.setUnifiedTitleAndToolBarOnMac(True)
+    main_window.setWindowIcon(QIcon(":icon.png"))
     main_window.show()
     main_window.activateWindow()
     main_window.raise_()
-    app.exec_()
+    try:
+        app.exec_()
+    except Exception as e:
+        raise
+    finally:
+        # Clean up on close
+        if not cef.GetAppSetting("external_message_pump"):
+            app.stopTimer()
 
-    # Clean up on close
-    if not cef.GetAppSetting("external_message_pump"):
-        app.stopTimer()
-    # TODO clean up threads when we switch threading model
-    del main_window  # Just to be safe, similarly to "del app"
-    del app  # Must destroy app object before calling Shutdown
-    cef.Shutdown()
-    sys.exit(0)
+        main_window.shutdownServer()
+        del main_window  # Just to be safe, similarly to "del app"
+        del app  # Must destroy app object before calling Shutdown
+        cef.Shutdown()
+        sys.exit(0)
 
 
 if __name__ == '__main__':

server/gui/utils.py

@@ -1,3 +1,4 @@
+import errno
 import platform
 
 from PySide2.QtCore import QObject, Signal
@@ -13,13 +14,66 @@ class WorkerSignals(QObject):
     Defines the signals available from a running worker thread.
     Supported signals are:
         finished
+        ready
         error - `str` error message
         result - `object` data returned from processing, anything
     """
     finished = Signal()
+    engine_error = Signal(str)
+    server_error = Signal(str)
     error = Signal(str)
     result = Signal(object)
+    ready = Signal()
+
+
+class SiteReadySignals(QObject):
+    """
+    Defines the signals available from a running worker thread.
+    Supported signals are:
+        timeout
+        ready
+        error - `str` error message
+    """
+    ready = Signal()
+    timeout = Signal()
+    error = Signal(str)
 
 
 class FileLoadSignals(QObject):
     selectedFile = Signal(str)
+    error = Signal(str)
+
+
+class FileChanged(QObject):
+    changed = Signal(bool)
+
+
+class Emitter:
+    def __init__(self, transport, signals):
+        self.transport = transport
+        self.signals = signals()
+
+    def _emit(self, signature, args=None):
+        if args is None:
+            getattr(self.signals, signature).emit()
+        else:
+            getattr(self.signals, signature).emit(args)
+
+    def run(self):
+        while True:
+            try:
+                signature = self.transport.recv()
+            except EOFError:
+                # Server done
+                break
+            except OSError as e:
+                if e.errno == errno.EBADF:
+                    break
+                else:
+                    self.signals.error.emit(str(e))
+                    break
+            except Exception as e:
+                self.signals.error.emit(str(e))
+                break
+            else:
+                self._emit(*signature)

server/gui/workers.py

@@ -1,47 +1,89 @@
-import traceback
+from multiprocessing import Process
+import time
 
-from server.gui.utils import WorkerSignals
+import requests
+
+from server.gui.utils import SiteReadySignals
 
 
-class DataLoadWorker():
-    def __init__(self, data_file, layout="umap", *args, **kwargs):
-        super(DataLoadWorker, self).__init__()
-        self.data_file = data_file
-        self.layout = layout
-        self.signals = WorkerSignals()
+class EmittingProcess(Process):
+    def __init__(self, parent_conn, child_conn, *arg, **kwargs):
+        super(EmittingProcess, self).__init__()
+        self.parent_conn = parent_conn
+        self.child_conn = child_conn
 
     def run(self):
-        if not self.data_file:
-            self.signals.finished.emit()
-            return
+        self.parent_conn.close()
 
-        # delayed import to speed load
-        from server.app.scanpy_engine.scanpy_engine import ScanpyEngine
-        args = {
-            "layout": self.layout,
-            "diffexp": "ttest",
-            "max_category_items": 100,
-            "diffexp_lfc_cutoff": 0.01,
-            "obs_names": None,
-            "var_names": None,
-        }
-        try:
-            data_results = ScanpyEngine(self.data_file, args)
-        except Exception as e:
-            traceback.print_exc()
-            self.signals.error.emit(str(e))
-        else:
-            self.signals.result.emit(data_results)
-        finally:
-            self.signals.finished.emit()
+    def emit(self, signal_name, *args):
+        message = (signal_name, *args)
+        self.child_conn.send(message)
 
 
-class ServerRunWorker():
-    def __init__(self, app, host, port, *args, **kwargs):
-        super(ServerRunWorker, self).__init__()
-        self.app = app
+class Worker(EmittingProcess):
+    def __init__(self, parent_conn, child_conn, data_file, host, port, title, engine_options, *args, **kwargs):
+        super(Worker, self).__init__(parent_conn, child_conn)
+        self.data_file = data_file
         self.host = host
         self.port = port
+        self.title = title
+        self.engine_options = engine_options
 
     def run(self):
-        self.app.run(host=self.host, debug=False, port=self.port, threaded=True)
+        super(Worker, self).run()
+        if not self.data_file:
+            self.emit("finished")
+            return
+        from server.app.app import Server
+        from server.app.scanpy_engine.scanpy_engine import ScanpyEngine
+        # create server
+        try:
+            server = Server()
+            server.create_app()
+        except Exception as e:
+            self.emit("server_error", str(e))
+            self.emit("finished")
+            return
+        # load data
+        try:
+            args = {
+                "max_category_items": 100,
+                "diffexp_lfc_cutoff": 0.01,
+                "obs_names": None,
+                "var_names": None,
+            }
+            args.update(self.engine_options)
+            data = ScanpyEngine(self.data_file, args)
+            server.attach_data(data, self.title)
+            self.emit("ready")
+        except Exception as e:
+            self.emit("engine_error", str(e))
+            self.emit("finished")
+            return
+        # launch server
+        try:
+            server.app.run(host=self.host, debug=False, port=self.port, threaded=True)
+        except Exception as e:
+            self.emit("server_error", str(e))
+        finally:
+            self.emit("finished")
+
+
+class SiteReadyWorker:
+    def __init__(self, location):
+        super(SiteReadyWorker, self).__init__()
+        self.signals = SiteReadySignals()
+        self.location = location
+
+    def run(self):
+        session = requests.Session()
+        for i in range(90):
+            try:
+                session.head(self.location)
+                self.signals.ready.emit()
+                break
+            except requests.exceptions.ConnectionError:
+                time.sleep(1)
+            except Exception as e:
+                self.signals.error.emit(str(e))
+        self.signals.timeout.emit()

server/requirements-gui.txt

@@ -0,0 +1,4 @@
+cefpython3>=66
+requests
+PyInstaller>=3.4
+PySide2>=5.12.3

server/requirements.txt

@@ -6,10 +6,10 @@ Flask-Compress>=1.4.0
 Flask-Cors>=3.0.6
 Flask-RESTful>=0.3.6
 flatbuffers>=1.10.0
+scanpy>=1.3.7
 matplotlib>=2.2
 numpy>=1.15.2
 pandas>=0.23.1
-scanpy>=1.3.7
 scipy>=1.1.0
 scikit-learn>=0.19.1,!=0.20.0
 tables==3.5.1

server/test/test_nan_scanpy_engine.py

@@ -13,7 +13,6 @@ class NaNTest(unittest.TestCase):
     def setUp(self):
         self.args = {
             "layout": ["umap"],
-            "diffexp": "ttest",
             "max_category_items": 100,
             "obs_names": None,
             "var_names": None,

server/test/test_scanpy_engine.py

@@ -16,7 +16,6 @@ class EngineTest(unittest.TestCase):
     def setUp(self):
         args = {
             "layout": ["umap"],
-            "diffexp": "ttest",
             "max_category_items": 100,
             "obs_names": None,
             "var_names": None,
@@ -78,8 +77,6 @@ class EngineTest(unittest.TestCase):
     def test_schema(self):
         with open(path.join(path.dirname(__file__), "schema.json")) as fh:
             schema = json.load(fh)
-            print(schema)
-            print(self.data.schema)
             self.assertEqual(self.data.schema, schema)
 
     def test_schema_produces_error(self):

server/test/test_scanpy_engine_data_load.py

@@ -16,7 +16,6 @@ class DataLoadEngineTest(unittest.TestCase):
     def test_delayed_load_args(self):
         args = {
             "layout": ["tsne"],
-            "diffexp": "ttest",
             "max_category_items": 1000,
             "obs_names": "foo",
             "var_names": "bar",

setup.cfg

@@ -1,4 +1,6 @@
 [flake8]
 max-line-length = 120
 ignore = E203, W503
-exclude = server/app/util/fbs/NetEncoding/
+exclude =
+    server/app/util/fbs/NetEncoding/,
+    server/gui/cellxgene_rc.py

setup.py

@@ -8,7 +8,7 @@ with open("server/requirements.txt") as fh:
 
 setup(
     name="cellxgene",
-    version="0.10.1",
+    version="0.11.1",
     packages=find_packages(),
     url="https://github.com/chanzuckerberg/cellxgene",
     license="MIT",
@@ -36,5 +36,5 @@ setup(
         "Topic :: Scientific/Engineering :: Bio-Informatics",
     ],
     entry_points={"console_scripts": ["cellxgene = server.cli.cli:cli"]},
-    extras_require=dict(louvain=["python-igraph", "louvain>=0.6"], gui=["PySide2>=5.12.3", "cefpython3>=66"]),
+    extras_require=dict(louvain=["python-igraph", "louvain>=0.6"], gui=["PySide2>=5.12.3", "cefpython3>=66", "requests"]),
 )