Bump version: 1.1.0 → 1.1.1 (#2572)

Co-authored-by: Seve Badajoz <severiano.badajoz@chanzuckerberg.com>

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.0.1
+current_version = 1.1.1
 commit = True
 parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(?:-(?P<prerel>rc)\.(?P<prerelversion>\d+))?
 serialize = 

README.md

@@ -7,27 +7,27 @@ _an interactive explorer for single-cell transcriptomics data_
 [![Compatibility Tests](https://github.com/chanzuckerberg/cellxgene/workflows/Compatibility%20Tests/badge.svg)](https://github.com/chanzuckerberg/cellxgene/actions?query=workflow%3A%22Compatibility+Tests%22)
 ![Code Coverage](https://codecov.io/gh/chanzuckerberg/cellxgene/branch/main/graph/badge.svg)
 
-cellxgene Desktop (pronounced "cell-by-gene") is an interactive data explorer for single-cell datasets, such as those coming from the [Human Cell Atlas](https://humancellatlas.org). Leveraging modern web development techniques to enable fast visualizations of at least 1 million cells, we hope to enable biologists and computational researchers to explore their data.
+CZ CELLxGENE Annotate (pronounced "cell-by-gene") is an interactive data explorer for single-cell datasets, such as those coming from the [Human Cell Atlas](https://humancellatlas.org). Leveraging modern web development techniques to enable fast visualizations of at least 1 million cells, we hope to enable biologists and computational researchers to explore their data.
 
-Whether you need to visualize one thousand cells or one million, cellxgene Desktop helps you gain insight into your single-cell data.
+Whether you need to visualize one thousand cells or one million, CELLxGENE Annotate helps you gain insight into your single-cell data.
 
 <img src="https://github.com/chanzuckerberg/cellxgene/raw/main/docs/images/crossfilter.gif" width="350" height="200" hspace="30"><img src="https://github.com/chanzuckerberg/cellxgene/raw/main/docs/images/category-breakdown.gif" width="350" height="200" hspace="30">
 
 # Getting started
 
-### The comprehensive guide to cellxgene Desktop
+### The comprehensive guide to CZ CELLxGENE Annotate
 
-[The cellxgene documentation is your one-stop-shop for information about cellxgene Desktop](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/README.md)! You may be particularly interested in:
+[The CZ CELLxGENE Annotate documentation is your one-stop-shop for information about CELLxGENE Annotate](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/README.md)! You may be particularly interested in:
 
-- Seeing [what cellxgene Desktop can do](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/explore-data/explorer-tutorials.md)
-- Learning more about cellxgene [installation](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/install.md) and [usage](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/quick-start.md#quick-start-1)
-- [Preparing your own data](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/data-reqs.md) for use in cellxgene Desktop
+- Seeing [what Annotate can do](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/explore-data/explorer-tutorials.md)
+- Learning more about Annotate [installation](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/install.md) and [usage](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/quick-start.md#quick-start-1)
+- [Preparing your own data](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/data-reqs.md) for use in Annotate
 - Checking out [our roadmap](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/roadmap.md) for future development
-- [Contributing](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/contribute.md) to cellxgene Desktop
+- [Contributing](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/contribute.md) to Annotate
 
 ### Quick start
 
-To install cellxgene Desktop you need Python 3.6+. We recommend [installing cellxgene Desktop into a conda or virtual environment.](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/install.md)
+To install CELLxGENE Annotate you need Python 3.6+. We recommend [installing Annotate into a conda or virtual environment.](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/install.md)
 
 Install the package.
 
@@ -35,19 +35,19 @@ Install the package.
 pip install cellxgene
 ```
 
-Launch cellxgene Desktop with an example [anndata](https://anndata.readthedocs.io/en/latest/) file
+Launch Annotate with an example [anndata](https://anndata.readthedocs.io/en/latest/) file
 
 ```bash
 cellxgene launch https://cellxgene-example-data.czi.technology/pbmc3k.h5ad
 ```
 
-To explore more datasets already formatted for cellxgene Desktop, check out the [Demo data](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/quick-start.md#example-datasets) or
+To explore more datasets already formatted for Annotate, check out the [Demo data](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/quick-start.md#example-datasets) or
 see [Preparing your data](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/desktop/data-reqs.md) to learn more about formatting your own
-data for cellxgene Desktop.
+data for CELLxGENE Annotate.
 
 ### Supported browsers
 
-cellxgene Desktop currently supports the following browsers:
+CELLxGENE Annotate currently supports the following browsers:
 
 - Google Chrome 61+
 - Edge 15+
@@ -62,11 +62,11 @@ For questions, suggestions, or accolades, [join the `#cellxgene-users` channel o
 
 For any errors, [report bugs on Github](https://github.com/chanzuckerberg/cellxgene/issues).
 
-# Developing with cellxgene Desktop
+# Developing with CZ CELLxGENE Annotate
 
 ### Contributing
 
-We warmly welcome contributions from the community! Please see our [contributing guide](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/contribute.md) and don't hesitate to open an issue or send a pull request to improve cellxgene Desktop. Please see the [dev_docs](https://github.com/chanzuckerberg/cellxgene/tree/main/dev_docs) for pull request suggestions, unit test details, local documentation preview, and other development specifics. 
+We warmly welcome contributions from the community! Please see our [contributing guide](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/contribute.md) and don't hesitate to open an issue or send a pull request to improve CELLxGENE Annotate. Please see the [dev_docs](https://github.com/chanzuckerberg/cellxgene/tree/main/dev_docs) for pull request suggestions, unit test details, local documentation preview, and other development specifics. 
 
 This project adheres to the Contributor Covenant [code of conduct](https://github.com/chanzuckerberg/.github/blob/master/CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to opensource@chanzuckerberg.com.
 
@@ -77,11 +77,11 @@ As such, we encourage other scientific tool builders in academia or industry to
 this project. All code is freely available for reuse under the [MIT license](https://opensource.org/licenses/MIT).
 
 
-Before extending cellxgene, we encourage you to reach out to us with ideas or questions. It might be possible that an 
+Before extending CELLxGENE Annotate, we encourage you to reach out to us with ideas or questions. It might be possible that an 
 extension could be directly contributed, which would make it available for a wider audience, or that it's on our 
 [roadmap](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/roadmap.md) and under active development. 
 
-See the [cellxgene extensions](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/community-extensions.md) section of our documentation for examples of community use and cellxgene extensions. 
+See the [CELLxGENE extensions](https://github.com/chanzuckerberg/cellxgene-documentation/blob/main/community-extensions.md) section of our documentation for examples of community use and CELLxGENE extensions. 
 
 ### Security
 

client/configuration/webpack/webpack.config.dev.js

@@ -9,6 +9,7 @@ const sharedConfig = require("./webpack.config.shared");
 const babelOptions = require("../babel/babel.dev");
 
 const fonts = path.resolve("src/fonts");
+const images = path.resolve("src/images");
 const nodeModules = path.resolve("node_modules");
 
 const devConfig = {
@@ -28,11 +29,11 @@ const devConfig = {
       {
         test: /\.(jpg|png|gif|eot|svg|ttf|woff|woff2|otf)$/i,
         loader: "file-loader",
-        include: [nodeModules, fonts],
+        include: [nodeModules, fonts, images],
         options: {
           name: "static/assets/[name].[ext]",
           // (thuang): This is needed to make sure @font url path is '/static/assets/'
-          publicPath: "/",
+          publicPath: "..",
         },
       },
     ],

client/configuration/webpack/webpack.config.prod.js

@@ -14,6 +14,7 @@ const CspHashPlugin = require("./cspHashPlugin");
 const sharedConfig = require("./webpack.config.shared");
 
 const fonts = path.resolve("src/fonts");
+const images = path.resolve("src/images");
 const nodeModules = path.resolve("node_modules");
 
 const prodConfig = {
@@ -43,11 +44,11 @@ const prodConfig = {
       {
         test: /\.(jpg|png|gif|eot|svg|ttf|woff|woff2|otf)$/i,
         loader: "file-loader",
-        include: [nodeModules, fonts],
+        include: [nodeModules, fonts, images],
         options: {
           name: "static/assets/[name]-[contenthash].[ext]",
           // (thuang): This is needed to make sure @font url path is '../static/assets/'
-          publicPath: "static/",
+          publicPath: "..",
         },
       },
     ],

client/index.html

@@ -3,7 +3,7 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1" />
-    <title>cell&times;gene</title>
+    <title>CELL&times;GENE | Annotate</title>
     <style>
       html,
       body,

client/index_template.html

@@ -3,7 +3,7 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1" />
-    <title>cell&times;gene</title>
+    <title>CELL&times;GENE | Annotate</title>
     <style>
       html,
       body,

client/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "cellxgene",
-  "version": "1.0.1",
+  "version": "1.1.1",
   "lockfileVersion": 2,
   "requires": true,
   "packages": {
     "": {
       "name": "cellxgene",
-      "version": "1.0.1",
+      "version": "1.1.1",
       "license": "MIT",
       "dependencies": {
         "@babel/eslint-parser": "^7.15.0",

client/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cellxgene",
-  "version": "1.0.1",
+  "version": "1.1.1",
   "license": "MIT",
   "description": "cellxgene is a web application for the interactive exploration of single cell sequence data.",
   "repository": "https://github.com/chanzuckerberg/cellxgene",

client/src/components/app.js

@@ -41,7 +41,7 @@ class App extends React.Component {
     const { loading, error, graphRenderCounter } = this.props;
     return (
       <Container>
-        <Helmet title="cellxgene" />
+        <Helmet title="CELL&times;GENE | Annotate" />
         {loading ? (
           <div
             style={{

client/src/components/framework/logo.js

@@ -1,17 +1,15 @@
 import React from "react";
-import * as globals from "../../globals";
+import icon from "../../images/icon.png";
 
 const Logo = (props) => {
   const { size } = props;
   return (
-    <svg width={size} height={size} viewBox="0 0 48 48" fill="none">
-      <rect width="48" height="48" fill="white" />
-      <rect width="48" height="48" fill={globals.logoColor} />
-      <rect x="19" y="19" width="22" height="22" fill="white" />
-      <rect x="24" y="24" width="12" height="12" fill={globals.logoColor} />
-      <rect x="7" y="19" width="7" height="22" fill="white" />
-      <rect x="19" y="7" width="22" height="7" fill="white" />
-    </svg>
+    <img
+      src={icon}
+      height={size}
+      width={size}
+      alt="CELLxGENE Annotate Logo"
+     />
   );
 };
 

server/__init__.py

@@ -2,7 +2,7 @@ import logging
 import sys
 from server.common.utils.utils import import_plugins
 
-__version__ = "1.0.1"
+__version__ = "1.1.1"
 display_version = "cellxgene v" + __version__
 
 try:

server/cli/annotate.py

@@ -4,7 +4,7 @@ import os.path
 import shlex
 import shutil
 import subprocess
-import sys
+from os.path import isfile
 from subprocess import STDOUT, PIPE
 from tempfile import NamedTemporaryFile
 
@@ -27,23 +27,42 @@ def annotate_args(func):
 
 @sort_options
 @click.command(
-    short_help="Annotate H5AD file columns. Run `cellxgene annotation --help` for more information.",
-    options_metavar="<options>",
+    options_metavar="<options>"
 )
-@click.option(
-    "-i",
-    "--input-h5ad-file",
+@click.argument(
+    "input_h5ad_file",
+    type=click.Path(exists=True, dir_okay=False, readable=True),
+    nargs=1,
+    metavar="<path to H5AD input file>",
     required=True,
-    type=str,
-    help="The input H5AD file containing the missing annotations.",
 )
 @click.option(
     "-m",
     "--model-url",
+    # Making this a required "option", rather than an "argument", since we support automatic model selection in the
+    # future, in which case the user would not need to specify this option at all and we can make it optional at
+    # that time.
     required=True,
     help="The URL of the model used to prediction annotated labels. May be a local filesystem directory "
     "or S3 path (s3://)",
 )
+@click.option(
+    "-o",
+    "--output-h5ad-file",
+    default="",
+    help="The output H5AD file that will contain the generated annotation values. If this option is not provided, "
+         "the input file will be overwritten to include the new annotations; in this case you must specify "
+         "--overwrite.",
+    metavar="<filename>",
+)
+@click.option(
+    "--overwrite",
+    default=False,
+    is_flag=True,
+    help="Allow overwriting of the specified H5AD output file, if it exists. For safety, you must specify this "
+         "flag if the specified output file already exists or if the --output-h5ad-file option is not provided.",
+    show_default=True,
+)
 @click.option(
     "-l",
     "--counts-layer",
@@ -53,8 +72,8 @@ def annotate_args(func):
 @click.option(
     "-g",
     "--gene-column-name",
-    help="The name of the `var` column that contains gene identifiers. The values in this column will be used to match "
-    "genes between the query and reference datasets. If not specified, the gene identifiers are expected to exist "
+    help="The name of the `var` column that contains gene names. The values in this column will be used to match "
+    "genes between the query and reference datasets. If not specified, the gene names are expected to exist "
     "in `var.index`.",
 )
 # TODO: Useful if we want to support discoverability of models
@@ -91,19 +110,6 @@ def annotate_args(func):
     "will store the predicted annotation values and confidence scores. This can be used to allow multiple "
     "annotation predictions to be run on a single AnnData object.",
 )
-@click.option(
-    "-u",
-    "--update-h5ad-file",
-    is_flag=True,
-    help="Flag indicating whether to update the input h5ad file with annotation values.  This option is mutually "
-    "exclusive with --output-h5ad-file.",
-)
-@click.option(
-    "-o",
-    "--output-h5ad-file",
-    help="The output H5AD file that will contain the generated annotation values. This option is mutually "
-    "exclusive with --update-h5ad-file.",
-)
 @click.option("--use-model-cache/--no-use-model-cache", default=True)
 @click.option(
     "--use-gpu/--no-use-gpu",
@@ -141,6 +147,9 @@ def annotate_args(func):
 )
 @click.help_option("--help", "-h", help="Show this message and exit.")
 def annotate(**cli_args):
+    """
+    Add predicted annotations to an H5AD file. Run `cellxgene annotate --help` for more information.
+    """
     _validate_options(cli_args)
 
     print(f"Reading query dataset {cli_args['input_h5ad_file']}...")
@@ -149,7 +158,11 @@ def annotate(**cli_args):
         filter(None, [cli_args.get("annotation_prefix"), cli_args.get("annotation_type"), cli_args.get("run_name")])
     )
 
-    output_h5ad_file = cli_args["input_h5ad_file"] if cli_args["update_h5ad_file"] else cli_args["output_h5ad_file"]
+    output_h5ad_file = (
+        cli_args["input_h5ad_file"]
+        if cli_args["overwrite"] and not cli_args["output_h5ad_file"]
+        else cli_args["output_h5ad_file"]
+    )
 
     model_url = cli_args.get("model_url")
     local_model_path = _retrieve_model(cli_args.get("model_cache_dir"), model_url, cli_args.get("use_model_cache"))
@@ -196,7 +209,7 @@ def annotate(**cli_args):
 
             p.wait()
             if p.returncode == 0:
-                print(f"Wrote annotations to {cli_args.get('output_h5ad_file')}")
+                print(f"Wrote annotations to {output_h5ad_file}")
             else:
                 print("Annotation failed!")
     else:
@@ -218,13 +231,11 @@ def _retrieve_model(model_cache_dir, model_url, use_cache=True):
 
 
 def _validate_options(cli_args):
-    # TODO(atolopko): Use cloup library for this logic
-    if cli_args["update_h5ad_file"] and cli_args["output_h5ad_file"]:
-        click.echo("--update_h5ad_file and --output_h5ad_file are mutually exclusive")
-        sys.exit(1)
-    if not (cli_args["update_h5ad_file"] or cli_args["output_h5ad_file"]):
-        click.echo("--update_h5ad_file or --output_h5ad_file must be specified")
-        sys.exit(1)
+    output = cli_args["output_h5ad_file"]
+    overwrite = cli_args["overwrite"]
+
+    if isfile(output) and not overwrite:
+        raise click.UsageError(f"Cannot overwrite existing file {output}, try using the flag --overwrite")
 
 
 if __name__ == "__main__":

server/common/utils/utils.py

@@ -98,8 +98,7 @@ def custom_format_warning(msg, *args, **kwargs):
 
 
 def jsonify_strict(data):
-    return json.dumps(data, cls=StrictJSONEncoder, allow_nan=False)
-
+    return StrictJSONEncoder().encode(data)
 
 def import_plugins(plugin_module):
     """

setup.py

@@ -14,7 +14,7 @@ with open("server/requirements-annotate.txt") as fh:
 
 setup(
     name="cellxgene",
-    version="1.0.1",
+    version="1.1.1",
     packages=find_packages(),
     url="https://github.com/chanzuckerberg/cellxgene",
     license="MIT",

test/unit/cli/test_annotate.py

@@ -1,7 +1,7 @@
 import os
 import shutil
 import unittest
-from tempfile import mkstemp, TemporaryDirectory
+from tempfile import mkstemp, TemporaryDirectory, NamedTemporaryFile
 
 import mlflow
 from click.testing import CliRunner
@@ -12,10 +12,8 @@ from test.unit.cli.fixtures.mlflow_model_fixture import FakeModel
 
 def write_model(model) -> str:
     with TemporaryDirectory() as mlflow_model_dir:
-        fixtures_path = os.path.join(os.path.dirname(__file__), 'fixtures')
-        mlflow.pyfunc.save_model(mlflow_model_dir,
-                                 loader_module='fixtures',
-                                 code_path=[fixtures_path])
+        fixtures_path = os.path.join(os.path.dirname(__file__), "fixtures")
+        mlflow.pyfunc.save_model(mlflow_model_dir, loader_module="fixtures", code_path=[fixtures_path])
         return shutil.make_archive(mkstemp()[1], "zip", mlflow_model_dir)
 
 
@@ -46,7 +44,6 @@ class TestCliAnnotate(unittest.TestCase):
         result = CliRunner().invoke(
             annotate,
             [
-                "--input-h5ad-file",
                 query_dataset_file_path,
                 "--model-url",
                 model_file_path,
@@ -54,7 +51,8 @@ class TestCliAnnotate(unittest.TestCase):
                 f"{query_dataset_file_path}.output",
                 # avoid having mflow create conda env or virtualenv when in test env;
                 # this avoids making pip remote requests and is also faster
-                "--mlflow-env-manager", "local"
+                "--mlflow-env-manager",
+                "local",
             ],
         )
 
@@ -74,31 +72,52 @@ class TestCliAnnotate(unittest.TestCase):
             result.stdout,
             "inputs passed correctly",
         )
-
-    def test__annotate__verifies_mutually_exclusive_options(self):
-        required_options = ["--input-h5ad-file", "some.h5ad", "--model-url", "some_url"]
-        result = CliRunner().invoke(
-            annotate,
-            required_options + [],
-        )
-
-        self.assertNotEqual(0, result.exit_code, "aborts with non-success code")
         self.assertIn(
-            "--update_h5ad_file or --output_h5ad_file must be specified",
+            f"Wrote annotations to {query_dataset_file_path}.output",
             result.stdout,
-            "error message displayed",
+            "success message is correct",
         )
 
-        result = CliRunner().invoke(
-            annotate, required_options + ["--output-h5ad-file", "some_arg", "--update-h5ad-file"]
-        )
+    def test__annotate__requires_overwrite_option_when_output_file_exists(self):
 
-        self.assertNotEqual(0, result.exit_code, "aborts with non-success code")
-        self.assertIn(
-            "--update_h5ad_file and --output_h5ad_file are mutually exclusive",
-            result.stdout,
-            "error message displayed",
-        )
+        with NamedTemporaryFile() as input_h5ad, NamedTemporaryFile() as existing_file:
+            required_options = [input_h5ad.name, "--output-h5ad-file", existing_file.name, "--model-url", "some_url"]
+            result = CliRunner().invoke(
+                annotate,
+                required_options + [],
+            )
+
+            self.assertNotEqual(0, result.exit_code, "aborts with non-success code")
+            self.assertIn(
+                "try using the flag --overwrite",
+                result.stdout,
+                "error message displayed",
+            )
+
+    def test__annotate__overwrite_option_allows_overwrite_of_existing_output_file(self):
+        model_file_path = write_model(FakeModel())
+
+        with NamedTemporaryFile() as existing_file:
+            required_options = [
+                existing_file.name,
+                "--output-h5ad-file",
+                existing_file.name,
+                "--overwrite",
+                "--model-url",
+                model_file_path,
+            ]
+            result = CliRunner().invoke(
+                annotate,
+                required_options + [],
+            )
+
+            print(result.stdout)
+            self.assertNotEqual(1, result.exit_code, "aborts with non-success code")
+            self.assertIn(
+                f"Wrote annotations to {existing_file.name}",
+                result.stdout,
+                "success message is correct on output file overwrite",
+            )
 
 
 # TODO:

test/unit/utils/test_jsonify_strict.py

@@ -54,5 +54,5 @@ class TestJsonifyStrict(unittest.TestCase):
         # the actual test!
         self.assertEqual(
             jsonify_strict(values),
-            '{"floating": [100.0, 101.0, 102.0], "integer": [0, 1, 2, 3, 4, 5, 6, 7]}',
+            '{"integer": [0, 1, 2, 3, 4, 5, 6, 7], "floating": [100.0, 101.0, 102.0]}',
         )