fork/cellxgene
1
0
Fork 0
mirror of https://github.com/chanzuckerberg/cellxgene.git synced 2026-09-15 20:57:56 +08:00
Code Issues Packages Projects Releases Wiki Activity

Refactor developer convenience scripts (#1377)

Browse Source 
* Cleanup the backend-dev convenience method

* Add the frontend_dev convenience method

frontend_dev is a soup-to-nuts convenience method for setting up the FE
development environment with node running a the client code on port 3000
with the a separate cellxgene package serving the API over port 5005 in
the background.

The script can be run from Finder.

* Update the developer scripts documentation

* Remove the 'test' make target in the client Makefile

Rationale:
* Given how long the smoke tests take to run, it is unlikely that
  developers will want to run all tests together.
* It is unlikely that developers will have set up the backend server
  properly for the tests to pass.
* Available commands should be safe-ish and not lend themselves to
  confusing errors.
* You can still group tests by concatenating them in a make command, as
  in `make unit-test smoke-test`.
* This target isn't used in any of our CI pipelines -- KISS.

* Some version of python3...

* Minor typos in docs

* Respond to feedback from @bkmartinjr

* Make adjustments so that DATASET path is predictable

* Simplify environment defaults a bit
This commit is contained in:
Matt Weiden
2020-04-15 10:32:19 -07:00
committed by GitHub
parent e5dfd6e8fa
commit 7d4d360e52
7 changed files with 146 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files

156
dev_docs/developer_scripts.md
View File

@@ -1,122 +1,114 @@
# Developer convenience methods
# Developer convenience scripts
## Makefile
This document describes scripts for accelerating cellxgene development.
Documentation for the `Makefile` targets in the project root directory.
Paths are relative to the root project directory. If you need to know what
this is, run `PROJECT_ROOT=$(git rev-parse --show-toplevel); echo
$PROJECT_ROOT`.
### Build commands
## Project-level scripts
builds source code
### Build
```
build - builds the whole app (client/ source and server/ source packaged together)
build-client - runs the client webpack build
build-for-server-dev - builds client and copies output directly into the server source tree (only for server devlopment)
```
**Usage:** from the `$PROJECT_ROOT` directory run:
* `make build` builds whole app client and server
* `make build-client` runs webpack build
* `make build-for-server-dev` builds client and copies output directly into
source tree (only for server devlopment)
### Clean commands
### Clean
deletes generated files
Deletes generated files.
```
clean - cleans everything including node modules (means build with take a while
clean-lite - cleans built directories
clean-server - cleans source tree
```
**Usage:** from the `$PROJECT_ROOT` directory run:
* `make clean` cleans everything including node modules (means build with take
a while
* `make clean-lite` cleans built directories
* `make clean-server` cleans source tree
### Dist commands
### Distribution
creates distribution for python module to upload to pypi
Creates distribution for python module to upload to pypi.
```
pydist - builds code and then builds sdist
```
**Usage:** from the `$PROJECT_ROOT` directory run:
* `make pydist` builds code and then builds sdist
### Release commands
### Release
see release_process.md
See `release_process.md`.
### Env commands
### Development environment
Installs requirements files
Installs requirements files.
```
dev-env - installs requirements and requirments-dev (for building code)
```
**Usage:** from the `$PROJECT_ROOT` directory run:
* `make dev-env` installs requirements and requirments-dev (for building code)
### install commands
### Installing cellxgene packages
Installs cellxgene from different locations
**Usage:** from the `$PROJECT_ROOT` directory:
* `install-dev` - installs from local source tree
* `install-release-test` - installs from test pypi
* `install-release` - installs from pypi
* `install-dist` - installs from local dist folder
* `uninstall` - uninstalls cellxgene
```
install-dev - installs from local source tree
install-release-test - installs from test pypi
install-release - installs from pypi
install-dist - installs from local dist folder
uninstall - uninstalls cellxgene
```
## Client-level scripts
## Client Makefile
The following phony `make` targets in `client/Makefile` are convenience methods for getting you up and developing.
### Runner scripts
### Running the client
#### start-frontend
**About** Serve the current client javascript independently from the `server` code.
**Requires**
* The server to be running. Best way to do this is with `make backend-dev`
* The server to be running. Best way to do this is with [backend_dev](#backend_dev).
* `make ci` to install the necessary node modules
**Usage** `make start-frontend`
**Usage:** from the `$PROJECT_ROOT/client` directory run `make start-frontend`
#### backend-dev
#### backend_dev
**About** This script enables FE developers to run the REST API necessary to back the development server for the front end. It is intended to ensure that the FE developer gets the current version of the backend with a single command and no knowledge of python necessary. It creates and activates a virtual environment and installs cellxgene from the current branch.
**About** This script enables FE developers to run the REST API necessary to
back the development server for the front end. It is intended to ensure that
the FE developer gets the current version of the backend with a single command
and no knowledge of python necessary. It creates and activates a virtual
environment and installs cellxgene from the current branch.
**Requires** Python3.6 - `virtual-env`, `pip`
**Requires** `Python3.6+`, `virtual-env`, `pip`
**Usage** `make backend-dev`. Optionally, you can then launch the node development server to serve the current state of
the FE with `make start-frontend`. You can also select a specific dataset using `DATASET=<dataset path> make backend-dev`.
You can also use `CXG_OPTIONS` to pass options to the `cellxgene launch` command, as in
`CXG_OPTIONS='--experimental-annotations --experimental-annotations-file annotations.csv' make backend-dev`.
**Usage:** from the `$PROJECT_ROOT` directory run `./scripts/backend_dev`
**Tips** Developers will probably want to run this in parallel with the node dev server. You can either do this by running each in a separate termanial window or by running `backend-dev` in the background (add an `&` at the end of the command to run in the background: `DATASET=<dataset> make backend-dev &`).
**Options:**
* In parallel, you can then launch the node development server to serve the
current state of the FE with [`start-frontend`](#start-frontend), usually in
a different terminal tab.
* You can also select a specific dataset using `DATASET=<dataset path> ./scripts/backend_dev`.
* You can also use `CXG_OPTIONS` to pass options to the `cellxgene launch`
command, as in `CXG_OPTIONS='--disable-annotations' ./scripts/backend_dev`.
**Breakdown**
| command | purpose |
| ---------------------------------------- | --------------------------------------------------------- |
| python3.6 -m venv cellxgene | creates cellxgene virtual environment |
| source cellxgene/bin/activate | activates virtual environment |
| yes \| pip uninstall cellxgene \|\| true | uninstalls cellxgene (if installed) |
| pip install -e . | installs current local version of cellxgene |
| command | purpose |
| ---------------------------------------- | ---------------------------------------------------------- |
| python3.6 -m venv cellxgene | creates cellxgene virtual environment |
| source cellxgene/bin/activate | activates virtual environment |
| yes \| pip uninstall cellxgene \|\| true | uninstalls cellxgene (if installed) |
| pip install -e . | installs current local version of cellxgene |
| cellxgene launch | launches cellxgene (must supply dataset as last parameter) |
### Test scripts
### Client test scripts
#### test
Methods used to test the client javascript code
**About** Run test locally. In order for this command to succeed you will need to give it a specific unit test to run. It won't pass if you run all tests as may be expected. This is because the unit tests and end to end (e2e) tests require different testing environments.
**Usage** `make test`
#### unit-test
**About** Runs all unit tests. It excludes any tests in the e2e folder. This is used by travis to run unit tests.
**Usage** `make unit-test`
#### smoke-test
**About** Starts backend development server and runs end to end tests. This is what travis runs. It depends on the `e2e` and the `backend-dev` targets. One starts the server, the other runs the tests. If developing a front-end feature and just checking if tests pass, this is probabaly the one you want to run.
**Usage** `make smoke-test`
#### e2e
**About** Runs backend tests without starting the server. You will need to start the rest api separately with the pbmc3k.h5ad file. Note you can use the `JEST_ENV` environment variable to change how JEST runs in the browser.
**Usage** `make e2e`
**Usage:** from the `$PROJECT_ROOT/client` directory run:
* `make unit-test` Runs all unit tests. It excludes any tests in the e2e
folder. This is used by travis to run unit tests.
* `make smoke-test` Starts backend development server and runs end to end
tests. This is what travis runs. It depends on the `e2e` and the
`backend-dev` targets. One starts the server, the other runs the tests. If
developing a front-end feature and just checking if tests pass, this is
probabaly the one you want to run.
* `make e2e` Runs backend tests without starting the server. You will need to
start the rest api separately with the pbmc3k.h5ad file. Note you can use
the `JEST_ENV` environment variable to change how JEST runs in the browser.