fork/cellxgene
1
0
Fork 0
mirror of https://github.com/chanzuckerberg/cellxgene.git synced 2026-09-24 08:58:12 +08:00
Code Issues Packages Projects Releases Wiki Activity

Collect all env vars in one, easy-to-find place (#1149)

Browse Source 
* Collect all env vars in one, easy-to-find place

Past state:
* Default environement variables were stored in both client/package.json
and client/__tests__/e2e/config.js
* Constants that should have been linked--like the cellxgene server port
during testing--were repeated.

With this commit:
* All environment variables are parameterized
* All environment variables are packaged in default env files

* Move npm scripts to client Makefile

* Respond to feedback from @seve and @bkmartinjr
This commit is contained in:
Matt Weiden
2020-02-12 12:50:48 -08:00
committed by GitHub
parent 5340a5f951
commit ac13b31e13
16 changed files with 777 additions and 287 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dev_docs/developer_guidelines.md
+21 -12
View File
@@ -9,6 +9,12 @@
**All instructions are expected to be run from the top level cellxgene directory unless otherwise specified.**
### Environment
For all `make` commands, `common.mk` automatically checks whether required environment variables are set and, if they are not set, assigns them default values from `environment.default`.
You can set these environment variables manually with the `export` shell command, as in `export JEST_ENV=debug`.
## Running test suite
Client and server tests run on Travis CI for every push, PR, and commit to master on github. End to end tests run nightly on master only.
@@ -18,21 +24,19 @@ Steps to run the all unit tests:
1. `make dev-env`
1. `make unit-test`
To run unit tests for the `client` code only:
1. Start in the project root directory
1. `cd client`
1. `make unit-test`
### End to end tests
End to end tests use two env variables:
* `JEST_ENV` - environment to run end to end tests. Default `dev`
* `prod` - run headless with no slowdown, chromium will not open.
* `dev` - opens chromimum, runs tests with minimal slowdown, close on exit.
* `debug` - opens chromium, runs tests with 100ms slowdown, dev tools open, chrome stays open on exit.
* `JEST_CXG_PORT` - port that end to end tests are being run on. Default `3000` (client hosted port).
To run E2E tests, run `cd client` and `make smoke-test`
On CI the end to end tests are run with `JEST_ENV` set to `prod` using the `smoke-test` make target.
To run end to end tests as they will be run on CI use the following command:
```
JEST_ENV=prod JEST_CXG_PORT=5000 make pydist install-dist dev-env smoke-test
```
The `JEST_ENV` environment variable enables the following E2E test options:
* `dev` - opens chromimum, runs tests with minimal slowdown, close on exit.
* `debug` - opens chromium, runs tests with 100ms slowdown, dev tools open, chrome stays open on exit.
* `prod` - run headless with no slowdown, chromium will not open.
Run end to end tests interactively during development
1. cellxgene should be installed as [specified in client dev](#install-1)
@@ -40,6 +44,11 @@ Run end to end tests interactively during development
1. Run `npm run e2e` or `make e2e` from the `client` directory
1. To debug a failing test `export JEST_ENV='debug'` and re-run.
To run end to end tests _exactly_ as they will be run on CI use the following command:
```
JEST_ENV=prod make pydist install-dist dev-env smoke-test
```
## Server dev
### Install
* Build the client and put static files in place: `make build-for-server-dev`
dev_docs/developer_scripts.md
+68 -51
View File
@@ -1,56 +1,8 @@
## Node scripts in package.json
# Developer convenience methods
#### backend-dev
## Makefile
**About** This scripts enables FE developers to run the rest API necessary to back the development server for the front end. It is intended to ensure that the FE developer gets the current version of the backend with a single command and no knowledge of python necessary. It creates and activates a virtual environment and installs cellxgene from the current branch.
**Requires** Python3.6 - `virtual-env`, `pip`
**Usage** `npm run backend-dev <dataset>` then you can launch the node development server to serve the current state of the FE `npm run start`
**Tips** Developers will probably want to run this in parallel with the node dev server. You can either do this by running each in a separate termanial window or by running `backend-dev` in the background (add an `&` at the end of the command to run in the background: `npm run backend-dev <dataset> &`).
**Breakdown**
| command | purpose |
| ---------------------------------------- | --------------------------------------------------------- |
| python3.6 -m venv cellxgene | creates cellxgene virtual environment |
| source cellxgene/bin/activate | activates virtual environment |
| yes \| pip uninstall cellxgene \|\| true | uninstalls cellxgene (if installed) |
| pip install -e .. | installs current local version of cellxgene |
| cellxgene launch | launches cellxgene (must supply dataset as last parameter |
### Test scripts
#### test
**About** Run test locally. In order for this command to succeed you will need to give it a specific unit test to run. It won't pass if you run all tests as may be expected. This is because the unit tests and end to end (e2e) tests require different testing environments.
**Usage** `npm run test <test file or pattern>`
#### unit-test
**About** Runs all unit tests. It excludes any tests in the e2e folder. This is used by travis to run unit tests.
**Usage** `npm run unit-test`
#### smoke-test
**About** Starts backend development server and runs end to end tests. This is what travis runs. It depends on the `e2e` and the `start-server-for-test` node scripts. One starts the server, the other runs the tests. If developing a front-end feature and just checking if tests pass, this is probabaly the one you want to run.
**Requirements** Must set env variable for port. In terminal `export JEST_ENV='dev'` `export JEST_CXG_PORT=5000`
**Usage** `npm run smoke-test`
#### e2e
**About** Runs backend tests without starting the server. You will need to start the rest api separately with the pbmc3k.h5ad file.
**Requirements** Must set env variable for port. In terminal `export JEST_ENV='dev'` `export JEST_CXG_PORT=5000`
**Usage** `npm run e2e`
## makefile
Documentation for the `Makefile` targets in the project root directory.
### Build commands
@@ -118,3 +70,68 @@ gui-spec-windows - creates the initial spec file for windows, do not run unless
gui-build-osx - builds the app from the osx spec file
gui-build-windows - builds the app from the windows spec file
```
## Client Makefile
The following phony `make` targets in `client/Makefile` are convenience methods for getting you up and developing.
### Runner scripts
#### start-frontend
**About** Serve the current client javascript independently from the `server` code.
**Requires**
* The server to be running. Best way to do this is with `make backend-dev`
* `make ci` to install the necessary node modules
**Usage** `make start-frontend`
#### backend-dev
**About** This script enables FE developers to run the REST API necessary to back the development server for the front end. It is intended to ensure that the FE developer gets the current version of the backend with a single command and no knowledge of python necessary. It creates and activates a virtual environment and installs cellxgene from the current branch.
**Requires** Python3.6 - `virtual-env`, `pip`
**Usage** `make backend-dev`. Optionally, you can then launch the node development server to serve the current state of
the FE with `make start-frontend`. You can also select a specific dataset using `DATASET=<dataset path> make backend-dev`.
You can also use `CXG_OPTIONS` to pass options to the `cellxgene launch` command, as in
`CXG_OPTIONS='--experimental-annotations --experimental-annotations-file annotations.csv' make backend-dev`.
**Tips** Developers will probably want to run this in parallel with the node dev server. You can either do this by running each in a separate termanial window or by running `backend-dev` in the background (add an `&` at the end of the command to run in the background: `DATASET=<dataset> make backend-dev &`).
**Breakdown**
| command | purpose |
| ---------------------------------------- | --------------------------------------------------------- |
| python3.6 -m venv cellxgene | creates cellxgene virtual environment |
| source cellxgene/bin/activate | activates virtual environment |
| yes \| pip uninstall cellxgene \|\| true | uninstalls cellxgene (if installed) |
| pip install -e . | installs current local version of cellxgene |
| cellxgene launch | launches cellxgene (must supply dataset as last parameter) |
### Test scripts
#### test
**About** Run test locally. In order for this command to succeed you will need to give it a specific unit test to run. It won't pass if you run all tests as may be expected. This is because the unit tests and end to end (e2e) tests require different testing environments.
**Usage** `make test`
#### unit-test
**About** Runs all unit tests. It excludes any tests in the e2e folder. This is used by travis to run unit tests.
**Usage** `make unit-test`
#### smoke-test
**About** Starts backend development server and runs end to end tests. This is what travis runs. It depends on the `e2e` and the `backend-dev` targets. One starts the server, the other runs the tests. If developing a front-end feature and just checking if tests pass, this is probabaly the one you want to run.
**Usage** `make smoke-test`
#### e2e
**About** Runs backend tests without starting the server. You will need to start the rest api separately with the pbmc3k.h5ad file. Note you can use the `JEST_ENV` environment variable to change how JEST runs in the browser.
**Usage** `make e2e`