fork/cellxgene
1
0
Fork 0
mirror of https://github.com/chanzuckerberg/cellxgene.git synced 2026-09-15 12:47:56 +08:00
Code Issues Packages Projects Releases Wiki Activity

Makefile modularity, test targets, and auto-formatting (#1070)

Browse Source 
* Fix Makefile whitespace and .PHONY use

* Fix Makefile filename

* Modularize Makefile into client and server Makefiles

Part of the reason that the Makefile in the root directory is a bit
complicated is that it tries to handle tasks that can be handled
separately in the client and server modules.

This commit pushes some of the make logic specific to each module into
their own makefiles and calls out to those makefiles from that in the
project root.

* Add auto-formatting to client and server modules

One thing that can make linting faster is auto-formatting. This commit
adds the yapf auto-formatting tool to the server module and uses
eslint's "fix" functionality to speed up the linting/formatting process.

* Add yapf for automatic code formatting

* Add a root test target that calls sub-tests

* Apply yapf to python files

* Do not duplicate npm commands, simply pass through

* Update documentation

* Do not shadow reserved word len

* Add general test target

* Fix make call in dev-env

* Use black instead of yapf

* Run flake8 from the root directory

* Revert "Apply yapf to python files"

This reverts commit cdca128a01.

* Apply black to python code

* Resolve lint errors resulting from black format

* Add explanation of server unit tests in dev guidelines
This commit is contained in:
Matt Weiden
2019-12-27 14:43:37 -08:00
committed by GitHub
parent ec79995be8
commit f3015cb9df
37 changed files with 738 additions and 806 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
dev_docs/developer_guidelines.md
View File

@@ -9,6 +9,38 @@
**All instructions are expected to be run from the top level cellxgene directory unless otherwise specified.**
## Running test suite
Client and server tests run on Travis CI for every push, PR, and commit to master on github. End to end tests run nightly on master only.
### Unit tests
Steps to run the all unit tests:
1. Start in the project root directory
1. `make dev-env`
1. `make unit-test`
### End to end tests
End to end tests use two env variables:
* `JEST_ENV` - environment to run end to end tests. Default `dev`
* `prod` - run headless with no slowdown, chromium will not open.
* `dev` - opens chromimum, runs tests with minimal slowdown, close on exit.
* `debug` - opens chromium, runs tests with 100ms slowdown, dev tools open, chrome stays open on exit.
* `JEST_CXG_PORT` - port that end to end tests are being run on. Default `3000` (client hosted port).
On CI the end to end tests are run with `JEST_ENV` set to `prod` using the `smoke-test` make target.
To run end to end tests as they will be run on CI
1. cellxgene should be built and installed as [specified in server dev](#install)
2. `export JEST_ENV='prod'`
3. `export JEST_CXG_PORT=5000`
4. Run `npm run --prefix client/ smoke-test`
Run end to end tests interactively during development
1. cellxgene should be installed as [specified in client dev](#install-1)
2. Follow [launch](#launch-1) instructions for client dev with dataset `example-dataset/pbmc3k`
3. Run `make smoke-test`
4. To debug a failing test `export JEST_ENV='debug'` and re-run.
## Server dev
### Install
* Build the client and put static files in place: `make build-for-server-dev`
@@ -21,11 +53,15 @@
If you install cellxgene using `make install-dev` the server will be restarted every time you make changes on the server code. If changes affects the client, the browser must be reloaded.
### Linter
We use `flake8` to lint code. Travis CI runs `flake8 server`.
We use [`flake8`](https://github.com/PyCQA/flake8) to lint python and [`black`](https://pypi.org/project/black/) for auto-formatting.
To auto-format code run `make fmt`. To run lint checks on the code run `make lint`.
### Test
1. Install development requirements `pip install -r server/requirements-dev.txt`
2. Run tests `pytest server/test`
If you would like to run the server tests individually, follow the steps below
1. Install development requirements `make dev-env`
1. Run `make unit-test` in the `server` directory or `make unit-test-server` in the root directory.
### Tips
* Install in a virtualenv
@@ -33,8 +69,8 @@ We use `flake8` to lint code. Travis CI runs `flake8 server`.
## Client dev
### Install
1. Install prereqs for client: `npm install --prefix client/ client`
2. Install cellxgene server: `pip install -e .` Caveat: this will not build the production client package - you must use the [server install](#install) instructions above to serve web assets.
1. Install prereqs for client: `make dev-env`
2. Install cellxgene server: `make install-dev` Caveat: this will not build the production client package - you must use the [server install](#install) instructions above to serve web assets.
### Launch
To launch with hot reloading you need to launch the server and the client separately. Node's hot reloading starts the client on its own node server and auto-refreshes when changes are made.
@@ -49,43 +85,12 @@ To build only the client: `make build-client`
We use `eslint` to lint the code and `prettier` as our code formatter.
### Test
In `client/` directory run `npm run unit-test`
If you would like to run the client tests individually, follow the steps below in the `client` directory
1. For unit tests run `npm run unit-test` or `make unit-test`
1. For the smoke test run `npm run smoke-test` or `make smoke-test`
### Tips
* You can also install/launch the server side code from npm scrips (requires python3.6 with virtualenv) in `client/` directory run `npm run backend-dev`
## Running tests
Client and server tests run on Travis CI for every push, PR, and commit to master on github. End to end tests run nightly on master only.
### Server unit tests
Install development requirements `pip install -r server/requirements-dev.txt`
Run tests `pytest server/test`
### Client unit tests
In `client/` directory run `npm run unit-test`
### End to end tests
End to end tests use two env variables:
* `JEST_ENV` - environment to run end to end tests. Default `dev`
* `prod` - run headless with no slowdown, chromium will not open.
* `dev` - opens chromimum, runs tests with minimal slowdown, close on exit.
* `debug` - opens chromium, runs tests with 100ms slowdown, dev tools open, chrome stays open on exit.
* `JEST_CXG_PORT` - port that end to end tests are being run on. Default `3000` (client hosted port).
On CI the end to end tests are run with `JEST_ENV` set to `prod` using the `smoke-test` npm script
To run end to end tests as they will be run on CI
1. cellxgene should be built and installed as [specified in server dev](#install)
2. `export JEST_ENV='prod'`
3. `export JEST_CXG_PORT='5000'`
4. Run `npm run --prefix client/ smoke-test`
Run end to end tests interactively during development
1. cellxgene should be installed as [specified in client dev](#install-1)
2. Follow [launch](#launch-1) instructions for client dev with dataset `example-dataset/pbmc3k`
3. Run `npm run --prefix client/ e2e`
4. To debug a failing test `export JEST_ENV='debug'` and re-run.